Synthetic Multimodal Dataset for Daily Life Activities

Outline This dataset is originally created for the Knowledge Graph Reasoning Challenge for Social Issues (KGRC4SI) Video data that simulates daily life actions in a virtual space from Scenario Data. Knowledge graphs, and transcriptions of the Video Data content ("who" did what "action" with what "object," when and where, and the resulting "state" or "position" of the object). Knowledge Graph Embedding Data are created for reasoning based on machine learning  This data is open to the public as open data Details Videos mp4 format 203 action scenarios For each scenario, there is a character rear view (file name ending in 0), an indoor camera switching view (file name ending in 1), and a fixed camera view placed in each corner of the room (file name ending in 2-5). Also, for each action scenario, data was generated for a minimum of 1 to a maximum of 7 patterns with different room layouts (scenes). A total of 1,218 videos Videos with slowly moving characters simulate the movements of elderly people. Knowledge Graphs RDF format 203 knowledge graphs corresponding to the videos Includes schema and location supplement information The schema is described below SPARQL endpoints and query examples are available Script Data txt format Data provided to VirtualHome2KG to generate videos and knowledge graphs Includes the action title and a brief description in text format. Embedding Embedding Vectors in TransE, ComplEx, and RotatE. Created with DGL-KE (https://dglke.dgl.ai/doc/) Embedding Vectors created with jRDF2vec (https://github.com/dwslab/jRDF2Vec). Specification of Ontology Please refer to the specification for descriptions of all classes, instances, and properties: https://aistairc.github.io/VirtualHome2KG/vh2kg_ontology.htm Related Resources KGRC4SI Final Presentations with automatic English subtitles (YouTube) VirtualHome2KG (Software) VirtualHome-AIST (Unity) VirtualHome-AIST (Python API) Visualization Tool (Software) Script Editor (Software)