CogSci 2023 · Source-reviewed citation guide
Physion++: Evaluating Physical Scene Understanding with Objects Consisting of Different Physical Attributes in Humans and Machines
Proceedings of the Annual Meeting of the Cognitive Science Society · Poster with abstract
Physion++ evaluates physical prediction when mass, friction, elasticity, and deformability must be inferred online from how objects move and interact.
When to cite this work
Cite this CogSci Physion++ record when motivating benchmarks for latent physical-property inference or human–model gaps in physical scene prediction.
- Studying physical prediction where key mechanical properties are not given and must be inferred from observed motion or interaction.
- Comparing human judgments with video, object-centric, or physical-state model predictions under changing latent properties.
Contribution
What this paper adds
- Introduces a benchmark focused on four latent mechanical-property families—mass, friction, elasticity, and deformability.
- Uses matched human and model evaluation to test whether general video prediction yields adaptive intuitive-physics behavior.
- Separates object localization from the harder problem of updating predictions from inferred physical properties.
Evidence
What the paper reports
- The exact CogSci abstract reports that models encoding objectness and physical state tend to perform better, while remaining far from human performance.
- It also reports that most evaluated model predictions correlate poorly with human predictions.
- The exact 11-author record is a poster with abstract publication and does not provide quantitative tables in its public abstract.
Boundary
What it does not establish
- The public CogSci record supports qualitative, not paper-table-level quantitative, claims.
- A separate nine-author NeurIPS technical paper has a different title and author list; its numbers and DOI must not be silently attributed to this 11-author CogSci record.
- The benchmark concerns prediction settings where properties are inferred from observed motion and interaction, not all forms of physical reasoning.
Paper abstract
The authors' summary
Human physical scene understanding requires more than simply localizing and recognizing objects—we can quickly adapt our predictions about how a scene will unfold by incorporating objects' latent physics properties, such as the masses of the objects in the scene. What are the underlying computational mechanisms that allow humans to infer these physical properties and adapt their physical predictions so efficiently from visual inputs? One hypothesis is that general intuitive physics knowledge can be learned from enough raw data, instantiated as computational models that predict future video frames in large datasets of complex scenes. To test this hypothesis, we evaluate existing state-of-the-art video models. We measured both model and human performance on Physion++, a novel dataset and benchmark that rigorously evaluates visual physical prediction in humans and machines, under circumstances where accurate physical prediction relies on accurate estimates of the latent physical properties of objects in the scene. Specifically, we tested scenarios where accurate prediction relied on accurate estimates of objects' mechanical properties, including masses, friction, elasticity and deformability, and the values of these mechanical properties could only be inferred by observing how these objects moved and interacted with other objects and/or fluids. We found that models that encode objectness and physical states tend to perform better, yet there is still a huge gap compared to human performance. We also found most models' predictions correlate poorly with that made by humans. These results show that current deep learning models that succeed in some settings nevertheless fail to achieve human-level physical prediction in other cases, especially those where latent property inference is required.
Canonical citation data
Use the source, not a guessed citation.
The downloadable BibTeX and RIS are derived from this site's canonical bibliography entry. The contextual guidance is editorial and is not part of the paper's formal citation.