Inconsistent objects in naturalistic scenes attract overt attention boosting the detection of changes from short-term memory

Object-scene integration is a hotly debated topic in vision science, as it relates to predictive cognitive processes. For example, objects inconsistent with their scene context (e.g., a torch vs. a toothbrush in a bathroom) are known to capture early attention and be better remembered. What is still unclear though is how short-term memory mechanisms for high-level properties of objects such as their identity or location could depend on their semantic integration with the scene. In this eye-tracking study, participants had to detect a change in one object which was either consistent or inconsistent with the scene. The object could change on three conditions: identity (an object becomes another object), location (an object moves to another position), and both (an object moves and becomes another object). Only for location changes, the object maintained the same consistency between encoding and recognition. Inconsistent objects were fixated earlier and for longer at encoding. During recognition, objects encoded as inconsistent had a higher probability of first fixation and their change was detected better and faster; especially for changes happening only on locations. Objects encoded as consistent instead were fixated proportionally less when a change happened on only its location as compared to changes in identity or both. A key insight is that objects semantically inconsistent with the scene are more strongly encoded in short-term memory because they elicit a prediction-error. In fact, when changes happen only in location, better detections and faster attentional responses are observed because the object-scene integration error is not yet resolved.