Spatial and temporal aspects in visual interaction

The formal modeling of visual interaction processes, where images are composed by placing elementary pictures on a rectangular space, has to take into account both temporal and spatial aspects. Time is relevant in that some moves are enabled or disabled by previous ones, while for the spatial aspect one can impose constraints on the positions which can be occupied by certain pictures. Interaction can then be seen as a sort of visual game, in which players have to synchronise their moves in order to make the interaction progress. However, due to this dual nature, traditional models of concurrent computation are not sufficient to give reason of specific types of non-determinism encountered in visual interaction processes. We propose a characterisation of visual interaction in terms of games, and show that the algebraic structure derived from the association of temporal and spatial structures is canonically associated with a system of logics, in which the limitations on the possible inferences imposed by this type of non-determinism can be adequately expressed. Moreover, the logics can be naturally extended by the introduction of operators which simultaneously model both temporal and modal qualifications of formulae. In particular, it becomes possible to give a formal account of the "surprises" experienced by users in visual interaction processes or, equivalently, by players of visual games. (C) 2011 Elsevier Ltd. All rights reserved.