Deductive Parsing of Visual Languages

Computational linguistics has largely focussed on written and spoken textual languages. However, humans use many other kinds of symbolic notations for communication, in particular, two-dimensional graphical notations such as mathematical notation, choreography notation, organizational charts and electrical circuit diagrams. We can term such multi-dimensional symbolic notations, visual languages. Like textual languages, many of these notations have a well defined syntax and semantics. The standard approach to computer interpretation of visual languages is to utilize parsing technologies based on multi-dimensional grammars. In this paper we investigate a new approach to parsing visual languages based on linear logic. The advantages of this logic-based approach are threefold: It provides a more adequate level for modelling the semantics of visual languages; it allows us to implement them based on automated deduction and it provides a good basis for the investigation of their formal properties. We show how attributed multiset grammars, one of the most widely used methods for multi-dimensional parsing, can be embedded into linear logic, demonstrate how parsing corresponds to linear proofs and prove the soundness and correctness of this embedding. Importantly, our embedding is into a subset of a linear logic programming language. Thus, we also demonstrate how multi-dimensional parsing can be implemented as a directly executable linear logic program.