XML-Based Languages for Multimodality in Mobile Environments

The development of multimodal tools and mobile devices in particular is producing great interest, especially for accessing Web information, performing transactions, and use of services in general. This article considers the different markup languages proposed by the working groups of W3C (World Wide Web Consortium) to manage multimodal interaction and perspectives of multimodal applications and services. The trend toward the convergence of various methodologies and technologies has developed new devices providing complex services, contributing to the sharing of experiences, and promoting the inclusion of people as community members (Paternò, 2004). This trend is based on the development of mobile devices and their usability, accessibility, portability, and versatility (Kvale, Warakagoda, & Knudsen, 2003). The usefulness and usability of services, and the ability to access them and information, are the basic elements in the diffusion of Web systems and development of Web multimodal languages. The diffusion and implementation of multimodal services is supported by the activities of the World Wide Web Consortium, aimed at extending interaction modes for different devices and particularly devoted to solving various problems connected with: (1) multimodal Web interaction through the different devices, and (2) practice Web navigation from different devices. Some W3C working groups focus their activities on issues such as independence from devices, multimodal Web access, and types of contents for multimodal messaging. These specifications allow rich multimodal contents to be transmitted, and are based on the power and extensibility of XML (eXtensible Markup Language) (Bray, Paoli, Sperberg- McQueen, Maler, & Yergeau, 2004). XML is highly important in a mobile application environment, as many applications have to manage multimediacontents and need dedicated tools for this. SMIL (Synchronized Multimedia Integration Language) (Solon, McKevitt, & Curran, 2004) was proposed to achieve this goal. In the early years W3C-MMI (W3CMultiModal Interaction) focused on multimodal interaction modes such as speech and pen interaction, and providing users with W3C technologies. W3C develops these technologies by orienting individual interaction modes in order to create mixed-namespace XML documents, such as SVG (scalable vector graphics) (Chatty, Lemort, Sire, & Vinot, 2005) and XHTML (Extensible HyperText Markup Language) (Musciano & Kennedy, 2003) for visual interaction, and VoiceXML (Voice Extensible Markup Language) (Lucas, 2000) for voice interaction. However, many other XML-derived languages have helped in the development of mobile services. The next target is the consideration of the mobile network as an extension of the global Internet network. This article explains the importance of XML and its dialects in a mobile application environment to enable their use by the “various applications/services” (today available on the Web). In fact, different dialects may be needed for different mobile devices depending on their characteristics (such as memory, CPU speed, integrated software engine, etc.). For example, two SVG profiles are defined for cellular phones and PDAs (personal digital assistants): SVG Tiny (SVGT) is suitable for the next generation of cellular phones especially, while SVG Basic (SVGB) is aimed at high-tech devices such as PDAs or smart phones (Andersson et al., 2003). The pervasive use of mobile devices will be the target for the near future (Branco, 2001), given the trend towards considering the mobile network as an extension of the Internet global network. This scenario promotes the development of new dialects for multimodal interaction through mobile devices. The dialects developed for speech, sketch, and visual interaction are discussed next. An area for future development might focus on interaction through gestures. XML (eXtensible Markup Language) is a simple, flexible, and powerful markup language, based on text format that allows the development of a potentially unlimited number of innovative multimodal services and applications. It was derived from the more complex, complete SGML (Standard Generalized Markup Language, ISO 8879) (Chamberlin & Goldfarb, 1987), designed for more general purposes. However, XML language is easier to manage, and is genuinely Web oriented and mobile oriented. In other words, XML language is an optimal subset of SGML, constructed in consideration of the possible Web services and applications. XML can be used to develop several languages taking the specific working context into account. It also plays an important role in the exchange of a wide variety of data, making them available and accessible by Web using computers and mobile devices.