A compositional model of human-computer interaction

The iterative and exploratory nature of user interface (UI) design determines the requirements for UI design tools: to provide good coverage of the UI design space and to support exploration. That is, the tools should facilitate going from one design to another, providing guidance and assistance in the process. The motivation for this research is to show that the model-based approach to UI design addresses all of the above requirements. Improved models can not only expand the range of interfaces a UI tool can produce, but can also capture the knowledge needed for the tool to assist in the design process. This research develops a compositional model of human-computer interaction as the basis for UI design tools. The model integrates the UI representation based on the compositional view of user interfaces with the domain-independent representation of application semantics. The UI representation identifies and classifies components of a UI and structuring principles for composing them into a UI. Both the components and the structuring principles are reusable across applications and different user interfaces; application specificity and different UI designs are achieved by configuring components and applying structuring principles. The UI representation can explain behavior and the look-and-feel of a UI in terms of properties of UI components and the way the components are structured. The compositional model establishes an explicit relation between the application semantics and the corresponding UI, captured in two sets of rules: composition rules that explicitly represent the mapping from the structure representing the application semantics to the structure representing the application's UI, and consistency rules that evaluate the UI structure with respect to the application semantics according to a given set of criteria. An integral part of the model is design transformations--abstractions for changing the application conceptual model and the user conceptual model of an application while preserving the application functionality. The compositional model provides a framework for a fully-functional UI tool. A prototype tool based on the model is implemented as a proof of concept. It automatically generates a UI from a high-level specification, supports a wide range of different UI designs, and can provide assistance and guidance in the process of exploring alternative designs.