An Axiomatic Model of Information Presentation

Gary Perlman
Wang Institute
Tyngsboro, MA 01879 USA

Table of Contents

Paper originally published in Proceedings of the Human Factors Society 31st Annual Meeting, 1987, 1229-1233, Santa Monica, CA: Human Factors Society. Republished in Perlman, Green, & Wogalter (Eds.) Human Factors Perspectives on Human-Computer Interaction, 1995, 120-124, Santa Monica, CA: HFES.

Abstract

The goal of information layout is to physically display information to reinforce the underlying structure of the information. In this paper, I describe an axiomatic model of information layout. The model has three levels:
  1. a device-independent representation for structured information,
  2. a set of axioms (or rules) relating information structure with display attributes,
  3. a set of device dependent display attributes used to distinguish differences and show similarities in information structure.
The model infers, using logical deductions from its axioms, how display attributes should be used to show the structure of information. A prototype software system exists that allows interactive design and evaluation of screen layouts. Future research is planned to develop an expert system to aid in the automatic design of layouts, and to refine the prototype into a usable system.

Goals of this Research

The fundamental goal of information presentation is to present information in a medium that people can readily understand and use. To accomplish this goal, we design displays that convey the structure of the information: the ideas and their relations. The form of the display reinforces its content.

The work presented here tries to formalize the rules we use to design effective information displays based on information structure. It is concentrated on visual information displays, but the methods may apply as well to other media, such as acoustic presentations of information.


Summary of Tullis' Work

Tullis (1983, 1985) predicted search times and subjective preferences of screen layouts using six display characteristics measured from screen dumps.
  1. Overall density (percentage of non-blank characters)
  2. Local density (how tightly packed a display is)
  3. Number of groups (separated by blank space boxes)
  4. Average size of groups (maximum visual angle)
  5. Number of items (labels, data items, etc.)
  6. Item uncertainty (use of vertical/horizontal alignment)
Using these six display characteristics, Tullis was able to obtain multiple correlation coefficients of .71 for predicting search times and .90 for predicting subjective ratings. For search times, the two most important predictors were the number and size of groups, combining for a correlation for prediction of .65. For subjective ratings, all characteristics were significant predictors.

Criticisms of Tullis' Model

There are several limitations of Tullis' model. (1) Tullis used plain character displays with no video enhancements nor quasi-graphic characters such as lines for drawing boxes. (2) Tullis' model does not make use of the information structure underlying the display. (3) The output of a predictive model based on regression analysis is not diagnostic nor prescriptive because it is a single number (ie. predicted search time or subjective rating).

Use of Simple Displays

Tullis restricted his analysis to plain character displays based on the valid assumption that any screen analysis tool would have to be able to deal with the simplest screens before it could deal with ones complicated by highlighting and boxing. While this may be a good simplifying assumption to explore the feasibility of making quantitative predictions of screen display quality, his model does not seem to generalize well to the use of more advanced display features.

Monochrome video enhancements like reverse video, underlining, different intensity and blinking, and color enhancements are very salient. With judicious use of these enhancements, users can find specific items among hundreds almost as quickly as if the items were displayed alone. For example, if only a few fields on a screen are in reverse video or blinking, then they can be found almost immediately. Similar effects can be observed for graphics displays that can present different type faces and font sizes. We should expect that any model of information presentation should apply equally well to printed media as to computer displays.

The lack of use of boxes or region-separating lines is another limitation of Tullis' model. Boxes (or window panes) are used to separate and highlight distinct items or groups of items, in many cases without the loss of display space. The combination of boxing and video attributes is used in almost every modern display. Boxing is such an attractive display alternative, that Tullis himself could not avoid using it in his sample display of an improved narrative display (1985), even though his own program misinterpreted the meaning of the box, identifying it as a separate group.

Any analysis of screen display quality must be able to work with these highly effective techniques of coding and discrimination.

Lack of Structure Representation

Tullis' model has no representation of the structure of the information the screen designer is trying to present. To decide if a display is a good one, we must know the structure of the information being displayed. Even if a program tells us that the number of groups is good, and so are the sizes of the groups, we need to know if the display parses into the right groups and if items are aligned as they should be. Tullis' model works from the final presentation of the information (a screen dump) and tries to infer the structure of the information. This is a good idea, because he tries to predict how a naive user would try to understand the display by modeling some human perceptual capabilities.

It is unrealistic to assume that people would try to use and evaluate a display without considering its content. For example, if we know that there are many items to display on a screen, and that they must appear in a particular order, such as in many data entry forms, then most random orderings would produce unacceptable displays, but perfectly acceptable to Tullis' analysis.

Lack of Diagnostic Information

Tullis' analysis predicts search time and subjective ratings but cannot provide much information about why one display is better than another. Averages may not be helpful to designers and are often misleading. Macdonald (1983) reports on Writer's Workbench software that produces readability scores for technical writing. These scores predict the number of years of schooling a reader would need to understand a text, but give little help in ways of improving the text. Berry & Meekings (1985) applied similar techniques to the readability of computer programs, but programmers are left with an empty feeling knowing that their program scored 22% but not how to improve it. Tullis (1986) describes an enhanced version of his display analysis program that makes some attempts at focusing attention to problem areas and suggesting heuristics to improve them.

Designers need kibbitzers that describe problematic regions of displays and prescribe remedies. To spot problem regions, or in extreme cases, disaster areas, a designer's aide must know the intentions of the designer (the structure of the information to be displayed), and the methods available to the designer to convey that structure (e.g., the video attributes, graphics, and positional controls).


A Detailed Example

I will use the following example throughout the rest of this paper to illustrate my points. It is relatively simple -- a form that you might use to keep track of an employee -- and familiar, but complex enough that it will show the importance of information representation and the use of axioms (rules) for displaying them. The following display is what I think is a well-designed form.
PERSONAL
  Name: __________________________ (Last, First, ...)
  Sex:  _ (M or F)   Birthdate:   __/__/__ (mm/dd/yy)
  SSN:  ___-__-____  Citizenship: ________

HOME INFORMATION
  Address: _____________________________   Apt: _____
  City:    ___________________  State: __  Zip: _____
  Phone:   (___) ___-____

WORK INFORMATION
  Title:   ___________________  Dept: _______________
  Company: __________________________________________
  Address: __________________________________________
  City:    ___________________  State: __  Zip: _____
  Phone:   (___) ___-____ Ext: _____
  Email:   __________________________________________
After seeing so many forms like this, it is hard to consciously notice all the relations of component parts. I will first describe them informally. The form has three sections: PERSONAL data, HOME address information, and WORK address information. Each section has subparts, so the form is a hierarchical structure. The sections are partially ordered -- PERSONAL data is first, and the order of the WORK and HOME data does not matter. The subparts of the major parts of the form, or fields as they are usually called, have their own structure. Each field has a label followed by a colon, and a fill-in area of some maximum size. The fill-in areas in a section tend to be aligned in columns. A fill-in area may contain punctuation delimiting further subdivisions and there may be a comment or example at the end of the field to help guide data entry and interpretation. There are recurring fields and groups of fields, like the City-State-Zip fields in the HOME and WORK sections. Some fields are repeated, but with exceptions; the HOME address has a field for apartment number, and the WORK phone number has a field for extension.

Now I will show how the structure of information in the form can be represented. Later I will show how differences and similarities among units of information can be mapped to differences and similarities in display. There are the following relations in the form:

	X   in    Y        the item X is part of Y
	X   isa   Y        X is an instance of Y
	X  after  Y        X must follow after Y
	X   is    P        X has some property P
To represent this information structure, I use semantic networks (Quillian, 1968) because of their ability to represent graphs with different relations among nodes. The basic data structures in a semantic network are nodes and arcs between nodes. Here, nodes represent items of information, and arcs represent relations between items of information. Uppercase names will be used to denote items in an information structure. Lowercase names will be used to denote types (such as dates) that are defined elsewhere. Literal text is in quotes. A prefix notation list structure will be used to denote relations between items. For example, if X is part of Y, I will write:
	(in X Y)
and when there is no ambiguity, I will also write:
	(X in Y)
More properly, in a semantic network notation, (X in Y) means to point from X to Y with an arc labeled in:
	(point X Y in).
Also, when there is no ambiguity, I will write multiple statements in one statement by adding multiple parameters around relations. For example:
	(X Y Z in W)
is read as X and Y and Z are all in W. In some cases, to avoid notation that contains details of programming languages, I will use English.

To describe parts of the example display, we must write many statements describing relations. For each item in the form, we need to indicate what it is contained in, its type, and a display string (such as a prompt for a field).

(PERSONAL HOME WORK in FORM)
There are three sections in the form.
(NAME SEX BIRTH SSN COUNTRY in PERSONAL)
There are five parts of the PERSONAL section.
(NAME SEX BIRTH SSN COUNTRY is field)
The parts of PERSONAL are all fields. Properties of fields will be provided later.
("Personal" isa label, in PERSONAL)
This is the sixth part of the PERSONAL section. Note that on the form, "Personal" is displayed as "PERSONAL" (all caps) because of its role as a high level label; a rule for this will be provided.
(SEX BIRTH SSN COUNTRY after NAME)
(SSN isa ssn)
(BIRTH isa date)

Next I declare that a field has a label, and a fill-in part, and that they must be in that order.

	(label fillin in field) (fillin after label)
	(":" entry-area in fillin)
Putting the colon as part of the fillin part of a field makes sense because it is the prompt for the data entry area.

Next I say that the string "Name" is the label for the NAME field,

	("Name" isa label, in NAME)
and define the fillin part of the field in NAME.
	((len 30) "(Last, ...)" isa fillin, in NAME)

When types are defined, they can be given properties that are inherited by all instances of these types. For example, by saying that BIRTH isa date. and defining what a date is, we encapsulate and reuse the structural properties of dates.

	("Birthdate" isa label, in BIRTH)
	(month day year in birth)
	(year after day after month)
	(month day year isa number)
	((len 2) "/" ... isa fillin, in date)
To represent the rest of the form takes declarations like the following.
	(HOME WORK after PERSONAL)
	("Home Information" isa label, in HOME)
	("Work Information" isa label, in WORK)
	(HADDRESS HAPT HCSZ HPHONE in HOME)
	(HADDRESS HAPT HPHONE isa field)
	(WTITLE WDEPT ... WEMAIL in WORK)
	(WTITLE WDEPT ... WEMAIL isa field)
Note how the City-State-Zip part of each field is identical in the HOME and WORK parts. In a display, we would want them to be displayed analogously, and the way to do this is to define a type that contains the common parts.
	(HCSZ WCSZ isa csz)
	(city state zip isa field, in csz)
	(zip after state after city)
	("City" isa label, in city)
	("State" isa label, in state)
	("Zip" isa label, in zip)
	...
	(HAPT isa apt)
	(apt after address)
	("Address isa label, partof address)

The Axiomatic Model

The orientation of the following model of information presentation is different than Tullis' in that I do not focus on the analysis of existing displays and try to infer their structure, but instead I begin with an information structure and try to decide how that information should be presented to best convey that structure. This approach is similar to that used by Mackinlay (1986), who used rules for displaying quantitative information stored in relational databases. I think my approach will prove more fruitful than Tullis' because it begins with the intentions of the designer (ie. the structure of the information to be presented) and helps map those intentions into the display domain.

Representing Information Structure

In the example, we have seen that the labeled relations in semantic networks can represent the structure of information including containment, typing, and temporal ordering. Part of the inspiration for this method of representation comes from the "box" model for screen layout of Coutaz (1984).

Once the structure of the information to be displayed has been represented, we can predict sources of complexity based on cognitive psychological principles, before we even begin to consider how that information will be displayed. This is important because we need to be able to evaluate the complexity of the display with respect to the complexity of the information to be displayed. For any display, there must be some minimum possible perceptual complexity based on the information structure to be displayed. Further complexity is added by suboptimal display design. Sometimes, a designer will try to put too much information, so that there is no acceptable layout. In such a case, redesigning the information presentation is hopeless; it is the information itself that must be redesigned. We can look at the number of structures at any level in a hierarchy that are to be displayed. Research such as Miller (1956) suggests that this number, which we may call the branching factor in this context should be kept within a 7 +/- 2 limit for many displays, but which should be traded off against potential hierarchical depth (Kiger, 1984).

If we assume that distinct top level groups are separated by blank space, then we are in a good position to use Tullis' regression models, because we know (1) the number of groups, (2) their sizes, (3) the number of items, and (4) the overall density. The particulars of item placement would affect item uncertainty and local density. This points out that much of Tullis' analysis is directed at figuring out what the designer should already know.

Axioms of Information Presentation

Only some of the axioms are presented here because of space limitations. There are two classes of axioms. The first axioms give substance to the relations used in representation. Later axioms map information structure into the display domain. The main idea is to map abstract information structures to abstract display methods, and when the concrete details of information structure are provided, then the exact methods of display are also decided. This is shown in the following mapping diagram.

Representational Axioms

Representational axioms provide details of the meaning of the relations used to represent information structure. They allow inferences to be made based on information structures without regard to display.

Axiom of Transitivity of Containment
(X in Y) & (Y in Z) -> (X in Z)
This axiom states that if an item is part of a second, which is part of a third, then the first item is part of the third.

Axiom of Transitivity of Instances
(X isa Y) & (Y isa Z) -> (X isa Z)

Axiom of Inheritance of Properties of Instances
(X isa Y) & (Y is P) -> (X is P)
This axiom, when combined with the axiom of transitivity of instances, allows the inference of recursive inheritance of properties. For example, if X isa field and fields are displayed in red, then we can infer that X is red, unless a specific color attribute has been assigned directly to X.

Axiom of Transitivity of Ordering
(X after Y) & (Y after Z) -> (X after Z)

Axiom of Asymmetry
(X isa Y) -> ~(Y isa X)
(X in Y) -> ~(Y in X)
(X after Y) -> ~(Y after X)
These axioms help stop infinite loop/recursion inferences. The symbol ~ is used to denote negation.

Display Axioms

The following axioms map structural similarities and differences to display similarities and differences.

Axiom of Proximity
(A in X) & (B in X) & ~(C in X)
-> D(A,B) < (D(A,C) + D(B,C))
This axiom states that items in the same group should be closer to one another than to items outside the group. The distance function D(X,Y) can be defined to meet specific design needs and device capabilities. Another interpretation of this axiom is that if two items are in the same group, then there should not be any outside items between them.

Axiom of Discrimination
(X rel A) & (Y rel A) & ~(Z rel A)
-> (disp X A) & (disp Y A) & ~(disp Z A)

(for some display attribute A)
If two items, X and Y, are similar based on some relation, and both differ from Z on that relation, then there should exist a display attribute, A, used to display X and YA, that differs for the display of Z. In practice, it is also important that this display attribute not be used to show similarity or differences based on any other relation. Some attributes can be used more than once, such as boxing or position, or temporal location, but others, such as color, are usually not effective. For example, if the relation is isa, and we have two labels in fields and one that is not part of a field, then we may want to use color to code label type: yellow on blue for input field labels, and yellow on red for others, and this would preclude the use of color to distinguish other aspects of information structure, such as the importance of some items.

There are several more general axioms of information presentation, but I will now turn to a specialized axiom, or rule, for a particular display. In the example form, I display section titles in all uppercase letters, at the top and to the left of each section's region. Here is a rule for doing part of this.


Axiom of Main Titles
(X isa label) & (X in* S) && (S in* ROOT)
-> (disp X allcaps) & (Z in S after X)
This rule says that for any label in S, where S is a top level part of the display, X should be in all uppercase letters, and should precede all parts of S. Some details are omited. A * is placed after the in relation to stop recursive transitive interpretation. To change the format of the display of such "main" titles, we could change the rule above. We could make other items look like main titles, change how titles look, or add new main parts to the display, and they would automatically be displayed according to current rules. Construction of local display rules simplifies the process of automatically choosing display methods to present information structure, and also give a designer control over critical parts of displays. We could also construct our rules to accept display attributes as parameters, and that is discussed in the next section.

Parameterizing Display Capabilities

The model presented here uses display characteristics as parameters. In so doing, it can adapt to any display type for which the properties of the display are well understood, even auditory displays. Mackinlay (1986) adapted Cleveland & McGill's (1984) work to rank order display methods such as position, size and color according to their perceptual salience for displaying data on different scales of measurement. These rankings may prove useful in a more advanced model of information display, but currently, the method of displaying differences must be hand-picked.

In the example form, structure is presented using display methods like position, alignment, upper and mixed case. I use spatial location to distinguish major sections of the form, alignment to show position of labels and fillins in fields, and similar formats to show similar structures (e.g., City-State-Zip).

Rather than use spatial separation to display personal, work, and home information, different regions could separate different types of data (e.g., addresses in one, phone numbers in another, etc.). This would produce an unsatisfactory display because it creates too many top level groups. This shows that in any automated system, we must provide designers with control over how general rules are applied.

Designers also need control when there are no rules to apply. I have no rule for when unrelated items may occupy the same line (e.g., Sex and Birthdate), so designers need the capability to place specific fields (ie. to attach specific display attributes to specific items). Note that there are rules that make City, State, and Zip items appear on the same line. Those fields are part of a specially defined structure so they can share a feature (the same line, or perhaps the same color).

Software Prototype

The software prototype of the axiomatic model will not be discussed in detail because of space limitations. The interactive system allows designers to selectively move and add attributes to items. The semantic network of the representation of the system is implemented in LISP running on a UNIX system. The LISP model creates a series of rules and database statements that are input to a Prolog interpreter, which produces statements of location and attributes of items.

Once a display is designed, the system has a kibbitzing component that looks for problems. It complains if it cannot find a display attribute to distinguish different items, and to a lesser extent if it cannot find common attributes for similar items. The similarity analysis is based on Tversky (1977). Because the general kibbitzer is not too smart, the screen display kibbitzer also complains about more mundane violations, such as sub-units being outside the physical region of a unit, or overlapping units.

With more experience with the model I hope to build a flexible system for a more common environment such as an IBM PC. Another goal is to be able to connect it to a runtime library of display routines, so that inferred changes of attribute assignments to structural patterns can be viewed interactively.

An Altered Example

The following example could be derived from the same relations in the original example, but with minor modifications to the rules for display:
====================== Personal =====================
|NAME: .......................... (LAST, FIRST, ...)|
|SEX:  . (M OR F)   CITIZENSHIP: ........           |
|SSN:  ...-..-....  BIRTHDATE:   ../../.. (MM/DD/YY)|
================== Work Information =================
|TITLE:   ...................  DEPT: ...............|
|COMPANY: ..........................................|
|EMAIL:   ..........................................|
|ADDRESS: ..........................................|
|CITY:    ...................  STATE: ..  ZIP: .....|
|PHONE:   (...) ...-.... EXT: .....                 |
================== Home Information =================
|ADDRESS: .............................   APT: .....|
|CITY:    ...................  STATE: ..  ZIP: .....|
|PHONE:   (...) ...-....                            |
=====================================================

Summary of Main Points

We need better tools for design and evaluation of information presentations. In particular, we need tools that allow interactive design with immediate online evaluation.

We need to be able to represent intentions of designers to determine goodness of a display. Such intentions are representable, at least in part, as the relations and properties of items in the display.

We need rules to map information structure into display attributes; these rules and a system for their application would be an expert system for display design. An eventual goal is to develop rules that allow automated (at least good first approximations of) presentations of complex information structures, much as database report generators can be used to automate displays of relational data. We need to be able to override expert system choices.

We need to be able to adapt display designs to different devices such as character displays, graphics displays, and even auditory displays. Any model of information display that does not use display device capabilities (or lack thereof) as display parameters will have limited applicability.

We need intelligent kibbitzers to tell us where we may be making mistakes or creating catastrophes and how to avoid them, suggesting alternatives and perhaps tradeoffs. Weighted averages are misleading evaluations because of their limited diagnostic and prescriptive abilities. We need to be able to tell kibbitzers that their advice has been noted but ignored, such as when we decide that it is acceptable that a window overlay obscures part of a screen, or we will always have to search for new complaints among the old.


References

  1. Berry, R. E., & Meekings, B. A. E. (1985) A Style Analysis of C Programs. Communications of the Association for Computing Machinery, 28:1, 80-88.
  2. Cleveland, W. S., & McGill, R. (1984) Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods. Journal of the American Statistical Association, 79, 531-554.
  3. Coutaz, J. (1984) The box, A Layout Abstraction for User Interface Toolkits. Pittsburgh, PA: Carnegie Mellon University Computer Science Department.
  4. Kiger, J. I. (1984) The Depth/Breadth Trade-Off in the Design of Menu-Driven User Interfaces. International Journal of Man-Machine Studies, 20, 201-213.
  5. Macdonald, A. H. (1983) The UNIX Writer's Workbench Software: Rationale and Design. Bell System Technical Journal, 62, 1991-2008.
  6. Mackinlay, J. (1986) Automating the Design of Graphical Presentations of Relational Information. ACM Transactions on Graphics, 5:2, 110-141.
  7. Miller, G. (1956) The Magical Number Sever Plus or Minus Two: Some Limits on Our Capacity for Processing Information. Psychological Review, 63, 81-97.
  8. Quillian, R. (1968) Semantic Memory. In M. Minsky (Ed.), Semantic Information Processing. Cambridge, MA: MIT Press.
  9. Tullis, T. S. (1983) Predicting the Usability of Alphanumeric Displays. (Unpublished Doctoral Dissertation.)
  10. Tullis, T. S. (1985) A Computer-Based Tool for Evaluating Alphanumeric Displays. In B. Shackel (Ed.), Human-Computer Interaction: INTERACT '84. (Paper presented in London, England in September, 1984.)
  11. Tullis, T. S. (1986) Display Analysis Tool User Manual. Lawrence, KS: Report Store.
  12. Tversky, A. (1977) Features of Similarity. Psychological Review, 84, 327-352.