Notes for Ben Shneiderman “Direct Manipulation: A Step Beyond Programming Languages”, 1983

Key concepts: direct manipulation, semantic learning, syntactic learning, virtuality, WYSIWYG.

Related theorists: Gee, Don Hatfield, Ted Nelson, Seymour Papert, Thimbleby.

Replace complex command language syntax by direct manipulation of the object; examples include display editors, VisiCalc, spatial data management, and video games.

(486) The central ideas seemed to be visibility of the object of interest; rapid, reversible, incremental actions; and, replacement of complex command language syntax by direct manipulation of the object of interest – hence the term “direct manipulation.”

Examples of Direct Manipulation Systems

Display Editors

VisiCalc

Spatial Data Management

Video Games

Video games easy to learn by analogy when commands are physical actions, and lessons from games can be transferred to office applications; compare to Gee.

(489) Because their fields of action are abstractions of reality, these games are easily understood – learning is by analogy.
(490) The commands are physical actions, such as button presses, joystick motions, or know rotations, whose results appear immediately on the screen. . . . Error messages are unnecessary because the results of actions are so obvious and easily reversed. These principles can be applied to office automation, personal computing, and other interactive environments.
(490) Game players compete with the system, but application-system users apparently prefer a strong internal locus of control, which gives them the sense of being in charge.

Computer-Aided Design/Manufacturing
(491) The design's basic strategy precludes the necessity of recalling complex commands in once-a-year emergency conditions.

Further Examples

Driving automobile as quintessence of direct manipulation.

(491) Driving an automobile is my favorite example of direct manipulation. . . . Imagine trying to turn by issuing a LEFT 30 DEGREES command and then issuing another command to check your position, but this is the operational level of many office automation tools today.

Xerox Star office automation user interface examples of direct manipulation, graphical versus command driven.

(491) Designers of advanced office automation systems have used direct manipulation principles. The Xerox Star offers sophisticated text formating options, graphics, multiple fonts, and a rapid, high-resolution, cursor-based user interface.

Examples of Direct Manipulation

Further examples of Hatfield WYSIWYG, Nelson virtuality.

(492) “What you see is what you get,” is a phrase used by Don Hatfield of IBM and others to describe the general approach. . . . The display should indicate a complete image of what the current status is, what errors have occurred, and what actions are appropriate, according to Thimbleby. Another imaginative observer of interactive system designs, Ted Nelson, has noticed user excitement over interfaces constructed by what he calls the principle of “virtuality” - a representation of reality that can be manipulated.

Problem-Solving and Learning Research

Problem solving and learning depend on suitable representation, such as Papert Logo mathematical microworld.

(492) Another perspective on direct manipulation comes from psychology literature on problem solving. It shows that suitable representation of problems are crucial to solution finding and to learning.
(493) Papert's Logo language creates a mathematical microworld in which the principles of geometry are visible.

Problems with Direct Manipulation

Graphic icons may still require learning of meaning for what their virtual manipulation performs.

(493) A second problem is that users must learn the meaning of the components of the graphic representation. A graphic icon, although meaningful to the designer, may require as much – or more – learning time as a word.

The Syntactic/Semantic Model

Syntactic/semantic model of user behavior based on kinds of knowledge in long-term memory: syntactic volatile, acquired through rote memorization, semantic memorable, acquired through explanation, analogy, example, hierarchically structured in matrix of concepts.

(494) My own understanding of direct manipulation was facilitated by considering the syntactic/semantic model of user behavior.
(494) The basic idea is that there are two kinds of knowledge in long-term memory: syntactic and semantic.
(494) This knowledge is arbitrary and therefore acquired by rote memorization. Syntactic knowledge is volatile in memory and easily forgotten unless frequently used.
(494) The concepts or functionality – semantic knowledge – are hierarchically structured from low-level functions to higher level concepts.
(495) Semantic knowledge, which is acquired through general explanation, analogy, and example, is easily anchored to familiar concepts and is therefore stable in memory. The command formulation process in the syntactic/semantic model proceeds from the user's perception of the task in the high-level problem domain to the decomposition into multiple, lower level semantic operations and the conversion into a set of commands.

Implications of the Syntactic/Semantic Model

Training manuals should be written based on semantic learning principles.

(495) The syntactic/semantic model suggests that training manuals should be written from the more familiar, high-level, problem domain viewpoint. The titles of sections should describe problem domain operations that the user deals with regularly. Then the details of the commands used to accomplish the task can be presented, and finally, the actual syntax can be shown.

Semantic learning explains success of direct manipulation versus difficulty of mathematics and programming.

(495) The success of direct manipulation is understandable in the context of the syntactic/semantic model. The object of interest is displayed so that actions are directly in the high-level problem domain. There is little need for decomposition into multiple commands with a complex syntactic form. On the contrary, each command produces a comprehensible action in the problem domain that is immediately visible. The closeness of the problem domain to the command action reduces operator problem-solving load and stress.
(495) Since mathematics and programming require abstract thinking, they are difficult for children, and a greater effort must be made to link the symbolic representation to the actual object. Direct manipulation is an attempt to bring activity to the concrete operational stage or even to the preoperational stage, thus making some tasks easier for children and adults.

Potential Applications of Direct Manipulation

Trick is appropriate representation of reality, especially when no physical parallel.

(496) The trick in creating a direct manipulation system is to come up with an appropriate representation or model of reality.
(497) It is possible to apply direct manipulation to environments for which there is no obvious physical parallel.
(497) Direct manipulation has the power to attract users because it is comprehensible, natural, rapid, and even enjoyable. If actions are simple, reversibility ensured, and retention easy, then anxiety recedes and satisfaction flows in.

Shneiderman, Ben. “Direct Manipulation: A Step Beyond Programming Languages.” The NewMedia Reader. Eds. Noah Wardrip-Fruin and Nick Montfort. Cambridge, Mass: MIT Press, 2003. 486-497. Print.