Chapter+9

__Images, Words, and Gestures (textbook)__ Coding Words and Images: - Bertin, in his seminal work, //Semiology of Graphics// (1983), distinguishes two distinct sign systems. One cluster of sign systems is associated with auditory information processing and includes mathematical symbols, natural language, and music. The second cluster is based on visual information processing and includes graphics, together with abstract and figurative imagery. More recently, the dual coding of Paivio (1987) proposes that there are fundamentally different types of information stored in working memory; he calls them //imagens// and //logogens.// Roughly speaking, imagens denote the mental representation of visual information, whereas logogens denote the mental representation of language information. - Logogens and imagens, although based on separate subsystems, can be strongly interlinked. For example, the word //cat// and language-based concepts related to cats will be linked to visual information related to the appearance of cats and their environment. The Nature of Language: - Sign Language: o It is also the case that even signs apparently based on similarity are only recognized correctly about 10% of the time without instruction, and many signs are fully abstract. Images vs. Words: - Static Images vs. Words: o images are better for spatial structures, location, and detail, whereas words are better for representing procedural information, logical conditions, and abstract verbal concepts. o image memory advantage cannot be relied on if the information is new and is represented abstractly and out of context. o Text is better than graphics for conveying abstract concepts, such as freedom or efficiency o Procedural information is best provided using text or spoken language, or sometimes text integrated with images - Animated Images vs. Words: o animation brings graphics closer to words in expressive capacity. The work of Michotte shows that with appropriate animation and timing of events, a causal relationship will be directly and unequivocally perceived. o graphical technique called //snakes// (Parker et al., 1998). Animation moves the head of the snake from one software component to the next as the locus of computation moves; the tail of the snake provides a sense of recent history. o An animation illustrating the task of disassembling a machine gun was compared to a sequence of still shots. The animation was found to be superior for complex motions, but verbal instructions were just as effective for simple actions, such as grasping some component part. - Links between Images and Words: o // dual coding/multimedia theory // suggest that if active processing or related material takes place in both visual and verbal cognitive subsystems, learning will be better. o In this way, process steps could be read immediately adjacent to the relevant visual information. Sweller et al. (1990) used the concept of //limited-capacity working memory// to explain these and similar results. They argue that when the information is integrated, there is a reduced need to store information temporarily while switching back and forth between locations. Gestures as Linking Devices: - When possible, spoken information--rather than text information--should accompany images, because the text necessarily takes visual attention away from the imagery. If the same information is given in spoken form, the auditory channel can be devoted to it, whereas the visual channel can be devoted to the imagery (Mousavi et al., 1995). The most natural way of linking spoken material with visual imagery is through hand gestures. - Deixis: o The deictic gesture is considered to be the most elementary of linguistic acts. o In human communication theory, a gesture that links the subject of a spoken sentence with a visual reference is known as a //deictic gesture,// or simply //deixis.// o Deictic techniques can be used to bridge the gap between visual imagery and spoken language. Visual Momentum in Animated Space: - In order for the viewer to link different camera shots together (a car moving off the screen onto another scene), there must be perceptual continuity. The example of the car must be traveling in the same direction, otherwise the audience may lose track of it and lose attention. o Wickens 4 principles for user interface design: § Use of consistent representations: same color of the car etc. § Use graceful transitions: smooth animations § Highlight anchors: certain visual objects may act as visual reference points. Ideally, several anchors should be visible from the previous frame. § Display continuous overview maps: use of an overview map places the user in a larger spatial context allowing for recognition. Ex: GPS and other navigation systems.

__Animation History and Techniques (Lecture)__ - 2 main categories: 1. Computer assisted animation and 2. Computer generated animation - Animation Techniques: o Squash and Stretch: Can relieve the disturbing effect of strobing. o Timing and Motion: Gave meaning to movement. Proper timing is critical to making ideas readable. o Anticipation: Preparation for an action. o Staging: A clear presentation of an idea. o Follow Through and Overlapping Action: 1. Termination part of an action and 2. starting a second action before the first has completed. o Straight Ahead Action and Pose to Pose Action: 1. Animator start from first drawing in the scene and draw all subsequent frames until the end of scene. 2. Animator plans actions, draws a sequence of poses, in between frames. o Slow in and Out: Spacing of in between frames to achieve subtlety of timing and movement. o Arcs: Visual path of action for natural movement. Makes animation much smoother and less stiff than a straight line. o Exaggeration: Accentuating the essence of an idea via the design and the action. Ex: An object made smaller to give the idea of a child. o Secondary Action: Action that results directly from another action. Used to increase the complexity and interest of a scene. Ex: Body movement is the primary action, facial expression is the secondary action. o Appeal: Refers to what an audience would like to see. Chapter cannot be too simple (boring) or too complex. Ex: Avoid mirror symmetry, asymmetry is interesting. - What makes graphics effective: o Congruence Principle: The structure and content of the external representation should correspond to the desired structure and content of the internal representation. The natural way for conveying concepts of change in space. o Apprehension Principle: The structure and content of the external representation should be readily and accurately perceived and comprehended. § Why do animations fail: 1. Animations may be hard to perceive. 2. Animations may be comprehended discretely. 3. Not universally preferred and often require expertise for understanding.