EMS 594J Scientific and Technical Visualization

From WolfWikis

Conceptual Framework for Scientific and Technical Visualization

(Here is a seed thought....It is meant to largely disappear by the time we're done)

Scientific visualization is all about what you 'see' and not what you 'think'. The most important aspect of a visualization is how it is processed by the automatic, perceptual centers in the brain. Past experience and other more cognitive factors are not as important.

(Bedward) Visualization implies more than just seeing or does it? When seeing we may have sufficient prior knowledge and domain knowledge in order to automate the biological processing of what we perceive. As Hegarty, Carpenter, et al suggest, a student has to learn how to read a linguistic tree diagram and interpret the links as indicating grammatical relations,.... Objects contain information/data, the mind references these symbols in memory in order to understand what is being perceived--a mental image that is referenced in our consciousness (Duchowski, 2003). Since consciousness is a process that involves both short term memory and attention (Georges, 2003) than I argue seeing cannot be separated from thinking since memory and attention are both cognitive processes. We must consider attention as an internally covert mechanism akin to imagination, anticipation, hence, where, what and how we see requires cognitive acts. Whether we take in the visual cues all at once (Gestalt) or sequentially ((Yarbus) does not negate the cognitive processes needed to visualize what we see. peace.

Georges, T. M. (2003). Digital Soul.: Intelligent Machines and Human Values. USA: Westview Press.

(Jessee) Something happens between your ears when looking at a visualization. When evaluating an image, Kosslyn stated that an image can be worth a thousand words, but only if you can make sense of the relationship presented. When learning to read, most people associate pictures with the words (learning to see the words with pictures in order to remember them – one quickly learns how to form the images with the words and does so without thinking). Similarly, the relationships that exist between data are of great magnitude. So, when assessing this field, cognitive factors are very important (reasoning, perception, intelligence, etc). Certain images may require fewer cognitive processes to occur (Vekiri), while others take more time, examples: concrete versus abstract, verbal versus visual, etc. I reason (with the support of Haber and Wilkinson) that (visual) cognition captures a sizeable role in scientific visualization due to the complex sequences of events between stimulation of the retina and storage of information in the memory, but other social, cultural, reasoning factors also largely affect ones understanding. Thus, there are a multitude of factors central to the understanding of scientific visualization.

(Jones) Visualization, as it has been defined over the course of this semester, involves more than just automatic, perceptual brain mechanics. The result of a stimulus sensed by the eye is merely sight. Visualization implies prior knowledge, memory, dual coding and other cognitive factors.

In the context of learning (as well as any other pragmatic context), scientific visualization is a two-way street. Gordin, Baker and others emphasize both interpretation and expression. Since visualization is a form of communication, students of visualization must analyze and create data visualizations to truly understand the impact of graphic representation.

If interpretation and expression are two different directions on our metaphorical street (Visualization Blvd.), then cognitive processes must be the pavement.

(Bedward)I'm struggling with the rhetorical nature of the production of scientific reality, visualization, because of what (Jessee) stated, the importance of making sense of the relationship presented as a precursor to evaluating the image. Meaning making which are strong in both cognitive and constructivist discourse is extremely complex. In fact, Tufte stresses aesthetic as being inextricably linked to moral, political and cognitive issues (Pauwels, 2006). Yet, Holton, Mathewson and others discuss how the current success of scientific research is based on aligning data within X-Y Cartesian plane. My struggle is one of reconciling the practice-theory-rhetoric of visualization with culture. There is an assumption, due to silence or omission in the literature, that culture does not play a role in image interpretation. I argue that culture allows for visualization to exist and evolve. There is a coloring or bias inherent, based on the "lens" we see the world through. We must not limit our scope to the mere reflecting of scientific fact, when, visualization can equally take the form of hypothesis that represent subjective knowledge (Pauwels, 2006). Subjectivity can be derived from myth, art, language etc. If a core aspect of learning is thinking and problem solving in all its dimensions, then we need to see visualization not only as a tool for replicating reality but also revealing reality. The value of visualization is in solving problems, filling gaps in our knowledge, or facilitating knowledge building or transfer (Pauwels, 2006). Mind and geography, affect our perception and representation of reality. To use (Jones') metaphor, maybe culture is the pavement by which visualization aids in revealing complex phenomena.

Pauwels, L. (2006). Visual Cultures of Science: Rethinking Representational Practices in Knowledge Building and Science Communication. New Hampshire: Dartmouth College Press.

(Jones) This post doesn't build on what John and Emily have laid out so far, but it is relevant. I guess I'm putting on my pragmatic shoes and taking a short step back. The Rhetorical Model is at the top of Dr. Wiebe's Scientific and Technical Graphic Communication Model. Rhetoric has become a dirty word in modern use, but its true definition, to me, should be the basis for all communication (Sci Vis or otherwise).

Rhetoric is the art of persuasion. The rhetorical situation as I have learned it has three components (Schriver 1997):

1) audience

2) invention

3) heuristics

The first in the list is the least prone to categorization. Audiences have prior knowledge, cultural bias (to John's point), and differing levels of verbal, visual and spatial ability. In the discourse of visualization, we assume that the communicator knows the audience (which is not always the case).

The second is intangible. Invention means original thought, which for Tufte is the theoretical goal of his work. Tufte suggests that the theory of graphics is supposed to derive new graphical forms. He credits Marey with accomplishing that with his La Methode Graphique on page 31 of our Tufte reading:

Although the resolution of this image is too low to see the labels, you may remember this schedule from our readings. It's a classic example of graphic invention.

The third rhetorical component consists of the rules and guidelines proposed by Tufte, Kosslyn and others. Heuristics, whether they pertain to use of blank space or font size of headers, are useful and proven for developing communications.

The point is that any communicator, whether graphic designer, educator, business person other other, must know the audience, introduce an "original thought," and adhere to fundamental rules when developing information content.

Tufte, E. R. (1983). The Visual Display of Quantitative Information. Cheshire, CT: Graphics Press.

Schriver, K. A. (1997). Dynamics in Document Design. John Wiley & Sons, Inc. Wiley Computer Publishing.

(Bedward) I don't see (Jones') comments as going off on a tangent. Clear definitions of each container is important if we are to contribute, change or modify Wiebe's original model. So, I would like to delve a little deeper on the audience portion of (Jones') definition of rhetoric and hopefully tie it back to Wiebe's original statement. The construction of scientific artifacts for audience consumption is in fact a selfish act and I contend the creator has little regard for the audience--this is not faulting the creator but instead suggesting the act of creation is an internal cognitive process that helps shape the creators understanding of the investigation. Representation becomes a visual shorthand for the scientist who is conducting scientific inquiry. Audience understanding is of secondary concern to the creator. I suggest audiences must develop a practice of visual literacy in order to appreciate scientific visualization. Components of visual literacy include: visual thinking (which is both a cognitive act and a set of skills), visual learning (whereby the reader develops meaning making and interpretation of the representation), and visual communication (the ability for the viewer to express their ideas visually). As da Vinci stated, his visualization process saper vedere, [was] knowing how to see. Sound rhetoric does not guarantee audience understanding of the visualization since it is a complex task requiring formal training, some may argue otherwise. I think understanding expectations of the creator's visualizations may help to untangle viewer expectations of the visualization--interpretation is muddy. "...visual representation of scientific information relies on the credibility of the source, the degree to which the image is comprehensible, and the degree to which the image is an accurate reflection of the scientific principle or data." As Latour has pointed out on many occasions, laboratory science study provides different insights into the nature and practice of science. Recipients of visualizations must be expected to look into the minds of the creators visualization in order to firmly understand the visualization in question. Developing visual literacy moves us closer to understanding.

Pauwels, L. (2006). Visual Cultures of Science: Rethinking Representational Practices in Knowledge Building and Science Communication. New Hampshire: Dartmouth College Press.

(Mapson) With that thought in mind (Bedward-looking into the minds of the creators)can't we revisit Lester's Model, adding Zimmerman's seventh approach (rhetoriacal approach)for analyzing visual images? Lester stated that we first analyze major elements such as color, form, depth, and movement. Then we move through the six perspectives; personal, historical, technical, ethical, cultural, and critical. Through this process we move beyond our initial reactions by analyzing and understanding images and content presented. This may be off tangent, but this process (Lester's Model) also makes me wonder whether or not this becomes part of the study of design. As the creator of an image or design, moving from visualization to what I tend to think of as applied visualization, McCracken stated that "to become literate in the design process requires acquiring the cognitive and procedural knowledge needed to create a design, in addition to familiarity with the processes by which a design will be carried out to make a product or system." I believe that ties directly back to Lester's Model and what (Bedward) stated about recipients of visualizations looking back into the minds of the creators.

McCracken, J. (2000). Design-The creative soul of technology. In E.Martin(Ed.) Technology education for 21st century:49th Yearbook, Council on Technology Teacher Education (pp.85-90). Peoria,Il: Glencoe/McGraw-Hill.

(Jones) Based on previous comments, it seems Dr. Wiebe's model must "draw a line in the sand." The communication model must require audience consideration at the beginning (as it does now), but does not include visual thinking (one component of visual literacy that Bedward referenced above).

Therefore, visualization in terms of da Vinci's work exists outside the model, even though he used his drawings, etc. to communicate ideas to others. First and foremost, da Vinci's visualizations were self-serving to his scientific thought.

On the flip side, when USA Today publishes a graphic in the Money section depicting the largest online retailers, "looking into the mind" of the graphic creator is not important to the rhetorical situation.

(Mack) If you define both terms - 'see' and 'think', it may be easier to relate them to the concept of scientific visualization. It is very easy to get bogged down in thereotical concepts, but if we go back to the heart of the matter - (Wiebe) how the 'important aspect of a visualization is how it is processed by the automatic, perceptual centers in the brain' - we can surmise that your initial viewing of an object and the connections and interpretations that immediately follow are what prompts the concrete knowledge base that we use to associate things from (I apologize for that run-on sentence!!)

(still Mack) Now, the definition for 'see' is to see something and/or to perceive with the eyes....while to think means to have a conscious mind, to some extent of reasoning, remembering experiences, making rational decisions, etc. (also to employ one's mind rationally and objectively in evaluating or dealing with a given situation)...so, after stating this, it seems to me that seeing something is just that - to look at something. To think, however, implies more than a passing glance...it requires connections and associations to prior knowledge, so I have to disagree with the statement that 'past experience and other more cognitive factors are not as important'. As important to scientific visualization? Is this not why experts say that it is important to stimulate babies, even though they may not remember everything, but it helps them to begin forming associations and a knowledge base? I would hesitate to discount anything that we see, because it would be virtually impossible to pinpoint when the thinking and connecting began.

(Bedward) To follow-up on (Mapson's) viewpoint. Lemke states, organizational relations requires a definition of the whole and parts of those wholes, both in the semiotic space of the text and in the (ecosocial) interactional space...He is concerned with the audience participation in interpreting visualizations. It supports Mapson's reiteration of the seven perspectives. I also see Jones' point that great minds may create visualizations without the audience in mind. But in the case of da Vinci, could communicating to self, be considered an audience of one. If so then I retract my earlier statement and submit that audience is always part of process of image creation.

Redirection--if visualizations are mainly graphical interpretations of mathematical knowledge--we spent considerable time viewing visualizations within an X-Y independent/dependent variable perpective--as well, Holton suggests that the success of science is in scientist ability to represent their knowledge within a X-Y cartesian coordinate system, which greatly facilitates scientifc communication. Should we then think of visualizations as only mathematical--empirical and rational--entities, in otherwords purely objective. Which as (Mack) has stated is the thinking portion of visualization. If this is the case then do issues of culture, ethics, historical and personal have any place in the ability and ways of understanding visualizations. Lester's model implies a certain level of subjectivity, which is counter to mathematical knowledge. To quote Jones', science is absolute, therefore, there should be no ambiguity in interpretations of the visualizations. Should we remove the design, artistic element of visualization since it is difficult to quantify. SciVis implies the scientific components of meaning-making only, or does it?

Lemke, J. L. (1998). Multiplying meaning: Visual and verbal semiotics in scientific text. In J. R. Martin & R. Veel (Eds.), Reading science: Critical and functional perspectives of the discourses of science (pp. 87-111). New York: Routledge.

(Bedward) I've been thinking about the model and believe visualization should be at the center of the Venn diagram. If we are going to explode and discuss the components of visualization (2D, 3D, Static, Dynamic, audience....tools) then we may need to do the same with the rhetoric, performance, information, theory and practice. Another way of thinking about visualization is not to make any delineation of when visualization occurs. Is Wiebe's model implying, visualization occurs only after processing, fact finding and analysis, or is visualization a concurring real time parallel processing event. I propose visualization is atemporal and not a temporal activity.

(Jessee) I would agree (Bedward) and the movement of the previous conversations in thinking visualization should be the center of this diagram. In working on the mid-term and final projects, our main goal is to evaluate visualization where the audience, goals, theories, etc, stem from our desire to explore and integrate scientific and technical graphical communications. I, too, think scientific visualization is still in its “infancy” (Najjar), room for exploration (practice may have permanent links with problem solving and critical evaluation) is desirable as this model continues to mature.

Najjar, L. J. (1995). A review of the fundamental effects of multimedia information presentation on learning (Technical Report GIT-GVU-95-20): Georgia Institute of Technology, Graphics, Visualization & Usability Center.

(Jones) I'd like to quickly weigh in on three points that have been made this afternoon. I certainly agree that Sci Vis is in its infancy -- probably half of the readings we've done this semester admit that individual differences within an audience are omnipresent, and have the potential to hinder Sci Vis instruction in a classroom. This problem seems to be accepted as a fact of life, at least until we know more about the human brain.

Second, visualization may be atemporal, but it's complementary trappings are not. For example, the label on a diagram, the caption under a photograph and the description of a time plot series are all textual elements. These textual elements, which according to Hegarty and Vekiri are guiding components of information design, must be interpreted serially and computationally.

Neither of these two observations remove visualization from the center of the "star model" that Bedward shared, but I wonder...what's the point of a model if it doesn't reflect a structure or process?

Hegarty, M., Carpenter, P. A., & Just, M. A. (1991). Diagrams in the comprehension of scientific text. In R. Barr, M. L. Kamil, P. B. Mosenthal & P. D. Pearson (Eds.), Handbook of reading research (Vol. 2, pp. 641­668). New York: Longman.

Vekiri, I. (2002). What is the value of graphical displays in learning? Educational Psychology Review, 14(3), 261-312.

(Bedward) Jones' I agree with your second point. My argument is the upper portion of the model does not read as a process, but instead as a series of containers. The understanding of the process is clear to the author and possibly to astute students in our class. But I'm challenged on how to navigate my way through each one of the models/submodels. I respect what Dr. Wiebe said about not forcing the creation of a laundry list for each area, but we seem very clear on the process and components once we reach the visualization stage, yet our wolfwikis indicate, that much still needs to be thought through (at least on my part) on what definitive processes are involved at the upper level. Would anyone consider this model monosemic? Should models embed multiplicity of meaning? Can someone help clarify the role of a model as a means of visualizing meaning.

just for grins :)

(Kim) We can say that seeing is more related to the perception and thinking is related to the working memory and long-term memory. If the visualization is all about just what I ‘see’ and not ‘thinking’, why a number of researchers are concerned with the limitations of the human perception and cognition when designing the graphics? Here is my personal story. A few days ago, my friend had a car accident and she tried to explain to me what happened to her. Even though we talked in Korean, it was hard to understand about the situation she tried to explain. Finally, she drew some pictures with her explanation verbally and after that I could understand completely and much more easily than only her verbal explanation. I realized that there might be answers about these questions in this situation: what is the visualization and what is the role or the purpose of the visualization. When I saw that picture she drew, I figured out what the picture represented (perception), and also I was thinking about my prior car accident experience occurred a few weeks ago and tried to associate my experience with that picture because that situation was similar with mine(thinking). This whole process occurred at the moment and made me understand the situation better. That is, the visualization is not just ‘see’ as well as more than that including cognitive factors such as recall, associating, reasoning, and problem solving, etc. With regard to the purpose and the role of the visualization as mentioned earlier, I can say that they would be facilitating the learning and helping the audience learning/understanding more accurately and quickly about some particular information with minimum mental cost. This is consistent with Bertin that he discussed an ‘image theory’ and emphasized the ‘efficiency’, which is a shorter period of perception.

The scientific visualization is not merely automatic perception, therefore just displaying to show a number of information in a small space without considering the human cognitive aspects could be useless. Basically, scientific visualization should not forget their basic purpose and role: ‘facilitating’ the learning of the audience and this process should be occurred ‘efficiently’ in terms of human performance (perception and cognition) and these factors should be considered first when designing the scientific visualization.

(Whitman) Cognitive factors, including past experience, play a key role in scientific visualization. They allow the viewer to not only perceive a visual representation that the eyes see, but to visualize and understand the scientific concept and message that the representation is communicating as well. Cognitive factors derive meaning from a representation so that it is more than just lines and dots on a surface that are viewed by the eyes, but a representation of a scientific concept that displays information and conveys a message. For example, our brain can receive messages from different channels, verbal and audio, (Mayer & Moreno, 2002) and interpret them in working memory by developing relationships between the messages and past experiences. Cognitive factors also allow us to determine the credibility of the visualization by comparing past experience and knowledge of scale to the messages that we see in the visualization.

Research has shown that cognitive factors are important to consider in the design of scientific visualization since they can influence how effectively the viewer understands the message being represented in the graphic. One should try to discover the cognitive abilities, individual differences, and prior knowledge of the audience in order to design the graphic so it will be perceived as intended. For example, if the audience has little prior knowledge of the subject matter, then the diagram should reiterate what is in the text, not introduce new information. Also, animation may be helpful for viewers in certain contexts. Individual differences of the viewers will make a certain type of diagram more effective in some situations than others (Hegarty, 1991; Vekiri, 2002; and Mayer & Moreno, 2002.

By applying cognition to the messages we perceive from a visual representation, we can use the representation as a tool. It can be used as a tool for learning as working memory integrates messages coming through one or more channels and relationships are made with past experiences in working memory in order to store the concept being communicated into long-term memory (Mayer & Moreno, 2002). A visual representation can also be used as a tool to solve problems. For example, in working memory we can remember visual symbols labeled in a legend and then draw meaning from a map by drawing relationships between the representations on the map and their corresponding symbols. Such relationships can help us understand the meaning of the concept being communicated and draw conclusions from it that will assist us in solving problems, such as how to get from point A to point B in the space represented in the map. Visualizations are useful tools for problem solving since they display multiple variables and therefore decrease the amount of information that is required to be held in working memory (Vekiri, 2002). It is our cognitive abilities that allow us to visualize, a process that can enable us to understand a more complex concept from a simple representation, such as a three-dimensional system from a two-dimensional diagram.

Hegarty, M., Carpenter, P. A., & Just, M. A. (1991). Diagrams in the comprehension of scientific text. In R. Barr, M. L. Kamil, P. B. Mosenthal & P. D. Pearson (Eds.), Handbook of reading research (Vol. 2, pp. 641¬668). New York: Longman.

Mayer, R. E., & Moreno, R. (2002). Animation as an aid to multimedia learning. Educational Psychology Review, 14(1), 87-99.

Vekiri, I. (2002). What is the value of graphical displays in learning? Educational Psychology Review, 14(3), 261-312.

(Jones) Now I see what Bedward is talking about -- it's the "containers" at the top of the model that don't seem to gel with the lower portion of the model. For me personally, the "theory" and "practice" containers complicate the diagram.

The rhetorical situation seems like a logical starting point, but it also seems that "audience/goals" belong in the "rhetorical model" container at the top. I also believe that considerations of "visual performance" must be accounted for before "information design" techniques are applied.

(Whitman) While having both theory and practice listed at the top of the Scientific and Technical Graphical Communication Model (shown below) may seem to complicate the model, both components must be shown since they are different but equal contributors to the design of a scientific visualization. For an effective scientific visual to be designed, one should research the theory behind visual perception, psychology, technical communication, scientific visualization, and other relevant disciplines in order to understand how the visual will be perceived and interpreted by the viewer and to design accordingly. Practice must also be considered since someone who is designing a scientific visual would benefit from learning from and expanding on the experience of other designers who have put their scientific visuals into practice and observed the reactions of viewers. A designer can therefore benefit from research that addresses both theory and practice of scientific visualization.

Like Jones, I do not necessarily agree with the model's lower portion. I think that Visualization Tools and Techniques should be moved to the upper portion of the diagram at the same level as Data/Information and Audience/Goals. I think that it is another contributing factor to the decision of what type of visualization should be selected since the tools available and the techniques or skills of the person developing the visualization are limitations or options that can effect the decision of what type of visualization should be used. For example, if someone does not have an animation software program like Flash, then that would influence their selecetion of visualization type, and they would choose a 2D static visual. I am confused by the arrows used in the model to show the decisions between 2D or 3D and static or dynamic. I feel that a matrix would more suitably show this decision making process, such as the one that I show below.

(Kim)Maybe I'm not the only one who was curious about the rhetorical model. I would agree with what Jones said that 'rhetorical situation seems like a logical starting point, but it also seems that "audience/goals" belong in the "rhetorical model" container at the top.' I thought that STGC model is a kind of a process or structure regarding design of the scientific visualization. According to this model, I believed that the rhetorical(the audience, original goals(or purpose),and heuristics) components should be considered first at the beginning of designing the visualization. Also I believed that this makes a foundation stone for the design of the visualization. However, like Jones said, the presence of 'audience/goals' again the below made me confused because I believed that those components are already considered at the beginning stage of the design. And also I am confused that several overlapped portions among three models. Is this for emphasizing a just combination between rhetorical model,the theory with centering the visual performance model,and the practice with focusing on the information design model everything together? Or does the overlapped portions imply other things I cannot catch?

(Earnshaw)I also agree with Jones. If we are following a traditional technical communication or instructional design model, Audience/Goals should be placed at the beginning, along with Visualization Tools and Techniques. Past experience, as well as learning styles or preferences, play a role in one's ability to interpret a graphic. I've included my take of the 2D/3D static/dynamic issue (from the research I did at IBM).

Image:Type of interaction.bmp (having issues uploading the .bmp, so I'm e-mailing it to everyone)