algorithm visualization · •two main learning approaches: –conceptualize algorithm at a high...
TRANSCRIPT
Algorithm VisualizationAlgorithm VisualizationReflections and Future DirectionsReflections and Future Directions
John StaskoJohn Stasko
Information Interfaces Research GroupInformation Interfaces Research Group
School of Interactive Computing & GVU CenterSchool of Interactive Computing & GVU Center
Georgia TechGeorgia Tech
[email protected]@cc.gatech.edu
http://http://www.cc.gatech.edu/~staskowww.cc.gatech.edu/~stasko
ICER – Sept. ‘07 2
IntroductionIntroduction
•• BS Math 1983, BS Math 1983, BucknellBucknell
•• PhD CS 1989, BrownPhD CS 1989, Brown
–– TANGO algorithm animation systemTANGO algorithm animation system
•• On faculty at Georgia Tech since ‘89On faculty at Georgia Tech since ‘89
ICER – Sept. ‘07 3
TANGO
Polka
Polka-RC
ConcurrentProgramsPVM/ConchPThreadsHPF
PVaniM
EvaluationPairheap
Intro Algos
BH & DFS
HW Scenario
Lens
Choreographer
Algorithm Animation Program Visualization
Trees
LargeDataSets
Dance
Information Mural
XTango
89
90
91
92
93
94
95
96
97
98
99
Polka-3D
Samba
ICER – Sept. ‘07 4
Algorithm Animation’s HistoryAlgorithm Animation’s History
•• Quick look back for those who may not Quick look back for those who may not
be so familiarbe so familiar
ICER – Sept. ‘07 5
Sorting Out SortingSorting Out Sorting
•• Seminal work in areaSeminal work in area
•• 30 minute video produced by Ron 30 minute video produced by Ron
BaeckerBaecker at Toronto in 1981at Toronto in 1981
•• Illustrates and compares nine sorting Illustrates and compares nine sorting
algorithms as they run on different data algorithms as they run on different data
setssets
ICER – Sept. ‘07 6
Sorting Out SortingSorting Out SortingVideo
ICER – Sept. ‘07 7
Early Early AlgoAlgo AnimAnim SystemsSystems
•• SmalltalkSmalltalk--based systems, Animusbased systems, Animus
–– Ralph London & Rob Ralph London & Rob DuisbergDuisberg, ,
Univ. of WashingtonUniv. of Washington
•• BALSABALSA
–– Marc Brown, Brown Univ.Marc Brown, Brown Univ.
ICER – Sept. ‘07 8
BalsaBalsa
•• Used in “electronic classroom” at BrownUsed in “electronic classroom” at Brown
•• Introduced use of Introduced use of
multiple views and multiple views and
interesting event interesting event
modelmodel
M. Brown Computer ‘88
ICER – Sept. ‘07 9
Brown’sBrown’sElectronicElectronicClassroomClassroom
Circa 1983
Apollo workstations
ICER – Sept. ‘07 10
TangoTango
•• Added smooth animation to algorithm Added smooth animation to algorithm
animationanimation
•• Simplification of the design/programming Simplification of the design/programming
process (easier programming model and process (easier programming model and
framework)framework)
•• Formal model of the animation, the PathFormal model of the animation, the Path--
Transition ParadigmTransition Paradigm
ICER – Sept. ‘07 11
TangoTango
StaskoIEEE Computer ‘90JVLC ‘90
Multipleframes frombubblesort
ICER – Sept. ‘07 12
POLKAPOLKA
Video Demo
Stasko & KraemerJPDC ‘93
ICER – Sept. ‘07 13
PolkaPolka--3D3D
•• 33--D & VR version, same animation modelD & VR version, same animation model
VL ‘93
Stasko & Wehrli
Quicksort side view
Blue dots are as in2-d view
Colored planes represent exhanges
ICER – Sept. ‘07 14
StudentStudent--Built AnimationsBuilt Animations
•• SambaSamba
–– Simple animation scripting languageSimple animation scripting language
circle 1 0.8 0.8 0.1 red halfline 2 0.1 0.1 0.2 0.2 green thinrectangle 3 0.1 0.9 0.1 0.1 blue solidtext 4 0.0 0.0 0 black Hellocircle 6 0.3 0.3 0.2 wheat solidtriangle 7 0.5 1.0 0.6 0.8 0.4 0.9 cyan solidbigtext 8 0.2 0.2 0 black Some Big Textmoveto 1 6moverelative 3 0.05 -0.4jumprelative 4 0.4 0.4lower 1color 6 blue
StaskoSIGCSE ‘97
- Embed print statements in any program to generate
ICER – Sept. ‘07 15
TransitionTransition
•• In the mid ’90’s, my work began to In the mid ’90’s, my work began to
transition from building systems to transition from building systems to
thinking about the pedagogical thinking about the pedagogical
implications of the technologyimplications of the technology
ICER – Sept. ‘07 16
EvaluationEvaluation
•• Big question:Big question:
–– Do algorithm animations help students learn Do algorithm animations help students learn
and understand better?and understand better?
•• I think that this issue has dominated I think that this issue has dominated
algorithm animation research of the last algorithm animation research of the last
~8 years~8 years
ICER – Sept. ‘07 17
How to Evaluate?How to Evaluate?
•• What’s a good way to evaluate the What’s a good way to evaluate the
pedagogical value of an algorithm pedagogical value of an algorithm
animation?animation?
Ponder
ICER – Sept. ‘07 18
My WorkMy Work
•• Conducted a number of empirical studies Conducted a number of empirical studies
to answer questionto answer question
–– Compare learning with animation to learning Compare learning with animation to learning
withoutwithout
–– An evolving methodologyAn evolving methodology
•• Concluded with a couple reflective Concluded with a couple reflective
articlesarticles
ICER – Sept. ‘07 19
1. Pairing Heaps1. Pairing Heaps
•• Classical experimental designClassical experimental design
–– Exam scenarioExam scenario
–– AnimAnim vs. no vs. no animanim, followed by test, followed by test
•• Results: Just having animation doesn’t make Results: Just having animation doesn’t make learning happenlearning happen
•• Reflection: Difficult for student to leverage Reflection: Difficult for student to leverage animation of complex algorithm when they animation of complex algorithm when they don’t understand algorithm and visual mapping don’t understand algorithm and visual mapping yetyet
•• Reflection: The better students benefit more…Reflection: The better students benefit more…
Stasko, Badre & LewisInterCHI ‘93
ICER – Sept. ‘07 20
2. Introductory Algorithms2. Introductory AlgorithmsSorting, graphsSorting, graphs
•• Lab style methodologyLab style methodology
•• Results: Benefits to animation, though Results: Benefits to animation, though
unevenuneven
•• Reflection: Interaction is the keyReflection: Interaction is the key
–– Students who enter their own data sets into Students who enter their own data sets into
the algorithm benefited from animationthe algorithm benefited from animation
Lawrence, Badre, & StaskoVL ‘94
ICER – Sept. ‘07 21
3. Binomial Heap & DFS3. Binomial Heap & DFS
•• Classical experimental design with Classical experimental design with
prediction addedprediction added
–– Is animation like prediction?Is animation like prediction?
–– Does it help the student to anticipate what Does it help the student to anticipate what
will happen next and learn from that?will happen next and learn from that?
•• Results: Mixed findings, some supportResults: Mixed findings, some support
•• Reflection: We need a new methodologyReflection: We need a new methodology
Byrne, Catrambone & StaskoComputers & Ed ‘99
ICER – Sept. ‘07 22
4. Binomial Heaps4. Binomial Heaps
•• Change to HW scenarioChange to HW scenario
–– Provide student with learning objectives, materials, Provide student with learning objectives, materials, and questions up frontand questions up front
–– Give unlimited work timeGive unlimited work time
•• Results: Significant benefit to animationResults: Significant benefit to animation
–– Students worked longerStudents worked longer
•• Reflection: Animations appear to help Reflection: Animations appear to help motivationmotivation
–– Make a complex algorithm less intimidatingMake a complex algorithm less intimidating
–– Students with animation were “happier”Students with animation were “happier”
Kehoe, Stasko & TaylorIJHCS ‘00
ICER – Sept. ‘07 23
5. Samba5. Samba
•• Students built their own animations Students built their own animations
during algorithms classduring algorithms class
–– Performance on final exam on questions Performance on final exam on questions
about those algorithms was excellentabout those algorithms was excellent
–– Anecdotal evidence that this helped Anecdotal evidence that this helped
understandingunderstanding
–– Students spent a lot of time tweaking Students spent a lot of time tweaking
graphicsgraphics
StaskoSIGCSE ‘97
ICER – Sept. ‘07 24
Unreported WorkUnreported Work
•• InIn--class assessment of class assessment of quicksortquicksort
•• Worked with focus groups of studentsWorked with focus groups of students
•• Developed task analysis methodology for Developed task analysis methodology for
ensuring that animations were illustrating ensuring that animations were illustrating
the “right” thingsthe “right” things
ICER – Sept. ‘07 25
Little Victories…Little Victories…From *******@felix.cc.gatech.edu Wed Feb 25 19:59:22 1998Date: Wed, 25 Feb 1998 19:52:17 -0500To: [email protected]: Susan ****** <*******@cc.gatech.edu>Subject: cs3158
I know you always wanted feedback on using algorith m animations...Well, I have another instance where I think it help ed me...
I was taking a sample Computer Science GRE,a question was asked that had five sorting algorith ms as thepossible answers.
I was pondering the question and thinking about the algorithms- some that we've been exposed to for years are easy to remember,almost inherent. But then I came to "shellsort," a nd I stopped fora second, going "aw geez... that was a complicated one..." butthen suddenly I could see that picture of the anima tion in my headand could remember it as clear as a bell.
Thought you'd enjoy it....Susan ******
ICER – Sept. ‘07 26
Let’s Step Back a MomentLet’s Step Back a Moment
•• Some important questions:Some important questions:
–– What does it mean to understand an What does it mean to understand an
algorithm?algorithm?
–– What makes a good algorithm animation?What makes a good algorithm animation?
–– How should algorithm animations be used?How should algorithm animations be used?
–– ……
ICER – Sept. ‘07 27
UnderstandingUnderstanding
•• What does it mean for a student to What does it mean for a student to
understand an algorithm?understand an algorithm?
•• What are we evaluating in these studies?What are we evaluating in these studies?
Ponder
ICER – Sept. ‘07 28
Understanding an Algorithm isUnderstanding an Algorithm is
•• Knowing which tasks, or types of tasks, for which the algorithm Knowing which tasks, or types of tasks, for which the algorithm is suited or appropriateis suited or appropriate
•• Knowing how to “run” the algorithm, knowing its procedures, Knowing how to “run” the algorithm, knowing its procedures, operations, steps and chronologyoperations, steps and chronology
•• Knowing the algorithm’s purpose and what it can accomplishKnowing the algorithm’s purpose and what it can accomplish
•• Analyzing the algorithm’s complexity and efficiencyAnalyzing the algorithm’s complexity and efficiency
•• Having the ability to code or program the algorithmHaving the ability to code or program the algorithm
•• Being able to articulate about the algorithm in one’s own highBeing able to articulate about the algorithm in one’s own high--level (natural language) wordslevel (natural language) words
•• Having the ability to transfer knowledge about it to another Having the ability to transfer knowledge about it to another algorithm or domainalgorithm or domain
•• Knowing where the algorithm fits into the bigger Knowing where the algorithm fits into the bigger taxonomy/methodology of algorithms in generaltaxonomy/methodology of algorithms in general
ICER – Sept. ‘07 29
Typical Student ApproachTypical Student Approach
•• Focus group findingsFocus group findings
•• Two main learning approaches:Two main learning approaches:
–– Conceptualize algorithm at a high level Conceptualize algorithm at a high level (English discourse). Students referred to (English discourse). Students referred to this as being able to describe it in their own this as being able to describe it in their own terms terms (top down)(top down)
–– Start with pseudo code and get good grasp Start with pseudo code and get good grasp of how it works mechanically, including of how it works mechanically, including tracing on data sets tracing on data sets (bottom up)(bottom up)
Newstetter
ICER – Sept. ‘07 30
Learning OrderLearning Order
•• A. Knowing which tasks, or types of tasks, for A. Knowing which tasks, or types of tasks, for which the algorithm is suited or appropriatewhich the algorithm is suited or appropriate
•• B. Knowing how to “run” the algorithm, B. Knowing how to “run” the algorithm, knowing its procedures, operations, steps and knowing its procedures, operations, steps and chronologychronology
•• C. Knowing the algorithm’s purpose and what C. Knowing the algorithm’s purpose and what it can accomplishit can accomplish
•• D. Analyzing the algorithm’s complexity and D. Analyzing the algorithm’s complexity and efficiencyefficiency
•• E. Having the ability to code or program the E. Having the ability to code or program the algorithmalgorithm
•• F. Being able to articulate about the algorithm F. Being able to articulate about the algorithm in one’s own highin one’s own high--level (natural language) level (natural language) wordswords
•• G. Having the ability to transfer knowledge G. Having the ability to transfer knowledge about it to another algorithm or domainabout it to another algorithm or domain
•• H. Knowing where the algorithm fits into the H. Knowing where the algorithm fits into the bigger taxonomy/methodology of algorithms in bigger taxonomy/methodology of algorithms in generalgeneral
•• A & CA & C
oror
B & FB & F
•• Other pair aboveOther pair above
•• EE
•• DD
•• HH
•• GG Ponder
ICER – Sept. ‘07 31
Algorithm Animation’s Value?Algorithm Animation’s Value?
•• For which of the learning objectives can For which of the learning objectives can
algorithm animations help?algorithm animations help?
•• Perhaps better: For which will algorithm Perhaps better: For which will algorithm
animation have greater learning animation have greater learning
potential?potential?
ICER – Sept. ‘07 32
Assess AA UtilityAssess AA Utility
•• A. Knowing which tasks, or types of tasks, for which the A. Knowing which tasks, or types of tasks, for which the algorithm is suited or appropriatealgorithm is suited or appropriate
•• B. Knowing how to “run” the algorithm, knowing its procedures, B. Knowing how to “run” the algorithm, knowing its procedures, operations, steps and chronologyoperations, steps and chronology
•• C. Knowing the algorithm’s purpose and what it can accomplishC. Knowing the algorithm’s purpose and what it can accomplish
•• D. Analyzing the algorithm’s complexity and efficiencyD. Analyzing the algorithm’s complexity and efficiency
•• E. Having the ability to code or program the algorithmE. Having the ability to code or program the algorithm
•• F. Being able to articulate about the algorithm in one’s own higF. Being able to articulate about the algorithm in one’s own highh--level (natural language) wordslevel (natural language) words
•• G. Having the ability to transfer knowledge about it to another G. Having the ability to transfer knowledge about it to another algorithm or domainalgorithm or domain
•• H. Knowing where the algorithm fits into the bigger H. Knowing where the algorithm fits into the bigger taxonomy/methodology of algorithms in generaltaxonomy/methodology of algorithms in general
Ponder
ICER – Sept. ‘07 33
What Makes a Good Algorithm Animation?What Makes a Good Algorithm Animation?
•• What are the elements of an effective What are the elements of an effective
visualization/animation?visualization/animation?
•• How does one design a visualization that How does one design a visualization that
students can learn from?students can learn from?
Ponder
ICER – Sept. ‘07 34
Good AA DesignGood AA Design
•• Very little work done on this topicVery little work done on this topic
•• Design has largely been a very individual, Design has largely been a very individual,
intuitive matterintuitive matter
•• Involves graphic design, pedagogy, Involves graphic design, pedagogy,
understanding of misunderstandings, understanding of misunderstandings,
story telling, artistic sense, …story telling, artistic sense, …
ICER – Sept. ‘07 35
Multiple ViewsMultiple Views
•• Can be quite valuable but also Can be quite valuable but also
problematicproblematic
–– We can only look one place at a time (people We can only look one place at a time (people
may not notice the text at the bottom of one may not notice the text at the bottom of one
important view)important view)
ICER – Sept. ‘07 36
HeapsortHeapsort Case StudyCase Study
•• Taught psychology faculty colleague the Taught psychology faculty colleague the
heapsortheapsort algorithm using an existing algorithm using an existing
animationanimation
•• Q & A session uncovered many problems Q & A session uncovered many problems
with the animationwith the animation
–– 20+ changes made20+ changes made
ICER – Sept. ‘07 37
HeapsortHeapsort ChangesChanges
•• Show numbers appear in array view Show numbers appear in array view first (reading them in). Then place first (reading them in). Then place them in the tree view and (somehow) them in the tree view and (somehow) highlight the correspondences. highlight the correspondences.
•• Put better text region somewhere for Put better text region somewhere for telling viewer what is happening. telling viewer what is happening.
•• Use circle with small lines out sides Use circle with small lines out sides (indicating (indicating subtreesubtree) to tell where a heap ) to tell where a heap is being checked.is being checked.
•• Change color of node "dropping down" Change color of node "dropping down" in in heapifyheapify operation. Rather than operation. Rather than putting the "H" there.putting the "H" there.
•• Put some text like "Comparing key Put some text like "Comparing key values" and "Swapping with biggest“ in values" and "Swapping with biggest“ in tree view when that occurs. Similarly, tree view when that occurs. Similarly, put some text in array view to draw put some text in array view to draw viewers attention to swaps there.viewers attention to swaps there.
•• Put up text to tell when valid heap has Put up text to tell when valid heap has been built. been built.
•• Put up text to indicate when root and Put up text to indicate when root and
last leaf are being swapped. last leaf are being swapped.
•• Change color of node when it has been Change color of node when it has been
moved out of tree and is in place (do in moved out of tree and is in place (do in
both views). both views).
•• Take circle away at this later stage of Take circle away at this later stage of
algorithm. algorithm.
•• Change color of node moving down in Change color of node moving down in
heapifyheapify..
•• Pause longer after Pause longer after heapifyheapify finishes. finishes.
•• Put text on both sides of line in array Put text on both sides of line in array
view, indicating regions that are sorted view, indicating regions that are sorted
and unsorted respectively. and unsorted respectively.
•• Take out that "Elevate #" text at top.Take out that "Elevate #" text at top.
•• Move line over in array view to very left Move line over in array view to very left
end.end.
Video Demo
ICER – Sept. ‘07 38
The Learner is Your UserThe Learner is Your User
•• Understanding an algorithm well Understanding an algorithm well
handicaps you in designing an animation handicaps you in designing an animation
of itof it
•• We should learn from HCIWe should learn from HCI
–– Have learners as part of your design teamHave learners as part of your design team
–– Iteratively prototype animation Iteratively prototype animation
–– Test it with other learnersTest it with other learners
ICER – Sept. ‘07 39
Algorithm Animation’s PlaceAlgorithm Animation’s Place
•• How should we use algorithm animations How should we use algorithm animations
for learner benefit?for learner benefit?
–– What’s the scenario of use?What’s the scenario of use?
–– When?When?
–– Where?Where?
–– How? How?
Ponder
ICER – Sept. ‘07 40
Potential Use ScenariosPotential Use Scenarios
•• In classIn class
–– When and how?When and how?
•• LabLab
•• OfflineOffline
ICER – Sept. ‘07 41
Typical ClassTypical Class
•• A discussion of the motivation for an algorithm and how it mightA discussion of the motivation for an algorithm and how it mightbe used in the real world. Frequently, analogy is used to give be used in the real world. Frequently, analogy is used to give the students a feeling for the practical value of the algorithmthe students a feeling for the practical value of the algorithm
•• A discussion of the high level methodology and general A discussion of the high level methodology and general operations of an algorithm. Here, a very informal description ioperations of an algorithm. Here, a very informal description is s provided, attempting to give the students a "feel" for the provided, attempting to give the students a "feel" for the algorithmalgorithm
•• A discussion of the detailed methodology and operations of an A discussion of the detailed methodology and operations of an algorithm, with a focus on tracing the algorithm in pseudoalgorithm, with a focus on tracing the algorithm in pseudo--code, code, understanding its control flow and memory usageunderstanding its control flow and memory usage
•• A discussion of the algorithm's computational complexity, that iA discussion of the algorithm's computational complexity, that is, s, an analysis of its space and time requirementsan analysis of its space and time requirements
•• A discussion revolving around the presentation of an algorithm A discussion revolving around the presentation of an algorithm animation illustrating the algorithm's methods and operationsanimation illustrating the algorithm's methods and operations
ICER – Sept. ‘07 42
RepresentationRepresentation
•• Note that each lecture activity involves a Note that each lecture activity involves a
different kind of representationdifferent kind of representation
•• Each uses a set of artifacts and Each uses a set of artifacts and
discussion terms from a different discussion terms from a different
domain/languagedomain/language
ICER – Sept. ‘07 43
ComponentsComponents
•• Motivation/ExampleMotivation/Example
•• HighHigh--level level
methodologymethodology
•• Detailed Detailed
methodologymethodology
•• Complexity analysisComplexity analysis
•• Algorithm animationAlgorithm animation
•• RealReal--world things, world things,
artifacts, problemsartifacts, problems
•• English discourseEnglish discourse
•• Pseudo code or Pseudo code or
programming lang.programming lang.
•• Math notationMath notation
•• Graphics and Graphics and
visualizationvisualization
ICER – Sept. ‘07 44
ReflectionsReflections
•• Running the sessions amazingly instructive to Running the sessions amazingly instructive to
meme
–– Often saw just how an animation wasn’t helping a Often saw just how an animation wasn’t helping a
studentstudent
•• Two papers reflect on the evaluation findings of Two papers reflect on the evaluation findings of
many researchersmany researchers
–– “Meta“Meta--EvalEval of AV,” of AV,” HundhausenHundhausen, Douglas, & , Douglas, & StaskoStasko, ,
JVLCJVLC , June ’02, June ’02
–– ““AlgoAlgo VisVis,” ,” StaskoStasko & & HundhausenHundhausen, appears in , appears in CS CS Education Research, Education Research, ’04’04
ICER – Sept. ‘07 45
ResultsResults
•• Findings definitely mixed, but some studies do Findings definitely mixed, but some studies do
show benefitsshow benefits
–– ------> Potential for pedagogical aid is there, but just > Potential for pedagogical aid is there, but just
can’t throw animation at algorithm and expect it to can’t throw animation at algorithm and expect it to
helphelp
–– Interaction & engagement is keyInteraction & engagement is key
•• Blindly watching algorithm animation not really helpfulBlindly watching algorithm animation not really helpful
•• Student must interact with animation and be engagedStudent must interact with animation and be engaged
–– Animations help motivation, can make algorithm less Animations help motivation, can make algorithm less
intimidatingintimidating
ICER – Sept. ‘07 46
Future WorkFuture Work
•• What’s still missing in algorithm What’s still missing in algorithm
visualization research?visualization research?
–– Further evaluation (somewhat)Further evaluation (somewhat)
–– Simpler animation construction (somewhat) Simpler animation construction (somewhat)
–– Design principlesDesign principles
–– Giant web indexGiant web index
–– Integrated curriculum Integrated curriculum
–– Interactive classroom blackboard toolsInteractive classroom blackboard tools
ICER – Sept. ‘07 47
Controversial ThoughtsControversial Thoughts
•• Age breeds cynicismAge breeds cynicism
•• Let me push a few buttonsLet me push a few buttons
ICER – Sept. ‘07 48
Teach, Don’t RationalizeTeach, Don’t Rationalize
•• The people who seem to benefit most The people who seem to benefit most
from algorithm animations are those who from algorithm animations are those who
already understand the algorithmalready understand the algorithm
•• MaybeMaybe algoalgo animanim helps instructors helps instructors
present better (or think they present present better (or think they present
better) and that’s why they like itbetter) and that’s why they like it
–– But do students learn better?But do students learn better?
Ponder
ICER – Sept. ‘07 49
No Good Use ScenarioNo Good Use Scenario
•• It takes too much time to use an It takes too much time to use an
algorithm visualization well in classalgorithm visualization well in class
•• Difficult to get students to use them Difficult to get students to use them
offlineoffline
Ponder
ICER – Sept. ‘07 50
Software VisualizationSoftware Visualization
•• Good introductionGood introduction
to field to field
•• Many AA chaptersMany AA chapters
–– HistoryHistory
–– DesignDesign
–– SoSSoS
–– EvaluationEvaluation
ICER – Sept. ‘07 51
AcknowledgmentsAcknowledgments
•• Research supported by National Science Research supported by National Science
Foundation and the GVU CenterFoundation and the GVU Center
•• More infoMore info
–– http://www.cc.gatech.edu/gvu/iihttp://www.cc.gatech.edu/gvu/ii
–– http://http://www.cc.gatech.edu/gvu/softvizwww.cc.gatech.edu/gvu/softviz
–– [email protected]@cc.gatech.edu
ICER – Sept. ‘07 52
Thanks for Your Attention!Thanks for Your Attention!
•• Questions?Questions?