a qualitative study of animation programming in the wild aniket dahotre, yan zhang, christopher...

A Qualitative Study of Animation Programming in the Wild

Aniket Dahotre, Yan Zhang, Christopher ScaffidiESEM 2010

22

Roles for animation programming

• A vehicle for getting kids excited about programming• An environment for teaching programming skills• A medium for communicating and entertaining• A platform for research aimed at raising usability of

programming tools

• Examples of animation programming tools:○ Logo, KidSim, AgentSheets, Alice, Hands, Scratch

Intro Technical Social Remixing Closing

33

Scratch as a particular programming tool

• Turing-complete language

• Events, loops, conditionals, sprites, sound…

• Drag-and-drop programming

+Online community

for sharing, trying, discussing

& remixing animation projects


44

Scratch online repository


55

Statements and anecdotesfrom related work

• Scratch site is “the YouTube of interactive media”.• Users are “a new generation of creative, systematic thinkers

comfortable using programming to express ... ideas”• Supports “creative appropriation… the utilization of someone

else’s creative work in the making of a new one” • “The site’s collection of projects is wildly diverse, including

video games, interactive newsletters, science simulations, virtual tours, birthday cards, animated dance contests, and interactive tutorials, all programmed in Scratch”

• One project obtained over 100 user comments. • Several programmers formed collaborative partnerships.• One particular Tetris game was remixed dozens of times.


66

Questions for our study

• To what extent is Scratch succeeding as a basis for developing programming skills in the wild?

○ Technical programming skills

○ Social skills

○ Remixing/reuse skills


77

Research methods

• Screen-scraped 100 randomly-selected animations○ Including their code, usage statistics, and user comments

• Developed coding schemes for several research questions○ One student examined subset of animations, proposed coding scheme○ This student and another independently applied scheme to half○ Negotiated modifications to coding scheme○ Then checked each other’s work on the other half of the animations○ < 10% disagreement; negotiated to resolve all disagreements○ Coalesce codes to increase clarity of presentation as needed

• In a few cases, related work provided relevant coding schemes○ Which we applied without modification

• A few research questions could be answered directly with quantitative (non-coded) data


88

Functional roles of animations

To teach skillor knowledge

To entertaininglychallenge

To communicatefictional plot

56%: No clear functional roleIntro Technical Social Remixing Closing

99

Use of programming constructs:repository vs. afterschool “Clubhouse”


1010

Apparent design patterns

ControllermethodWithLoop()

SpriteSpriteSpriteonMessage()

CollisionHandleronHit(type2)

Sprite of type 1

CollisionHandleronHit(type1) onHit(type2)

Sprite of type 2

AbstractHandleronHit(type1) onHit(type2) onHit(type3)


1111

Conclusion regardingtechnical skill development

• Relatively successful platform for development of technical programming skills

○ Comparable primitive use relative to Clubhouse users○ Some demonstration of patterns (perhaps subconscious)○ Comparable in complexity to spreadsheets and other programs

created by end-user programmers (see paper for details)


1212

Kinds of user comments exchanged

Approval + idea

Approval, no suggestion

Disapproval

Other commentsfrom users, other

than projectcreator

Comments fromproject creator


1313

Active collaboration on projects

• By “collaboration,” we mean ○ joint design or implementation ○ by a team of multiple people ○ consciously working on a common intellectual goal

• Reviewed user comments○ E.g., to find comments like “I used your suggestion – thanks!”○ E.g., or like “Thanks, my friend helped me with that.”○ Also examined source code to look for any action on suggestions○ Looked for comments revealing code edits by multiple people

• We found no such indications of any collaboration at all.


1414

Conclusion regardingsocial skill development

• Uneven success as platform for development of social programming skills

○ Half of projects received comments○ Most comments were complimentary and offered suggestions○ But unable to find evidence of actual collaboration


1515

Frequency of downloading and remixing

• Related work: 15% of projects were created by remixing• But what fraction of projects are used to create remixes?

• Of our 100 projects…○ 50% were downloaded at least once○ 10% were remixed at least once○ 5% were remixed at least once by other users


1616

Changes made to programs during remixing

0

2

4

6

8

10

12

14

multimedia changes

script changes

feature add/change

bug removed

bug added

Nu

mb

er o

f re

mix

es

• 20 of our 100 projects were created by remixing. Of these…


1717

Conclusion regardingremixing skill development

• Uneven success as platform for development of remixing skills

○ Most remixes simply involved multimedia tweaks○ Few attempts at script modification during remixing○ Nearly half of script modifications led to major bugs○ Even the biggest modifications were still fairly small (see paper)○ Frequency of remixing unimpressive vs other systems (see paper)


1818

Results and opportunities for future work

• Technical skill development: relatively successful○ Perhaps still a need for helping animators to do higher-level design

• Social skill development: uneven success○ Definitely a need for helping animators to collaborate

• Remixing skill development: uneven success○ Definitely a need for helping animators to remix code


1919

Additional empirical questions for future work

• Do animators consciously understand design patterns?

• What usually happens after code downloads?

• Is some hard-to-detect collaboration somehow occurring?

• What kinds of interaction happen in the online forums (outside the context of particular projects)?

• How well do these programming skills transfer to other programming languages and tools?


2020

Thank You…

• For the opportunity to present.

• For your questions, thoughts, and constructive feedback.


2121

Frequency of interactions between users

0

10

20

30

40

50

60

70

80

90

100

1 2 5 7 10 20 50 70 100

% o

f pro

ject

s w

ith ≥

this

ma

ny

eve

nts

Number of events

views

comments

downloads

remixes


2222

Size of repository animationsby animations’ functional role

88 82 8756

13

144

95 6873

120

50

100

150

200

250

Pro

ject

siz

e (#

prim

itives

) general programming

animation-specific

a qualitative study of animation programming in the wild aniket dahotre, yan zhang, christopher...

Documents

teaching programming

remixing animation projects

programming online community

use of programming constructs

research questions

related work scratch

functional roles of

proposed coding scheme