Special Section

ArtScience: The Essential Connection

Guest Editor: Robert Root-Bernstein

The seventh installment of a Leonardo special project exploring the work and writings of artistic scientists who find their art avocation valuable; scientifically literate artists who draw problems, materials, techniques or processes from the sciences; or others interested in such interactions.

10-oneill.ps - 3/14/2007 2:59 PM

10-oneill.ps - 3/14/2007 2:59 PM

A long-standing connection exists betweenthe worlds of animation and natural history, in particular inthe studies of evolution and anatomy. This article addressesthe use of natural history topics as inspiration for narrative el-ements in animation and the advisory work of scientists re-garding the accuracy of images depicted. Conversely, I alsodiscuss the influence of artists, designers and engineers uponthe methods of scientists. One of the earliest examples ofAmerican animation (produced in Brooklyn, New York) wasWinsor McCay’s Gertie the Dinosaur (1914), in which the powerof animation was used to represent an extinct entity and toportray it being “tamed” by a modern human, McCay himself.

Disney Studios encouraged animators to study anatomy,both by bringing live animals into the studio and by send-

ing artists to visit zoos. The study oflive animals and consultation withexperts have been long-standingpractices in the world of animation.The realistic grounding that the re-sultant knowledge brings is unpar-alleled and reflected in some of thebest animation. Furthermore, as 3D animation becomes increasinglycomplex and often realistic, tech-nical animators are turning toanatomy-driven systems to re-createand simulate the underlying struc-tures (bones and muscles) that drivethe look of these hyper-real charac-ters. In doing so, they rely on the consultations of experts inthe fields of anatomy and evolution. Similarly, the techniquesused in animation and computer graphics are being increas-ingly used by researchers not only to visualize their data butalso as a means of collecting new information from fossil skele-tal material and living species. An interesting feedback loophas developed, in which anatomy and animation inform eachother in order to achieve compelling results.

©2007 ISAST LEONARDO, Vol. 40, No. 2, pp. 168–173, 2007 169

Emerging Congruence between Animation and Anatomy

Rob O’Neill

Rob O’Neill (researcher, artist), Digital Arts Research Lab, Pratt Institute, ISC 101A, 200 Willoughby Avenue, Brooklyn, NY 11205, U.S.A. E-mail: <roneill@dal.pratt.edu>. Web site: <www.morphometric.com>.

A B S T R A C T

The worlds of animation andanatomy have a long-standingconnection based on both direct and indirect collaboration.The author surveys a number of projects in which anatomistshave consulted on animationprojects or animation techniqueshave been used for data gather-ing and analysis. The authordescribes his own work in lightof this connection.

ARTS

CIE

NC

E: T

HE

ESS

ENTI

ALC

ON

NEC

TIO

N

Fig. 1. Stills from the animation The Theory of Transformations, a 3D rumination based on the work of D’Arcy Went-worth Thompson. (© Rob O’Neill) A 3D model is warped by a procedurally generated grid.

Article Frontispiece. dataProjections 003. (© Rob O’Neill) The data,which has been processed as in Figs 2 and 3, here is mirrored tocreate a semblance of the source object (a female gorilla’s skull). All skulls used in the dataProjections series are housed in the collec-tion of the Division of Anthropology of the American Museum of Natural History. Data collected by Ken Mowbray.

10-oneill.ps - 3/14/2007 2:59 PM

ANATOMY FOR ANIMATORSScientists are often called in to consulton animation projects, and their workand advice is incorporated to varying de-grees. Willy Ley, Wernher von Braun andHeinz Haber were consulted for the1950s Disneyland space series that in-cluded Man in Space (1955), Man and theMoon (1955) and Mars and Beyond (1957).In these films, animators created charac-ters and situations in outer space basedon interviews with the scientists. MerianCooper (on behalf of Willis O’Brien)consulted with the American Museum ofNatural History for “gorilla specifica-tions” [1] on King Kong (1933). Greystoke:The Legend of Tarzan (1984), although notan animated film, employed the knowl-edge of Roger Fouts, co-director of theChimpanzee and Human Communica-tion Institute, whose expertise in teach-ing chimpanzees human language in theform of American Sign Language for theDeaf was inverted when he was asked toguide human actors to act more like non-human apes [2]. The Jurassic Park filmseries, which featured 3D animated di-nosaurs in a live-action film, benefitedgreatly from the consistent input of pa-leontologist James Horner, Curator of Pa-leontology at the Museum of the Rockies,acting as a technical consultant. Horner’sinput included a range of information,

tion of a character still remains groundedin realistic biomechanics to some degree.This grounding is where animation oftenbegins; from there the style of motionneeded for the project is designed andthe limits of motion, or lack thereof, aredefined. Sumida’s studio lectures inter-weave details of bone structure and mus-cle activity with species idiosyncrasies toillustrate topics such as animal postureand gait. These lectures are attended byanimators as well as by artists who imple-ment this information in the character’sconstruction.

The position on modern productionsknown as the character technical direc-tor (or, more commonly, character rig-ger) takes all this information and buildsthe control rig that animators manipu-late to pose the character in the mannerneeded for the shot. The skeleton in-side drives the deformation of the digi-tal sculpture. Much thought, sculptingand technology is put into how the char-acter looks while undergoing every posi-tioning of the joints by the animator.These motion and deformation systemsare the foundation of all digital char-acters, and the character technical di-rectors are the individuals who bridgeanatomy and technology to create an in-tuitive animatable character rig for usein the project. An anatomy text is alwayson hand, and the position of every joint

from skin texture and coloration to themechanics of dinosaur movement. It haseven been suggested that Horner was apartial inspiration for the original bookand subsequent movie’s lead character,Dr. Alan Grant. For the Pixar film Find-ing Nemo (2003), a 3D animated feature,the animators were treated to a special12-lecture course on fish behavior and lo-comotion by Adam Summers, a professorin the Ecology and Evolution departmentat the University of California at Irvine.Summers was asked to discuss such top-ics as fish shapes and colors but in theend taught what he referred to as a “graduate-level ichthyology course” tothe Pixar staff [3]. In all these examples,scientific accuracy was taken into accountto some degree, but the end result canbest be described as an artist’s interpre-tation of this information.

Stuart Sumida [4] is a paleontologistat California State University at SanBernardino who consults with animationand visual-effects companies. His spe-cialty is comparative anatomy and bio-mechanics of vertebrates with regard tolocomotion. Providing animators with anunderstanding of the underlying physi-ology of animals helps them to betterunderstand the nuances involved in ani-mating characters in a way that looksnatural. Even on projects in which thecharacter design is very stylized, the mo-

170 O’Neill, Animation and Anatomy

ARTS

CIE

NC

E: T

HE

ESS

ENTI

ALC

ON

NEC

TIO

N

Fig. 2. dataProjections 001, digital print developed from scientist-collected morphometric data. (© Rob O’Neill) The simulated object on theright is the 3D polygonal structure created by the raw data collected from a male gorilla skull. The object on the left is the shadow producedby casting simulated light through the object on the right.

10-oneill.ps - 3/14/2007 2:59 PM

in the character is scrutinized while staticand in motion to achieve the desired look for the character. As noted above,the complexity of these characters is in-creasing at a dramatic rate, and most fea-tured characters utilize some form oflayered muscle system to deform the over-lying geometry. The complexity and an-atomical abstraction of these systemsvaries from studio to studio and from pro-duction to production, but the principlesremain the same.

Consultation by experts in anatomy orthe behavior of animals depicted variesdepending on the film and the level ofrealism needed. Certainly, on films suchas the Jurassic Park series and programscreated for educational benefit, expertsare brought in early to help define themotion of the creatures developed forthe production. It is likely that we will seemuch more collaboration on this front inthe future. Every animation student stud-ies figure drawing [5] and the dynamicanatomical work of Leonardo da Vinci,but turning this information into some-thing that moves over time often takes ex-pert advice from those who study theintricacies of anatomy. Similarly, anima-tors and computer-graphics program-mers, as well as the techniques they havedeveloped, are starting to find their wayinto research projects and work previ-ously informed exclusively by scientificdata.

ANIMATION FOR ANATOMISTSThere are a number of cases emerging in which scientists are delving into theworld of animation and computer graph-ics not only for the purposes of datavisualization but also for data collec-tion and analysis. Four research projectsconducted in the last few years are worthexploring. The first is that of StephanGatesy [6] of Brown University and the“Scientific Rotoscoping” technique. Ro-toscoping is an animation technique inwhich the motion of the animated char-acter is created by the tracing of live-action reference footage. This is a com-mon technique in both traditional 2D animation and modern digital 2D/3Danimation. Walt Disney even used roto-scoping on such classic films as SnowWhite (1937), for which he filmed an ac-tress acting out the role and then pro-vided the film to his animators as a 2Dbaseline of realistic motion from whichthey could begin work. The motion de-rived from this method has a very realis-tic and controlled look when comparedwith that done without this frame-by-frame reference. Gatesy and his team are

possible solutions for the position ofbones from frame to frame. Evaluatingthese multiple possibilities has been aneye-opening experience, and Hutchin-son and Gatesy are employing strict con-straints and innovative techniques tonarrow the realm of possibilities [8] andshed some light on how these methodscan develop into documented analyticaltechniques.

While scientific rotoscoping takes acue from animation techniques, a col-laborative project between paleontolo-gists at the American Museum of NaturalHistory and computer scientists at theUniversity of California at Davis takes its

creating accurate 3D animations of thelocomotion of animals (often birds) byfilming them on treadmills and in windtunnels with an X-ray camera. These X-ray movies are then imported into 3Danimation programs in which hierarchi-cal models of the bones, which have beenmeticulously sculpted (or scanned), arematch-moved on a frame-by-frame basisto re-create the motions and thus createaccurate animated 3D models of the mo-tion. Scientists are now using what Dis-ney and many others have relied on foryears to re-create and study motion.

Similarly, the modern-day analogue of rotoscoping is motion capture, often

O’Neill, Animation and Anatomy 171

ARTS

CIE

NC

E: T

HE

ESS

ENTI

ALC

ON

NEC

TIO

N

Animators and computer-graphicsprogrammers are starting to find their way into scientific research projects

derided as “The Devil’s Rotoscope” (re-flecting the perception that it is ”cheat-ing” and not true animation), in whichmarkers are placed on a performer in amulti-camera calibrated space. The mo-tion of these markers is then convertedto 3D point positions, thus re-creatingthe motion of the performer. This tech-nique is used by John Hutchinson [7] ofthe Royal Veterinary College of the Uni-versity of London where, as in Gatesy’swork, the motion of extant species suchas elephants and other large animals iscaptured and compared to informationcollected from dinosaur anatomy (with-out the benefit of motion capture). Thisis done to gauge what generalities aboutlarge animal stance and gait emerge fromcomparisons of these different lineages.Problems with motion retargeting, thetransformation of one creature’s pro-portions to those of another, and dif-ferences in anatomy prevent a directtransfer of joint angles from elephants todinosaurs, but the connections gleanedfrom traditional and emerging biome-chanical analysis tools have shed an in-novative light on the subject. These tworesearchers are working intensely withanimation techniques and methodolo-gies on various research endeavors to in-form their analysis and to visualize theirresults with great success. They are alsodiscussing the limitations and restraintneeded in making assumptions based onanimation techniques. Merely animatingthe bones does not amount to a scientificanalysis and results in a vast number of

lead from classic computer graphics tech-niques, in particular morphing. Morph-ing is the transformation of one shapeinto another. The work being carried out,labeled “Evolutionary Morphing” [9], is actually the interpolation of shapetransitions between extinct and living pri-mate species in an attempt to mathe-matically fill in the gaps in the fossilrecord. With the advent of 3D laser-scan-ning technologies, the ability to capture,at high resolution, the shape and detail of anatomical specimens is changing the way data are collected and analyzed.Databases of specimens can be stored and compared using methods that relynot on the resolution of the scan but onthe shape of the specimen. Work withanatomical landmarks is also changing.Anatomical landmarks are definedanatomical locations typically used formeasurement and are denoted by skele-tal features such as the nasion, which isthe midline point where the two nasalbones meet the frontal bone (forehead).These markers can now be identifiedvisually by the investigator (as they havebeen traditionally) or, increasingly, bysoftware via algorithmic analysis of thesurface. In addition, the use of semi-landmarks or interpolated landmarksplaced by the software between true land-marks is becoming more prevalent asthese markers are used to define shapeand drive analytical morphing from onespecies to another. The power and, I would say, artistry of this technique al-low researchers to investigate the hypo-

10-oneill.ps - 3/14/2007 2:59 PM

thetical shapes (and thus hypotheticalspecies) interpolated in between them.While most morphing operations are lin-ear—the points of shape A are moved in a linear manner to their correspond-ing position on shape B—the evolution-ary morphing technique (encapsulatedin software called Landmark) bases itsmorphing on the biometric mathemati-cal underpinnings of the work of F. JamesRohlf of the State University of New Yorkat Stony Brook. This leaves the work in-formed by the database of shapes and theprinciples of biometry and geometricshape analysis.

A step closer to the world of computergraphics, and also approaching those ofanatomy and natural history, is the soft-ware Dinomorph [10]. It is essentially abiomechanical simulator through whichscanned anatomical elements are articu-lated and simulated in a physical envi-ronment. This allows a researcher tointeractively hypothesize the mechanicsof an extinct species with the intent ofgaining a better sense of stance, postureand motion. The physics of many mo-tions, in animals ranging from long-necked sauropods to modern giraffes,can be simulated and explored from abiomechanical point of view. These arethe subtle details that are difficult to dis-cern from raw anatomical study and willultimately help bring extinct species tolife by the hand of the animator.

tion between this influential volume andmy work is my series entitled The Theoryof Transformations (named for a chapterof On Growth and Form) (Fig. 1), com-posed of 3D interpretations of Thomp-son’s work in animated and print form.In many ways, these ruminations onshape represent a direct animation ofThompson’s work and have been a semi-nal exploration in my work.

dataFace is a multi-component workthat I created to undertake simple ex-periments with a large morphometricdata set. The data set is one collected byW.W. Howells (1908–2005, professor ofanthropology at Harvard University) be-tween 1968 and 1980 and now main-tained by the University of Tennessee.The data set contains 82 craniometricmeasurements from over 3,000 humanskulls housed at various museums. I havebeen fascinated by this data set sincediscovering it as an anthropology major at Brooklyn College. Over 10 years laterand after much time studying computergraphics and animation, I have begun togenerate software and art pieces using it.The first piece in this series is a simpletower of measurements in which eachbizyomatic breadth (essentially the fullwidth of the face) is generated as a linethat is added to a stack of lines repre-senting each member of the data set. Thecolumn of data is then printed to scale sothat the widths are accurate representa-

FINDING THEMIDDLE GROUNDAs an observer of all these projects (and a consultant on the Evolutionary Morphing project), as well as a charactertechnical director, graphics programmerand artist, I am intrigued by these rela-tionships and the results emerging fromthem.

A great influence on my work, as wellas that of many others in the field, hasbeen the writing and illustrations ofD’Arcy Wentworth Thompson in his 1917volume On Growth and Form [11]. Thebook outlines some early principles in bi-ological morphology, or the study ofshape. Thompson found geometry andpattern in the natural world and intro-duced Cartesian morphs (grid-based geo-metric deformations of 2D surfaces) tothe thinking of biologists. Thompson es-sentially applied the idea of morphing tobiological structures and thereby plottedthe profiles of various species, warpedthem to look like other species, andgenerated deformed grids based on theresults to illustrate the difference. His il-lustrations were compelling at the timeof their publication and continue to be asource of inspiration. In my work as anartist I have always returned to this vol-ume, which has challenged me to un-dertake experiments that compare theshapes of biological objects. A connec-

172 O’Neill, Animation and Anatomy

ARTS

CIE

NC

E: T

HE

ESS

ENTI

ALC

ON

NEC

TIO

N

Fig. 3. dataProjections 002. (© Rob O’Neill) As in dataProjections 001, the object depicted is the simulated shadow of the object created, in this case a female gorilla’s skull.

10-oneill.ps - 3/14/2007 2:59 PM

tions of the measurements. In a seconditeration of this piece, I programmed ananimated line that moves through thedata as an act of meandering across thewidth of all the faces in much the sameway that the eye would track across faces.In these pieces the line reveals nothingbut the variation in the data set. Taken astep further is a data-generated facesketch based on many of the data pointsplotted on a 2D plane, illustrated by ani-mated points that are drawn procedurallyto give an impression of the face of theindividual (Color Plate A). This proce-dure is carried out for each of the indi-viduals in the data set. I am captivated bythe idea that all these collected numbersreflect the person that they were col-lected from.

The Morphology Project is a series ofartistic ruminations on these data withthe intention of bringing some life to the cold numbers that were so meticu-lously, I would venture to say artistically,captured. The gestures required to cap-ture these measurements entailed a great deal of discipline and precision, and thedata were collected for hours, days andmonths on end at various anatomical col-lections at museums all over the world.Where in the past, measurements werecollected with a wide array of appara-tuses—such as calipers, protractors orhand-made measuring boxes—much oftoday’s data is collected with digital tools such as 3D point plotters. The datacollected by these devices provide a 3Dcoordinate point cloud of the samelandmarks that were used to capture dis-tances. Once these landmarks are cap-tured in 3D space, a large number ofmeasurements can be calculated fromthem. Many of these measurementswould be impossible to record on a phys-ical specimen due to the position of andangles between the landmarks. In this useof 3D landmark data collection, whatemerges is the path traveled by the datacollector. The points are stored as a lineardata set of the number of the points col-lected, followed by the x-y-z coordinates.By reading these points in and drawing aline from one to the next, one follows thehand of the researcher and ends with an-other example of the artistic gesture ofdata collection. Data Projections 001 is anexample of work created from this ex-ploration of scientific data collection.The shape procedurally drawn is that ofa gorilla skull—the shape data was col-lected via a 3D point plotter, in this casean Immersion MicroScribe, and modeled

References and Notes

1. Tom Huntington, “King Kong: How the GreatestSpecial-Effects Movie Was Made with the SimplestTechnology,” American Heritage of Invention & Tech-nology 21, No. 3, 21–31 (Winter 2006).

2. “Biology of King Kong,” panel, 2006 Tribeca FilmFestival, New York (6 May 2006).

3. “The Making of Finding Nemo,” CG Society,<http://features.cgsociety.org/story_custom.php?story_id=1389>.

4. Stuart S. Sumida, <www.stuartsumida.com>.

5. See, for example, Kimon Nicolaides, The NaturalWay to Draw: A Working Plan for Art Study (Boston:Houghton Mifflin, 1941).

6. Stephen M. Gatesy, <www.brown.edu/Departments/EEB/gatesy>.

7. John R. Hutchinson, <www.rvc.ac.uk/SML/People/jhutchinson.cfm>.

8. John R. Hutchinson and Stephan M. Gatesy, “Be-yond the Bones,” Nature 440 (16 March 2006) pp.292–294.

9. David F. Wiley et al., “Evolutionary Morphing,” Pro-ceedings of IEEE Visualization 2005, 23–25 October2005, Minneapolis, Minnesota, <http://graphics.idav.ucdavis.edu/research/EvoMorph>.

10. Dinomorph, <www.dinomorph.com>.

11. D’Arcy Wentworth Thompson, On Growth andForm, Rev. Ed. (Cambridge, U.K.: Cambridge Univ.Press, 1942).

Glossary

character rigger (or character technical director)—aperson who merges anatomy with computer graph-ics to create a character rig for animated films.

landmarks (or anatomical landmarks)—definedanatomical locations typically used for measurement,ideally denoted by features on skeletal elements.

motion capture—a technique of digitally recordingmovements for entertainment, sports and medicalapplications. In general, markers are placed on a per-former and tracked, in real-time, in a multi-cameraspace.

motion retargeting—adaptation of an animated mo-tion from one character to another to compensatefor differences in size or limb proportion.

rig—the architecture of a modern digital 3D ani-mated character that defines how it moves and howit appears when in motion.

rotoscoping—an animation technique wherein themotion of the animated character is created by thetracing of live-action reference footage.

Manuscript received 9 May 2006.

Rob O’Neill is a researcher, animator and pro-grammer in New York City. He is a researchassociate in the Digital Arts Research Lab andfaculty in the Department of Digital Arts atPratt Institute. Formerly, he was the stu-dio technical director at Eyebeam, where hecontinues to consult on projects. Prior to that he was a character technical director atPDI/Dreamworks and a researcher in Cul-tural Resources at the American Museum ofNatural History. He holds a BA in anthro-pology from Brooklyn College and an MFA indigital design and technology from ParsonsSchool of Design.

using custom tools in Autodesk Maya. A3D surface is generated, and the object’sshadow is then projected onto anothersurface, producing a flat, almost graffiti-like graphic image (Fig. 2) representingan analysis and presentation of data as astudy in how hypotheses are generatedby projecting the data into hypotheticalspaces (Fig. 3 and Article Frontispiece).

My future projects will involve map-ping the aforementioned 2D Howells dis-tance measurements as 3D landmarkpoints to visualize historic data throughmodern means. Concurrently, I am de-veloping genetic algorithms to treat thedata as a population and cause it toevolve. In the end, my true goal is to col-laborate with scientists and to developfunctional crosstalk between the fields ofart/animation and research. Current de-velopments in animation, biomechanicsand morphology suggest a number of po-tential collaborations. As artists becomemore technically oriented and scientificanalysis of large data sets is increasinglyperformed graphically, the possibility ofcollaboration increases. For example, Iam working on the above-mentioned Evo-lutionary Morphing project to developgeometric techniques to “fix” 3D scans ofgeologically deformed fossil specimensso that they can be analyzed. This work iscurrently a mix of graphics-tools devel-opment, animation technology and dig-ital sculpture. For me, all of these threadsare paths that lead to personal artisticoutput in the form of animations, printsand sculptural installations that emergefrom collaborations with scientists andfrom following published research. Thework is typically personal rumination onthe scientific process, but it often leadsto the development of software that couldbe used for research. I plan to release thesoftware to researchers, as the algorith-mic processes that I use to analyze thedata are very similar, if not identical, tothe processes utilized in research-specifictools. Much of this software will be avail-able via open-source distribution.

Looking forward, collaborations withscientists and data gleaned from researchare important for me as a working art-ist in addition to my work in developingtools for character-animation technology.This is merely the beginning of uncover-ing and fostering the balance betweenanatomists impacting animation produc-tion in a meaningful way and the toolsand techniques of animation evolvinginto truly accurate and usable tools forscientific research.

O’Neill, Animation and Anatomy 173

ARTS

CIE

NC

E: T

HE

ESS

ENTI

ALC

ON

NEC

TIO

N

10-oneill.ps - 3/14/2007 2:59 PM