Interac(veStorytelling

DerekWoodgate

Interac(veStorytellingisaformofdigitalentertainmentinwhichuserscreateorwhereuserscaninfluenceadrama(cstorylinethrough

ac(ons,eitherbyissuingcommandstothestory'sprotagonist,orac(ngasageneral

directorofeventsinthenarra(ve.Interac(vestorytellingisamediumwherethenarra(ve,

anditsevolu(on,canbeinfluencedinreal‐(mebyauser.

Redefiningmeaningandcontext

EngagementPlot

Story

Userinterac3on

Storytelling

Driversandinfluencers Newtechnologies Designskills/approaches Socialchange Hybridiza(onandconvergence Cogni(veandthebehavioralsciences Userinterfaces

Theuseristhestory

Fromgamingtointerac(venarra(vespaces

  Non‐linear–post‐structuralist  Abstract–nomadic

  Remixlifestyles

  Transmedia

 Mixedreality

  Collabora(veinterplayandimprovisa(on

  Extensionofhownarratologyandludologyintersect

Fromprogrammablemoviestosimulatedreali(es  Interferenceandguidance Alterna(veplotsandstorylines  Spa(altransforma(on

  Programmableenvironments

 Dialogmanagement

  Cogni(vefeedbackandexplora(on

Theuserdesigner

TheISdesigner  Narra(vegenera(onengine/system(includinginbuiltconstraint

selectorandpoten(alfordifferentPOV)

  Authoringtools  3DComputergraphicsoXware/3Dmodelers/anima(on

  camera

  visualrenderingsystem

  interac(onmechanismbetweenmul(pleagents,etc

  interac(veauthoringinterface/hap(cs  representa(onsoXware/scriptlanguage  iden(fica(onand“registra(on”system,

  A.I.planner,etc.Thenever‐ending

list

AIandmachinelearning  Humancomputerinterac(on:pro‐ac(veandresponsiveac(ons

  Convergenceofskills,performance,analysisandadjustment–improvingtheuserexperience

  Pathfindingandplanning(incl.nonplayercharacters)  Naturalbehaviorallanguageforstory‐basedbelievableagents  Interac(veactors  Affec(vecompu(ng  Socialengineeringofcharacters  Reinforcementlearning  Resourcegathering

Characterbuilding/agents

LANoir:addingemo(on• Facialemo(on

• Real‐worldactorsonthedigitalstage

• Moralitysystems

• Goodconversa(ons

TheWatchman_synthe(ccharacters

 CGmovesintothefuture

 Digitally‐replicatedactorswho'llgoonperforminglongaAertheactorisgone

 Rewri3ngthestory Bringingbackdeceasedactors

Avatar–superimposingactors  Crea(nganen(reworldwithcomputergraphicsandthendirectlyintegra(ngac(ngperformances

  JapaneseTelexistencesurrogateAntropomorphic

 Headgear,gloves,vestswithcontrolgear

On‐setmo(oncapture

AI–humaninterac(on Music Creatures: Synthetic AI animals act as an

interface to direct and act with humans in movies.

Thehologram,theperformer,theprojec(on

Touchableholograms Anarrayofcomputer‐controlledultrasonicwaveemiaerswithfinepointsthatemitpulsesofairthatfeellikepressureonyourhand.

  Systemknowswhereyourhandis–behavioralfeedback.

  Thesensa(onofweightandtexture.

Interac(ngwithholograms

body>data>space

Next‐Genera3onSynthe3cPerformers

  Introducinglevelsofautonomyintothesynthe(cperformer(suchaseyecontact,facialandgesturalmovement,andlipsync),

  enablingmoreflexible,engaging,anddirectablerobot‐mediatedperformanceontheset.

  SecondSkinisaprojecttobuildawearablefabricthatsupportsmillimeter‐accurateloca(onandbio‐parametertrackingatthousandsofpointsonthebody.

  Suchafabriccancomputeandpredict3‐Drepresenta(onsofhumanac(vityandusetheinforma(ontoaugmenthumanperformance.

Remoteinterac(onwitharobot Robo(cavatarswillallowhumanstofullyexperiencetheenvironmentofotherworlds.

  Throughtheeyesofrobo(cavatarswewillwatchthesunriseovertherusty,redcraterrimswithouthavingto"experiencesuffoca(on,theicydeathof‐200degreesContheirskinorthes(ngofmicroscopicdustintheireyes.”

Learningthecharacter’ssoul

Revisi(ng“Creatures”andBlackandWhite”–theinterac(vestorytellinglearningversions.

Gamecharactersthatcanpredictthebehaviorofplayers

Thefantas3cflyingbooksofMr.MorrisLessmore

Interac(vefortheipadversion

Interac(veStorytellinginmusicvideos MoonbotStudiosandThePolyphonicSpreetogetherhavecreatedaninterac(ve,character‐basednarra(vemusicvideo.

Features:‐Seamlessworldofinterac(on‐Touchclouds,plantsandanimalstolife‐Evolvingvisualsthatchangewiththemusic‐FollowandinteractwithacreaturenamedYou‐Me‐TiltyourdevicetoswayYou‐Meashefalls‐ChoosebetweentheStudioorAcous(csongtracks

Contextbasedinterac(vestorytelling

HeavyRain  Interac(venarra(veifthechoiceswecanmakearemerelyalteringspecificeventsratherthantheactualcontext.Surelythewholepointofhavingthestoryplayoutdifferentlydependingonplayerchoicesistoaddreplayvalue.

Programmablemovies Moviesthatchangewithcontext(observer,emo(ons,place,or(me).

  Long‐DistanceBarcodesworktomakebothcamerasandtheworldmoreintelligent,byallowinguserstopiecetogetherandmergeseparateimagesusingmetadataencodedintotheimage.

  Thisworkwillallowforthemergingofmul(pleviewpointstocreatericherstoriesfromvariedstorytellers.

SensingSpaces

 Interac(vereconfigurablespaces

 Integratedvisual,virtual,realspaces Useofrobo(candAIs(mula(onandsimula(on

 Sensorclustersandnetworkstodriveperformancecuesandaudio‐visualinterfaces

 Gravitytocreateperformanceeffects

 Wearablecompu(ngtocreateperformanceandeffectssynchronicity

 Narra(ve‐changingsensorsusingaudiencefeedback

 Useofavatarsasperformerinterfaces

SensingSpaces

Dynamic spatial interfaces

Interac(veinstalla(onusesdynamicspa3alinterfacesandahuman‐scaledisplaysystemtoallowvisitorstoexplorethephotographicworkofar(stTa(anaParceroonmul(plelevelsoftheirownchoosing.

Interac(vestorytellinginannotatedenvironments  iRiS(ImmediateRemoteInterac(onSystem

  Usercan“paint”,createagamelikeajigsawpuzzle,aninterac(vestorywithoneormorepar(cipantsonafaçadefromamobiledevice.  

  ANW(ar(culatedNaturalityWebandcomputervision

Interac(vehumanbodymuseum•  Tac(leinterac(vemuseumexperiencethattakesvisitorsthroughthespacesinsidetheirownbodies.

•  Atthemuseumyoucanexperiencehayfeverfrominsidethenose,repletewitharollickingsneeze.

•  Youcandon3Dglassesandwatchascartoonspermfer(lizeanegg.Youcanbounceyourwayacrossarubbertongueasyounavigatetastebudsandhearburpswellingupinthedeep.

Addingaffectandatmosphere  Programmablesurfacesandaddi(onaldisplays

  Newhap(cs,withgesture,smell,vibra(onandtemperatureamplifica(on

  Immersiveandresponsive/Reac(vevideoandportraits

  Sociallyawaresystems,ambientandinterac(onenhancerssuchas:causalmappingsbetweenpastandfutureeventsorbetweenanac(onanditsreac(on

  Programmedatmospheresandaesthe(cvisualeffects

Situa(onawarenessandBehavioraliden((es  Ambientdetec(on‐toolstosharehabits

  AnnotatedenvironmentsandDistributedInterac3veVideo

ArraysforEventCaptureandEnhancedSitua3onalandInterac3veAwareness

  Humandynamicsandemo(onmodeling‐affec(veandcogni(vepercep(on,feedbackandawareness‐modeling

socialinterfacingandfeedback‐crea(ngbehavioraliden((es

You are the performance

‐ SpherAleasdevice–mobiledevicewithsensorsthatcreatesonorous

shapes

‐ TheTheateroftheSenses‐"EchooftheShadow,”Audiencepar(cipa(oninalabyrinthofperformances

‐HiddenWorldsofNoiseandVoice

(voicetovision)

Projec(ngyourselfintothegame

  BeingThere

  Thisprojector‐basedapproachprovidesawaytovisualizere‐crea(onsofrealandimaginarysitesthatarebothvisuallyandspa(allyrealis(c.

  Usershaveastrongsenseofimmersionandnaturalinterac(onastheywalkaroundavirtualsite.

Cameras  Computa(onalCinematographyandDisplay:  ‐futurecamerasforabletosharevisualcontent,andcrea(ng4‐Dand6‐Ddisplaysforrichercollec(onandpresenta(onofinforma(on.

  Videoofthefuture(videomanipula(on,videocoding,3D,andperceptualstudies),computa(onalcinematography,humanandfacialanima(on,andcapturetechnologies.

  Addi(onalstrengthsincludewirelessnetworkingandcomputa(onalmaterials

Sensorrobots  Capturing3Dmodelsofenvironments,outdoorandindoor,andcapturingac(vitythatishappeninginenvironments,usingcamerasmountedonmobilerobots.Anewmodeofdeployingcomputervisionisnowappearingasautonomousrobots

Displays  Interac(ve/HolographicTV Noborderscreens  RolloutportableOLEDscreens

Morphablemoviestudios This technology allows one physical space to represent a variety of things.  

 Projects involve techniques to augment and programmatically change the appearance of physical objects. 

 The physical object is illuminated with a data (or video or slide) projector.  

 The images to be projected are computed with a 3‐D graphics‐rendering program. This allows you to change the appearance of real objects, adding special effects to the world around you.  

Mobileprojec(on:ElectricPoetryAwebinterfaceforconstruc(ngpoemstobebroadcastinreal‐(meonthewallsofLondon'sundergroundsta(ons,forLondonTransport'sPlalormforArt

Similartothepopularfridgegame,peopleassemblewordsintopoemswhichthenappearinpublicorcommercialspacesaroundthecity

Storyteller Storyteller:ToolsandtechnologiesdevelopedintotoolkitsforchildrentoempowerthemcraXtheirowncompellingstoriesandcharacterstofostercrea(vityandlearninggoals.

 Researcherswillalsoleveragesynthe(cperformertechnologiestocreatecompellingrobo(corvirtualcharactersthatserveaslearningcompanionsforchildren

Educa(on

Interac(vestorytellinginfancy  Interac(veStorytellingtechnologiesintermsofperformanceandscalability

  Tomakethenextgenera(onofInterac(veStorytellingtechnologiesmoreaccessible

  Todevelopamoreintegratedapproachtointerac(vestorytellingtechnologies

 Methodologiestoevaluateinterac(vestorytellingsystemsaswellasthemediaexperienceofinterac(venarra(ve

Goingforward  Performanceandscalability Adap(ngtonewISanduserdesires Adap(ngnewhybridsofconvergedmixmediaandtechnologies

  Pumngthehumaninthelandscape.

  Therearerolesoftheintegra(onofISinsomanydifferentareas–thinkbeyondgamingandfilm

Past–Present‐Future

AssassinCreed3