IOWA STATE UNIVERSITYIOWA STATE UNIVERSITYDepartment of Electrical and Computer Department of Electrical and Computer EngineeringEngineering

Computer Control of Computer Control of Theater Performance Theater Performance ElectronicsElectronics

SD May06-18SD May06-18

May06-18 TeamMay06-18 Team

Tarun BhatiaTarun Bhatia Amanda FarniokAmanda Farniok Sheng LySheng Ly Alex SillsAlex Sills

Project ClientProject Client

Iowa State DanceIowa State Dance– Janice BakerJanice Baker

Co-Motion Dance CompanyCo-Motion Dance Company– Valerie WilliamsValerie Williams

Project Advisory BoardProject Advisory Board

Faculty AdvisorsFaculty Advisors– Dr. Julie DickersonDr. Julie Dickerson– Dr. Gerald ShebleDr. Gerald Sheble

Emeritus FacultyEmeritus Faculty– Dr. David Stephenson Dr. David Stephenson

PurposePurpose

Provide clients with better control Provide clients with better control of their performance environmentof their performance environment

ReasonReason

The environment can be The environment can be manipulated as an interpretive, manipulated as an interpretive, active portion of the artistic dance active portion of the artistic dance performance. With this design, up performance. With this design, up to four dancers can be identified to four dancers can be identified on the stage by their transmitter on the stage by their transmitter and their input will manipulate the and their input will manipulate the environment settings.environment settings.

MethodMethod

Using IsadoraUsing Isadora– It takes input from “sensors” to It takes input from “sensors” to

“Actors” “Actors”

Method ContdMethod Contd

Use electronic sensors on Use electronic sensors on dancer’s body to transmit dancer’s body to transmit information to a computerinformation to a computer

The computer will take the data The computer will take the data and use the information to control and use the information to control settings of video, sound, and settings of video, sound, and lighting predetermined by the lighting predetermined by the performersperformers

Previous Research Previous Research ProjectProject VrJuggler was previously used on a research VrJuggler was previously used on a research

project to manipulate input from sensors project to manipulate input from sensors and display 3D video on projectors.and display 3D video on projectors.

Certain aspects of previous research was Certain aspects of previous research was taken into consideration. taken into consideration. – The old hardware was reviewed to be used for The old hardware was reviewed to be used for

the initial development of our hardware solutionthe initial development of our hardware solution– Software setup to read sensors was managed Software setup to read sensors was managed

through past documentation and research papersthrough past documentation and research papers

Hardware LayoutHardware Layout

Hardware Design - Hardware Design - ResearchResearch Analog audio wireless Analog audio wireless

transmission methodtransmission method– Problems: Modulation/compression Problems: Modulation/compression

issues, capacity/channel issues, issues, capacity/channel issues, possible interference problemspossible interference problems

Hardware Design - Hardware Design - ResearchResearch Industrial SCADA-type wireless Industrial SCADA-type wireless

transmission methodtransmission method– Problems: Bulky packaging, insufficient Problems: Bulky packaging, insufficient

channels/capacity, possible software channels/capacity, possible software (Isadora) compatibility issues(Isadora) compatibility issues

Hardware Design - Hardware Design - XBowXBow Crossbow “Mote” Transmission Crossbow “Mote” Transmission

MethodMethod– Reasons for choice: Compact sizing, Reasons for choice: Compact sizing,

2.4GHz band avoids interference 2.4GHz band avoids interference and allows for multiple transmitters, and allows for multiple transmitters, TinyOS accompanying software TinyOS accompanying software suited for project goalssuited for project goals

– Possible problems: TinyOS/Isadora Possible problems: TinyOS/Isadora communication, line of sight issuescommunication, line of sight issues

Hardware Design - Hardware Design - XBowXBow

Analog Output CircuitAnalog Output Circuit

Crossbow motes accept analog inputsCrossbow motes accept analog inputs Flex sensor variable resistors used in a Flex sensor variable resistors used in a

simple analog circuit to give 1.0-2.5 simple analog circuit to give 1.0-2.5 Volt output to mote boardVolt output to mote board

Circuit Diagram - Four Circuit Diagram - Four Sensor OutputSensor Output

MDA300 Sensor BoardMDA300 Sensor Board

Accepts analog inputs via screw terminalsAccepts analog inputs via screw terminals Provides excitation voltages, power supplies Provides excitation voltages, power supplies

for analog circuitfor analog circuit Converts analog to digital signal for wireless Converts analog to digital signal for wireless

transmissiontransmission

MICAz TransceiversMICAz Transceivers

Connects to sensor board and serial Connects to sensor board and serial interface with 51-pin connectorinterface with 51-pin connector

Transmits on 2.4 GHz Zigbee compliant Transmits on 2.4 GHz Zigbee compliant band (good range, multiple channels, band (good range, multiple channels, minimal interference problems)minimal interference problems)

Powers sensor board, simple AA battery Powers sensor board, simple AA battery power supplypower supply

MIB510 Serial MIB510 Serial InterfaceInterface

Receives signal from transceiver and Receives signal from transceiver and transmits to computer through TinyOS transmits to computer through TinyOS softwaresoftware

Powered by external AC adapterPowered by external AC adapter Through TinyOS, programs/configures Through TinyOS, programs/configures

transceivers and sensor boardtransceivers and sensor board

Hardware Software Hardware Software IntegrationIntegration TinyOS is a platform bridges the TinyOS is a platform bridges the

hardware serial input and xmidi.hardware serial input and xmidi. Xmidi - XML Musical Instrument Xmidi - XML Musical Instrument

Digital InterfaceDigital Interface– XML Representation of MIDI dataXML Representation of MIDI data

Isadora recognizes MIDI as an Isadora recognizes MIDI as an input.input.

Isadora SDKIsadora SDK

Provides overall design needed Provides overall design needed for working Isadora Plugin/Actor.for working Isadora Plugin/Actor.

Provides data structures and Provides data structures and functions that are needed to functions that are needed to implement.implement.

Isadora Data Structure Isadora Data Structure ImplementationImplementation

How Do We Do It?How Do We Do It? Important Data StructuresImportant Data StructuresPluginInfo {PluginInfo { // Struct that contains variables unique to this plugin// Struct that contains variables unique to this plugin

ActorInfo* mActorInforRef.ActorInfo* mActorInforRef. //ptr to our ActorInfor//ptr to our ActorInfor

MessageReceiveRef mMessageReceiverMessageReceiveRef mMessageReceiver //ptr to our msg reciever//ptr to our msg reciever

int sensorInputint sensorInput //input sensor to read from//input sensor to read from

int lowerThresholdint lowerThreshold //input min limit//input min limit

int maxThesholdint maxTheshold //input max limit//input max limit

float triggerThresholdfloat triggerThreshold //input threshold for on/off//input threshold for on/off

float scaledOutputfloat scaledOutput //output scaled for flex//output scaled for flex

float triggerfloat trigger //output for on/off//output for on/off

}}

Isadora SDK FunctionIsadora SDK Function

Unique Function LogicUnique Function Logic

static void ReceiveMessage(static void ReceiveMessage(IsadoraParameters*IsadoraParameters* ip,ip, //ptr to //ptr to MessageMask MessageMask // Message to listen to// Message to listen toPortIndexPortIndex // applicable to midi// applicable to midiconst MsgData*const MsgData* // data to read// data to readUInt32UInt32 // length of data for the message// length of data for the messagelong inRefConlong inRefCon //ptr to our Actor info//ptr to our Actor info

))

Other Data StructuresOther Data Structures

Isadora Actor Group IdentificationIsadora Actor Group Identification- static const OSType- static const OSType kActorClass kActorClass

Unique 4 letter id for Isadora actorUnique 4 letter id for Isadora actor- static const OSType- static const OSType kActorIDkActorID

Name for the actor used in IsadoraName for the actor used in Isadora- static const char*- static const char* kActorNamekActorName

Defines the input and output for the actor in Isadora static const char*Defines the input and output for the actor in Isadora static const char*- static const char*- static const char* sPropertyDefinitionStringsPropertyDefinitionString

Contains information regarding the input and output and Actor infoContains information regarding the input and output and Actor info- const char* sHelpStrings[]- const char* sHelpStrings[]

Isadora Functions - Isadora Functions - MinorMinor Isadora Standard functions with Isadora Standard functions with

minor dataminor data

Called when an input parameter value is changed, just reload with new dataCalled when an input parameter value is changed, just reload with new data

- static void HandledPropertyChangeValue()- static void HandledPropertyChangeValue()

Function that gets the area needed to draw the actor in Isadora, bases on # of Function that gets the area needed to draw the actor in Isadora, bases on # of input and outputsinput and outputs

- static Boolean GetActorDefinedArea()- static Boolean GetActorDefinedArea()

Isadora Functions Isadora Functions StandardStandard

Function called when actor is required to be drawn in IsadoraFunction called when actor is required to be drawn in Isadora- static void DrawActorDefinedArea()- static void DrawActorDefinedArea()

Gets Actors class and id and also all the ptrs to all the plugins functionsGets Actors class and id and also all the ptrs to all the plugins functions- EXPORT_ void GetActorInfo()- EXPORT_ void GetActorInfo()

Getter function for the Parameter StingGetter function for the Parameter Sting- static const char* GetParameterString()- static const char* GetParameterString()

Getter function the Help StringsGetter function the Help Strings- static void GetHelpString()- static void GetHelpString()

Function that instantiates the actorFunction that instantiates the actor- static void CreateActor()- static void CreateActor()

Function that destroys the actor when deleted, scene is closed, or actor is cutFunction that destroys the actor when deleted, scene is closed, or actor is cut- static void DisposeActor()- static void DisposeActor()

Sets the activate flag true or false depending if the actors scene is activatedSets the activate flag true or false depending if the actors scene is activated- static void ActivateActor()- static void ActivateActor()

General Flow of the General Flow of the FunctionsFunctions

Isadora Actor MockupIsadora Actor Mockup

Our Actor Mockup

Actor Link Connection

Putting the Pieces Putting the Pieces TogetherTogether Integration between hardware and Integration between hardware and

software will be completed using the software will be completed using the research and documentation available.research and documentation available.

Testing will be done with trial Testing will be done with trial components on lend from the components on lend from the Department of Electrical Engineering Department of Electrical Engineering to assure products to be purchased to assure products to be purchased will function correctly under all will function correctly under all requirements.requirements.

Expected Product Expected Product DeliverablesDeliverables An alpha version delivered An alpha version delivered

February for use in March.February for use in March. Consist of a transmitter/receiver Consist of a transmitter/receiver

set for one dancerset for one dancer By the first week of May, a By the first week of May, a

finalized version of software and finalized version of software and hardware will be releasedhardware will be released

Coming Up NextComing Up Next

Writing SDK for actors specified Writing SDK for actors specified by our clientby our client

Find testing software for Find testing software for borrowed hardwareborrowed hardware

Implementing hardware design Implementing hardware design for final product demonstrationfor final product demonstration

Evaluate using test casesEvaluate using test cases

Any Questions?Any Questions?