Electronic Conducting System

By: Kenzo Abrahams

Supervisor: Mehrdad GhaziasgarCo-supervisor: James ConnanMentored by: Diego Mushfieldt

OverviewIntroductionDesign Decisions and System ChangesImplementationTools UsedProject PlanDemo

IntroductionInteractive Conducting SystemTracks hands using webcamReal-time alterations depending on hand gestures

Change volumeChange tempo

Design Decisions and System changesUse Haar classifiers to detect the faceDynamically check for skin colourTrack left hand only when needed

ImplementationDetect face and block it out

ImplementationConvert frame into HSV colour space

Implementationset hand as region of interest

ImplementationHSV image split up into separate components

ImplementationCreate histogram of skin colour

ImplementationTrack the right hand (CAMshift)

ImplementationPlay music

ImplementationChoose ensemble or entire orchestra

ImplementationTrack left hand

Tools UsedUbuntu 12.04Qt Creator with Qt 4.7OpenCV 2.4Rtcmix 4.0.1

DemoDemo Overview

Track right handChange the tempo of musicTrack left handDecrease volume of symphonyIncrease volume of one ensemble

Project PlanGoal Due Date

Learn how to use OpenCV and its tools.Elicit the requirements and define a designers interpretation of the problem. CompletedFrom the users requirements design a prototype for the system CompletedConstruct the system with all its functionality present CompletedTest the system and deployment Term 4

References O'Niel, D L. (2008). Music Theater Jobs. Available:

http://www.ehow.com/list_6547349_music-theater-jobs.html. Last accessed 29th March 2012.

Bradski, G Kaehler, A. (2008). Getting to Know OpenCV. In: Loukides, M Learning OpenCV. United States of America: O’Reilly. p31-87.

Gibson, J. (2009). RTcmix Tutorial. Available: http://iub.edu/~emusic/509/rtcmixtut.htm. Last accessed 10th Sep 2012.

Nakra, T., Ivanov, Y., Smaragdis, P., Ault, C.: The USB Virtual Maestro: an Interactive Conducting System, p.250-255, NIME2009 (2009).

BORCHERS, J., HADJAKOS, A., AND M¨UHLH¨AUSER, M. MICON (2006) A music stand for interactive conducting. Proceedings of the 2006 conference on New interfaces for musical expression, p254–259.

Viola, P. Jones, M. (2001). Rapid Object Detection using a Boosted Cascade of Simple Features. Conference on computer vision and pattern matching 2001.

Lansky, A . (2005). An Open-Source, Digital Signal Processing and Sound Synthesis Language. Available: http://rtcmix.org/. Last accessed 27th May 2012.

Nokia. (2009). Qt. Available: http://qt.nokia.com/products. Last accessed 10th Sep 2012.

Question and Answers