![Page 1: Sonification - unipi.it · PDF fileUniversity and Bad Sassendorf knee and ankle ... “The sonification handbook” - Hermann, Hunt, Neuhoff For a more neurology oriented approach](https://reader036.vdocuments.mx/reader036/viewer/2022062413/5a7536e67f8b9a63638c5282/html5/thumbnails/1.jpg)
Sonification
Ambient IntelligenceCourse 2012/2013
13/05/2013
![Page 2: Sonification - unipi.it · PDF fileUniversity and Bad Sassendorf knee and ankle ... “The sonification handbook” - Hermann, Hunt, Neuhoff For a more neurology oriented approach](https://reader036.vdocuments.mx/reader036/viewer/2022062413/5a7536e67f8b9a63638c5282/html5/thumbnails/2.jpg)
Un
iversità
di P
isaD
otto
rato
in In
g. d
ell'In
form
azio
ne
2
IntroductionTo introduce sonification with a “crash”,I will refer directly to the slides kindly provided to me for this course by Dr. Thomas Hermann
![Page 3: Sonification - unipi.it · PDF fileUniversity and Bad Sassendorf knee and ankle ... “The sonification handbook” - Hermann, Hunt, Neuhoff For a more neurology oriented approach](https://reader036.vdocuments.mx/reader036/viewer/2022062413/5a7536e67f8b9a63638c5282/html5/thumbnails/3.jpg)
Un
iversità
di P
isaD
otto
rato
in In
g. d
ell'In
form
azio
ne
3
A sport for visually impaired
Goalball
Show video
![Page 4: Sonification - unipi.it · PDF fileUniversity and Bad Sassendorf knee and ankle ... “The sonification handbook” - Hermann, Hunt, Neuhoff For a more neurology oriented approach](https://reader036.vdocuments.mx/reader036/viewer/2022062413/5a7536e67f8b9a63638c5282/html5/thumbnails/4.jpg)
Un
iversità
di P
isaD
otto
rato
in In
g. d
ell'In
form
azio
ne
4
Sounds for HCIWe will concentrate on a particular application field for sonification. In particular it will be regarding Human Computer Interaction, mainly in the sport field.
Moreover one of the main usages we make of sounds is to convey real-time or quasi-realtime feedbacks.
Sounds act differently on the brain …
![Page 5: Sonification - unipi.it · PDF fileUniversity and Bad Sassendorf knee and ankle ... “The sonification handbook” - Hermann, Hunt, Neuhoff For a more neurology oriented approach](https://reader036.vdocuments.mx/reader036/viewer/2022062413/5a7536e67f8b9a63638c5282/html5/thumbnails/5.jpg)
Un
iversità
di P
isaD
otto
rato
in In
g. d
ell'In
form
azio
ne
5
Sensors and soundsNow I present a few applications of sensing systems, designed or modified to enable the construction of a sonification of sensed data
- SoniFeet- PdFun- SofiRow and PERSEO (sample video and audio)- eMGeeA (sound events)- SPINE-Sonifier (SPINE on Shimmer + libPD on Android)- SeeColor Mobile (with Guido Bologna, Geneve)- Swimming (with Bodo Ungerechts and Thomas Hermann)
![Page 6: Sonification - unipi.it · PDF fileUniversity and Bad Sassendorf knee and ankle ... “The sonification handbook” - Hermann, Hunt, Neuhoff For a more neurology oriented approach](https://reader036.vdocuments.mx/reader036/viewer/2022062413/5a7536e67f8b9a63638c5282/html5/thumbnails/6.jpg)
Un
iversità
di P
isaD
otto
rato
in In
g. d
ell'In
form
azio
ne
6
SoniFeet 1/3Sonification of gait characteristics acquired through a sensorized (pressure) threadmill
Gait is a cyclic 2-phase act:- contact- oscillation
![Page 7: Sonification - unipi.it · PDF fileUniversity and Bad Sassendorf knee and ankle ... “The sonification handbook” - Hermann, Hunt, Neuhoff For a more neurology oriented approach](https://reader036.vdocuments.mx/reader036/viewer/2022062413/5a7536e67f8b9a63638c5282/html5/thumbnails/7.jpg)
Un
iversità
di P
isaD
otto
rato
in In
g. d
ell'In
form
azio
ne
7
SoniFeet 2/3The used threadmill is the Zebris GmbH FDM-T
Original GUI from Zebris
![Page 8: Sonification - unipi.it · PDF fileUniversity and Bad Sassendorf knee and ankle ... “The sonification handbook” - Hermann, Hunt, Neuhoff For a more neurology oriented approach](https://reader036.vdocuments.mx/reader036/viewer/2022062413/5a7536e67f8b9a63638c5282/html5/thumbnails/8.jpg)
Un
iversità
di P
isaD
otto
rato
in In
g. d
ell'In
form
azio
ne
8
SoniFeet 3/3SoniFeet is now a Client – Server application:
– server • C++, using proprietary Zebris Drivers to
access Hardware, Windows• Preprocessing (extraction of main
parameters using for the sonification)– client
• PureData, on Windows, Linux or Mac• Sonification of parameters (dual parameter
mapping):– Position of CoM → Panning – Total load → Pitch
Tests of SoniFeet being carried out in Hamburg University and Bad Sassendorf knee and ankle surgery klinik.
![Page 9: Sonification - unipi.it · PDF fileUniversity and Bad Sassendorf knee and ankle ... “The sonification handbook” - Hermann, Hunt, Neuhoff For a more neurology oriented approach](https://reader036.vdocuments.mx/reader036/viewer/2022062413/5a7536e67f8b9a63638c5282/html5/thumbnails/9.jpg)
Un
iversità
di P
isaD
otto
rato
in In
g. d
ell'In
form
azio
ne
9
PureData Intro
![Page 10: Sonification - unipi.it · PDF fileUniversity and Bad Sassendorf knee and ankle ... “The sonification handbook” - Hermann, Hunt, Neuhoff For a more neurology oriented approach](https://reader036.vdocuments.mx/reader036/viewer/2022062413/5a7536e67f8b9a63638c5282/html5/thumbnails/10.jpg)
Un
iversità
di P
isaD
otto
rato
in In
g. d
ell'In
form
azio
ne
10
PureData on AndroidPdFun is an application developed to explore the possibilities offered by the libPD PureData multiplatform library, executing on an Android phone.
Idea: discover new ways of using and perceiving the signals coming from the sensors embedded in smartphones:- accelerometer: Maracas, Scooter, Guess- magnetometer: Magnetic Field- tilt: Theremin- proximity: Theremin
Live Demo
![Page 11: Sonification - unipi.it · PDF fileUniversity and Bad Sassendorf knee and ankle ... “The sonification handbook” - Hermann, Hunt, Neuhoff For a more neurology oriented approach](https://reader036.vdocuments.mx/reader036/viewer/2022062413/5a7536e67f8b9a63638c5282/html5/thumbnails/11.jpg)
Un
iversità
di P
isaD
otto
rato
in In
g. d
ell'In
form
azio
ne
11
RowingSome basics of rowing ...
![Page 12: Sonification - unipi.it · PDF fileUniversity and Bad Sassendorf knee and ankle ... “The sonification handbook” - Hermann, Hunt, Neuhoff For a more neurology oriented approach](https://reader036.vdocuments.mx/reader036/viewer/2022062413/5a7536e67f8b9a63638c5282/html5/thumbnails/12.jpg)
Un
iversità
di P
isaD
otto
rato
in In
g. d
ell'In
form
azio
ne
12
Where to act on humans?
![Page 13: Sonification - unipi.it · PDF fileUniversity and Bad Sassendorf knee and ankle ... “The sonification handbook” - Hermann, Hunt, Neuhoff For a more neurology oriented approach](https://reader036.vdocuments.mx/reader036/viewer/2022062413/5a7536e67f8b9a63638c5282/html5/thumbnails/13.jpg)
Un
iversità
di P
isaD
otto
rato
in In
g. d
ell'In
form
azio
ne
13
SwimmingSome basics of swimming ...
![Page 14: Sonification - unipi.it · PDF fileUniversity and Bad Sassendorf knee and ankle ... “The sonification handbook” - Hermann, Hunt, Neuhoff For a more neurology oriented approach](https://reader036.vdocuments.mx/reader036/viewer/2022062413/5a7536e67f8b9a63638c5282/html5/thumbnails/14.jpg)
Un
iversità
di P
isaD
otto
rato
in In
g. d
ell'In
form
azio
ne
14
Web synth
http://matt.west.co.tt/music/jasmid-midi-synthesis-with-javascript-and-html5-audio/
http://cssdeck.com/labs/2j1pofsj/
https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html
http://codebase.es/riffwave/
Various software and libreries are emerging in the web:HTML5 exports an API for sound manipulation
![Page 15: Sonification - unipi.it · PDF fileUniversity and Bad Sassendorf knee and ankle ... “The sonification handbook” - Hermann, Hunt, Neuhoff For a more neurology oriented approach](https://reader036.vdocuments.mx/reader036/viewer/2022062413/5a7536e67f8b9a63638c5282/html5/thumbnails/15.jpg)
Un
iversità
di P
isaD
otto
rato
in In
g. d
ell'In
form
azio
ne
15
ReferencesFor a complete reference and introduction to sonification see:“The sonification handbook” - Hermann, Hunt, Neuhoff
For a more neurology oriented approach to music and sounds see:“Musicophilia: Tales of Music and the Brain” by Oliver Sacks
Papers on sports and movement oriented sonification:“Sonification of Pressure Changes in Swimming for Analysis and Optimization” - Hermann, Ungerechts, Toussaint, Grote“Examining effects of acoustic feedback on perception and modification of movement patterns in on-water rowing training” - Schaffert, Mattes, Effenberg
![Page 16: Sonification - unipi.it · PDF fileUniversity and Bad Sassendorf knee and ankle ... “The sonification handbook” - Hermann, Hunt, Neuhoff For a more neurology oriented approach](https://reader036.vdocuments.mx/reader036/viewer/2022062413/5a7536e67f8b9a63638c5282/html5/thumbnails/16.jpg)
Un
iversità
di P
isaD
otto
rato
in In
g. d
ell'In
form
azio
ne
16
Experimenting with PD and SC
Best way to learn:Experiment …
Next lesson we will use PureData (and maybe) SuperCollider ...