A Survey of Tools for Sonic and Music Visualization

Janet G. Lazar

Data Conveyed by Music

• Pitch• Dynamics• Key• Tempo• Intellectual content• Emotional content

Metadata About Music

• Style• Context• Description of work• Details about recording• Details about collection

Clusters of Visualization Tools

• Pitch rendering• Music transcription• Music analysis• Music illustration• Music discovery

Pitch Rendering Tools

• AP Tuner (pitch matching for tuning)• Tarsos (pitch rendering for analysis)• Theta Music Trainer (vocal pitch matching)

AP Tuner for Windows

• Frequency meter simulation

• Harmonics graph

AP Tuner (more)

• Meter simulation: position, orientation, motion

• Harmonics graph: shape, color, continuity• Both: Good data-ink ratio, direct

manipulation, immediate feedback, linked displays

• Overall: Above-average visualization

Music Transcription Tools

• Finale (commercial)• Finale Notepad (free)• Sibelius (commerical)• Noteflight (free, hosted online)• Noteflight Crescendo (commercial, online)

Noteflight

Noteflight (more)

• Music notation: position, orientation, shape, shading, stereo, proximity, similarity, continuity, connectedness, closure, containment; data-ink ratio maximized.

• Noteflight: motion, direct manipulation, immediate feedback, animate shift of focus, dynamic sliders

• Overall: excellent visualization

Music Analysis Tools

• Hooktheory Analyzer (composing/learning)• Audacity (free music editor)• Melodyne (commercial music editor)• CLAM Chordata (chord structure analysis)

CLAM Chordata

CLAM Chordata (more)

• Uses every tool in the box except zoom, focus, and details on demand

• High data density, perhaps too high• Overall: very good tool

Music Illustration Tools

• Music Animation Machine (clear, stable)• WhiteCap (customizable)• MAGIC (new, customizable)• Infinite Jukebox (beat relationship maps)• Baroque.me (harp-string emulation)

Baroque.me

Baroque.me (more)

• Uses position, size, shape, motion, and continuity

• Data-ink ratio high, data density low• Simplicity is appealing• Overall: good visualization

Music Discovery Tools

• Affecticon (proposed sorting by emotion)• LivePlasma (node-link map for Amazon data)• TuneGlue (node-link map for last.fm data)• MusicBox (prototype scatterplot)• Hitlantis (interactive bubble map)• Tamas’s map (last.fm network representation)

Tamas’s network map

Tamas (more)

• Uses position, size, color, depth cues, proximity, and connectedness

• Provides direct manipulation, immediate feedback, animated focus shift, zoom, focus plus context, and details on demand

• Data density much too high to be useful• Nevertheless, a good beginning

Conclusion

• Music visualization: a lively, growing field with a promising future

Pitch Rendering Tools

• AP Tuner for Windows (Version 3.08) [Software]. Available from http://www.aptuner.com

• Tarsos (2012) (Version 1.0) [Software]. Available from http://tarsos.0110.be

• Six, J. and Cornelis, O. (2011) Tarsos - A platform to explore pitch scales in non-western and western music. 12th International Society for Music Information Retrieval Conference. Retrieved from http://ismir2011.ismir.net/papers/OS2-4.pdf

• Vocal Match [Online application]. Theta Music Trainer. Retrieved from http://trainer.thetamusic.com/en/content/vocal-match

Music Transcription Tools

• Finale (2012) [Software]. Available from http://www.finalemusic.com

• Sibelius 7 [Software]. Available from http://www.sibelius.com

• Noteflight and Noteflight Crescendo [Online applications]. Retrieved from http://www.noteflight.com

Music Analysis Tools

• CLAM Chordata 1.0 (2010) [Software]. Available from http://clam-project.org

• Hooktheory Editor [Online application]. Retrieved from http://www.hooktheory.com

• Audacity (2012) (Version 2.0.2) [Software]. Available from http://audacity.sourceforge.net

• Melodyne (Version 2) [Software]. Available from http://www.celemony.com

Music Illustration Tools• Music Animation Machine Music Player (2006) [Software].

Available from http://www.musanim.com/player• WhiteCap (2011) [Software]. Available from

http://www.soundspectrum.com/whitecap• MAGIC: Multitrack Audio & Graphics Interactive Composer (2012)

[Software] Available from http://colorandmusic.com/home/• Lamere, P. (2012, November 11). The infinite jukebox [Web log

entry and online application]. Retrieved from http://musicmachinery.com/2012/11/12/the-infinite-jukebox

• Chen, A. (2011, October 26). Baroque.me: Bach Cello Suites No. 1, Prelude [Web log entry and online demonstration]. Retrieved from http://blog.chenalexander.com/2011/baroque-bach-cello

Music Discovery Tools• Yoo, M. and Lee, I. (2011). Affecticon: Emotion-based icons for music

retrieval. IEEE Computer Graphics and Applications, 31(3), 89-95. doi:10.1109/MCG.2011.36

• LivePlasma [Online interactive map]. Retrieved from http://www.liveplasma.com

• TuneGlue [Online interactive map]. Retrieved from http://audiomap.tuneglue.net

• Lillie, A. (2008). MusicBox: mapping and visualizing music collections. Retrieved from http://thesis.flyingpudding.com

• Hitlantis [Online interactive map]. Retrieved from http://hitlantis.com• Tamas, N. (2008) [Web page and online interactive map].

Reconstructing the structure of the world-wide music scene with Last.fm. Retrieved from http://sixdegrees.hu/last.fm