Information Visualization:

Application to Musical Alphabets

By Jonathan Wesel

Information Visualization

• “Unnatural” graphical representations of abstract data– Musical staff example

– vs non-visual alternative?

Musical Representation

• Standard modern musical notation is credited to Pythagoras (Hall, 1928)

• Representation by human processing (Deutsch/Feroe, 1981)

Enhanced Visualization

• Gestalt Laws of Visual Perception– Similarity– Proiximity– Continuation

• Color– Color coding– Color and sound

• Visual cues– Music and motion.

Gestalt Laws

• Similarity : group similar elements

Gestalt Laws

• Similarity : group similar elements

Gestalt Laws

• Similarity : group similar elements

problem…

Gestalt Laws

• Similarity : group similar elements

problem…

…solution

Gestalt Laws

• Proximity: group close elements

Gestalt Laws

• Proximity: group close elements

Two groups…

Gestalt Laws

• Proximity and Similarity

– comparison to similarity solution:• Same result• Proximity was used to solve similarity problem

Gestalt Laws

• Continuity: group elements to form smooth curves or lines

– Meaningful color:• Pitch ascent = rising colors• Pitch descent = falling colors

Gestalt Laws

• Similarity, Proximity, & Continuity

– Use/Effect requires experimentation

Color

• Pythagoras:– “Through vibration comes motion; through motion comes color; through color comes tone.

• Newton: – Split spectrum of light into 5 colors, then

added 2 to “complete the octave”

Color

• The color of music (Schloss, 2008)– Participants chose colors that were most

and least consistent with classical music they were presented with

– A strong agreement was found between the colors chosen to be most consistent with the same musical selections.

Color

• Color in Music-Reading (Rogers, 1996)– Elementary students were taught rhythm

by using notations in both color and black and white.

– Hand clapping was used to demonstrate the students interpretation.

– Students who used the colored notation scored higher in accuracy as well as noting greater enjoyment from the lessons.

Music and Motion

• Study, Barnes-Burroughs, 2004– Students of music would follow the contour

of a melody with their head and neck producing a negative result in the sound.

– Can this be fixed by an alternative representation?

Music and Motion

•Suggestion–Mapped notes to colors

–New representation

Music and Motion

• This visualization seems to solve the problem, but creates many more problems. (octaves ect…)

• This visualization is unlikely to be effective because it is too far from what is well known.

Conclusion

• Further research and experimentation must be done to determine effectiveness of…– Using Gestalt Laws– Using Color– Removing Contour

SourcesAllchin, Douglas. "Newton's Colors." SHiPS Resource Center. 10 Oct. 2008

<http://www1.umn.edu/ships/updates/newton1.htm>.Barnes-Burroughs, Kathryn, Christopher Watts, Oren L. Brown, and Jeannette LoVetri.

"The Visual/Kinesthetic Effects of Melodic Contour in Musical Notation as it Affects Vocal Timbre in Singers of Classical and Music Theater Repertoire." Journal of Voice 19 (2004): 441-19.

"Color Music--Found Images and the Second Perfect Number." Weblog post. Ptak Science Books. 18 May 2008. 10 Nov. 2008 <color music--found images and the second perfect number.>.

Deutch, Diana, John Feroe, "The Internal Representation of Pitch Sequences in Tonal Music.” Psychological Review 88 (1981): 504-522.

Doleisch, Helmut. Visual Interactive Analysis Research Group. n.d. 21 Sept. 2008 http://www.vrvis.at/via/.

Ellis, Willis D. A Source Book of Gestalt Psychology. New York: Harcourt, Brace & Co, 1938.

Hall, Manly P. "The Pythagorean Theory of Music and Color." The Secret Teachings of All Ages. 1928. 81-84.

Sources

Havre, Susan L, Anuj Shah, Christian Posse, et al. Diverse Information Integration and

Visualization. n.d. 21 Sept. 2008 http://infoviz.pnl.gov/pdf/SPIE_proceedings_6060_22.pdf.

Keller, Tanja, Peter Gerjets, Katharina Scheiter, and Barbel Garsoffky, “Information visualizations for knowledge acquisition: The impact of dimensionality and color coding.” Computers in Human Behavior 22 (2006): 43-65

Lengler, Ralph and Martin J. Eppler. A Periodic Table of Visualization Methods. n.d. 21 Sept. 2008 http://www.visual-literacy.org/periodic_table/periodic_table.html.

Li, Xuelong Li, Dacheng Tao, Stephen J. Maybank, and Yuan Yuan, “Visual music and musical vision” Neurocomputing 71 (2008): 2023-2028

Rogers, George L. "Effect of Colored Rhythmic Notation on Music-Reading Skills of Elementary Students." Journal of Research in Music Education 44 (1996): 15-25

Sahling, Gerald Nikolaus. Interactive 3D Scatterplots - From High Dimensional Data to Insight. n.d. 21 Sept. 2008

Schloss, Karen B., Partrick Lawler, and Stephen E. Palmer. "The color of music." Journal of Vision 8 (2008): 580.