1. Xiao Hu, Ph.D.Assistant ProfessorLibrary and Information ScienceUniversity of Denver1/11/2012 1

2. An ExerciseWhat term(s) do you describe the mood of the song? 21/11/2012 3. Agenda Background Music Mood: the Question Taxonomies as Organization System Developing Music Mood Taxonomies Taxonomy from Editorial Labels User Evaluation Automatic Classification Prototype System Taxonomy from Social Tags Comparisons to Psychological Models Automatic Classification User Evaluation1/11/20123 4. Music Moodmain reason behind most peoples engagement with music--Juslin and Sloboda(Lee & Downie,2004)(Lamere, 2008)Lee, J. H. & Downie, J. S. (2004). Survey of music information needs, uses, and seeking behaviours: Preliminary findings. InISMIR. Lamere, P. (2008). Social tagging and music information retrieval. Journal of New Music Research 37, 2, 101114.4 5. Moods in Previous Research Directly borrowed from psychologyThayers stress-energy model gives 4 clusters Farnsworths 10 adjective groups Grounded in music perception research, but lack social context of music listening (Juslin & Laukka, 2004) Tellegen-Watson-Clark model Juslin, P. N. and Laukka, P. (2004). Expression, perception, and induction of musical emotions: a review and a questionnaire study of everyday listening. Journal of New Music Research.5 6. Taxonomy as An Organization SystemFrom Linnaean taxonomy in BiologyDomain oriented controlled vocabulary Contain labels (metadata)Commonly used on websites Pick list; browsable directory, etc. Develop taxonomies to organize music information Methods can be applied to other informationtypes Stewart, D. (2008) Building Enterprise Taxonomies, Mokita Press 6 7. Agenda Background Music Mood: the Question Taxonomies as Organization System Developing Music Mood Taxonomies Taxonomy from Editorial Labels User Evaluation Automatic Classification Prototype System Taxonomy from Social Tags Comparisons to Psychological Models Automatic Classification User Evaluation1/11/20127 8. Taxonomy from Editorial LabelsEditorial labels: Given by professional editors of online repositories Rooted in realistic social contexts Have a certain level of control allmusic.com: the most comprehensive music reference source on the planet 179 mood labels created and assigned to music works1/11/20128 9. Mood Label Clustering 179 labels are too many Need a more concise, higher level view of the mood space Solution: clustering Automatically group similar items Similarity defined: Mood labels assigned to the same pieces of music are similar Data allmusic.com applies mood labels to albums and songs 7134 album-mood pairs and 8288 song-mood pairs Two independent data sources provide more robust andmeaningful clustering resultsHu, X., & Downie, J. S. (2007). Exploring Mood Metadata: Relationships with Genre, Artist and 9Usage Metadata. In Proceedings of ISMIR 10. Clustering Results Mood labels for albums Mood labels for songsC1 C2 C3 C4C5C4C1C3 C2 C51/11/2012 10 11. A Taxonomy of 5 Mood ClustersCluster_1: passionate, rousing, confident, boisterous, rowdyCluster_2: rollicking, cheerful, fun, sweet, amiable/good naturedCluster_3: literate, poignant, wistful, bittersweet, autumnal, broodingCluster_4: humorous, silly, campy, quirky, whimsical, witty, wryCluster_5: aggressive, fiery,tense/anxious, intense, volatile,visceral111/11/2012 12. Verifications of the 5 Clusters Survey users of different groups in labeling a set of songs using the taxonomy Developed an online music recommendation system based on the taxonomy1/11/201212 13. User Evaluation: Experts 1,250 music clips 21 MIR researchers Each clips had three judges % of clips with agreementsMost disagreements areC140.2%C260.2%between:C370.5% C2 (cheerful) and C4 (humorous)C439.6% C1 (passionate) and C2 (cheerful)C570.8% Other16.9%Hu, X., Downie, J. S., Laurier, C., Bay, M., & Ehmann, A. (2008). The 2007 MIREX Audio Mood13Classification Task: Lessons Learned. In ISMIR. 14. User Evaluation: Amazon Mechanic TurkAMT:crowdsourcing 1,250 music clips Each clips had twojudges% of clips with agreements C139.6%Most disagreement are C248.9%between: C369.5%C1 (passionate) and C2 (cheerful) C446.3%C2 (cheerful) and C4 (humorous) C560.0%C1 (passionate) and C5 (angry)Other21.3%Lee, J. H. & Hu, X. (Under Review) Generating Ground Truth for Music Mood Classification 14Using Mechanical Turk 15. Verifications of the 5 Clusters Survey users of different groups in labeling a set of songs using the taxonomy Developed an online music recommendation system based on the taxonomy1/11/201215 16. Prototype SystemMoodydb.comHu, X., et. al (2008). MOODY: A Web-Based Music Mood Classification and16Recommendation System, (Demonstration). ISMIR 17. Summary of the 5 Clusters Taxonomy Grounded in a real-world music repository The first music mood taxonomy undergone verifications by multiple approachesLimitations Other: not sufficiently comprehensive Confusions across clusters: multi-label Editorial labels vs. end user perspectives1/11/201217 18. Agenda Background Music Mood: the Question Taxonomies as Organization System Developing Music Mood Taxonomies Taxonomy from Editorial Labels User Evaluation Automatic Classification Prototype System Taxonomy from Social Tags Comparisons to Psychological Models Automatic Classification User Evaluation1/11/201218 19. Taxonomy from Social TagsSocial tagsThe largest music tagging Pros:site for Western music Users perspectives Large quantity Cons: Non-standardized Linguistic ResourcesHuman Expertise Ambiguous Hu, X. (2010). Music and Mood: Where Theory and Reality Meet. In Proceedings of the 5th iConference, (Best Student Paper).19 20. The Method1,586 terms in WordNet-Affect 202 evaluation terms in General Inquirer(good, great, poor, etc.) 135 non-affect/ ambiguous terms by experts( cold, chill, beat, etc.)= 1,249 terms 476 terms are last.fm tags group the tags by WordNet-Affect and experts => 36 categories201/11/2012 21. Verifications Compared to influential psychological models Developed multimodalclassification systems1/11/201221 22. Russells 2-D Model (1980) 1/11/2012 22 23. Comparison to Russells 2-D Model1/11/2012 23 24. Verifications Compared to influential psychological models Developed multimodalclassification systems1/11/201224 25. Automatic Classification Multimodal classification using audio and lyrics Most comprehensive study MUSIC on lyric mood classification Significantly outperformed top-ranked single-source systemsAudioLyrics Binary classification Each mood has its own classification model Allow one song to have multiple mood labels A data set of 18 categoriesHu, X., & Downie, J. S. (2010). Improving Mood Classification in Music Digital Libraries by25Combining Lyrics and Audio. In JCDL (Best Student Paper). 26. 0.750.60.70.8 0.550.5 0.85 0.45 0.65 calm sadgladromantic gleefulgloomyangryAngry mournful dreamy1/11/2012 cheerful broodingaggressiveAggressive anxious confident Accuracy across Categories hopefulearnest cynical exciting26 27. Summary of the 2-D Taxonomy Grounded in social tags (end users perspectives) Leverage linguistic resources and experts Complement psychological models Some categories are easier to predict than others Next step: Cultural dependency Focus groups and survey of people from different cultures Proposal: Developing A Music Mood Taxonomy: TowardsUnderstanding Emotion and Culture in the Fast ChangingInformation Environment1/11/2012 27 28. Question time!1/11/201228