big data in the music industries, dagfinn bach, bach technology
DESCRIPTION
VERDIKT conferenceTRANSCRIPT
the evolution of music continues
Big Data in the Music Industries -‐ MusicDNA From B2B data capturing to sophis3cated B2B2C Services The Norwegian Council of Research, October 16th, 2013
Dagfinn Bach
History
• Before 2007 (founders background): • Online MP3 scenarios test-‐cases (1991-‐1994) (before the commercial WWW)
• First European Music Online Service (1995-‐1997) (6 countries)
• ConsulLng Nokia Ventures (1998-‐1999) (feasibility study on music on mobile)
• Music aggregator Artspages (1999-‐2007) (Today Phonofile)
• From 2007:
• Founding Bach Technology AS in Bergen and Bach Technology GmbH in Ilmenau in the building of Fraunhofer Ins3tute (2007)
• R&D and product development search/recommenda3on/metadata (2007-‐2010)
• R&D and product development audio recogni3on and enhanced players/plug-‐ins for OEM products; smartphone, tablets etc.. (2010-‐2012)
• Consolida3ng into two business areas: Airplay monitoring (Radio/TV) and MetaData/BigData powered products for OEM and Automo3ve industries (in-‐car audio)
2
VERDIKT Project
• “The Future of P2P” (Sustainable and green solu3ons for online media in enhanced networks).
• Key R&D elements:
• Op3mized large scale audio recogni3on
• Audio analysis and tagging • Legal P2P solu3ons with automa3c metadata updates
• Budget: 16 MNOK (4,5 MNOK from NFR)
• Partners: • Bach Technology AS (Bergen, Norway) • University of Bergen, Department of Informa3cs (Bergen, Norway)
• Fraunhofer Ins3tute for Digital Media Technology (Ilmenau, Germany)
• Other contributors: • Hewleb Packard Norge AS (HW and business models) SERIT/Fjordane IT (Data Centre)
• MediArena, Bergen (match-‐making for poten3al product partners)
3
MusicDNA today
MusicDNA offers today a method for:
• Capturing audio • Analysing and producing metadata (MusicDNA descriptors)
• fingerprin3ng and capturing more data
• structuring • storing
”Big Data” about music
for creaIng:
1. Stand-‐alone B2B services 2. U3lizing the database to power services targeted for end consumers to enhance the
user experience within search, sharing, transfer and visualiza3on.
4
The MusicDNA Database
Three components:
1. A powerful database containing 2 fingerprints and 15 MPEG-‐7 descriptors of each segment within each sound tracks within a collec3on of 18 Million tracks is one of the most extensive opera3ve meta databases for music in the market.
2. 20.000 Radio Channels indexed in an addi3onal radio-‐monitoring database, currently running a real-‐3me monitoring of 4.500 the radio channels across Europe, and another 1.500 channels across Canada, Japan, Australia.
3. Recognizing audio of airplay every 10 seconds (fingerprin3ng of en3re track) and matches and display rights data and other associated data, and creates a new database showing the history of airplays across the world.
All databases are growing incrementally
5
MusicDNA Data
• Genre, subgenre • Tempo-‐/Beat determina3on
• Aggressiveness • Mood
• Hardness • Speech-‐/music discrimina3on
• Music color
• Segmenta3on
• Solo Instrument
• Instrument Density
• Percussiveness
6
• Vocal detec3on • Key • Synthe3city • Rhythm pabern*
• Vocal Detec3on; singer type (male, female, child, choir)*
• Vocal style (singing, rap, opera, screaming etc...)*
• Cover Song detec3on* • + 4-‐6 new descriptors every year enabling incrementally more advanced recommenda3on and recogni3on;
• ID3 Data • Soundslike Fingerprint
* to be launched in 2014
Radio Airplay Data
• For each airplay recogni3on: • Track Title • Ar3st Name
• ISRC (similar as ISBN)
• Channel Name
• Country of Channel • AirPlay (dd.mm.yyyy), 3me and dura3on (from hh.mm.ss to hh.mm.ss)
• City of channel loca3on (including GPS data) • More fields to be added by means of MusicDNA tracking/matching
7
Big Data Basis
• 18 million tracks
• 15 tags • Average 5 segments per song
• 1,35 Billion “data points” for describing/classifying music
• Can be matched and combined with 6 to 7 data fields for Radiomonitoring
• Over 6.000 channels -‐> soon increasing to 20.000 channels across the world
• Can be further matched and combined with data from affiliated par3es
8
Feasible products and use cases
OEM/Automo3ve plug-‐ins:
• Radio-‐monitoring for iden3fica3on of broadcast airplays
• Radio channel profiler (MusicDNA Radio profile) for smart radio-‐tuner apps
• Linking on-‐demand music to radio channels with similar profile
• “From radio music to on-‐demand music” recommenda3on
• “From radio to radio” recommenda3on
Other:
• Radio-‐plugging tools for pre-‐selected releases
9
Feasible products and use cases
Charts not uIlising MusicDNA aSributes:
• Inter-‐/na3onal/regional (city) airplay charts: I.E.: Top 10, 20, 50, 100 songs on weekly, monthly basis on World, Europe, Country, City level.
Premium:
• The next genera3on of charts: real 3me charts: Top 10, 20, 50, 100 songs constantly on World, Europe, Country, City level.
• specific genre charts : Combining the previous one with MusicDNA Abributes
• daily charts: last day (24 hours) • daily chart tendency last month, year: Visualised by graph
• daily airplay (24) tendency for one ar3st during one month: I.E: visualizing by map (one per day put together as an anima3on of 30 days)
• day3me/ nigh-‐3me charts, preby interes3ng since radios have a format where they only play interes3ng/indie music in the evening or at night. Otherwise similar as 4.
• independent charts: Indie music
• newcomer charts: Charts for new releases (i.e. last week, last month)
ConfidenIal info for GVL 10
Examples on charts
• most played in big city charts: combining with popula3on numbers to sort out big ci3es only
• style/genre tendency in different countries and during the year: display difference between music profile (based on MusicDNA) in different countries, and month by month
• trend charts: showing trends with respect to geographical spread and volumes for one ar3sts, one genre, etc.. in one defined defined territory
Extended with genre/style detecIon uIlizing the MusicDNA ASributes:
• up-‐tempo charts
• ballad charts: • instrumental charts
• vocal charts Even possible to make further extension with the following MusicDNA aSributes:
• dark/bright • hard/som • full/sparse (size of the ensemble)
Significant potenIal for VizualizaIon
•
ConfidenIal info for GVL 11
Demos
• MusicDNA Radio monitor:
• Ar3st Centric (see abached screen-‐dump)
• Chart (see abached screen-‐dump)
• Vizualisa3on ideas: • Ylvis Map (from screen shots)
• Vizrt vizualisa3on sketches: • Ylvis • David Gueba • Emmelie de Forrest (Eurovision) 1
• Emmelie de Forrest (Eurovision) 2
12
Social media scenarios (in progress)
A: IntegraIng Apply Magic Sauce with the MusicDNA mobile player
• Descrip3on: • Giving users the op3on to connect with facebook, get their personality score instantly, see which performing ar3sts in their library have a similar psychological profile, see links to discover music or purchase 3ckets for other ar3sts which they may not know about but which also share their profile. Users could also opt in to submit their data anonymously for academic research.
• Usage of data: • Process the personal informa3on and aggregate it before sending the analy3cs on the personality, IQ, life sa3sfac3on, etc. of the users who connected and use the player. We can then use these insights in any way you find useful informa3on, whether to understand the users beber, i.e. for UI personalisa3on, or presen3ng this informa3on to clients
13
B: AddiIonal dimensions in the music profiler and recommender
• Descrip3on: • A huge poten3al to add an addi3onal personality level to the exis3ng profiler and recommenda3on plaporm. We could analyse the profiles and listening stats of different radio channels and online plaporms such as Mixcloud, Soundcloud and Last.fm to target recommenda3ons more accurately. Combined with MusicDNA this informa3on could be presented not only as channels or songs that the user would like, but as a MusicDNA+personality profile of a user's en3re collec3on, which they have the op3on to rec3fy and thus tell you even more about the kind of music they want to listen to.
• Usage of data: • It would then be very easy to use this informa3on to suggest concert 3ckets, merchandise and other products to the user as we would have a far more detailed understanding of what they are likely to purchase or which gig they are likely to abend
14
Social media scenarios (in progress)
ArIst Centric Radio Monitor -‐ Screen-‐dump
16
Chart Radio Monitor -‐ Screen-‐dump
17