from power chords to the power of models

From Power Chords

to the Power of

Models

@aliostadAli Kheyrollahi

8

Local pop music ?

9

Local pop music “Cheelee pom!”

10

Boney M “Rasputin”

11

Blondie “Heart of Glass”

Short Wave Radio (SW2)

Data Acquisition

Data Source - Wiki

4,990,2794,990,279 English Articles

37,583,879 Articles

Data Source - What about …

2012

http://money.cnn.com/2012/03/13/technology/encyclopedia-britannica-books/

Data Source - Wiki vs BritannicaFeng Zhu (assistant prof at Harvard):

There has been lots of research on the accuracy of Wikipedia, and the results are mixed—some studies show it is just as good as the experts, others show [that] Wikipedia is not accurate at all.

… the editors [of Britannica] are still not found to be more objective than the crowd in articles that are sufficiently revised.

Data Source - Wikipedia in scholar papers

0

45000

90000

135000

180000

2005 2006 2007 2008 2009 2010 2011 2012 2013 2014Source: Google Scholar

Data Source - Dark side of Wiki The Doggs: A mid-1960s British rock band, noted for barking while playing music

Lord Byron kept a crocodile and a honey badger as a pet

Ted and the Treble Tones, a rock band from Fresno with a 1959 hit single that appeared nowhere on the Billboard charts

The Shaggs all female (sister) band Hacked-at drumbeats, whacked-around chords, songs that seem to have little or no meter to them… Zappa’s favourite…

Data Source - Content vs. Data

… or … Alt-country

Alternative country

… or … Alt.country

… or … Alt. Country [mind the gap]


comma

slash

new line

period?no, greek middle dot U+00B7


Hyphen U+002D

figure dash U+2012

minus sign U+2015

em dash U+2014

en dash U+2013

Data Source - DBpedia• http://wiki.dbpedia.org/

• bziped N-Triples or N-Quads RDF entries

• Different datasets:

• Titles

• Categories

• Infobox data

• Short Abstracts

• In the end was not used

http://wiki.dbpedia.org/

Data Acquisition - Wiki

List of Rock Genres Rock Genres Rock Artists

Store

Store HTML

Capture Links

Store HTML

Python scripts

Postgres

Data Exploration

Data Exploration

“I personally … literally just look at the screen, just like the matrix”

Claudia Perlich, multi-award winner Data Scientist

Data Exploration

“… the dirty little secret that I have won all of them because I have found something wrong with the data… I would like to play around with dataset and get initimately familiar with dataset and its properties.“

Claudia Perlich

Album Genre

50 Years of Rock (1965-1989)

50 Years of Rock 1965

1. rock (23) 2. folk rock (15) 3. rock and roll (13) 4. folk (12) 5. rhythm and blues (12) 6. blues rock (10) 7. country (9) The Who

My Generation

“Rock” as genre was king with The likes of The Who and Beatles

Rock and Roll, popular then, soon to disappear from the list


1. folk rock (25) 2. garage rock (21) 3. psychedelic rock (20) 4. rock (19) 5. folk (14) 6. protopunk (11) 7. pop (11) Bob Dylan

Blonde on Blonde

Folk Rock was popular all of the sixties with a resurgence in the early 70s

This is obviously named retrospectively but along with Garage rock to influence punk


1. psychedelic rock (56) ↑ 2. folk rock (30) 3. pop (19) 4. folk (19) 5. rock (18) 6. garage rock (12) 7. psychedelic pop (12) The Beatles

Sgt. Pepper's Lonely Hearts Club Band

67, a big year with Monterey Pop Festival, was all about psychedelia

Even for psychedelic pop


1. psychedelic rock (70) ← 2. rock (29) 3. folk rock (26) 4. blues rock (25) 5. folk (18) 6. acid rock (18) 7. progressive rock (15) Jimi Hendrix

Electric Ladyland

No comment, listen to Jimi!

Note Acid Rock and emergence of progressive rock


1. psychedelic rock (60) ← 2. rock (40) 3. folk rock (33) 4. hard rock (27) 5. folk (26) 6. progressive rock (25) 7. country (22)

Led Zeppelin Led ZeppelinNot much changed… progrock

moving up


1. rock (57) 2. folk rock (47) ↑ 3. psychedelic rock (44) !! 4. hard rock (40) ↑ 5. progressive rock (31) ↑ 6. country rock (25) 7. folk (24) The Velvet Underground

LoadedArtists leaving psychedelia for a harder sound


1. rock (47) 2. hard rock (34) ↑ 3. folk rock (33) ! 4. progressive rock (33) ↑ 5. psychedelic rock (26) ! 6. folk (24) 7. country rock (18) CAN

Tago MagoProgressive rock and hard rock moving up


1. rock (54) 2. folk rock (34) 3. progressive rock (29) 4. hard rock (29) 5. country rock (27) 6. folk (25) 7. country (20) Jethro Tull

Thick as a Brick


1. rock (60) 2. hard rock (44) ↑ 3. progressive rock (40) 4. folk rock (29) 5. country (24) 6. glam rock (22) 7. country rock (22)

Flashy dresses and make-up, bring it on!

Pink Floyd Dark Side of the Moon


1. rock (58) 2. hard rock (52) 3. progressive rock (40) 4. glam rock (29) ↑ 5. folk rock (28) 6. blues rock (20) 7. country rock (20) Supertramp

Crime of the Century


1. rock (53) 2. hard rock (52) 3. progressive rock (37) 4. folk rock (23) 5. country (20) 6. blues rock (20) 7. country rock (18) Patti Smith

Horses


1. rock (71) 2. hard rock (56) 3. progressive rock (28) 4. heavy metal (19) 5. pop (19) 6. folk (18) 7. country (18)

Did you notice?! Heavy Metal is now a thing…

AC/DC High Voltage


1. rock (77) 2. hard rock (43) 3. progressive rock (29) 4. punk rock (21) 5. pop (19) 6. folk rock (15) 7. country rock (14)

Oh yeah, punk rock in full swing and pretty much mainstream.

Sex Pistols Never Mind the Bollocks


1. rock (77) 2. hard rock (40) 3. progressive rock (24) 4. new wave (23) 5. pop (21) 6. country (20) 7. punk rock (19)

New wave with a more layered sound replacing punk rock

Talking Heads More Songs About Buildings and Food


1. rock (68) 2. new wave (49) ↑ 3. post-punk (42) 4. hard rock (38) ! 5. country (22) 6. punk rock (20) 7. progressive rock (15) !

Only after two years of punk, we have post-punk… phew

Gary Numan Replicas


1. rock (66) 2. new wave (57) 3. post-punk (49) 4. hard rock (38) 5. country (24) 6. heavy metal (22) 7. punk rock (21) Joy Division

Closer


1. new wave (75) ↑ 2. rock (71) 3. post-punk (68) 4. hard rock (41) 5. synthpop (25) 6. heavy metal (21) 7. country (19)

Another movement is already in motion with softer keyboard-based sound

Klaus Nomi Klaus Nomi


1. new wave (79) 2. post-punk (61) 3. rock (61) 4. hard rock (37) 5. heavy metal (28) ↑ 6. synthpop (28) 7. hardcore punk (21) Iron Maiden

Number of the BeastOther than new wave and post-punk, two completely different sounds growing, heavy metal on one hand and synth pop on the other…


1. rock (74) 2. new wave (71) 3. post-punk (45) ! 4. synthpop (39) ↑ 5. heavy metal (36) 6. hard rock (35) 7. pop rock (24) The Police

Synchronicity


1. new wave (73) 2. rock (61) 3. post-punk (44) 4. heavy metal (37) 5. synthpop (37) 6. hard rock (26) 7. alternative rock (24)

In case you haven’t noticed, New Wave has been on the top spot for a few years

Echo and the Bunnymen Ocean Rain


1. new wave (66) 2. rock (56) 3. post-punk (45) 4. heavy metal (33) 5. pop rock (33) 6. alternative rock (28) ↑ 7. synthpop (28) !

Alternative Rock appearing… soon to label much of the rock music we have now…

The Fall This Nation’s Saving Grace


1. rock (68) 2. new wave (48) ! 3. alternative rock (45) ↑ 4. heavy metal (42) 5. post-punk (40) 6. hard rock (31) 7. synthpop (28)

Synthpop is over… only to come back after 25 years.

The Smiths The Queen Is Dead


1. rock (62) 2. alternative rock (50) ↑ 3. hard rock (36) 4. thrash metal (35) 5. heavy metal (31) 6. new wave (29) ! 7. pop rock (24)

Wow… where did this come from?!

Testament The Legacy


1. alternative rock (59) ↑ 2. rock (58) 3. heavy metal (42) 4. hard rock (40) 5. thrash metal (33) 6. synthpop (24) 7. new wave (23) !

New wave is out of favour after almost 10 years…

Sonic Youth Daydream Nation


1. rock (80) 2. alternative rock (67) 3. hard rock (41) 4. thrash metal (39) 5. heavy metal (34) 6. new wave (26) 7. pop (22) The Cure

Disintegration

Data Models

Data Models Model?!

Data Models Model

Mathematical representation of a concept based on parameters that impact that concept

• Rating of a native app • Stackoverflow score • Credit score • Fraud check

Data Models Model Example

(Average)

Data Models Graph 101

Social Network Analysis and Graph Theory

• Nodes/vertices and edges/lines • Directedness:

• Directed • Undirected

• Degree, InDegree/OutDegree • Weight

A B

Data Models Centrality

API - Related artists

12

4

2

2

1

Same degree Different betweenness

Degree

Graph Codez

import networkx as nx

g = nx.Graph() g.add_edge(‘a’, ‘b’) g.add_edge(‘b’, ‘c’) … print len(g[‘b’]) # degree c = nx.betweenness_centrality(g, normalized=True) # c -> dictionary of node names and their score

DiGraph()

Modelling Influence using Wiki

Data Models Cited Influence

Howlin’ Wolf

Captain Beefheart

1940 1964

Data Models Cited InfluenceMost influential Rock Artists Based on out-degree

The Beatles => 188 Black Sabbath => 127 Led Zeppelin => 118 Jimi Hendrix => 114 Bob Dylan => 94 Pink Floyd => 86 Iron Maiden => 77 Metallica => 77 The Rolling Stones => 66 The Beach Boys => 65 Neil Young => 63 Nirvana => 62 Slayer => 60 Queen => 59

Data Models Cited InfluenceMost influential Rock Artists Based on Betweenness Centrality

Jimi Hendrix => 53476.2014921 The Beatles => 47511.7957531 Bob Dylan => 38107.0298185 Led Zeppelin => 32701.7223273 Nirvana => 29733.9066836 Metallica => 29356.6009213 Queen => 28989.2844223 Robert Smith => 28880.670718 Elvis Presley => 28463.2891497 Slade => 27656.487307 Iron Maiden => 22449.6697023 Ramones => 22437.6112965 Rush => 21125.9481602 Neil Young => 19913.887522

Data Models Cited InfluenceMost influential Artists Based on Betweenness Centrality

Metallica => 566.06 Iron Maiden => 419.21 Corey Taylor => 146.0 Led Zeppelin => 122.73 Slipknot => 116.58 King Diamond => 94.7 Machine Head => 85.12 Rush => 70.41 Black Sabbath => 68.0 Van Halen => 54.56 Deep Purple => 53.5 Megadeth => 42.63 Guns N' Roses => 24.25

Heavy MetalNirvana => 490.08 Muse => 114.5 Weezer => 97.33 Pixies => 94.17 Sonic Youth => 78.5 Rivers Cuomo => 69.5 Siouxsie and the Banshees => 51.67 The Smiths => 51.5 Jeff Buckley => 46.17 The Offspring => 43.0 Placebo => 42.0 My Chemical Romance => 34.0 The Smashing Pumpkins => 32.33

Alternative RockRush => 54.0 Marillion => 34.0 Pink Floyd => 33.0 Yes => 20.0 Porcupine Tree => 19.5 Dream Theater => 19.0 Chris Squire => 16.5 Primus => 15.0 Tool => 12.0 Mahavishnu Orchestra => 8.0 Geddy Lee => 7.0 Neil Peart => 5.0 Keith Emerson => 5.0

Progressive Rock

Shaping a Model

Shaping a Model•Start with Literature survey •Draw parallels with similar domains •If you still cannot find one, define a new model

•Try to refine your model with a quantitative measure if available

Shaping Visionaries ModelVisionary: Influencing vs. being influenced

V: visionary factor deg+: out-degree deg-: in-degree e: the exponential function

Shaping Visionaries ModelJoni Mitchell => 44.0 Roxy Music => 31.0 Leonard Cohen => 31.0 Josh White => 31.0 Robert Johnson => 24.0 Etta James => 23.0 Discharge => 22.0 Sam Cooke => 21.0 Duke Ellington => 20.0 Marc Bolan => 19.0 Hank Marvin => 17.0 Jonathan Richman => 16.0 Scott Walker => 16.0 Testament => 16.0 The Police => 16.0 Skip James => 15.0 Albert King => 15.0 The Cure => 14.7151776469 Nico => 14.0

Data Models Visionaries ModelMost influential Artists in genres

Radiohead => 15.0 Depeche Mode => 12.0 Morrissey => 11.0 Robert Smith => 11.0 The Replacements => 10.0 The Cure => 8.83 XTC => 8.0 Stone Temple Pilots => 6.0 Duran Duran => 6.0 Nick Cave => 6.0 Wipers => 5.0 Johnny Marr => 4.0 Jane's Addiction => 4.0

Alternative RockThe Stooges => 6.0 The Jam => 6.0 Sex Pistols => 4.05 Jawbreaker => 4.0 The Runaways => 4.0 The Dickies => 4.0 Circle Jerks => 3.0 Big Black => 3.0 Patti Smith => 3.0 Elvis Costello => 3.0 The Only Ones => 3.0 Kevin Seconds => 3.0 Descendents => 2.94

Punk RockCaptain Beefheart => 7.0 The Magic Band => 6.0 Genesis => 5.0 Scott Walker => 5.0 Scott Engel => 3.0 Talk Talk => 3.0 Sparks => 2.0 Richard Wright => 2.0 Roxy Music => 1.84 Pink Floyd => 1.49 Mark Hollis => 1.0 Nico => 1.0 Lou Reed => 1.0

Art RockDave Lombardo => 11.0 Napalm Death => 8.0 Morbid Angel => 3.68 Possessed => 3.31 Carcass => 2.21 Immolation => 2.0 Vulcano => 2.0 Hellhammer => 2.0 Autopsy => 1.47 Repulsion => 1.1 Entombed => 1.1 Impetigo => 1.0 Nihilist => 1.0

Death Metal

Shaping Impact ModelImact: Influencing many in a short span

Imp: impact factor deg+: out-degree L: longevity (initial continuous years active)

Shaping Impact ModelImact: Influencing many in a short span

Cream => 3.33 Sex Pistols => 2.19 Joy Division => 2.0 Jimi Hendrix => 1.83 The Beatles => 1.59 The Smiths => 1.19 Big Star => 1.19 Parliament => 1.11 Nirvana => 1.0 Ritchie Valens => 0.78 Led Zeppelin => 0.73 Traffic => 0.67

Shaping Impact ModelImact: Influencing many in a short span in a specific genre

Nirvana => 0.5 The Smiths => 0.5 Pixies => 0.28 Happy Mondays => 0.11 Jeff Buckley => 0.1 Jane's Addiction => 0.08 Soundgarden => 0.08 Violent Femmes => 0.06 The Replacements => 0.06 A Perfect Circle => 0.06 Sublime => 0.05 The Hush Sound => 0.04 The Get Up Kids => 0.03

Alternative RockSex Pistols => 0.69 Fuel => 0.22 Descendents => 0.16 The Runaways => 0.16 The Clash => 0.16 Hot Water Music => 0.13 The Stooges => 0.09 Buzzcocks => 0.08 Misfits => 0.06 Ramones => 0.05 The Jam => 0.05 Dead Kennedys => 0.05 The Replacements => 0.05

Punk RockCream => 1.67 Jimi Hendrix => 0.56 Thin Lizzy => 0.39 Led Zeppelin => 0.33 Deep Purple => 0.27 Ram Jam => 0.25 The Stooges => 0.11 Jim Morrison => 0.09 The Runaways => 0.08 Alice Cooper => 0.08 The Doors => 0.06 The Who => 0.04 Judas Priest => 0.02

Hard RockJoy Division => 0.56 Bauhaus => 0.028 The Police => 0.02 Siouxsie and Banshees => 0.02 Robert Smith => 0.006 Ultravox => 0.005 New Order => 0.005 My Bloody Valentine => 0.004 Sonic Youth => 0.004 Talking Heads => 0.003 Nick Cave => 0.002 The Cure => 0.001 David Byrne => 0.001

Post-punk

Other Models

Weighted graph Album GenresKrautrock

Psychedelic Rock

Experimental Rock

1

1

1

Genre Affinity

Alternative Rock

Art Rock

Experimental Rock

Progressive Rock

29

47

3127

Indie Rock

Genre Affinity

Indie Rock

Shoegazing

Alternative Rock

Dream Pop

22

25

2412

Post-rock

Genre Affinity

Gothic Metal

Doom Metal

Black Metal

Heavy Metal

13

34

2712

Stoner Metal

Clustering in Networks


u1 u2 u3 u4 u5u1 2u2 3u3 2u4 2u5 3

Spectral Clustering: Using Eigenvectors of the Laplacian Matrix

−u1 u2 u3 u4 u5

u1 1 0 0 1u2 1 1 1 0u3 0 1 0 1u4 0 1 0 1u5 1 0 1 1

=u1 u2 u3 u4 u5

u1 2 -1 0 0 -1u2 -1 3 -1 -1 0u3 0 -1 2 0 -1u4 0 -1 0 2 -1u5 -1 0 -1 -1 3

Degree MatrixAdjacency Matrix (Similarity Matrix)

Laplacian Matrix


Eigenvector: a vector (v) that by getting multiplied in matrix A does not result in changing its direction (similar to being multiplied by scalar λ)

u1 u2 u3 u4 u5

-0.7 0.3 -0.2 -0.1 0.7-0.7 0.3 -0.2 -0.1 0.7

Spectral Clustering Codez

from sklearn.cluster import spectral_clustering import numpy as np

A = [[0.0 for x in n] for x in n] … # build adjacency matrix res = spectral_clustering(np.matrix(A), n_clusters) # res -> list of cluster indices e.g. [1,1,0,5,…]

Spectral Clustering Results

Folk Rock Country Rock

Blues Folk

Country Americana Roots Rock Blues Rock

Southern Rock

Power Metal Progressive Metal Symphonic Metal Black Metal Melodic Death Metal Groove Metal Nu Metal Thrash Metal

Death Metal Metalcore Industrial Metal Gothic Metal Christian Metal Doom Metal Speed Metal

Alternative Rock Indie Rock

New Wave Synthpop

Electronica

Rock R&B Pop

Pop Rock Funk Soul

Heavy Metal Hard Rock

Alternative Metal

50 Years of Rock (1990-2014)


1. alternative rock (69) 2. rock (59) 3. hard rock (48) 4. thrash metal (45) 5. heavy metal (36) 6. punk rock (25) 7. country (23)

This will stay at this spot pretty much most of the remaining 25 years…

The Black Crowes Shake Your Money Maker


1. alternative rock (82) 2. rock (68) 3. heavy metal (44) 4. hard rock (33) 5. thrash metal (30) 6. country (23) 7. pop (19) Nirvana

Nevermind


1. alternative rock (83) 2. rock (56) 3. heavy metal (37) 4. hard rock (35) 5. thrash metal (29) 6. death metal (27) 7. indie rock (25)

A new genre is born….

Rage Against The Machine Rage Against The Machine


1. alternative rock (125) 2. rock (72) 3. indie rock (39) 4. pop (28) 5. hard rock (28) 6. punk rock (28) 7. grunge (26)

It is all about the West Coast now, especially Seattle…

Tool Undertow


1. alternative rock (123) 2. rock (81) 3. indie rock (52) 4. punk rock (46) ↑ 5. heavy metal (45) 6. hard rock (34) 7. country (24)

It is popular again and will be for the next 10 years…

Soundgarden Superknown


1. alternative rock (139) 2. rock (68) 3. punk rock (49) 4. heavy metal (46) 5. indie rock (44) 6. hard rock (37) 7. grunge (28) The Smashing Pumpkins

Mellon Collie and the Infinite SadnessAs it is clear here, grunge was really

undercurrent of what was going at the time although first half of the 90s known to be the grunge half-decade


1. alternative rock (132) 2. rock (98) 3. indie rock (60) ↑ 4. punk rock (44) 5. heavy metal (42) 6. hard rock (28) 7. country (27)

Another generic label not fully describing a genere…

Nick Cave and the Bad Seeds Murder Ballads


1. alternative rock (119) 2. rock (78) 3. indie rock (60) 4. punk rock (45) 5. heavy metal (33) 6. alternative metal (23) 7. pop punk (22) Radiohead

OK Computer


1. alternative rock (120) 2. rock (73) 3. indie rock (69) 4. punk rock (41) 5. heavy metal (37) 6. hard rock (29) 7. black metal (28)

Harder sound is more popular and we have a new genre in the list

Neutral Milk Hotel In the Aeroplane Over the Sea


1. alternative rock (146) 2. indie rock (73) 3. rock (72) 4. heavy metal (40) 5. hard rock (40) 6. alternative metal (34) 7. punk rock (33) The Beta Band

The Beta BandWe really did not see Britpop making an appearance… although the impact and influence would exist


1. alternative rock (103) 2. indie rock (88) 3. rock (86) 4. punk rock (45) 5. heavy metal (41) 6. pop punk (37) 7. alternative metal (37) Grandaddy

The Sophtware Slump


1. alternative rock (117) 2. rock (94) 3. indie rock (83) 4. alternative metal (49) 5. nu metal (45) 6. hard rock (44) 7. punk rock (36)

Another metal subgenre appearing and will not be the last.

System of a Down Toxicity


1. alternative rock (126) 2. indie rock (107) 3. rock (73) 4. heavy metal (54) 5. alternative metal (50) 6. hard rock (48) 7. punk rock (45) Interpol

Turn on the Bright Lights


1. alternative rock (169) 2. indie rock (117) 3. rock (91) 4. alternative metal (63) 5. punk rock (53) 6. hard rock (51) 7. power metal (46) The White Stripes

Elephant


1. alternative rock (155) 2. indie rock (130) 3. rock (96) 4. hard rock (64) 5. alternative metal (54) 6. punk rock (51) 7. pop punk (49)

Not much changing in the list…

Arcade Fire Funeral


1. alternative rock (168) 2. indie rock (164) 3. rock (107) 4. hard rock (69) 5. heavy metal (57) 6. pop punk (55) 7. alternative metal (53) Sufjan Stevens

Illinoise


1. alternative rock (188) 2. indie rock (179) 3. rock (105) 4. hard rock (70) 5. pop punk (58) 6. progressive metal (55) 7. power metal (46)

And yet another metal subgenre hitting mainstream…

MUSE Black Holes and Revelations


1. alternative rock (208) 2. indie rock (201) 3. rock (109) 4. hard rock (67) 5. alternative metal (64) 6. heavy metal (62) 7. pop punk (59) Radiohead

In Rainbows


1. alternative rock (203) 2. indie rock (197) 3. rock (101) 4. hard rock (60) 5. pop (56) 6. metalcore (51) 7. heavy metal (50)

And we have another metal variant that is here to stay…

Underoath Lost in the Sound of Separation


1. alternative rock (200) 2. indie rock (194) 3. rock (72) 4. hard rock (68) 5. indie pop (59) 6. pop rock (59) 7. metalcore (56)

Softer sounds come back…

Animal Collective Merriweather Post Pavilion


1. alternative rock (176) 2. indie rock (163) 3. rock (86) 4. hard rock (77) 5. indie pop (64) 6. heavy metal (62) 7. metalcore (61)

… and will be around… Arcade Fire The Suburbs


1. alternative rock (169) 2. indie rock (162) 3. metalcore (63) 4. indie pop (60) 5. alternative metal (58) 6. heavy metal (57) 7. rock (57) Tinariwen

Tassili


1. alternative rock (145) 2. indie rock (127) 3. rock (71) 4. hard rock (69) 5. alternative metal (50) 6. metalcore (49) 7. indie pop (45) Goat

World Music


1. alternative rock (126) 2. indie rock (124) 3. rock (73) 4. hard rock (57) 5. synthpop (56) 6. metalcore (51) 7. indie pop (51)

And synthpop makes a come back after 30 years or so…

Queens of the Stone Age ...Like Clockwork


1. alternative rock (108) 2. indie rock (106) 3. rock (50) 4. hard rock (48) 5. pop (41) 6. metalcore (41) 7. pop rock (40) Mac DeMarco

Salad Days

Intelligent Models

What is an Intelligent Model?

•Intelligent model is actually not a technically accurate term (butwe use it to denote shift in technique)

•Models learn from data instead of arbitrary definition •Can sometimes even distinguish and filter noise •Can represent a far more complex data relationships •Examples include HMM, SVM and recently Deep Learning •We review word2wec which uses Neural Network technique but not as deep as “Deep Learning”

word2vec ModelA very powerful model representing words (tokens) as vectors

based on their proximity to other words (tokens).

Once represented as vectors, all algebraic operations can be applied to the vector representations.

word2vec Model

Skip-gram: a proximity-based probability model trained using Neural Networks (Deep Learning)

Pink Floyd were an English rock band formed in LondonX XX

word2vec Model

Break the text to

sentences

Tokenise each

sentence to word/

phrases

Feed the data to engine

Store the

Model

Training

Load the model

Construct the query

Run against the model (Filter noise)

Querying

word2vec Codz (Training)import gensim

def train_and_save(multiword_dic_path, stopword_path, corpus_files_glob_pattern, mode_file_name = ‘rock_music.w2v'):

tknsr = dictionary_tokenization.DictionaryBasedMultiwordFinder(dictionary_path=multiword_dic_path)

filtr = stopword_filtering.StopwordFilter(stopword_path)

tokenizze = lambda text: filtr.filter(tknsr.tokenise(text))

wvc = gensim.models.Word2Vec(EnglishSentences(corpus_files_glob_pattern, tokenizze, True), min_count=2)

wvc.save(mode_file_name)

word2vec Codz (Querying)

sims = word_sim.most_similar(positive=pos, negative=neg, topn=topn) # pos is an array of positive e.g. [“Nick Mason”, “The Rolling Stones”] # neg is an array of negative e.g. [“Pink Floyd”] # result is an array of tuples of results for # “Nick Mason” + “The Rolling Stones” - “Pink Floyd” # the first entry of which is “Charlie Watts”

word2vec Demo

Wrap-up

Mystery missing artist

?

References•All pictures from wikipedia.org used under Creative Commons •Source of all data is from wikipedia.org collected online using a single call and then stored and processed •Efficient Estimation of Word Representations in Vector Space. Mikolov et. al. http://arxiv.org/abs/1301.3781 •Gensim's word2vec •networkx lib •word2vec blog post (500K docs): Five crazy abstractions my Deep Learning word2vec model just did •word2vec on Rock music blog: Daft Punk+Tool=Muse: word2vec model trained on a small Rock music corpus •code for word2vec on wiki data •Highcharts: highcharts •word2vec paper: PDF •Automatic real-time road marking recognition using a feature-driven approach PDF •Video of the road marking recognition: here and here and here •Future of Programming - Rise of the Scientific Programmer (and fall of the craftsman)

http://arxiv.org/abs/1301.3781

https://radimrehurek.com/gensim/models/word2vec.html

https://networkx.github.io/

http://byterot.blogspot.co.uk/2015/06/five-crazy-abstractions-my-deep-learning-word2doc-model-just-did-NLP-gensim.html

http://byterot.blogspot.co.uk/2015/07/daft-punk-tool-muse-word2vec-model-trained-36K-rock-music-corpus-wiki-NLP-gensim.html

https://github.com/aliostad/WikiRockWord2Vec

http://www.highcharts.com/

http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.227.8116&rep=rep1&type=pdf

https://www.youtube.com/watch?v=NeQvcdRPxoI

https://www.youtube.com/watch?v=3E3bcLqniDw

https://www.youtube.com/watch?v=NqYPJELHkhA

http://byterot.blogspot.co.uk/2015/01/future-of-programming-rise-of-the-scientific-developer-bigdata-datascience-machine-learning-and-fall-of-the-craftsman.html

from power chords to the power of models

Software