ainl 2016: rykov, nagornyy, koltsova, natta, kremenets, manovich, cerrone, crockett
TRANSCRIPT
![Page 1: AINL 2016: Rykov, Nagornyy, Koltsova, Natta, Kremenets, Manovich, Cerrone, Crockett](https://reader035.vdocuments.mx/reader035/viewer/2022062523/58749dd51a28abfc5f8b6a2b/html5/thumbnails/1.jpg)
Yuri Rykov Oleg Nagornyy
Olessia Koltsova Herbert Natta
Alexander KremenetsLev Manovich
Damiano CerroneDamon Crockett
Semantic and Geospatial Mapping of Instagram Images in Saint-Petersburg
AINL FRUCT Artificial Intelligence and Natural Language Conference
November 11, 2016
![Page 2: AINL 2016: Rykov, Nagornyy, Koltsova, Natta, Kremenets, Manovich, Cerrone, Crockett](https://reader035.vdocuments.mx/reader035/viewer/2022062523/58749dd51a28abfc5f8b6a2b/html5/thumbnails/2.jpg)
BACKGROUNDDigital urban studies is research field that combines issues and methods of urban sociology, computer science, digital humanities, linguistics and design to retrieve knowledge about everyday life and social organization of cities from diverse data sources.
The availability of large geolocated visual social media data creates new opportunities for studying cities.
The important task is to extract meanings of human experience in different urban areas.
New way to do it is to analyze relations between visual content of shared images and their geographical locations.
![Page 3: AINL 2016: Rykov, Nagornyy, Koltsova, Natta, Kremenets, Manovich, Cerrone, Crockett](https://reader035.vdocuments.mx/reader035/viewer/2022062523/58749dd51a28abfc5f8b6a2b/html5/thumbnails/3.jpg)
DATASET
47,410 Instagram items from Saint-Petersburg during one year period from July 1 2014 to June 30 2015.
Data = image + time stamp + geographical coordinates + user ID + user-generated #hashtags.
Instagram API was used to collect the data.Google vision API was used to recognize entities.
![Page 4: AINL 2016: Rykov, Nagornyy, Koltsova, Natta, Kremenets, Manovich, Cerrone, Crockett](https://reader035.vdocuments.mx/reader035/viewer/2022062523/58749dd51a28abfc5f8b6a2b/html5/thumbnails/4.jpg)
• Can Instagram images be clustered into meaningful categories reflecting human experience?
• Can this experience be related to certain urban areas in a meaningful way?
RQs
![Page 5: AINL 2016: Rykov, Nagornyy, Koltsova, Natta, Kremenets, Manovich, Cerrone, Crockett](https://reader035.vdocuments.mx/reader035/viewer/2022062523/58749dd51a28abfc5f8b6a2b/html5/thumbnails/5.jpg)
Step I: Google Tag Networks and Clusters
GOOGLE TAGS NETWORK AND CLUSTERS (15)
flowers
hair & style
sunrise & sea
dish
drink
clothing
animals
document & mobile device
automobile
portrait
facade & palace
strength training
Art
![Page 6: AINL 2016: Rykov, Nagornyy, Koltsova, Natta, Kremenets, Manovich, Cerrone, Crockett](https://reader035.vdocuments.mx/reader035/viewer/2022062523/58749dd51a28abfc5f8b6a2b/html5/thumbnails/6.jpg)
CLUSTER SAMPLES
Portrait
Cars
Flowers
![Page 7: AINL 2016: Rykov, Nagornyy, Koltsova, Natta, Kremenets, Manovich, Cerrone, Crockett](https://reader035.vdocuments.mx/reader035/viewer/2022062523/58749dd51a28abfc5f8b6a2b/html5/thumbnails/7.jpg)
GOOGLE TAG CLUSTERS
& USER
#HASHTAG CLUSTERS
intersection
![Page 8: AINL 2016: Rykov, Nagornyy, Koltsova, Natta, Kremenets, Manovich, Cerrone, Crockett](https://reader035.vdocuments.mx/reader035/viewer/2022062523/58749dd51a28abfc5f8b6a2b/html5/thumbnails/8.jpg)
Chi^2cluster label
co-occ(corresp.analysis)
TOPICAL SIMILARITY OF IMAGE CLUSTERS
![Page 9: AINL 2016: Rykov, Nagornyy, Koltsova, Natta, Kremenets, Manovich, Cerrone, Crockett](https://reader035.vdocuments.mx/reader035/viewer/2022062523/58749dd51a28abfc5f8b6a2b/html5/thumbnails/9.jpg)
‘PORTRAIT’cluster images on St.Petersburg map
![Page 10: AINL 2016: Rykov, Nagornyy, Koltsova, Natta, Kremenets, Manovich, Cerrone, Crockett](https://reader035.vdocuments.mx/reader035/viewer/2022062523/58749dd51a28abfc5f8b6a2b/html5/thumbnails/10.jpg)
‘ANIMALS’ cluster images on St.Petersburg map
![Page 11: AINL 2016: Rykov, Nagornyy, Koltsova, Natta, Kremenets, Manovich, Cerrone, Crockett](https://reader035.vdocuments.mx/reader035/viewer/2022062523/58749dd51a28abfc5f8b6a2b/html5/thumbnails/11.jpg)
‘DISH’cluster images on St.Petersburg map
![Page 12: AINL 2016: Rykov, Nagornyy, Koltsova, Natta, Kremenets, Manovich, Cerrone, Crockett](https://reader035.vdocuments.mx/reader035/viewer/2022062523/58749dd51a28abfc5f8b6a2b/html5/thumbnails/12.jpg)
Drink‘DRINK’
cluster images on St.Petersburg map
![Page 13: AINL 2016: Rykov, Nagornyy, Koltsova, Natta, Kremenets, Manovich, Cerrone, Crockett](https://reader035.vdocuments.mx/reader035/viewer/2022062523/58749dd51a28abfc5f8b6a2b/html5/thumbnails/13.jpg)
Sunraise‘SUNRISE & SEA’
cluster images on St.Petersburg map
![Page 14: AINL 2016: Rykov, Nagornyy, Koltsova, Natta, Kremenets, Manovich, Cerrone, Crockett](https://reader035.vdocuments.mx/reader035/viewer/2022062523/58749dd51a28abfc5f8b6a2b/html5/thumbnails/14.jpg)
GEOSPATIAL SIMILARITY FOR CLUSTER HEATMAPS
![Page 15: AINL 2016: Rykov, Nagornyy, Koltsova, Natta, Kremenets, Manovich, Cerrone, Crockett](https://reader035.vdocuments.mx/reader035/viewer/2022062523/58749dd51a28abfc5f8b6a2b/html5/thumbnails/15.jpg)
GEOSPATIAL SIMILARITY FOR CLUSTER HEATMAPS
![Page 16: AINL 2016: Rykov, Nagornyy, Koltsova, Natta, Kremenets, Manovich, Cerrone, Crockett](https://reader035.vdocuments.mx/reader035/viewer/2022062523/58749dd51a28abfc5f8b6a2b/html5/thumbnails/16.jpg)
The most similar to many other clusters are “hairstyle” and “animals” clusters, because they are relatively evenly distributed in space: they are geographically independent topics.
The most dissimilar to other clusters are “clothing & fashion”, “power training” and "facade & palace". It indicates that related human experience occurs in the most detached urban places probably reflecting social heterogeneity and exclusiveness.
Such combination of methods has not been applied to digital urban studies before.
Such results can be used to study urban segregation or to rate city areas in terms of their consumer/tourist attractiveness or cultural/entertainment development.
CONCLUSION
![Page 17: AINL 2016: Rykov, Nagornyy, Koltsova, Natta, Kremenets, Manovich, Cerrone, Crockett](https://reader035.vdocuments.mx/reader035/viewer/2022062523/58749dd51a28abfc5f8b6a2b/html5/thumbnails/17.jpg)
Yuri Rykov - National Research University Higher School of Economics, Russia Oleg Nagornyy - Olessia Koltsova - Herbert Natta - University of Rome Tor Vergata, ItalyAlexander Kremenets - makeomatic, RussiaLev Manovich - City University of New York , USADamiano Cerrone - Spatial Intelligence Unit , EstoniaDamon Crockett - Software Studies Initiative, USA
The study was implemented in the framework of the Basic Research Program at the National Research University Higher School of Economics (HSE) in 2016. This research was started at Digital Traces Summer Lab “Meta-Morphologies of St.Petersburg 2016” directed by Lev Manovich and Damiano Cerrone.
ACKNOWLEDGMENTS
WORKING GROUP
![Page 18: AINL 2016: Rykov, Nagornyy, Koltsova, Natta, Kremenets, Manovich, Cerrone, Crockett](https://reader035.vdocuments.mx/reader035/viewer/2022062523/58749dd51a28abfc5f8b6a2b/html5/thumbnails/18.jpg)
THANK YOU FOR ATTENTION!