Visualizing a Large Spatiotemporal Collection of Historic Photographywith a Generous Interface

Taylor Arnold*

University of Richmond, Math and Computer ScienceNathaniel Ayers†

University of Richmond, Digital Scholarship Lab

Justin Madron‡

University of Richmond, Digital Scholarship LabRobert Nelson§

University of Richmond, Digital Scholarship Lab

Lauren Tilton¶

University of Richmond, Rhetoric and Communications

ABSTRACT

Museums, libraries, and other cultural institutions continue to priori-tize and build web-based visualization systems that increase accessand discovery to digitized archives. Prominent examples exist thatillustrate impressive visualizations of a particular feature of a col-lection. For example, interactive maps showing geographic spreador timelines capturing the temporal aspects of collections. By wayof a case study, this paper presents a new web-based visualizationsystem that allows users to simultaneously explore a large collectionof images along several different dimensions—spatial, temporal,visual, textual, and through additional metadata fields including thephotographer name—guided by the concept of generous interfaces.The case study is a complete redesign of a previously released dig-ital, public humanities project called Photogrammar (2014). Thepaper highlights the redesign’s interactive visualizations that are nowpossible by the affordances of newly available software. All of thecode is open-source in order to allow for re-use of the codebase toother collections with a similar structure.

Index Terms: Human-centered computing—Visualization—Information visualization; Human-centered computing—Visualiza-tion—Geographic visualization

1 INTRODUCTION

Over the past few decades, cultural institutions have made extensivecommitments to digitizing and making their collections availableover the web. While some collections are only able to provide lim-ited access to metadata or through an institutional login screen forlegal reasons, many archives have been able to open up digitized ma-terials to the public at large. Prominent examples of institutions thathave made large collections digitally available include the Rijksmu-seum, the British Library, the Library of Congress, the MetropolitanMuseum of Art, and the Smithsonian [2, 15]. Making materials digi-tally available is an exciting step in improving access and discovery.However, additional contextual framing and digital tools are oftenneeded to guide users through the material in order to engage with abroad public audience.

In response to the increase in access to digital collections, expertsacross various fields including digital humanities and data sciencehave begun to think of collections as data [26]. Many tools focuson a specific mode of analysis in order to explore collections from

*e-mail: tarnold2@richmond.edu†e-mail: nayers@richmond.edu‡e-mail: jmadron@richmond.edu§e-mail: rnelson2@richmond.edu¶e-mail: ltilton@richmond.edu

a specific point of view. For example, the Knight Lab’s Timeline.jsprovides a library for building custom visualizations of data dis-tributed over time. Tools created by companies such as ArcGISand Carto provide similar options for spatial data. Textual data canbe visualized through the use of keywords (e.g., Voyant) or topicmodels (e.g., jsLDA). General metadata fields can be visualized withlibraries such as d3 and cube.js. While not specifically designedfor web-base usage, Gephi provides a range of options for show-ing and exploring network relationships. The WebGL-based libraryPixPlot [16] provides an impressive framework for exploring largephotographic collections using state-of-the-art neural network mod-els. In most of these cases, data must be loaded into a series of toolsto visualize the data from different angles and for different purposes.

Significantly less attention has been paid to approaches for visu-alizing many unstructured components of a collection at the sametime. Digital storytelling applications, such as the knight lab’s Sto-ryMap.js, have shown the power of putting together images, text,and other metadata within a specific narrative structure. However,web-based tools for the open exploration of such collections arelimited. Exploring the many dimensions of multimodal collectionsgenerally requires the use of programming software, which has asignificantly higher barrier to entry that limits its audience. Thishampers the ability of cultural institutions to allow digital visitors oftheir collections to understand the subtle interplay of an archive’scomponents.

In this paper, we present a web-based visualization system de-signed with the concept of generous interfaces for a collection ofover 170,000 historic photographs through a case study. Photogram-mar, a digital public humanities project and our case study, buildsoff of our previous work with the collection to allow users to explorespatial, temporal, textual, visual, and structured components of thecollection within a single interactive generous interface. This sig-nificantly increases the kinds of analysis available to users directlywithin the site while visually communicating the interconnectednessof data and metadata.

The remainder of the article is organized as follows. We start witha brief history of the archive we are working with and describing ouroriginal digital website (published in 2014) along with its strengthsand challenges. We then present the integrated design of the new siteand illustrate the new analyses afforded by the redesign. Particularattention is given to the use of newly available image analysis algo-rithms and how they have changed the possibilities for visualizingand therefore accessing and interpreting material. A concluding sec-tion discusses open challenges and our proposed path for developinga general-purpose visualization library from our specific case study.

2 BACKGROUND: FSA-OWI AND PHOTOGRAMMAR

The FSA-OWI photographic archive is the result of a commissionby the United States Farm Security Administration and Office ofWar Information (FSA-OWI) to document American life duringthe Great Depression and World War II [8]. Physically housed in

arX

iv:2

009.

0224

2v1

[cs

.HC

] 4

Sep

202

0

Figure 1: Original interactive map from the Photogrammar website(2014 through mid-2020), implemented using slippy maptiles. County-level objects show the number of photographs taken in each region;the photographs can be subset and the counts updates by using thepull-down menu to select a photographer or the timeline to selecta range of years. Clicking on a county—as shown here for Jasper,Iowa—reveals the exact number of photographs and a provides a linkto a search page.

Figure 2: Search page on the original Photogrammar website (2014through mid-2020), showing the results of clicking on the link in Fig-ure 1. Thumbnails are displayed in a grid along with limited metadata(location, photographer, month, and year). Images are displayed ingroups of 60. A more specific faceted search can be run by clickingon the ‘Start New Search’ prompt in the upper-lefthand corner of thepage.

the Library of Congress (LoC), the archive serves as an importantvisual record for scholars across the academy and the public-at-large. The collection includes many iconic photographs, includeDorothea Lange’s Migrant Mother, Gordon Parks’ American Gothic,and Arthur Rothstein’s photos of the American Dust Bowl. Thecollection’s 170,000 monochrome and color photographs residesentirely within the public domain and offers a unique snapshot ofthe nation in the 1930s and 1940s [37].

From 2012 to 2016, Arnold and Tilton, in collaboration withLaura Wexler, constructed an interactive website called Photogram-mar as a new approach to accessing and visualizing the FSA-OWIphotographic collection. The full team came to include Trip Kirk-patrick, Peter Leonard, Stacey Maples, and Ken Panko. The projectbegan as a response to the challenges of navigating the digital andphysical archive at the LoC. Users either had to search through theLoC’s filing cabinets of over 88,000 prints or LoC’s online site,which allows for limited exploration. Through collaboration withthe collection’s curator to harness the collection’s rich metadata, theproject developed creative and innovative new approaches to the

Figure 3: Photograph page from the original Photogrammar website(2014 through mid-2020). This is the result of clicking on the fourthimage shown in the results from Figure 2. Most of the metadata fieldsare hyperlinks to further searches to show other images from thesame location, topic, or by the same photographer. A set of similarphotographs are suggested based on the similarity of the imagecaptions.

collection using digital and public humanities methods. By cleaningand transforming the metadata, standardized the fields and geotaggedthe collection, the site allows users to map photographs, run facetedsearches, and see historic cataloging systems used to organize thephysical collection [5].

The original incarnation of the Photogrammar project can beseen in Figures 1 through 3. The core of the site was implementedas a MySQL database containing records for every photograph inthe collection. One page implemented an advanced faceted search,Figure 2, which would produce a sorted list of results on a newwebpage. Clicking on a result opened a new dynamically producedrecord page, Figure 3, displaying a large version of the image and as-sociated metadata. This interface offered a similar search experienceto the existing site at the Library of Congress, while providing morecontrol over the search parameters and improved cleaning of theavailable metadata [4]. Most users, however, entered the collectionthrough one of the available interactive visualizations on the site.These include an interactive map, Figure 1, a crosstab metadataexplorer, and a hierarchical description of an historic classificationsystem used by the Library of Congress. These visualization pro-vided different aggregate views of the 170,000 photographs. Theyare coded using individual JavaScript libraries; access to the indi-vidual photographs was provided by pre-filled links to the advancedsearch interface.

The original version of the Photogrammar website provided manydifferent ways of viewing the collection, opening up the FSA-OWIcollection to interactive exploration to a wide public audience. Inour redesign, described in the remainder of this article, our goal wasto build on new technology stacks to address several design issuesin the original site, with a particular focus on integrating the varioussearch and exploration pages into a single integrated web-basedvisualization of the archive.

3 GENEROUS INTERFACES AND THE DIGITAL PUBLIC HU-MANITIES

The concept of a ‘generous interface’ was introduced by MitchellWhitelaw in 2015 [40]. Focused on the collections held by culturalheritage institutions, he called for a conceptual shift from the singu-lar, narrow interaction of the keyword search (usually in the form ofa blank box such as Google Search) to interfaces that communicatethe scale, scope, and richness of a collection. Visualizations are a keycomponent and a strategy. Centering exploration, interactivity, andscalability, generous interfaces are an ethos, praxis, and outcome.

Figure 4: Landing page of the new Photogrammar website, which shows the currently selected images (left), a chloropleth map (right), andselected photographers and years (bottom). The elements are linked together to allow users to explore the interactions of different dimensions ofthe collection simultaneously.

Figure 5: The result of clicking on the state of Texas and photographer Russell Lee on the landing page shown in Figure 4. All three elements—theselected photographs, and spatial-temporal chloropleth visualizations—are updated to reflect only images by Russell Lee taken in Texas.

Figure 6: The map in Figure 4 can be replaced by other visualizationsby selecting an option at the top. This figure shows a hierarchicaltreemap of the historic themes assigned to the images in the collectionby Paul Vanderbilt. As with the map, the selected photos on the leftand interactive timeline at the bottom interactively update with theselected Themes.

Figure 7: The result of clicking on an image in the new Photogrammarwebsite. A larger version of the image is shown, along with metadataand similar photographs. A small map allows users to retain thespatial dimension of the collection.

Generous interfaces can also be understood as a strategy forthe digital, public humanities (DPH). The rapidly expanding set ofprojects online that identify as DPH—projects that seek to engage,and often co-create, with a set of publics beyond the academy andcommonly are built to expand access and interpretation using meth-ods from the digital humanities—has brought questions about howto engage with audiences and users to the fore. As Sheila Brennanargues, “It is important to recognize that projects and research maybe available online, but that status does not inherently make the workdigital public humanities or public digital humanities” [9]. As Jor-dana Cox and Lauren Tilton have argued, DPH can “forefront visual,nonlinear, and interactive argumentation in oder to engage publicsin generative humanistic inquiry” [14]. One way is projects de-signed with generous interfaces that further generative and generoushumanistic inquiry.

Through a more detailed explanation of the redesign of Pho-togrammar, we demonstrate how generous interfaces and DPH aretheoretical frameworks and methods for accessing and interpretingcultural heritage data through interactive visualizations. The newversion of Photogrammar forges an integrated approach to gener-ous interfaces. Rather than treat each interactive visualization as adiscrete task, we demonstrate how we can connect different typesof visualizations with concepts from information retrieval such assearch, browse, and recommender systems. What is considered data

and metadata shifts based on the community and their interests;therefore, we show how toggling between and integrating ‘data’ and‘metadata’ offers a strategy for creating generous interfaces builtaround integrated visualizations that further access and interpretationof collections.

4 A MORE GENEROUS REDESIGN

The new version of Photogrammar provides a complete redesign ofthe web-based visualization system, which offers a more integrateduser experience through a generous interface built on a modern tech-nology stack. Figure 4 shows the new landing page for the site,featuring a selection of photographic thumbnails (left), an interac-tive map (top right), and interactive timeline split by photographer(bottom right). The entire page is now implemented as a React.jsapp, allowing all of the elements of the page to be linked together.Clicking on the state of Texas on the map, for example, zooms tothe state of Texas. At the same time, the photographs on the left andtimeline at the bottom are updated simultaneously. Further clickingon a photographer at the bottom updates the map and photos fur-ther. The result of clicking on the state of Texas and photographerRussell Lee is given in Figure 5. Linking these elements togetherhelps to center the photographs themselves, the main attraction formost users, by putting them on the landing page and updating themdynamically rather than through a secondary page. It also allowsfor transdisclinary analyses at the intersection of space, time, pho-tographer, and themes that were not available through the originalsite.

A major limiting factor in offering many coupled visualizationsis the available screen area available for multiple visualizations. Sig-nificant effort has been put into compressing the amount of spaceneeded for the default visualizations. For example, the dynamicslippy maptile approach of the original site has been replaced with afixed zoom level map served by a GeoJSON file [18]. To compressspace and improve the visibility of the entire collection, Hawaii,Alaska, Puerto Rico, and the U.S. Virgin Islands have been reposi-tioned into the main map. For the timeline of photographers, onlythe most prolific photographers are included in the default visualiza-tion. Additional photographers may still be selected, however, byclicking the ‘other photographers’ button.

Even with these space saving efforts, it is not possible to includeall of the visualizations in a single window that is still visible on moststandard computer monitors. In order to access other visualizations,the default interactive map can be replaced with a different visual-ization by selecting one of the options at the top of the page. Theseinclude a more granular points-based map and a theme visualization(Figure 6). As with the original map, all of these visualizations arecoupled with the timeline and search results on the side bar.

The search results interactively within the page as the maps,timelines, and themes are selected provide a way of making visiblethe images themselves from the beginning. The search results onthe side bar contain some minimal metadata, and a small thumbnailof the image. Pages of the results can be scrolled through to seeall of the relevant images. It is still needed, however, to use morescreen space to see a larger version of the image and all of theassociated metadata. In order to not lose the search results, andto visually emphasis that all of the elements (i.e. metadata andphotos) are connected, this graphic is overlaid over the map andtimeline on the right-hand side of the page. An example is shownin Figure 7. In order to continue to stress the connection to thespatial visualization, a small in-lay map of the United States is shownwithin the photograph page. By aggregating and connecting differenttype of visualizations, the system creates a generous interface thatamplifies digital public humanities inquiry into a collection.

5 RECOMMENDATION SYSTEMS

The interactive maps, timeline, and theme treeplot are all designedto allow users of the website to explicitly select subsets of the FSA-OWI collection that they are interested in viewing. Typical behaviorinvolves selection a region of the map that a user is most familiarwith—such as their current location or a hometown—or selectingone of the better known photographers. Opening familiar lines ofsearch for users has many benefits: it creates a personal connectionbetween the (meta)data and the user, it allows users to bring theirown knowledge into their interpretation of the collection, and itincreases the level of user engagement. The downside of this wayinto the archive, however, is that it risks getting users stuck in a smallsubset of the larger collection. In order to overcome this limitation,Photogrammar also incorporates a set of recommendations that cutacross the fixed metadata fields that drives the initial discovery stepof exploring each collection.

A recommendation system is an algorithm for predicting whatrecords in a collection a user may be interested in [27]. Thesesystems feature prominently in commercial services. For example,product pages on Amazon features other ‘similar products’, Netflixrecommends new releases of interest, and Spotify creates artificallyconstructed playlists based on prior listening behavior. In the contextof a digital archive, recommendation systems are most commonlyused to recommend records that are most similar to the particularrecord that a user is currently looking at. It is possible to determinethe most similar records based on the search behavior of other users.This method for building a recommendation system, a process calledcollaborative filtering, requires a large amount of data, creates pri-vacy concerns, and necessitates constantly updating models basedon changing behaviors [12]. An alternative method, content filtering,bases recommendations entirely on the content of the records them-selves [36]. We utilize content filtering in our generous interface ofthe FSA-OWI archive.

The initial version of the Photogrammar website utilized a recom-mendation system that recommended similar records on the individ-ual photograph pages. The interface is shown at the bottom of Fig-ure 3. These similarities are based on using the text in the captions ofthe photographs. Specifically, we compute term frequency-inversedocument frequency (TF-IDF) scores and identify captions that arethe most similar to the caption of interest. Pervious work has dis-cussed the various iterations that this system has gone through. Forexample, a process of named entity recognition (NER) was used toremove geographic names from the captions [4]. A word-embeddingmodel was used to to find similar captions that use slightly differentwords, such as ‘apple orchard’ and ‘pear trees’ [6]. This caption-based system helps find connections that would otherwise be hiddenthrough other interfaces. However, it does have some limitations.Less than half of the images have captions, and many of these aretoo short to be useful for recommendation purposes. Furthermore,the captions only describe one main object of study and rarely men-tion elements of style and composition. In order to capture theseelements, a different approach is needed.

The new Photogrammar website implements a content filteringrecommendation system based on visual similarity. Every imagein the collection was projected into the penultimate layer of a largeconvolutional neural network in order to obtain an embedding of theentire collection into a high-dimensional space [33]. Each record isthen associated with other photographs that have the most similarset of embedding values [7]. This method tends to create connec-tions between images that contain similar objects, events, or sharesimilar forms of composition [16]. While the neural network usedis trained entirely on color images, the implicit connections createdby the embedding produce surprisingly consistent and meaningfulresults on the predominantly black-and-white images in the FSA-OWI collection [29]. The new recommendation system is shown inthe bottom of Figure 7. The visual search method overcomes many

of the difficulties of the entirely text-based method, such as beingable to produce recommendations for all images in the corpus andfinding links between elements of the images not explicitly men-tioned in each caption. The new recommender system also respondsto consistent feedback from users who asked about visual search.

Computing embedding-based visual similarities for each image isa computationally intensive task. Rather than running this each timea user requests a particular image, the graph of similar images hasinstead been precomputed and stored as static files on the website.In addition to the benefit of reducing the computational load of thesite, this approach also provides a simple path forward for furtheriterations of the recommendation system, such as integrating jointtext-image embeddings [3, 31]. A new recommendation system canbe pushed into the site by simply updating the static files describingthe most similar photographs.

6 CONCLUSIONS AND FUTURE WORK

The new, completely redesigned version of the Photogrammar web-based visualization and search interface offers numerous improve-ments over the already successful original version of the site. Inline with the goals of the digital public humanities and generousdesign, the new site allows for a more integrated user-experiencewith a number of tightly connected elements that all interactivelyupdate with one another. Additional modifications built into the newmaps, timeline, and recommendation system further improve theability of users to explore and connect with the photographs fromthe FSA-OWI archive. As a result, the visualizations come together,harnessing the relationships between and across the metadata, torealize a generous interface for the photographs. At the same time,the visualizations allows the user to toggle between and across dif-ferent objects of study. Metadata fields, as classified by the Libraryof Congress and through the structure of the database serving Pho-togrammar, can become the object of study, such as interest in thework of a specific photographer such as Marion Post Wolcott or astate such as as Texas, and therefore the ‘data’. Generous interfaces,therefore, provide a theoretical and methodological frame for de-signing systems that disrupt classifications of data and metadata inthe way that humanistic inquiry often demands.

In our redesign of the site, a key concern was building main-tainable and reusable components. Future work on the project willfocus on decoupling the elements specific to this collection (such asthe theme visualizer) and turning the codebase into a documentedframework that can be used by other archival projects in the digitalpublic humanities that feature rich, multimodal data.

REFERENCES

[1] H. Abdollahpouri, R. Burke, and B. Mobasher. Managing popularitybias in recommender systems with personalized re-ranking. In TheThirty-Second International Flairs Conference, 2019.

[2] V. Alexiev. Museum linked open data: Ontologies, datasets, projects.Digital Presentation and Preservation of Cultural and Scientific Her-itage, (VIII):19–50, 2018.

[3] M. Alikhani, S. Nag Chowdhury, G. de Melo, and M. Stone. CITE:A corpus of image-text discourse relations. In Proceedings of the2019 Conference of the North American Chapter of the Association forComputational Linguistics: Human Language Technologies, Volume 1(Long and Short Papers), pp. 570–575. Association for ComputationalLinguistics, Minneapolis, Minnesota, June 2019. doi: 10.18653/v1/N19-1056

[4] T. Arnold, P. Leonard, and L. Tilton. Knowledge creation through rec-ommender systems. Digital Scholarship in the Humanities, 32(3):151–157, 2017.

[5] T. Arnold, S. Maples, L. Tilton, and L. Wexler. Uncovering latent meta-data in the FSA-OWI photographic archive. DHQ: Digital HumanitiesQuarterly, 11(2), 2017.

[6] T. Arnold and L. Tilton. Cross-discourse and multilingual explorationof textual corpora with the DualNeighbors algorithm. pp. 50–59, 2018.

[7] T. Arnold and L. Tilton. Distant viewing: Analyzing large visualcorpora. Digital Scholarship in the Humanities, 34(1):i3–i16, 2019.

[8] S. Baldwin. Poverty and politics; the rise and decline of the farmsecurity administration. 1968.

[9] S. A. Brennan. Public, first. In M. Gold and L. Klein, eds., Debates inthe Digital Humanities. University of Minnesota Press, 2016.

[10] J. Buolamwini and T. Gebru. Gender shades: Intersectional accuracydisparities in commercial gender classification. In Conference onfairness, accountability and transparency, pp. 77–91, 2018.

[11] H. Caesar, J. Uijlings, and V. Ferrari. COCO-stuff: Thing and stuffclasses in context. In Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition, pp. 1209–1218, 2018.

[12] J. A. Calandrino, A. Kilzer, A. Narayanan, E. W. Felten, andV. Shmatikov. You might also like: Privacy risks of collaborativefiltering. In 2011 IEEE symposium on security and privacy, pp. 231–246. IEEE, 2011.

[13] C. Concordia, S. Gradmann, and S. Siebinga. Not (just) a repository,nor (just) a digital library, nor (just) a portal: A portrait of Europeanaas an API. In World Library and Information Congress: 75th IFLAGeneral Conference and Council, 2009.

[14] J. Cox and L. Tilton. The digital public humanities: Giving newarguments and new ways to argue. vol. 19, pp. 127–146. 2016.

[15] C. Dijkshoorn, L. Jongma, L. Aroyo, J. Van Ossenbruggen,G. Schreiber, W. Ter Weele, and J. Wielemaker. The Rijksmuseumcollection as linked data. Semantic Web, 9(2):221–230, 2018.

[16] D. Duhaime. PixPlot: Visualize large image collections with WebGL.https://github.com/YaleDHLab/pix-plot, 2019.

[17] R. V. Guha, D. Brickley, and S. Macbeth. Schema.org: evolution ofstructured data on the web. Communications of the ACM, 59(2):44–51,2016.

[18] T. Horbinski and D. Lorek. The use of Leaflet and GeoJSON files forcreating the interactive web map of the preindustrial state of the naturalenvironment. Journal of Spatial Science, pp. 1–17, 2020.

[19] M. W. Khaw, L. Stevens, and M. Woodford. Individual differences inthe perception of probability. Available at SSRN 3446790, 2019.

[20] A. Kirillov, K. He, R. Girshick, C. Rother, and P. Dollar. Panopticsegmentation. In Proceedings of the IEEE conference on computervision and pattern recognition, pp. 9404–9413, 2019.

[21] A. Kumar and R. K. Singh. Comparative analysis of Angular.js andReact.js. International Journal of Latest Trends in Engineering andTechnology, 7(4):225–227, 2016.

[22] T.-Y. Lin, A. RoyChowdhury, and S. Maji. Bilinear CNN models forfine-grained visual recognition. In Proceedings of the IEEE interna-tional conference on computer vision, pp. 1449–1457, 2015.

[23] J. McAuley, C. Targett, Q. Shi, and A. Van Den Hengel. Image-basedrecommendations on styles and substitutes. In Proceedings of the 38thInternational ACM SIGIR Conference on Research and Developmentin Information Retrieval, pp. 43–52, 2015.

[24] T. Mensink and J. Van Gemert. The Rijksmuseum challenge: Museum-centered visual recognition. In Proceedings of International Conferenceon Multimedia Retrieval, pp. 451–454, 2014.

[25] G. Miyakita, S. Arima, M. Yasui, and K. Okawa. Exploring digital cul-tural heritage beyond moocs: Design, use, and efficiency of generousinterfaces. In 2019 IEEE Learning With MOOCS, LWMOOCS 2019,pp. 42–46. Institute of Electrical and Electronics Engineers Inc., 2019.

[26] T. G. Padilla. Collections as data: Implications for enclosure. Collegeand Research Libraries News, 79(6):296, 2018.

[27] D. H. Park, H. K. Kim, I. Y. Choi, and J. K. Kim. A literature reviewand classification of recommender systems research. Expert systemswith applications, 39(11):10059–10072, 2012.

[28] J. C. Platt. Using analytic QP and sparseness to speed training ofsupport vector machines. In Advances in neural information processingsystems, pp. 557–563, 1999.

[29] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma,Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. ImageNetlarge scale visual recognition challenge. International journal of com-puter vision, 115(3):211–252, 2015.

[30] O. Seitsonen. Crowdsourcing cultural heritage: public participationand conflict legacy in Finland. Journal of Community Archaeology &Heritage, 4(2):115–130, 2017.

[31] K. Singhal, K. Raman, and B. ten Cate. Learning multilingual wordembeddings using image-text data. In Proceedings of the Second Work-shop on Shortcomings in Vision and Language, pp. 68–77. Associationfor Computational Linguistics, Minneapolis, Minnesota, June 2019.doi: 10.18653/v1/W19-1807

[32] R. Speakman, M. M. Hall, and D. Walsh. User engagement with gener-ous interfaces for digital cultural heritage. In International Conferenceon Theory and Practice of Digital Libraries, pp. 186–191. Springer,2018.

[33] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Er-han, V. Vanhoucke, and A. Rabinovich. Going deeper with convolu-tions. In Proceedings of the IEEE conference on computer vision andpattern recognition, pp. 1–9, 2015.

[34] J. Tagg. The disciplinary frame: Photographic truths and the captureof meaning. U of Minnesota Press, 2009.

[35] L. Thompson and D. Mimno. Computational cut-ups: The influence ofDada. The Journal of Modern Periodical Studies, 8(2):179–195, 2017.

[36] P. B. Thorat, R. Goudar, and S. Barve. Survey on collaborative filtering,content-based filtering and hybrid recommendation system. Interna-tional Journal of Computer Applications, 110(4):31–36, 2015.

[37] A. Trachtenberg. Reading American Photographs: Images as History-Mathew Brady to Walker Evans. Macmillan, London, England, 1990.

[38] O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. Show and tell: Lessonslearned from the 2015 MS COCO image captioning challenge. IEEEtransactions on pattern analysis and machine intelligence, 39(4):652–663, 2016.

[39] M. Wevers and T. Smits. The visual digital turn: Using neural networksto study historical images. Digital Scholarship in the Humanities, 2019.

[40] M. Whitelaw. Generous interfaces for digital cultural collections. Digi-tal Humanities Quarterly, 9(1), 2015.

[41] Y. Wu, A. Kirillov, F. Massa, W.-Y. Lo, and R. Girshick. Detectron2.https://github.com/facebookresearch/detectron2, 2019.

[42] M. Zimmer. The Twitter archive at the library of congress: Challengesfor information practice and information policy. First Monday, 20(7),2015.