Running head: VIRTUALLY THERE 1

Virtually There:

An Evaluation of the Usability of American Art Museums’ Digital Collections

Carolyn English

17:610:553:90

School of Communication and InformationRutgers University

April 27, 2015

VIRTUALLY THERE 2

Abstract

American art museums display some of the most beautiful works of art on their own walls, but

how do their virtual collections look? In this paper, the usability of four museums’ digital

collections—The Art Institute of Chicago, Museum of Fine Arts, Boston, National Gallery

of Art, and Philadelphia Museum of Art—were reviewed, rated and compared to reveal some

of the most user-friendly features of current digital collections. Specific elements of the user

experience in this case relate to searching and browsing capabilities, metadata, images, social

media, and help resources. Both qualitative and quantitative methods were employed to paint the

most detailed picture of each collection. Efficient faceted searching and detailed images were

some of the author’s favorite features, while poor navigation through search results and a lack of

help resources were some of the most troublesome areas.

VIRTUALLY THERE 3

Virtually There:

An Evaluation of the Usability of American Art Museum Digital Collections

American art museums display some of the most beautiful works of art on their own

walls, but how do their virtual collections look? Museums of the future will still be revered and

visited for the masterpieces on display, but they will also be accessed virtually for detailed

information and images of their entire collections, both on and off display. These virtual

museums will need to satisfy a range of users—from travelers planning visits to those who wish

to reflect on their visits to artists to students and educators. It is critical that they provide a user

experience that satisfies people with a variety of abilities and intentions while also promoting the

museum’s reputation through good design and an efficient interface.

The author was inspired by the Smithsonian Institute’s impressive 3D scanning project

(http://3d.si.edu/), which will provide virtual visitors with as close to an actual physical

experience as possible by intricately capturing its artifacts using 3D technology. Art museums

likely have a long way to go before taking on a project like this, but they face the same future

with the need to develop and promote a strong online presence. It is this prospect that calls for a

review of the current usability of art museums’ digital collections.

In this study, four traditional American art museums’ digital collections were explored—

The Art Institute of Chicago, Museum of Fine Arts, Boston, National Gallery of Art, and

Philadelphia Museum of Art. Several specific factors relating to the user experience were

described, quantified, analyzed and compared to determine strong and weak user experience

features. The author walked through each website examining the overall design, like color

scheme and interface, searching and browsing navigation, metadata, images, social media

presence, and help resources. Certain elements, like number of advanced search fields, were

VIRTUALLY THERE 4

measured, while other features, like quality of images, were subjectively quantified on a 5-point

scale to provide a clear comparison of the databases. Commentary is also included to paint a

picture of the usability of these four digital collections.

Literature Review

There has not been extensive research completed on this exact topic, but there are many

discussions surrounding the concepts of usability, user experience, digital libraries and cultural

heritage collections, social media, and visual resource management.

Before discussing the specifics of the user experience, we should fully understand the

need for it in the art museums of today and especially the future. In a study of use policies on art

museum libraries, Esther Roth-Katz (2012) stated that the benefits are two-fold, “Not only is

there a belief that the public is entitled to access to resources, but that increasing awareness of

these resources can be mobilized as a marketing tool for the museum itself.” But while such

access is expected, “Art museums must consider the implications of allowing open access to

valuable and fragile resources” (Roth-Katz, 2012). Such precious information has never been so

freely available, and museums must adjust to the new infrastructure. Unfortunately, there has not

been much formal research completed on the matter. Roth-Katz calls for more, which should

ultimately benefit both the institutions and the users of their websites, so the author would like to

contribute to the remote access conversation.

It is also important to understand who the users are. Pallas and Economides (2008)

described the diversity of virtual museum visitors in their study of the evaluation of art museum

websites in which they developed a framework to do so, “A wide number of users, from pre-

schoolers to art historians and researchers, may visit digital museums.” Kris Wetterlund’s

research from 2008 focused on bringing the art museum to the classroom. The development of

VIRTUALLY THERE 5

their digital collections has been a welcome resource to educators. “Before the existence of

digital images of works of art from art museums, teachers who wanted to use works of art in the

classroom arranged field trips to the museum or were restricted to slides, overhead

transparencies, or printed reproductions” (Wetterlund, 2008). So if educators are utilizing their

collections, so might the students for assignments or for independent use should they be inspired

by their education. Other users may include people planning a trip or revisiting a work of art,

time period, or style of art from a previous visit.

Now users have been established, what they wish to do online also determines the

requirements of the digital collection. A recent study exploring web searching behavior for

cultural heritage institution visitors (Skov and Ingwersen, 2014) described four motivational

categories of visitors to remote access visitors to the Science Museum in London in 1998:

Gathering information for planning an upcoming visit to the physical museum;

Self-motivated research for specific content information;

Assigned research for specific content information; and

Engage in casual browsing.

Again, the finding is that a digital collection must be comfortable for a novice user to browse,

but rich enough to benefit a serious student or professor in the field.

This growth of virtual visitors might make one question the consequences to physical

visitation. Wetterlund (2008) calms museum professionals’ worries by citing a study performed

by the Minneapolis Institute of Arts that was funded by the Institute of Museum and Library

Services, which “found that 78% of online visitors to their Web site had been to the real museum

as well.” So if anything, a strong virtual presence only encourages visitation or enhances the

overall experience by extending the access beyond the buildings’ walls. “Another study found

VIRTUALLY THERE 6

that most visitors to the museum Web sites are scholars and teachers” (Wetterlund, 2008). Since

many of the visits seem to have an educational focus, the need for a digital collection similar to a

library is likely the experience these users are used to and would expect from a similar

institution, because a museum is quite comparable to a library of art.

Interestingly, just like information literacy for librarians, museum literacy remains an

issue when utilizing their digital collections, especially when it comes to K-12 instruction. This

tells us that any supplemental information, particularly in a narrative form, would greatly

enhance the experience for those who are not well-versed in technical art terms or have a pre-

existing knowledge of the works. Wetterlund (2008) explained that museums have posted more

educational resources on their websites. Seamlessly including supplementary information within

the search database would enhance the user experience for both novices and experts. The need

for museum or information literacy doesn’t just apply to users but museum professionals too.

Paul Marty (2006) conducted interviews with 21 museum professionals to determine what skills

were necessary to enhance their work. He discovered that “there is a growing sense among

museum professionals that the skills taught in Library and Information Science (LIS) programs

are increasingly relevant to their everyday needs” (Marty, 2006). This type of expertise would

directly contribute to the type of information available in digital collections and their design,

which is critical, since “the emergence of this new role has coincided with the growing belief

that museum information resources should be as readily available as information resources are in

most libraries” (Marty, 2006). This is why a review of art museums’ digital collections from the

perspective of a budding information professional is relevant. But while this is simply a review

of the current state of affairs, future direction should also be considered. Collections will only

continue to improve.

VIRTUALLY THERE 7

Eventually, as technology is developed and more readily accessible by a range of

budgets, museums will likely follow the Smithsonian in providing even more interactive and

realistic virtual visit. Patel et al. (2005) described a new project called ARCO, or Augmented

Representation of Cultural Objects:

It is notable that although many museums have now established an online presence on the Internet, currently this presence is almost invariably a 2D one; that is associated Web sites comprise 2D images and textual descriptions. ARCO [Augmented Representation of Cultural Objects] on the other hand recognizes that objects are 3D in nature, that they have a front and back, top and bottom, mass and volume. ARCO seeks to enhance the awareness and experience of cultural objects by providing technologies for creating 3D digital surrogates of artefacts and allowing user to interact with them.

The trend is catching on, so museums must be prepared to update their digital collections

to become even more dynamic and user-friendly.

So, when user experience is mentioned, what exactly does it mean? Hariri and Norouzi

(2011) performed an extensive literature review to determine specific criteria for evaluating

digital libraries, which helped direct the author’s checkpoints. What is interesting is that they

admit that specific identification of such information is lacking. At the same time, “the quality of

user interface design has improved dramatically” (Hariri & Norouzi, 2011). They listed 22

specific factors in order of usage in evaluations, of which 11 matched the purposes of this study

(highlighted in bold font): navigation, searching, design, guidance, error management,

presentation, learnability, user control, consistency, language, feedback, ease of use, match

between system and the real world, customization, user support, user workload, interaction,

compatibility, visibility of system status, user experience, flexibility, and accessibility. Xie

(2006) also provided a detailed list of usability. Not only is there the idea of usability in general,

but interface usability pertains specifically to search and browse, navigation, help features, view

VIRTUALLY THERE 8

and output, and accessibility (Xie, 2006). Clearly, there are many factors that overlap and

contribute to the overall experience.

Such factors also require some context within the concept of usability. Even evaluation

itself requires context, as it can pertain to technical or conceptual ideas. The question could be

posed as whether or not a system performed a certain function, how it performed it, or how its

performance affects its user. This is an idea of a user-centered evaluation versus a system-

centered evaluation. Saracevic (2000) provided two questions to demonstrate the difference. A

user-centered evaluation might ask, “How well does a given interface provide and support

access, searching, navigation, browsing, and interaction with a digital library?”, whereas a

system-centered evaluation might ask, “How well is the collection or information resources

selected, represented, organized, structured, and managed?” He encourages the use of both, and

for this, both quantitative and qualitative functions will be examined so both areas are

incorporated into the overall comparison.

Another layer in the user experience is the user’s point-of-view. Xie (2008) reported on

the user perspective for evaluating user experience and stated that, “Digital libraries are designed

for users to use. However, most of the research on evaluation of digital libraries has applied

criteria from researchers themselves.” While this study is performed by a student of library and

information science, it could also be considered a user’s perspective as someone who is a

museum visitor (albeit, infrequent) and not highly practiced in the area of website evaluation,

which matches a novice user profile. While the evaluation criteria is quite specific, the author

will also note general sentiments towards each collection. For the purposes of formal research,

each carefully selected factor will be explored, but it does not require a professional to

understand them. “The best way to evaluate digital libraries is to actually use them” (Xie, 2008).

VIRTUALLY THERE 9

This study could be considered an inspiration for others to simply use websites and take notes. If

specific criteria are selected, it becomes easier to compare strengths and weaknesses. Ultimately,

each element builds up to an overall opinion, but many times usability is subconsciously decided

when users either like or do not like their experience on a certain website.

Usability specific to visual resources differs from a traditional library because of the

nature of the content. Zhang, Liu, Li, and Zhang (2013) referred to a study by Yee et al. in 2003

about faceted metadata in image searching, “Overall the participants preferred more faceted

metadata display, and evaluated it as more informative, flexible, and easier to use.” Faceted

searching would give specific control to the users to pick and choose select metadata from which

to expand or eliminate search results. Another area of control for users applies to the look of the

interface itself. In another study mentioned by Zhang et al. (2013), in 2005, Chen, Magoulas and

Dimakopoulos “suggested a flexible interface (e.g. switching visual cues, offering successive

options) to accommodate the preferences of users with different cognitive styles.” This would

take a lot of planning and sophistication to create such a dynamic website, but it would

ultimately provide a user experience that’s practically guaranteed to satisfy all users.

Menard and Smithglass (2014) also addressed issues in regards to access to digital

images with the intention of creating bilingual interfaces. Unfortunately, there are many studies

about general behavior, but no specific studies on current systems, so this evaluation should

begin to fill in the gaps. They did note that “Museums offering images online structure the

information almost exclusively according to collections and using traditional descriptive

methods. We suggest this is one reason for a high level of consistency between museum websites

in terms of search functionalities” (Menard & Smithglass, 2014). This study will reveal that

VIRTUALLY THERE 10

while there are basic structures in common with the following four museums, there are many

other subtleties that do affect the ultimate user experience of each collection.

Metadata is also facing changes with multiple sources of information in the advent of

social tagging. This allows users to provide their own metadata in addition to the descriptive

metadata provided by museum professionals. In a study focused on folksonomies in art

museums, the authors concluded that a blend of professionally provided metadata with user-

supplied tags created the best atmosphere for navigating through images (Boston-Clay, Mahoui,

& Jaebker, 2013). To understand what the user-supplied information is, Boston-Clay, Mahoui,

and Jaebker (2013) explained that “tagging is commonly referred to as ‘social tagging’ whereby

typical users describe and classify resources in a shared environment.” While metadata is critical

in terms of providing accurate information about works of art, when it is complemented with

keywords and terminology more broadly understood by the average user, the search process is

enhanced. What will be interesting to note is whether or not these highly prestigious institutions

allow users to interact with their works.

Methodology

Four museums were selected for this study—The Art Institute of Chicago (AIC),

Museum of Fine Arts, Boston (MFA), National Gallery of Art (NGA), and Philadelphia Museum

of Art (PMA). After gathering a list of major art museums’ websites from across the country,

candidates were narrowed down through quick browsing and searching sessions. Those with the

most sophisticated websites and unique features were included. Two other museums were almost

considered in the study—Los Angeles County Museum of Art and the Museum of Modern Art—

but due to the extensiveness of the qualitative selections and the realization that the last two

VIRTUALLY THERE 11

featured more contemporary and modern approaches (to both their collections and their

websites), the four finalists seemed like a suitable group to evaluate together.

While these collections are not being evaluated by their ability to replicate a literal virtual

tour through the museums’ halls, it was important to examine how well they could provide a

digital tour of their collection, relating to information access and easy navigation through the

virtual halls. Online visitors are likely researching information about artists, specific works,

styles, origin, media, or time periods. How well each museum could aid them in these quests is

critical, as they house so much of this information in both their staff members found at the

museums and in their extensive databases.

For this evaluation, the author used a combination of quantitative and qualitative

measures. Specific checkpoints gathered from the literature review were established before

touring each museum’s digital collection. To allow for a user’s point of view, which was also

discussed in the literature review, some factors for evaluation developed somewhat organically.

The author began with a basic outline of elements to explore, such as interface, searching and

browsing functionality, metadata, social media, and image quality and availability. As the tours

ensued, new concepts or specific measures became apparent. Depending on the experience of the

collection or consistency of results across the board, new questions were created or rating criteria

was altered to create the optimal comparison tool.

Criteria fell into two categories: subjective and objective. Objective measures included

counting the number of advanced search fields and social media links, while subjective measures

included the author making a judgment call on ease of use of the interface and quality of help

resources. Quantitative results are presented in charts in the Findings section, along with detailed

VIRTUALLY THERE 12

qualitative descriptions of each usability element, while the exact steps from the walk-throughs

are available in the Appendix.

The initial general outline included basic starting points for each usability category.

Overall design and layout of the interface was considered. For search and browsing capabilities,

basic and advanced search options were to be explored, as well as faceted search navigation

through results. The ability to control how results are sorted and the transparency so that users

can understand how results are sorted was also noted. The number and efficiency of search

refinements, and similarly, the availability and presentation of metadata was assessed. The

number, size and accessibility of images for downloading, as well as the number of social media

links was also important. Finally, the presence of help resources was also examined. These basic

points were the author’s initial guide, and as the virtual tours began, they were refined and

updated so that they would create the most value and insight into the evaluation.

The specific criteria are included below and are categorized into their respective

subjective and objective headings.

Objective

Presence of elements were counted, and their significance is explored in Findings.

Browse Are there clearly labeled collections to browse from main page?collections Yes or No

Advanced How many advanced search fields are available?search fields Number

Sort results How many ways can users sort results?Number

Search How many categories are available to refine search results?refinements Number

Metadata How many metadata categories actively link to similar works (i.e. author)?links Number

VIRTUALLY THERE 13

Create a Can users create a profile and curate their own collections?Collection Yes or No

Social media How many social media links are available to share images?Number

Subjective

Items were ranked on a 1 to 5 scale, as follows:1 = Missing (if applicable), 2 = Poor, 3 = Fair, 4 = Good, 5 = Excellent

Interface The overall ease of use, layout, and color scheme

Navigate The ease of moving through a list of results, which is particularly importantresults when browsing through hundreds of works or multiple pages of results

Metadata The number of metadata fields included, their presentation, and any supplemental information to enhance the user’s knowledge of the specific piece of art

Image quality The number of images, including angles if the artwork is three-dimensional,as well as size, clarity, and ease of downloading

Help The presence of help resources, either on the search page or with clear links toa help page, or contact information specific to the use of the digital collection

Findings

This section includes both quantified evaluations presented in table formats, accompanied

by quantitative evaluations below. For a detailed walk-through of each museum, please refer to

Appendix A: Walk With Me (page 30). Each category section provides an in-depth comparison

of each collection’s features. Each museum had strengths and weaknesses, but keep in mind,

however, that this is all purely from the point of view of the author. Xie (2008) reminds us,

“Users are not the same. Their evaluation of digital libraries is affected by their preferences,

experiences and knowledge structure. Different users have different preferences.” What might be

intuitive to one might not be clear to others, and vice versa. It would likely benefit the study to

VIRTUALLY THERE 14

include others who might not be as comfortable with this concept, but is interested in searching

the collections, to get a feel for their preferences with visual resource collections.

Objective Results

Table 4.1Objective Scores*Usability factors

AIC MFA NGA PMA

Browse collections

Yes Yes Yes No

Advanced search fields

9 0 3 8

Sort results 1 4 3 1Search refinements

5 3 7 0

Metadata links 2 3 3 4Create a collection

Yes No Yes Yes

Social media 2 3 4 3Total 21 14 22 17

*To quantify yes/no results, yes = 1, no = 0.

Browse collections. In general, all of the museums had clear paths to browse their featured

collections, so this was not a major difference maker in overall user experience. It did detract

from the Philadelphia Museum of Art, though, as this option was missing. Their advanced search

fields would allow the user to search for a classification or curatorial department from a

dropdown menu, which in essence is similar to browsing a collection, but it might be less

carefully curated than the collections of other museums. It also did not provide a different

experience than the act of searching. MFA, on the other hand, seemed to have the most focus on

browsing collections with carefully created unique presentations of these tours.

Some museums are more focused on specialized collections and the experience of a

virtual tour, where some are more concerned with powerful searches, enabling users to create

their own tour through easy access to their entire collection. How is this decided? It would be

VIRTUALLY THERE 15

interesting to explore whether the museums have chosen to do this based on their own decision

or through the wants and needs of their users.

Advanced search fields. The presence of an advanced search may or may not affect a user’s

ability to browse or search a museum’s collection. There was a wide range of approaches in

terms of providing an advanced search. AIC offered nine options, while MFA only offered a

basic search. Both offered a number of search refinements to enable users to quickly focus their

results as necessary. Browsing could be sufficient for some. Menard and Smithglass (2014)

explained, “Image retrieval for most cultural organisations is initiated with user keywords or an

advanced search screen displaying drop-down menus of predefined categories directly related to

collections, and browsing is presented with equal importance.” Pallas and Economides (2008)

added, “Of course, an advanced search engine that returns accurate and relevant results is

important.” Perhaps the effectiveness of the information retrieval is more important than

specified search fields.

AIC had nine advanced search fields, the most of all the collections. This is an adequate

amount for basic users and likely adequate for more advanced users of the collection. There

wasn’t anything particular outstanding about the options, but there wasn’t anything particularly

bothersome either. MFA did not offer an advanced search. This seemed a bit unusual, but their

focus on browsing specific collections made the author think this was a conscious decision to

encourage discovery rather than highly focused searches. NGA was considered to have three

advanced search fields, as this was the max number of fields that could be searched together.

They offered six other specific fields that could be searched separately, so this was not counted

towards an advanced search. PMA offered eight advanced search fields; however, the way they

formatted the keyword search in relation to the advanced search options made the search process

VIRTUALLY THERE 16

a little unclear. Even to an experienced user, the presence of two separate Search buttons made it

unclear that both keywords and advanced search options could be searched together.

The option of an elaborate advanced search versus a basic search paired with excellent

search refinements might not make a difference to overall usability as long as there are many

clear options for users to browse or search these expansive collections. The author’s experience

was that more options available make for a better user experience, but it’s up to the museum to

decide when and where they want these choices to be available for their users—before and/or

during the search process.

Sort results. Zhang et al. (2013) “suggested a flexible interface (e.g. switching visual cues,

offering successive options) to accommodate the preferences of users with different cognitive

styles.” An easy way to do this is in simple adjustments like sorting results.

AIC offered no customization in terms of sorting results, and the default setting was

unclear, which might be frustrating for users moving through a high number of results. There

was a small window in which the user could jump between pages, and there was no way to jump

to the end or know how many pages there are without dividing the total number of results by 10

(after counting the number of results per page). They also had no customization in terms of how

many results may be viewed per page. MFA had an excellent design for sorting results. In all,

they provided eight options—relevance, title, date, and artist—and each may be sorted ascending

or descending. It was also clear what the default setting was and that the user may control how

her results are sorted. Again, MFA placed emphasis on discovery through browsing rather than

through exact searches. NGA had three ways in which to sort results—default, chronological,

and title. It was unclear what the default setting was, but it was still advantageous to have two

other choices, although users could not control the direction of the sorting. PMA gave no

VIRTUALLY THERE 17

flexibility in terms of sorting results. This is a big disadvantage when sifting through many

pages, although they are the only museum to allow users to change the number of results per

page, which does lighten the load in this regard.

Search refinements. Search refinements are another way to allow users to sift through results

efficiently. Zhang et al. (2013) referred to another study that “demonstrated some advantages of

faceted metadata display over a standard image retrieval/browsing interface.” The author was

interested in how many choices the user had in narrowing search results and the experience when

using them.

AIC displayed their search refinements on the right side of the page, which is a little

unusual, but this allows for consistency of page layout, and they draw your attention by

displaying choices in maroon hyperlinks, so the user can easily understand their purpose. There

were a fair number of choices, but perhaps a refinement around date or period of time would be

helpful, especially for an artist like Picasso who covers a wide range. MFA only offered three

basic search refinements. Again, they seem to want to create a browsing atmosphere rather than

offer a fine-tuned search engine. The options were still clear, though their Classification pop-up

list is a little awkward to use (see Appendix A for a detailed description of this feature). On the

other hand, it was easy to manipulate the search refinements with their search summary

information presented on the top of the page. NGA had the nicest search refinements with the

most categories—seven—and the ability of the user to control which refinements they see by

expanding or collapsing each category in the left navigation. Styles appeared to be an ineffective

search refinement, though, as some searches resulted in all choices that had “No Style defined.”

Perhaps the museum is updating the metadata for this feature. PMA had no search refinements,

VIRTUALLY THERE 18

so while the advanced search options are plentiful, there is a big lack of control once those

results are presented.

Metadata links. Metadata is critical to users to properly identify a piece of art, but it can also

connect users to similar works of art by including links to searches for works by the same artist,

in the same style, medium, time frame, etc. Dynamically linked metadata can add another

beneficial layer of discovery for users.

AIC had three linked metadata fields: artist, format, and gallery (if item is on display).

This seems to be the average number of linked metadata fields among museums, so this must be

one element of consistency among museums as mentioned in the literature review. MFA followd

these same guidelines. NGA provided three linked metadata categories, but they are slightly

different—artist, collection or gallery, and on-view items. PMA was the only museum to offer

four linked metadata pieces—artist, classification, curatorial department, and country of origin.

Unfortunately, the placement of these links on the bottom of the page in a separate section with

the label “Explore the Collections” did not seem as user-friendly as the other collections which

included the hyperlinks in the primary metadata next to the image.

Create a collection. In order to appeal to online visitors, including both educators, students, and

enthusiasts, an interactive feature such as creating a personalized collection is critical. The author

did not fully explore the functionality of each, but rather the option to do so.

AIC, NGA and PMA were all onboard with this notion. Unsurprisingly, MFA, with its

extensive collections already available, did not appear to offer a way to create a unique user

collection. It seems that they want to control the curation, which can be seen as a positive or

negative depending on who is looking to browse or use the collection for educational purposes.

VIRTUALLY THERE 19

Some might appreciate the work the MFA has put into presenting ready-made examples, while

others might want the opportunity to pick and choose their own pieces.

Social media. Social media is another indicator that a museum is committed to contributing to

the online world in which we live and that they want to encourage this virtual interaction with

the user. Through social media networks, the user can connect with the museum or connect the

museum with other users.

AIC offered just two links to share their works on social media—Facebook and Twitter.

They only included these links on individual results pages, whereas the MFA included their three

social media icons—Facebook, Twitter, and Google+—immediately on their Collections home

page, and they are present on every page moving forward except for Collections Search Results.

NGA had four social media links—Twitter, Facebook, Instagram, and Pinterest—but they were

not prominently displayed for easy use. They are part of the page footer, so they do appear on

every page, but many times people do not scroll all the way down, and the author tends to look to

the top of the page for social media links. The light gray icons did not contrast much with the

light page background, so they were not eye-catching. PMA included three social media links—

Twitter, Facebook, and Tumblr—but they were somewhat hidden, so they could also easily be

missed by a user. The icons were located on the bottom of the right sidebar, so users would have

to move around the page to find them. For a museum that places so much emphasis on social

interaction, this seems like an unsocial place to put them.

VIRTUALLY THERE 20

Subjective Results

Table 4.2Subjective Scores*Usability factors

AIC MFA NGA PMA

Interface 4 4 5 3Navigate results 4 4 3 3Metadata 5 5 5 4Image quality 4 4 4 4.5Help 3 1 5 4Total 20 18 22 18.5

*These results reflect a ranking on a scale of 1 to 5.1 = Missing (if applicable), 2 = Poor, 3 = Fair, 4 = Good, 5 = Excellent

Interface. The design of a usable interface is key to creating a good user experience. Simply put,

“The design of its user interface (e.g. menus, toolbars, buttons, icons, frames) should facilitate

the visitor in his exploration” (Pallas & Economides, 2008). Design, layout, labeling, and system

performance were all considered when rating each collection’s interface.

Overall, the AIC interface, receiving a score of 4 out of 5, had soothing neutral tones

consistent with the museum’s branding, and accents of maroon, which help highlight actionable

items, such as search refinements and hyperlinked metadata. Generally, each step moved

quickly, and the author was not frustrated with load time, system errors or confusing labels. Still,

the lack of exciting or unexpected features kept it from receiving a full score. The author did

prefer AIC's use of different color backgrounds in its left navigation to MFA’s which help

designate which options are collections and which are other pages, such as provenance, as

opposed to MFA’s left navigation where each link looks exactly the same. The MFA interface,

also with a score of 4 out of 5, was straightforward and quite easy to use. Their advanced search

options and refinements were less powerful than in other museums, but they did offer dynamic

sorting options. Overall, this was a comfortable experience, though nothing was particularly

VIRTUALLY THERE 21

notable. NGA had the most enjoyable interface to use, so it received a perfect score. Layout and

labels, which are critical to navigating a website, were intuitive and clearly explained. Even the

style of the fonts aided in understanding every piece of information—category headers were in

small caps, hyperlinks were presented in a contrasting blue color, artwork titles were slightly

larger and italicized, and bonus information was often bolded to bring attention. Additionally,

having search results update quickly as refinements were selected without waiting for the entire

page to refresh made the process seamless and enjoyable. PMA offered some powerful features,

but the author did not care for the initial size of the interface, which was quite overwhelming on

the screen. There were also many features that were functional, but were not initially clear how

to use. Labeling was either lacking or unclear, and placement of certain elements made for a less

usable interface. For instance, the carousel of images at the top of the individual artwork page

first seemed like a unique way to move through search results, when it was actually displaying

multiple angles. In the particular example, the angles were of the bottom of the piece, so it was

not obvious that it was the same work rather than a separate piece (see Appendix A for a more

detailed description of this feature). Additionally, the Social Tagging page, while novel, could

still be reworked for a more user-friendly presentation. Their apparent bigger-is-better ideal

seemed to come through in their uncomfortably long list of social tags. PMA received a 3 out of

5 for these issues.

Navigate results. Navigating results overlaps with the subjective measures of sorting results and

search refinements, but takes into account the user’s experience in doing so.

AIC, scoring a 4 out of 5, offered nine pages of results at a time, so it was fairly easy to

move between results, but the author would have liked to have seen choices to sort the results

and to understand how the results were sorted by default. The author also would have liked to

VIRTUALLY THERE 22

jump to the last page of results and worked backwards but was unable to do so. The author did

appreciate the ability to move between results from the individual artwork page, but the labeling

was not initially clear. This feature could have been missed had the author not been examining

every element on the page for discussion. MFA also scored a 4 out of 5 for navigation. As

mentioned previously, users had a lot of control over sorting results, but not as much with

refining them. To navigate between pages was not the easiest, but they did show an improvement

over AIC by being able to jump to the first or last page of results. The user cannot move between

results from the individual artwork page, though. NGA scored only a 3 for navigating results due

to the inability to move between results efficiently. While the control offered to the user through

effective search refinements and several sorting options was highly appreciated, if the user was

left with many pages of results, it was hard to move more than a couple of pages at a time. The

lengthy list of search results also made scanning each page of results a slightly longer task. PMA

was also lacking in its navigation, scoring only a 3. While it was the only collection to offer

multiple options for displaying results per page, there were no search refinements or sorting

options, as well as no explanation of the default sort setting. It was better at jumping between

pages than some of the other collections, though.

Metadata. The existence and presentation of metadata is critical to these collections, as many

visitors are using these items for educational purposes. Its presentation is critical for users to

identify works without having to search all over the page, and extra information, particularly in a

narrative form, would be welcomed by users with a little or a lot of pre-existing knowledge. As it

might not be necessary for all users, though, its presentation is also key—the ideal situation

would be a clear indication that supplemental information is available without it overwhelming

the rest of the page.

VIRTUALLY THERE 23

AIC, with a perfect score, presented critical metadata in a concise format, and in some

cases, provided supplemental descriptive information, which will be greatly appreciated by

educators or researchers. The user does not have to exert a lot of effort to find this information,

which is another bonus. One feature of metadata that the author would have changed would have

been the hyperlink for the author. If there were multiple creators attached to the work, along with

country of origin and dates, the entire list was highlighted, but it was only one link for the main

creator, so the exact linking value of metadata was unclear. MFA also received a perfect score.

Metadata was fully available and presented clearly. Where available, supplemental information

was provided. The way they linked creator names, which highlighted only the specific name

rather than all information surrounding it, was preferable to AIC’s linking. NGA is the third

museum to receive a perfect score for metadata. They packed a lot of information into critical

spots, like the search results page, without sacrificing functionality of the website. The metadata

was also presented clearly, the hyperlinked metadata was easy to understand, and the

supplemental information was plentiful and displayed nicely. They even include a map of the

museum for easy location of the physical work if it is on display. PMA received a 4 out of 5 for

metadata. While they included just as much information as the others, even including

supplemental information, the placement of linked metadata and narrative text could easily be

missed by the user.

Image quality. The main reason that users would visit these digital collections would be to see

the art. Therefore, the number of images and their size, quality, and availability for download are

all critical to the overall usability of each collection.

AIC received a 4 out of 5 on this measure since not all of their images enlarge, and some

sculptural pieces only had one angle to view. They performed well for downloading, as the

VIRTUALLY THERE 24

author had no problem saving the biggest available image, which was fairly large, to her

computer. MFA also received a 4 for image quality. While images were very large when

available and could be downloaded, it would have been nice to have multiple angles for three-

dimensional pieces. It would be preferable not to see an option to zoom when the zoom function

is unavailable. On the results page, it also felt unnecessary to provide an additional image pieces

when hovering over an individual result, as many of the images hardly changed. It created a

busier atmosphere than necessary as the mouse moved around the page. Also confusing was the

ability to download the image by right-clicking, while there was also a link to license the image

with specific instructions and a 2-3 week processing timeline, so the user was a little unclear as

to which option to follow depending on usage. NGA received an almost perfect score for image

quality. Not all images were available in very large sizes or with the zoom function, which

prevented it from receiving a 5. The zoom feature for artwork that did have it was amazing, and

the level of detail was incredible to see up close. For users who want to save an image, NGA’s

lightbox feature did make it easy for the user to responsibly download it, but the extra steps

required to visit another page to do so might be a nuisance for some. PMA received the highest

score for image quality. In the same vein as their oversized interface, they offered oversized

images, which is a good thing. While a user cannot download by right-clicking, the download

icon made it just as seamless a process. This was the only museum to offer multiple angles and

detailed views of three-dimensional pieces that the author could find. Unfortunately, pieces that

did include multiple angles were not available to download, which is why it received a 4.5, as the

author wanted to give the museum credit for going above and beyond in this area.

Help. What happens when a user has a question? Xie (2008) wrote, “Very few users used Help

and other features when they encountered problems in the search process.” She explained that

VIRTUALLY THERE 25

help resources had the biggest range of opinions in terms of usability, which helps explain the

disparity of results for this element. It is the author’s opinion that help features are integral to the

usability of each digital collection. One would hope that users might not need help if the

collections are perfectly designed, but given the range of experience of potential visitors, it is

never a bad thing to include help resources. Perhaps the lack of contact information in many

cases is due to the lack of staff time to assist online users.

AIC received a 3 out of 5. They did offer a few key help pointers below their advanced

search fields, but it was unclear if there were any other resources available on the website, and

they did not include any contact information within their digital collections for additional

assistance. MFA received a 1 for help. There did not appear to be any help resources displayed

or linked while moving through the collections pages. NGA received a perfect score for its help.

It offered quick tips in key areas, like the search page, while also including links to more help

guides. There were a number of topics covered in their FAQ section, but the NGA even offered

contact information for people who needed more. Their website was so easy to use, so users

might not need a lot of help, but the diversity of users could present a range of questions not only

relating to use of the database. Either way, help was provided and further assistance was

available. PMA received a strong score for its help resources, a 4 out of 5. It provided an in-

depth list of search tips in an expandable format, which the author felt was a very user-friendly

presentation. They even included a special help section on accented characters for both PC and

Mac users, which was extremely unique.

VIRTUALLY THERE 26

Overall Usability Results

Table 4.3Overall Usability Scores

AIC MFA NGA PMATotal 41 32 44 35.5

Generally, collections that scored the highest on objective measures also scored the

highest on subjective measures, thus resulting in the highest overall usability scores. That being

said, just because a collection might feature many advanced search fields does not translate to the

collection’s performance as a whole or reflect its visual appeal. This is why it was important to

perform qualitative measures in addition to quantitative ones. According to this study, NGA was

the most usable digital collection, followed closely by AIC, while MFA received the lowest

score for usability. NGA’s interface was extremely user-friendly and intuitive but also included

the most help resources. Though they did not offer an extremely complicated advanced search

interface or efficient navigation through results, they excelled in most other categories. AIC was

another strong contender with a slick interface and high scores all around. MFA took a different

approach to their digital collections by providing curated collections for a user to browse and a

less exacting search experience. While it resulted in a lower score for the purposes of this study,

this type of experience might be perfect for some users. Finally, PMA had some outstanding

features, such as excellent images with multiple angles, well-placed help resources and an

extensive advanced search interface, but the overall design and layout proved more difficult to

use and the oversized scale was overwhelming.

Conclusions

The lack of research surrounding the current state of digital collections for art museums is

slim, but relatively straightforward assessments such as these can easily create a discussion of

best practices and areas for improvement. Roth-Katz (2012) mentioned one big hurdle in

VIRTUALLY THERE 27

determining requirements for an excellent digital collection, “Andrew J. Pekarik [from the Office

of Policy and Analysis at the Smithsonian Institution] argues that museums as a field have not

reached consensus on the purpose of their websites.” Without a purpose, it can be difficult to

assess good functionality. At this point, usability can only be assessed in terms of what we know

about current visitation behavior. These users are likely previous or future visitors to the museum

itself, so it’s important to create a strong overall experience with the institution. It is also

important to meet the needs of those who are unable to visit but still want the reliable and

accurate information provided by the museum. Each museum’s digital collection has many

admirable features and seems to be meeting the needs of many users, but there were many less

than perfect scores, so there is room for improvement and innovation in terms of presenting

information and providing access to it.

Though museums have not always had to share so much information publicly, virtual

access should be encouraged. If people are searching for information and images responsibly, we

should honor this practice, rather than forcing them to resort to all-purpose search engines for

unreliable information and illegally reproduced images. It also benefits the museum by

respecting their assets, responsibly promoting their art and services, and proving their

commitment to the community and to the future that blends the physical world with the virtual

one. For these reasons, it seems as though museums are virtually there in providing dynamic and

informative online visits, and for this, visitors can feel like they’re virtually there, enjoying the

art of the digital collections.

VIRTUALLY THERE 28

References

Boston-Clay, C., Mahoui, M., & Jaebker, K. (2013). Exploring the usability of folksonomies in

the art museum community. Proceedings of the 13th ACM/IEEE-CS Joint Conference on

Digital Libraries (JCDL '13), pp. 387-388. http://doi.acm.org/10.1145/2467696.2467760

Hariri, N., & Norouzi, Y. (2011). Determining evaluation criteria for digital libraries' user

interface: A review. Electronic Library, 29(5), 698-722.

Marty, P. F. (2006). Meeting user needs in the modern museum: Profiles of the new museum

information professional. Library & Information Science Research, 28(1), 128-144.

http://dx.doi.org/10.1016/j.lisr.2005.11.006

Menard E., & Smithglass, M. (2014). Digital image access: An exploration of the best practices

of online resources. Library Hi Tech, 32(1), 98-119. http://dx.doi.org/10.1108/LHT-05-

2013-0064

Pallas, J., & Economides, A. A. (2008). Evaluation of art museums' web sites worldwide.

Information Services & Use, 28(1), 45-57. http://dx.doi.org/10.3233/ISU-2008-0554

Patel, M., White, M., Mourkossis, N., Walczak, K., Wojciechowski, R., & Chmielewski, J.

(2005). Metadata requirements for digital museum environments. International Journal

on Digital Libraries, 5(3), 179-192. http://dx.doi.org/10.1007/s00799-004-0104-x

VIRTUALLY THERE 29

Roth-Katz, E. (2012). Access and availability: A study of use policies on art museum library

websites. Art Documentation: Journal of the Art Libraries Society of North America,

31(1), 123-140.

Saracevic, T. (2000). Digital library evaluation: Toward evolution of concepts. Library Trends,

49 (2), 350-369. Special issue on Evaluation of Digital Libraries.

Skov, M., & Ingwersen, P. (2014). Museum web search behavior of special interest visitors.

Library & Information Science Research, 36, 91-98.

Xie, H. I. (2006). Users' evaluation of digital libraries (DLs): Their uses, their criteria, and their

assessment. Library & Information Science Research, 28(1), 433-452.

http://dx.doi.org/10.1016/j.lisr.2006.06.002

Xie, H. I. (2008). Users’ evaluation of digital libraries (DLs): Their uses, their criteria, and their

assessment. Information Processing & Management, 44 (3), 1346-1373.

Wetterlund, K. (2008). Flipping the field trip: Bringing the art museum to the classroom. Theory

Into Practice, 47(2), 110-117. http://dx.doi.org/10.1080/00405840801992298

Zhang, X., Liu, J., Li, Y., & Zhang, Y. (2009). How usable are operational digital libraries: A

usability evaluation of system interactions. EICS '09 - Proceedings of the ACM SIGCHI

Symposium on Engineering Interactive Computing Systems, 177-186.

VIRTUALLY THERE 30

Appendix

Walk with Me

The Art Institute of Chicago

The Art Institute of Chicago (AIC) Collections page has a strong emphasis on browsing

art by collection rather than searching for specific works, but this is a logical choice given the

title of the section. The color scheme is muted gray, tan and white, and the layout is grid-like,

mirroring the top navigation, which is represented in blocks, in this case, mirroring the

museum’s logo.

You may browse collections in two ways: by selecting a representative image in a 15-

image grid at the top of the page, whose exact topic is revealed upon hovering over the image—

and whose functionality I did not realize until I moved my mouse over them. Or you may select

one of the same options from the left navigation.

There are four blocks underneath the collection images: Collection Updates, Quick

search, Interact with the Collections, and About the Collections. Collection Update brings you to

the latest updates. The search section includes a quick search bar with a link to an Advanced

search option. Interact with the Collections allows you to create your own gallery. About the

Collections features a concise description of the page’s purpose.

The left navigation features quite a few options in which to enter the collections: Explore

Themes, Browse A-Z, Search Online Collection, or Search Interpretive Resources, which are

highlighted in a contrasting taupe to the gray page background. Underneath these options are the

15 collection titles, in the gray background color, seemingly to bring more attention to surround

left navigation options. Below the 15 collections are Conservation, Provenance Research Project,

and Books, which I will not explore in this study.

Explore Themes brings you to a list of eight selected themes, again with a representative

image, but this time shown to the right a short paragraph description about each so you can

informatively choose your path. Browse A-Z refers to artist name, which, to the author, was not

intuitive. There is an alphabet grid to begin exploring, but they also provide their search options

in case you decide to change your approach. Search Online Collections brings you to their

advanced search interface. The following free-text fields are present: Keyword, Artist or Culture,

Title, and Accession Number. There are two dropdowns where you may select one choice:

Collection Category and Type. Finally, you may check all or one of the following three choices:

VIRTUALLY THERE 31

On Display, Has Image, and Has Interpretive Resource. Below the search fields, there are five

bullet points with five search tips.

I performed a simple search and used one of their suggestions from the help section: the

artist Picasso. This brings me to the Search Collection Results page, which includes the quick

search bar at the top, with the option to expand and display the advanced search fields if I wished

to change my search immediately. Underneath, I have the message: Displaying records 1 – 10 of

359.

Search refinements are available on the right side of the page, which is an unusual option,

but they are prominently displayed in maroon-colored hyperlinks with “Refine Search” at the

top, so it is obvious what their purpose is. There are five categories in which I can refine my

results, and each refinement includes a number in parentheses indicating how many results I will

get for each choice: On Display or Not On Display, Show only artworks with: Images or

Interpretive Resource, Artist or Culture (Picasso, Pablo being the only choice), Collection

Category, and Object Type. Collection Category includes three categories, but each has a “[+]”

beside it, a clear indication that there are subcategories available to select. Sub-collections

immediately appear when [+] is clicked. When a refinement is selected, the page refreshes with

updated results. You may remove selections by clicking on the maroon “[x]” next to the

refinement element which remains in the navigation, but which is not hyperlinked anymore for

selection.

Again, results are presented in a grid-like display—five rows of two images that expand

based on the length of the descriptive metadata. A small image is displayed in each grid with a

bit of metadata about each record—artist (country and birth/date dates), title, and date, and at the

bottom, I am presented with the choice to Add to my collection (which you have to create an

account to do). Below the results are Pages 1, 2, 3, up to 9, of my results, which I can select

directly or I can use the right arrow to go to the next page. There is no way to jump to the end of

the results, and at most you may move up or down within nine surrounding pages of results.

There is no option to sort my results, and it is unclear what the default sort setting is. You may

go to an individual result page by clicking on the maroon hyperlinked title or by clicking on the

image.

The individual result page, About This Artwork, includes a slightly larger image than

before, again on the top right of the page’s primary content. The metadata includes artist name

VIRTUALLY THERE 32

with country of origin and birth/death dates (in a maroon hyperlink), title, date, format, extent,

and rights information. Underneath there are maroon hyperlinks to its collection and specific

gallery. Below the metadata is a two-paragraph description and history of the work. Below that,

there is a link to Exhibition, Publication and Ownership Histories. Again, I am presented with the

option to add the work to my collection. And finally, there are two more blocks of related

content: Browse Related (which includes collections and styles of art) and related Interpretive

Resources. The image is clickable and expands so that it fills up my entire computer screen as an

overlay with a block of quick metadata at the top, including artist, artist information, title, date,

and copyright information. I am able to right click and save the JPG file to my computer. Back

on the result page, there are four icons underneath the image, which include a description when

hovered over, though they are clear already: Email to a friend, Print this page, Share this artwork

on Facebook, and Share this artwork on Twitter. Below the icons, there are three more choices in

text: View this enlargement, Add to my collection, and Shop for related gifts. To go back to my

search results, I can either click Browse selections, which is in a separate box above the page

content, or I can browse to the previous or following result by clicking on a left or right arrow

next to this, which is a really nice feature.

Museum of Fine Arts, Boston

The Museum of Fine Arts, Boston (MFA) digital collection appears to be a browsing-

focused database. The color scheme is similar to AIC—in this case, it is a combination of soft

grays with red accents, which help direct the user to actionable navigation throughout the

collections. The Collections page features a list of 11 collections down the middle of the page,

with a small representative image on the left, a collection title in red and an abbreviated

description (indicated by a cut off sentence and ellipsis at the end of each paragraph). The left

navigation features a quick search bar at the top, and links to Featured Galleries and the 11

collections, followed by Provenance, Conservation and Collections Management, Libraries and

Archives, Publication and MFA Images. There is a sidebar on the right highlighting a featured

gallery in the same format as the main collections—one image with the collection title and an

abbreviated description. Users may also opt to view this gallery or More Featured Galleries. On

the top right of the page, round icons for easy sharing of artwork are already displayed for email,

Facebook, Twitter and Google+, so users are immediately aware that images can be shared

socially.

VIRTUALLY THERE 33

When you go straight to a collection, you are presented with a larger version of the image

from the Collections page with a lengthy description of the collection. This time, the right

sidebar highlights a related exhibition. Below the description is a red hyperlink leading you to

provenance research. The next section of content below this includes a manually-controlled

carousel of collection highlights with two featured pieces of art and their titles underneath, which

you may click on to bring you to a new page featuring a much larger version of the image with a

description and metadata (when available), along with the same images from the carousel. Back

on the collection page and below the carousel is a list of Collection Tours links. These will bring

you to the same type of image carousel, this time with the specialized tour created by the MFA.

The design of the collection tour differs from the main pages, which gives the feeling of a

special experience. This time, the content is presented on a black background, which really

emphasizes the art and gives the feeling of stepping into a new gallery with painted walls. The

image is large and can be downloaded by right-clicking on it. When you hover over the image, a

few options pop-up on the bottom, including License this Image and Zoom. Both are in red text,

which indicates that they are active links. License This Image brings you to a separate page with

specific instructions on image reuse and how to request permission. I was unable to zoom on the

particular image used for this study, so it would be preferable that it were not presented as an

option, but I assume it is the default display. Metadata is presented on the right side, including

the title, date, artist (another red hyperlink that brings you to a search for that artist with all

his/her works) and artist information, and description. Below this initial information are several

elements that can be expanded or condensed, as indicated by the up arrows next to the section

headers, Description, Details and Multimedia. Description includes a few paragraphs about the

artist and specific work. Details includes dimensions, medium, classification, accession number,

collection and display status. Similar to the artist feature, this item’s classification (paintings)

and collection (Europe) can be clicked on to bring you to all works with this features. At the

bottom of the Details section is another link for MORE INFO, which brings you to the individual

Artwork page, with a large image and all its metadata listed underneath with the same hyperlinks

as before. This page resembles the standard color palette of grays and red accents.

It does not appear that MFA offers an advanced search option, which is unusual. I

performed a simple keyword search. On the Collections Search results page, the left navigation is

replaced with search refinements. The search bar is again at the top, and it displays my search

VIRTUALLY THERE 34

terms. There are three search refinements under the heading “Refine Your Search By”:

Collection, Classification, and On View. Again, MFA is very collection-focused. Collection

offers a clickable list with the number of results for each choice. You may select one at a time,

and the page refreshes as you do. Classification displays a bar with “Select options”, and when

you click on it, a scrollable list pops up with checkboxes next to each choice allowing you to

select more than one option,. This returns items that feature any of these items, rather than every

option. I understand why they have this as a pop-up as the list is less structured and longer than

the categories, so it would make the left navigation longer than the search results, which is messy

and not user-friendly. The number of hits for each option is not included. You must click submit

underneath to apply Classification refinements. Finally, you may select No or Yes for On View,

and these choices also display number of matches.

The search results are displayed with four sorting options at the top. The default setting,

Relevance, is indicated by its gray text, while Title, Date, and Artist are presented in red text.

There is an arrow next to the selected sort option indicating which direction the results are

sorted, and you may change the direction by clicking on it. This is a nice feature. Below the sort

options, a wide gray box tells you your number of results and repeats your exact search options,

including your provided keywords and any refinements you may have added. You may remove

refinements by clicking on the red x to the left of each choice, so you have a lot of control here to

manipulate your results. Results are presented in a 3 x 3 grid, so you have 9 results per page, and

below them, you may navigate through results in a few ways. Pages 1 through 9 are clickable,

and if you want to move beyond those nine pages, you may go to the last choice and move

forward from there, but it does not bring you to the next set of 9 pages. You may also click

NEXT to go to the next page, or LAST, which brings you to the last page of results, so you can

move somewhat swiftly through your results if you have a lot.

Each result is represented by a medium-sized image, where available (if not, a gray box

with the MFA logo is the default placeholder). Underneath the image is the title in a red

hyperlink, the artist (if applicable), date and accession number. Each element except for the

creator is clickable, although date and accession number are not displayed in red—when hovered

over, they display a red underline. Each brings you to the individual result page. When you hover

over the image, it displays another size image (more zoomed out than the display image, which

VIRTUALLY THERE 35

sometimes is a big change, but some images look basically the same) or angle, if available. The

artwork page is the same as described above.

National Gallery of Art

The National Gallery of Art (NGA) has a very neutral palette of white and gray with navy

accents and a clean but traditional website, which matches the tone of the institution. NGA

seems to strike a balance between offering users the choice to browse or search the collection

upon entering to the Collection home page. The main content of the page sits on the left with a 3

x 3 grid of tall rectangles, each with a title, small image, description, and an actionable link, like

View All or More, in cases where there is a link to view only what is On View (a blue hyperlink

located directly underneath the description). The featured collections on this page are mainly

divided by genre—Paintings, Sculptures, Prints, Drawings, Photographs, Decorative Arts, and

Media Arts—in addition to Highlights, Search the Collection, Artists and Recent Acquisitions.

The right sidebar on the page is what I consider to be the traditional left navigation menu,

including Search the Collection, Recent Acquisitions, Provenance Research, Tours and Guides,

Building Maps, Support Us, National Lending Service, and FAQs. Each section has a small “+”

icon to the right of each header, indicating that you can expand or contract for further

information on the subject matter or for links. This is also indicated by the Search section, which

is already expanded by default and includes a brief preview of the searching parameters available

to users.

Upon visiting a collection, in this case Highlights, you arrive on a new page with a 9 x 5

grid of small square- and circle-shaped images, which I like since it allows you to get a feel for

the collection quickly without having to scroll down the page or through the collection only after

you enter it. There is no text, which really highlights the art. When you hover over an image, a

box pops up with a small, but full-size view of the image, author, title, date, viewing status, and

the collection to which it belongs. At the bottom of the page it says “43 objects arranged

chronologically,” which a nice piece of information to include. On the bottom right, there are

three icons. The first, which has a dark gray background indicating this is the current setting, is a

mini grid, so you can tell you are able to change your view. The other two options appear to be a

list view with small images and descriptions or a slideshow view. This is a powerful tool

presented in a clear and concise manner. Finally, on the page footer, there are four social media

links—Twitter, Facebook, Instagram, and Pinterest.

VIRTUALLY THERE 36

Clicking on an image brings you to a new page about the individual artwork. This

includes a larger image on the left, which is featured within a prominent content area with a

slightly darker background at the top of the page. Once again, key metadata is included for

immediate reference, including the artist, artist information, title, date, format, extensions in both

metric and standard measurements (overall and framed), rights information, accession number

(though it is not labeled), and on-view status. The artist, collection and on-view status (only if

the item is on view) are all blue hyperlinks. Clicking on the artist name, collection or on-view

status brings you to a new search results page, executed by the piece of metadata selected.

Clicking on On View brings you to a new page with a simple map of the gallery, and the room in

which this work of art is displayed is highlighted in blue. The specific room information is

clearly labeled at the top, and the same metadata and small image are displayed again underneath

the map. The map is also clickable by room. Embedded at the bottom of the page is a list of

search results for other works on display in the same room. You may click on any other room,

and the header changes to that room title, and the search results update. All of this is extremely

fast, and it is a wonderful feature for both people who might wish to visit the museum in the

future and people who are reflecting on their previous visit.

Directly below this list are four icons paired with text descriptions—zoom, download,

share, and add to favorites. Zoom brings up an overlay of the image with a black background

where you can zoom in to extreme detail. In the upper left corner of the page, a round icon with

IIP displays information about the image viewer you click on it, and it also displays a question

mark and “click for help” when you hover over it. This information is also displayed if you right-

click on the image. After testing a few different works of art, the zoom doesn’t work with each,

but you can zoom in to extreme detail on some. Since you can’t right-click on this image, the

download option allows you to save the image.

Clicking on download brings you to their lightbox page, as indicated by the page footer,

which reads: “Lightbox: My First Lightbox (0)” with a dropdown that would display each

lightbox if you had created any previously. A search has been executed by accession number to

give you the exact piece of art you were just viewing. It is a small image with artist, title, date

and classification underneath, as well as four icons. When you hover, the image is highlighted in

a red border and basic artwork information appears. Hovering over each icon underneath the

image shows that the options at this point are: Add to front of lightbox, Click to view detailed

VIRTUALLY THERE 37

object information and preview image (if available), Click to download a high-resolution image

suitable for publication use, and Quick Download – Click to download an image suitable for

presentation and screen use (1200 pixels). If you choose to download a high-resolution image,

you are prompted to log in, so this is their way of ensuring people are properly downloading

these images. Clicking on more information opens a small window with a slightly larger image, a

quick reference of appropriate metadata on the left hand side (artist, artist info, title, date,

medium, classification, dimensions, credit, accession number, digitization, and image use)

underneath the following options: Preview (Image Only), Add to front of lightbox, Add to back

of lightbox, Print preview with details, and Download lecture image. Again, they are providing a

lot of options and there are a lot of steps, but I think it enhances the experience as the steps are

intuitive, the labels are clear, and the page functions well. At the bottom of the lightbox pop-up,

you can click on the link to View this object record on nga.gov, which brings you to a search

results page in a new tab, retrieved by accession number again.

Back on the individual record page, share displays a small pop-up to e-mail, Tweet, or

Share (on Facebook). In the second section of the page content, there is a small left navigation

menu in the same darker gray background, including Overview, Explore This Work, Inscription,

Provenance, Exhibition History, Bibliography, Technical Summary, and Related Content. The

default display is Explore This Work, which included a lengthy description. What’s nice is that

all of the major identification for the piece is readily available at the top, but there is extensive

additional information on the page, presented in a very usable manner.

Other specific collections have a slightly different design. For instance, Paintings, which

includes multiple sub-collections based on nationality and timespan, offer a landing page with a

featured work of art at the top. There is a left sidebar (in the same format as the right sidebar on

the main Collections page) with the Paintings section expanded, displaying all sub-categories,

while other media are available to visit underneath (like sculpture and prints), and may be

expanded to visit there instead. There is another section of content down the middle of the page

with a smaller featured image, followed by a lengthy history of the collections. The right sidebar

includes several Related Features (expanded by default), Provenance Research, and FAQs (both

collapsed). The FAQs link brings you to a new page with a diverse array of 20 questions about

artwork and an option to e-mail NGA by clicking on “I still have a question.” Clicking on one of

the Paintings collections brings you to another landing page in the same format with a history of

VIRTUALLY THERE 38

the specific collection. A prominent link at the top of the description allows you to “View all

items in [the collection]”. This simply brings you to a search results page for the search

parameters related to the selected collection.

NGA’s Search the Collection page offers what I will call a semi-advanced search. They

have eight free-text fields in which you may enter your own keywords, but you cannot search

them collectively, as indicated by separate Search buttons next to each field. This is comparable

to an enhanced basic search. NGA includes a couple of suggestions for searches in the right

sidebar—by searching only items that are on view now, or works by category—drawings,

paintings, photographs, prints, and sculptures. Specifically, you may search a combination of

artist name, key words in title, and check yes or no for searching for only objects that have

images available. Below, in a section called “More ways to search,” you may search individually

for key words in object information, credit line, provenance name, accession number, exhibition

history, or catalogue raisonné. Each choice has a short explanation to guide the user in her

selection. In general, these seem to offer much more targeted searches.

As you begin to type in the main search fields, artist name or key words in title, matching

options are presented in a pop-up form, so that you may click on it for immediate fill-in. If there

are more than 10 matches, there is a line at the bottom, More Options…, which brings up the

next 10 matches in the list. As you go further down, “Previous Options…” appears at the top to

return to the previous 10 results. This is an unexpected and powerful feature. You may also opt

to use your own search term by clicking outside of the pop-up to ignore the choices.

Now on the Collection Results page, the content is split into two almost equal sized

columns. The search refinements are clearly labeled on the left sidebar with the header, Filter

results by:. There are seven categories with a variety of refining methods: Works on View,

Online Images, Medium, Nationality, Online Editions, Time Span, and Styles. Most choices are

a checkbox, and each refinement choice includes the number of matching results. Beside each

refinement header is a minus sign that indicates you may collapse (and a plus to expand) each

choice, so you have a lot of control over what you see on the page. Works on view and online

images each have one option, to select only items that are on view or that have images online.

Medium and nationality offer multiple results, and underneath the headers are a blue text feature

to Clear All, so you don’t have to manually delete multiple choices, although you can uncheck

them manually. As you select options, the results update quickly without a whole page refresh, as

VIRTUALLY THERE 39

do the remaining matching search refinement options. Time span is a preset dropdown, which

defaults to Before 1300 until After 2000. There are 18 dates available to create your own time

span on either end. Finally, Styles is interesting. In my current search, I can only refine by one

option: “No Style defined,” but I don’t see any styles. Perhaps this metadata is still in progress.

As each refinement is selected or removed, on the right side of the page above the search results

in a prominent gray box, the search terms are updated, which shows you your search parameters

as Filters:. There is a round X icon next to each, so you can also adjust your search here.

Directly below this, you have the choice to sort works by: default, chronological or title.

Below that, the number of results is displayed, and it is clear that there are 20 per page (e.g. 1-20

of 300 results). Just to the right of this are the pages of results. Only the first two are displayed

with an ellipsis, and you must click on next> to move forward in the results, so it isn’t easy to

jump around between pages. As you advance in the pages, you may go back to the first page or

the pages directly preceding or following your current page (e.g. <prev, 1 … 5 6 7 … next>).

The results page is displayed as a long list, which requires you to scroll quite a bit to see

everything, but you also get to see 20 results at a time, which is an improvement over other

collections. Each result is separated by a light gray dividing line. There is a small image on the

right side, if available, and on the left there is a short list of essential metadata, including the

heading “Work of Art,” artist and artist information (in blue, indicating a hyperlink), title (in a

blue, larger italicized font), date, format, extent, collection (in blue) and accession number, and

its on-view status (only On View is in blue).

Clicking on the artist name brings you to a new page with a search for all works by this

artist. At the top of the page is the artist’s name, and the line below includes nationality and birth

and death dates. Between this and the search results (which have been pushed to a lower place on

the page) is a small horizontal navigation menu, including biography, works of art, and related

content. I realize Works of Art is the default display. When you click on Biography, the page

content updates to a detailed history of the artist. Related Content lists any other features on the

website that mention this particular artist. It is noted that not every artist has all of these features.

Clicking on the title of a piece of art brings you to the individual artwork page, which was

previously described.

Philadelphia Museum of Art

VIRTUALLY THERE 40

My initial impression of the Philadelphia Museum of Art (PMA) interface is that it is

large. Images and text are quite overwhelming, and the immediately displayed advanced search

options are quite extensive. There is no landing page for their digital collections—users must go

directly to Search Collections. There is a large horizontal image across the top of the page, and

users may visit its individual page by clicking on the “i” icon in the bottom right corner of the

image. PMA emphasizes the notion of being big in the first sentence of the search page, by

stating, “94,855 objects from the Museum’s collection of over 227,000 are available in the online

collections database.” They want to make it clear just how many items are in their digital

collection, which is a lot. Interestingly, this is the first outright statement of the size of the digital

collection and the entire collection of the four museums reviewed.

The PMA website features a white background with gray and black text and light blue

hyperlinks. This allows the artwork to pop, much like the other collections we’ve seen. There are

two layers to the search page, as indicated by the tab-shaped outline around the content header

“Search”, with a second page devoted to “Social Tagging”, another unique feature to the four

collections. Again, PMA is forthcoming with information to its users, as it includes links to

search help and examples in the introductory text to aid users in their search requests. Clicking

on “search help” updates the content in the main section of the page by expanding a hidden

section below the search bar with search tips and examples. After scrolling down the page, the

advanced search options have just been moved down rather than replaced by the search tips. I

discovered that clicking on Search Help again, in either the introductory paragraph or the

hyperlink below the search bar, collapses the information, if you choose not to have this much on

the page at once.

There is a right sidebar with more navigational options. At the top is a large museum

logo, which brings you back to the homepage, and a mini navigation menu with five pages. The

first, Search Collections, is highlighted in gray, indicating “you are here,” followed by My

Gallery, New Acquisitions, Audio Tours, and ART 24/7. New Acquisitions provides narrative

features about recent news from the museum. Audio Tours includes lists of all the available self-

guided audio tours, which require users to subscribe to the museum’s free podcasts on iTunes.

ART 24/7 explains the process of creating this digital collection, which provides this 24/7 access

to their works. They give a background and encourage people to contribute to the extensive

project. The next section includes links to Support ART 24/7 and Guides to the Museum & its

VIRTUALLY THERE 41

Collections. Then there is a prominent button to Purchase Admission Online. Finally, there is a

clickable image of baked goods with “Dining” overlaid on top. This is the first time I’ve seen a

collection with an emphasis on converting virtual visitors to actual visitors. PMA also

encourages social interaction through their user tagging feature.

Before we move on to the search experience, let’s explore the social tags. First, the

museum supplies a brief overview of social tagging. Below this explanation, they offer the

option to browse by selecting any one of the long list of user tags displayed below. The larger the

word, the more often the tag was used. The list, which is a large run-on sentence of words

separated by more than a traditional space so that users may identify breaks between choices.

The tags also change color as you hover—from light blue to a brighter blue—so you can also

check for words versus strings of words this way. It took me 13 scrolls of my mouse to reach the

bottom of the page, so it is an extensive list. It even includes a number of Chinese symbols.

While the idea is great, and it’s nice to see how much user tagging has happened, perhaps a

better organized presentation of words would be easier to scan. It might be up to the user to

search the page for a social tag in mind.

Back on the Search page, there is a basic search bar at the top of the Search section,

labeled with “Keywords”. There are Search and Reset buttons after the search bar and links to

Search Help and Typing Accented Characters below, which is presented in the same way that

search help is—the section expands directly below the search bar. There are seven advanced

search options which appear to a separate searched from just keywords, as indicated by the

separate Search and Reset buttons below the search fields, but after testing, the user may enter

both a keyword and advanced search terms and search for them all together by using either

search button. The choices are split into two types of refinements: Artist/Maker, Classification,

Country of Work, and Curatorial, in which you select one choice from a dropdown menu, and

Provenance, Audio, and On View, in which you may select only items that have each option.

Artist/Maker is unique in that it appears to offer two search options, but really the first choice

(Artist/Maker beginning with: A, B, C, etc.) controls the dropdown display for Artist/Maker. The

user must select the appropriate letter to view resulting artists, but selecting a letter alone without

selecting from the dropdown does not provide results from all artists matching that letter. Artists

are displayed in last name, first name when applicable; however, commas are missing, so it can

be confusing to read the list.

VIRTUALLY THERE 42

At the top of the search results page, the number of results are displayed. On the next

line, exact search terms are repeated with “Phrase:” preceding them. The text that follows

encourages users to “Create, organize, and share your own galleries with My Museum.” My

Museum is a blue hyperlink. The next line of text tells me that, in this case, I am “Viewing [1-18

of 83], and I have the option to update my results per page via a dropdown from the default 18 to

36 or 54. In between these options and above the search results, I see that there are 5 pages of

results, and I can go directly to any one of them. If there are more than 10 pages, the first 10 are

listed, followed by an ellipses and the last page. There is an option to jump to the next 10 pages.

This is the only way to jump forward. Each set of 10 pages is then displayed, no matter whether

you are at the beginning of the 10 or the end. Again, this is different than other collections. As

you move to the middle of the pages, both the first and last pages are displayed, with options to

move back to the previous 10 pages or the next 10 pages. This is not perfect, but much preferable

to page navigation in other collections. The results are displayed as medium-sized images in a

3x6 grid (since I have left it at 18 results per page). Each image has a light gray border that

changes to bright blue as the image is hovered over, so I know it is clickable. Underneath each

image, the title and artist (or closest information relating to artist if unknown, such as country of

origin) are listed. The title is also in blue, so I can click either the image or the title to go to the

individual work page. If the title includes the keyword that was searched for, it is in bold. This is

a nice feature. Unfortunately, though, there are no search refinements at this point to narrow the

results, nor any way to sort results or understand what the default sort option is.

On the individual work of art page, there is a carousel of four thumbnail images

displaying multiple images of the work. The author thought this was a search result browsing

feature, but upon further inspection, realized that these were simply detailed images of other

angles of the work. Like the results page, each image’s border changes to blue when hovered

over, so the user may go directly to one of the images. Arrows on the left and right sides of the

images allow the user to go to the previous or following set of images if there are more than four.

It is a permanently revolving carousel, so the user will not get stuck on the first or last image.

Underneath, there is a very large image on the left side of the page. There are two icons in the

upper corners. On the left corner are four arrows pointing to each corner, indicating that I can

expand the image even more. This expands the image so that it takes up the entire width of the

main content on the page (the right sidebar remains the same as the initial collection search

VIRTUALLY THERE 43

page), and the only metadata remaining is the title (and alternates, if applicable) underneath the

image. The icon then inverts the arrows to indicate that it will collapse if I click on it again. The

icon in the top right corner is an arrow pointing down to an open box, which indicates the

download image (and says “Download this image” when hovered over). Though I could right-

click on both size images, the download resulted in an empty space .gif file. The downloaded

image, through the download icon, appears to be the same size as the initial image on the

individual result page (a large, rather than extra-large image).

The metadata along the right side includes title, artist and artist information, which is

separated by a light gray horizontal line from the following labeled categories: Geography, Date,

Medium, Dimensions, Curatorial Department, Object Location, Accession Number, and Credit

Line. Underneath the image are two separate boxes of content. On the left side is a Social Tags

box with a light blue background and a “[?]” linked icon, which brings you to the Social Tagging

page. The author would have preferred a brief explanation in a pop-up with a link to the page

rather than immediately going to a new page when it was unclear that this would happen. This

area displays any social tags, or includes a message that there are not any tags associated with the

object. At the bottom of the section, when “[Add Your Own Tags]” is clicked, a message

displays underneath about adding tags appears, along with a free-text field and a button to Save

Tags. There does not appear to be any requirements for setting up an account to do this. I wonder

what kind of monitoring the museum does for this freely available feature. The second area of

content is titled “Explore the Collections.” There is a small box that says 1 love it, and an icon of

a heart underneath, which implies that I could favorite this piece of art by clicking on the heart,

and assumingly, this would update the top to say 2 love it. Underneath is a list of three links:

This Artist/Maker, Classification, Curatorial Department, and Country of Origin with a map of

the world and a small circle indicating this work of art’s origin (and which displays the name of

the country when hovered over). Each links to a search related to the specific link, much like

linked metadata exists in other collections, but provides it in separate links, which could be

easily missed by users. Another piece that could be missed by users is a paragraph of additional

information, which is the narrative supplemental information also included in other collections.

The author almost missed it as it was placed all the way at the bottom of the page.

Users may go back to search results by using the back button, or they have a few options

for navigating results at the top of the page. In the top right corner of the page content, “Viewing

VIRTUALLY THERE 44

[1 of 48]” appears in small gray text. To the right, there are two small icons. The first is a small

3x2 grid of bright blue boxes. There is no information when hovering, but it brings you back to

the grid of search results, which makes sense only after the fact. To the right, there is a small

blue arrow pointing to the right, which brings the user to the next search result. As one moves

forward in the search, a left arrow appears to allow the user to move backwards or forwards.