vsa 2011 exhibit files v.5

Visitor Studies Association Conference -- Tuesday, July 26, 2011 -- 9:00 to 10:15 a.m.

1

Knowledge, Identity, and Networks in the Informal Learning Community

Wendy Pollock, Wendy Pollock LLCCarey Tisdal, Tisdal Consulting


2

Improving Practice?


3

Three Big Ideas

• Professional knowledge and identity are defined by and shared through complex networks of people and media

• These networks, people, and media are constantly evolving and changing

• Ways of understanding networks, people, and media are evolving and changing


4

ExhibitFiles


5

ExhibitFiles

Emergence in a Changing Media

Landscape

Evaluating Social Media Designed

to Improve Practice

Lessons Learned

ExhibitFilesEmergence in a changing

media landscape

[email protected]

ExhibitFiles Development TeamWendy PollockKathy McLean Ideum, led by Jim Spadaccini Administrative HomeAssociation of Science-Technology CentersAdministrative support: Wendy Hancock FundingNational Science FoundationGrant No. ESI-054026

Advisors and Core ContributorsDave BaileyAndrea Bandelli Joanna FisherGretchen JenningsJeff KennedyKathy KrafftTom NielsenPaul OrselliClaire Pillsbury Jenny Sayre RambergChristine ReichBeth Redmond-JonesJustine RobertsBeverly SerrellEric SiegelJason Jay StevensJoan WaltermireMartin WeissKatherine WhitneyNAME Board and membersThe 1,950+ members of ExhibitFiles

"Learning is least useful when it is private and hidden; it is most powerful when it becomes public and communal." Lee S. Shulman, 1999

"One of the things that might be done is some kind of collecting activity to retrieve memories of things that were wonderful." Mike Spock

1980s

WorkshopsConferences

Print publications

DesktopsFiles, folders

1990s

Pages, links, webs

Early 2000s

Web 2.0Forums

User-contributed content

Community

"Communities of practice are formed by people who engage in a process of collective learning in a shared domain of human endeavor." - Etienne Wenger

Collection

"Learning within [communities of interest] ... requires externalizations in the form of boundary objects." - Gerhard Fischer

Community + Collection =

"ExhibitFiles is an online community of exhibit practitioners building a shared collection of exhibition

records and reviews."

Simplicity

LanguageCulture

• one niche in learning ecology • flows, not walls

• emergent design

• requires ongoing commitment of resources


23

Evaluating Social Media Designed to Improved Professional Practice

Remedial Evaluation

Summative Evaluation


24

Design

Questions• How did the design team assume

the site would work to produce impacts?

• To what extent and in what ways did users value and use the functions provided by the website?

• To what extent did users perceive themselves as members of a community

• To what extent and in what ways were strategies successful in accomplishing the project impacts?

Methodology & Methods• Naturalistic Inquiry• Methods

– Program theory Weiss (1998)– Analysis of the website

database (N = 1357)– Comparison to an association

membership database (N = 830)– Online survey of registered

members (N = 286– In-depth interviews with the

project team and registered members (N = 18).

Remedial Evaluation


25

Program TheoryRemedial Evaluation


26

Target Audience

Who used ExhibitFiles?• 59.1% exhibition

professionals• 40.9% roles connected to

exhibition development (e.g. administrators, educators, evaluators)

Where did ExhibitFiles members work?

• 37.7% work in science-technology centers/museums, natural history museums, zoos or aquaria, botanical gardens/arboreta, and nature or interpretive centers

• 63.3% work in other settings including other museum types (art, history, children’s) and design firms.

Remedial Evaluation


27

Frequency of UseRemedial Evaluation

Figure 22. How often do you visit ExhibitFiles? (N = 264)


28

Visit PromptsRemedial Evaluation

Figure 23. What prompts you to visit ExhibitFiles? (N = 245)


29

Social Connections

Who on ExhibitFiles has influenced your work?

Remedial Evaluation

Already know them, however I did connect with [name] due to fascinating case history in ExhibitFiles

I have not contacted them via ExhibitFiles - I knew them before.

I knew all of these individuals before the creation of ExhibitFiles.

[Name] is my manager and mentorWorked together


30

Level of Participation

ExhibitFiles (online survey)• 10.0% contributed at least

one case study• 10.0% contributed at least

one review• 90.0% neither.• Low levels of favoriting and

comment• Site somewhat difficult to

search

Neilson’s 90-9- 1 rule (2006)• 90% of users are lurkers

(i.e., read or observe)• 9% of users contribute from

time to time, but other priorities dominate their time.

• 1% of users participate a lot and account for most contributions

Remedial Evaluation


31

Influence on WorkRemedial Evaluation


32

Reasons for Contributing Remedial Evaluation

Figure 30. Reasons for publishing a review (N = 23)


33

Remedial Findings

• How could ExhibitFiles be more useful and relevant? – Email reminders to return to site– Improved search functions on site– Connect search with other sites– Greater range of contributors– More non-science contributions– More contributions from outside the US– More critical discussion/edgier content– More discussion of what didn’t work

Remedial Evaluation


34

Revised TheoryRemedial Evaluation


35

Focusing

• What information was most critically needed? • By whom? (stakeholders)• What is the future of ExhibitFiles?

(sustainability)• What evaluation approach will help answer

these questions? • Do we need a revised theory?

Summative Evaluation


36

Theory -- EcosystemSummative Evaluation


37

Levels of EcosystemSummative Evaluation

Individual


38


Groups


39


Fields


40

EcosystemSummative Evaluation

Fields

More than one Fields?

Lessons Learned• not just a site - part of learning ecology - multiple

flows/stream • breaks down traditional walls (e.g., between

novice and expert,) • emergent design, never finished, must continue to

evolve • caring for the emergent

collection/resource/community requires ongoing commitment of resources

vsa 2011 exhibit files v.5

Education

complex networks of

mediathese networks

wendy pollock llccarey

exhibitfiles advisors

learning ecology flows

process of collective

changing media

memories of things