apollon - eparticipation re-adjusted pilot techniques and methodology

DELIVERABLE

Project Acronym: APOLLON

Grant Agreement number:

250516

Project Title: Advanced Pilots of Living Labs Operating in Networks

D.5.4 Re-adjusted pilot techniques and methodology

Authors: Tanguy Coenen, Bram Lievens (IBBT) Claudio Vandi (UP8) Dave Carter (MDDA)

Project co-funded by the European Commission within the ICT Policy Support Programme Dissemination Level

P Public X C Confidential, only for members of the consortium and the Commission Services

Apollon – Deliverable 5.4

2

Table of Contents

1. Introduction ................................................................................................................ 3 2. Intial pilot description ............................................................................................. 5 2.1 Paris pilot ........................................................................................................................... 5 2.2 Manchester pilot .............................................................................................................. 5 2.3 Brussels pilot .................................................................................................................... 5

3. Pilot re-‐adjustments ................................................................................................. 6 3.1 Paris pilot ........................................................................................................................... 6 3.1.1 Use scenario ................................................................................................................................ 6 3.1.2 Pilot set-‐up ................................................................................................................................... 9 1.1.2 Technology adjustments .................................................................................................... 13 3.1.3 User involvement ................................................................................................................... 16

3.2 Manchester pilot ............................................................................................................ 17 3.2.1 Use scenario ............................................................................................................................. 17 3.2.2 Pilot set-‐up ................................................................................................................................ 18 3.2.3 Technology adjustments .................................................................................................... 20 3.2.4 User involvement ................................................................................................................... 21

3.3 Brussels pilot .................................................................................................................. 22 3.3.1 Use scenario ............................................................................................................................. 22 3.3.2 Pilot set-‐up ................................................................................................................................ 25 3.3.3 Technology adjustments .................................................................................................... 31 3.3.4 User involvement ................................................................................................................... 32

4. Evaluation tracking ................................................................................................ 33 4.1 Transfer Living Labs evaluate SME technology based on user feedback in cross-‐border settings .............................................................................................................. 33 4.2 SME evaluation of the collaboration with Living Labs in the cross-‐border experiments ............................................................................................................................... 34

Statement of originality: This deliverable contains original unpublished work except where clearly indicated otherwise. Acknowledgement of previously published material and of the work of others has been made through appropriate citation, quotation or both.

The information in this document is provided as is and no guarantee or warranty is given that the information is fit for any particular purpose. The user thereof uses the information at its sole risk and liability.


3

1. Introduction The main issues addressed by APOLLON (Advanced Pilots Of Living Labs

Operating in Networks) are the present lack of Living Lab harmonisation and

collaboration, and the serious difficulties of SMEs in engaging in cross-‐border

innovation.

APOLLON will demonstrate the positive impacts of cross-‐border domain-‐specific

Living Lab networks, by setting up an advanced pilot composed of 4 thematically

focused European-‐wide Living Lab experiments. SMEs are enabled to take part in

cross-‐border Living Lab experiments beyond their home markets, and are

supported by large industrial companies, academic centres and other

stakeholders.

The APOLLON pilot aims to share and to harmonise the Living Lab approaches

and platforms between exemplary European networks as well as the subsequent

evaluation results and the set up of sustainable domain-‐specific networks on a

European and global level.

APOLLON addresses 4 major domains in which ICT products and services

innovation may benefit most from cross-‐border Living Lab networking. These

are:

• Homecare and Independent Living

• Energy Efficiency

• eManufacturing

• eParticipation

The project consortium of the domain 4 is composed of:

Issy Media ( France), Université de Paris VIII (France), IBBT (Belgium),

Manchester City Council ( United-‐Kingdom), 3D2+1 (France), Navidis (France),

and People’s Voice Media (France)

The objectives of Work Package 5 are the following:

1 The name of this SME has changed to Virdual. However, for the sake of conformity with the other documents of the Apollon project, we will keep referring to them as 3D2+.


4

• Sharing and comparing technologies/ methodologies in order to

understand to which local, regional, national results can be extended to

other contexts and which common technology/methodology can be built

for generalization.

• Adapting technologies/methodologies to the European context.

• Integrating technologies/methodologies to overcome fragmentation of

services

• Promoting citizens’ innovation to eMedia participation in Europe and

evaluating if cross-‐border user testing can help existing projects to open

to the European audience.

In this report we describe how the technologies and the methodologies have to

be adapted in order to carry out the pilot. The different pilot descriptions within

the media and e-‐participation workpackage were the subject of D5.3. In this

report, we document how and why some methodologies have been withheld and

others rejected.

In other words, this deliverable describes in more detail the way in which the

existing technologies will be integrated and how they need to be adapted in

order to do so. In addition, this document also describes the adjustments from a

living lab research point of view. Concerning living labs, we will not only focus on

the actors involved and the context, but also on research activities related to user

evaluation.

The structure of this document is as follows. In section 2, the description of the

initial pilots, as described in Deliverable 5.3, is re-‐stated. If changes to the pilot

description have been made, this is mentioned, but the actual changes are

detailed in section 3. This section provides detailed descriptions on the different

pilots. Finally, in section 4, a number of ways to follow up on evaluations of both

SME collaboration and user experience are described.


5

2. Intial pilot description

2.1 Paris pilot The French pilot is identical to what was described in deliverable 5.3. As

described there, the pilot involves the aggregation of Media technologies using

3D multimedia, context aware mobile applications and Community Reporting,

within the framework of the Digital Fort project in Issy Les Moulineaux.

All WP5 partner's technologies will be involved: 3D2+ provides ‘Real and More’,

Navidis provides ‘Issy 3D’ and ‘Urbadeus’, IBBT provides ‘Air Graffiti’, and People

Voice Media provides ‘Community Reports’. 2

The main change with regard to the initial set-‐up (described in deliverable 5.3) is

a more elaborate and detailed scenario for the pilot as well as the decision at

which level the different technologies will be integrated in the pilot. Between a

fully functional integration and a simulated integration we opted to adopt an

intermediate solution that we call the aggregation approach. In this approach,

partners' technologies are not fully integrated on a common platform, but still

exchange data and are mutually accessible.

2.2 Manchester pilot As outlined previously in Deliverable 5.3, the Manchester pilot will use the

refurbishment of Manchester Central Library and a public art trail in city centre

Manchester, managed by the Manchester City Galleries department, to focus

activities related to the testing and evaluation of products from IBBT, Navidis

and Peoples Voice Media and the evaluation of Living Labs processes and

methodologies created by Work Package 1. In addition to the scenario set out in

D5.3, the Manchester pilot will also involve a project being run by Greater

Manchester County Records Office to engage citizens in local history archives.

2.3 Brussels pilot The initial pilot in Brussels, as described in Delverable 5.3, revolved around the

campus of the future project. The aim was to experiment with e-‐participation

regarding the future of the university as an institution and the campus as a

2 For a full description of these technologies see deliverable 5.3


6

physical social space. Therefore, various components would have been deployed

at the university campus, like an open Wifi network and access to the open data

sources from the University. The purpose of the Brussels pilots as initially

envisioned in APOLLON was to conduct an open envisioning exercise based on

and using this existing infrastructure. This would have been done in

collaboration with as many stakeholders as possible. The foreseen outcome was

to create a computer-‐assisted methodology that, by using future scenarios

wrapped in a context-‐aware application, helps to define possible ways to record

scenarios for change of a certain entity involving the users of that entity.

However, due to the fact that some of the initial technologies and core services

on the University level, related to this smart-‐university concept, are not yet

implemented, we are forced to refocus the initial pilot-‐scenario and setup. This

will be detailed in section 3.3.

3. Pilot re-adjustments

3.1 Paris pilot

3.1.1 Use scenario

The Use scenario for the French pilot is built around the City of Issy Les

Moulineaux and the Digital Fort new district. The Digital Fort in Issy-‐les-‐

Moulineaux is a new Green and Hi-‐Tech district that is being built in the place of

the old Issy Fort (constructed between 1840-‐1845 and nearly destroyed in the

war of 1870).

The first objective is to encourage the citizens to get involved in the life of their

city, to make them discover and increase their awareness on the rich history of

their town. They will not get involved in delicate political issues (for example,

expressing their anger regarding the works at the construction site), but they

will be able to express themselves and have an active role in their town, via a

series of innovative media technologies. Using a 3D interactive documentary on

the history of the Fort and a 3D interactive cartography of Issy, they will discover

the city's history. Using two context-‐aware mobile application for creating and

viewing geolocalized content they will participate in an Urban Game, become


7

History Explorers and thus contribute to a shared representation of the city.

Through Community Reports, they will become Community Reporters and create

User Generated Content.

The pilot aims to test the aggregation of various new media technologies for

involving and engaging the citizens during the construction works and to help

them discover and re-‐appropriate the Fort's history and become a part of its

future.

The realization of the pilot will involve two steps:

• In a first step, the Awareness phase, citizens of Issy-‐les-‐Moulineaux will

first be invited to discover the History of the Fort through a cross-‐media

Documentary on the Fort and a 3D interactive plan of the City.

• In a second step, citizens will act as History Explorers and Community

Reporters and will be involved in an Urban Game and Community

Reports.

Figure 1: Illustration of the Paris pilot scenario

3.1.1.1 First Phase : raising awareness

To engage with citizens, it is important to raise awareness on the object of their

participation activity and to provoke the citizens' curiosity on the proposed

subject. A group of citizens will be invited to participate in a workshop in which

they will discover the history of the Fort in an innovative way using 3D

technologies.

In this phase, the following technologies will be used to create awareness and

provoke the citizens' curiosity:

• Real and More will be used to provoke the citizen’s curiosity about the

Fort and its history.

• Issy 3D will be used as the entry level for the documentary to help

citizens visualize the Fort on the City plan.

Using these technologies, a mediator will explain the history of the Fort and its

role in the city. Citizens will be invited to actively participate in the presentation

by contributing their knowledge and memories and by interacting with the 3D

environment.


8

During this workshop, citizens will be motivated to take an active role in the

project and become History Explorers and Community Reporters in the second

phase. At the end of the workshop, citizens will be asked to express their opinion

about the technologies they have been using.

3.1.1.2 Second Phase: User Involvement through an Urban Game and Community Reports

In this second phase, citizens will take a more active role and participate as

History Explorers and Community Reporters.

During two Urban Games, the citizens will be able to discover and contribute to

the history of their city in a vivid, interactive and playful way. Using Urbadeus

and Air Graffiti, they will follow a discovery trail, an Urban Game on the history

of Issy. Citizens will have to find a number of hints and clues that will take them

through time and legend, back to the War of 1870 and in emblematic places and

streets around the city that still show traces of the old times.

The starting point of the game will be Real and More through which

users/players will be introduced to the game scenario. With the help of an

iPhone or another Smartphone, users will be able to find POI (points of interest)

in the surrounding neighborhoods of the Fort of Issy (the interior of the Fort will

be closed to public access until 2013). The POI will offer access to information on

the history of the Fort and its surroundings before and during the 1870 war.

The available resources in the POI will be:

• Engravings presenting the Fort during the war

• Engravings of the surroundings

• 3D images based on the digital reconstruction of the Fort

• Videos of the 360 ° panorama realized from the digital reconstruction of

the Fort

• On-‐site additional information in the form of small texts

These POI will be the starting point of the urban game, which will take users

from one historical point to another, leading them finally to the coveted treasure.

Throughout the trail, users will have to solve various riddles and puzzles and

they will have to report constantly on their journey via the Urbadeus platform.


9

This will be done by “capturing reality” with their mobile phones through

emoticons, photos, videos or sounds.

Using Air Graffiti, 2D barcodes (tags) will be placed on certain historical

buildings that users will have to find by solving the various riddles along the

way. The QR codes will link to online content and information about the

building/object they are placed on.

During the game, citizens will have to snap photos and take videos that will be

uploaded to Issy 3D, thus creating a shared cartography of the City History.

During and after the game, players will also have to produce reportage materials

using Community Reports methods. They will have to comment on what they do

and what they see around them as they progress in the game.

After the game, content produced by the citizens will be accessible both on Issy

3D through Urbadeus and on mobile through Air Graffiti. Starting from the

contents produced during the game, a Community Report will be realized to keep

track of the events and to build a shared representation of the City's past and

future.

3.1.2 Pilot set-up

3.1.2.1 Technologies

In this pilot we will use all the technologies of the SME’s involved in WP5. We will now elaborate the role of each SME in the pilot.

3D2+

3D2+ provides Real and More, a 3D documentary on the history of the Fort of Issy

Les Moulineaux and 3D interactive reconstruction of the Fort. The documentary

proposes a guided visit of the place enriched with documents such as video

sequences, engravings, sounds and pictures. The 3D reconstruction of the place

allows the user, in the form of an avatar or a subjective camera, to go around and

make a tour as one pleases in a free visit. Real and More will be used to raise

citizen’s interest and make them aware of the place of the Fort in Issy's history.

Navidis


10

Naividis provides Issy 3D, a 3D cartography of Issy that allows a user to travel in

space and time. Issy 3D will be used as the entry level for the documentary to

help citizens visualize the Fort on the City plan.

Navidis also provides Urbadeus, allowing citizens to post with a smart phone

into the 3D Model of the city additional points of information for sharing city

experience and providing relevant feedback. An example would be to build a

memory of the city or to raise any issues that will be monitored by the technical

service of the city. Urbadeus will be used to geo-‐localize content from the 3D2+

product Documentary and enrich it with geo-‐localized user-‐generated content.

IBBT

IBBT’s Air Graffiti is a context-‐aware urban mobile service that allows users to

describe, experience and discover objects and locations. The application allows

for dynamic in-‐ & outdoor trails and uses GPS, QR-‐codes, RFID-‐tags and short -‐

urls.

Air Graffiti will be used to distribute context-‐aware content (geo-‐localized, 2D-‐

barcodes) regarding the Fort’s history around the city. The Fort played an

important role in the Franco-‐Prussian War of 1870-‐71 and references to that era

are still to be found across other parts of Issy and the wider Parisian area (e.g.

museums in Paris).

People's Voice Media

People's Voice Media will provide Community Reporters programmes, which are

an innovative way for people to build confidence in the use of social media tools,

learn new skills and tell a story about themselves or their communities.

Community Reporters Programmes will be used to engage with Community

Reporters and creating Community Reports and work with and train Issy

Media’s staff as well as recruit members of the public to act as community

reporters. They will create and upload content on the social media (e.g. YouTube,

Flickr). The content will be geo-‐localized using AirGrafitti and projected via

Navidis’s Urbadeus onto the Issy 3D plan.


11

3.1.2.2 Non-SME Actors

We aim at involving Citizens, Schools and local public authorities. There is not a

specific age range or profile. The Urban Game scenario will be adapted to satisfy

different user groups (kids-‐adults).

An important actor for the pilot that wasn't involved in previous pilot set up is

the City Museum, which provided a detailed list of 9 Points of Interest that bear

some traces of the history of the City related to the Fort. These will be used as a

base for both Community Reports and the Urban Game.

3.1.2.3 User research

Users are at the core of the pilot since they will act both as History Explorers and

Community Reporters. During the pilot we will have two ways of collecting users

feedback :

1. During the pilot, as citizens use the technologies, a Living Lab

representative will be with them to collect their feedback on two

dimensions: usability and acceptability. This will be done through

questionnaires using both Likert scales and open ended questions. These

questionnaires will be designed at the beginning of the pilot (by M14 of

Apollon).

2. Towards the end of the pilot (M22 of Apollon) we will organize a Co-‐

innovation workshop to record and analyze users feedback on the

technologies they have been using. UP8 will be responsible for this task

that will be carried out using a methodology that has been used widely

before APOLLON in national projects in which we brought together the

creator of the technology and some representatives of the target user

groups: the Eneide project about New Technologies for Education

(Observus workshops with teachers), the RANUTER project about Digital

Terrestrial Radio (workshops with drivers), the Tablets for seniors

project (workshop with senior citizens). These workshops are intended to

bring together technology providers (SMEs) and end users and guide

them through a participative innovation workshop in which:

a. Users can express their needs, provide new ideas for technology

development and test and highlight what they like or don't like.


12

b. SMEs can collect input on what is needed to improve acceptability,

to make the learning curve less steep, to adapt their tools to a

group of users and to extend the potentials of their tools.

The workshops will be conducted along the following lines :

1. Participants gather in the same room

2. Each participant introduces himself

3. During a first phase, the SME’s make a quick presentation of the proposed

service

4. During the second phase, users are invited to generate ideas, focusing on

two aspects :

a. On the product side, starting from existing services, possible new

usages of these services are investigated. The general question is

“What would you like to do with this service ?”

b. On the users side, starting from existing needs new services

(inside the product) to satisfy user needs are focused upon.

Different methods exist to boost the idea production process, like free

brainstorming, guidance by images, by scenarios, …. Yet in order to record ideas,

what we propose is for each participant to write down his idea on a post-‐it note.

Ideas should be formulated clearly and include a verb (ex: use the service to do

X).

The mediator (LL) then takes the post-‐its and organizes them on a wall, creating

groups associated to “events” in the brainstorming (e.g. ideas generated while

discussing a given service, or presenting an image).

At the end of the brainstorm, each group of idea is commented together, trying to

identify the best ideas and increase the coherence of what has been proposed.

Finally, the results are analysed. Recurrent ideas are the most requested from

users. But these are not always the most relevant for the SME. In order to have

the most relevant ideas we need to organize the ideas along categories

associated to services and/or along a typology of ideas (e.g. ideas for innovation

/ ideas for business opportunities).


13

3.1.2.4 Timeline

1. End of M12: Identify POI in Issy related to the History of the Fort (done)

2. End of M13: Adapt the Urban Game to include these POI

3. 15th of December (M14)– People’s Voice Media will train Issy Média’s staff

to use their technology and become Community Reporters

4. 12th of December (M14) – Real and More will be presented at the Cité des

Sciences et de l’Industrie in Paris

5. 2nd half of M15 – Test the exchange of data between the apps

6. M16: Preparation and Communication for the first Urban Game

7. M17: Urban Game and Community Report during the “Fête de l'Internet”

8. M18: First evaluation including user feedback and SMEs (what we

integrated and how it worked, what are the new perspectives)

9. M19: Preparation and Communication for the second Urban Game

10. M20: Second Urban Game and Community Report during “Futur en Seine”

Festival

11. M21: second evaluation including user feedback and SMEs (what we

integrated and how it worked, what are the new perspectives)

12. M22: Final evaluation

1.1.2 Technology adjustments

The best scenario would have been all technologies providing APIs for easily

exchanging data with each other. Since this is the case only for Air Graffiti, we

decided to adopt an Aggregation Approach that avoids a functional integration of

technologies but still allows them to exchange data. Table 1explains what will be

exchanged.

Table 1 : What and how -‐ data exchanged across technologies

To

From

Pilot Real and More

Urbadeus Air Graffi t i Community Reports

Real and More

Real and

More is used

to explore

What: Inside

Real and

More users

What: Inside

Real and

More users

What: Inside

Real and

More users


14

the history of

the Issy Fort,

provoke

curiosity and

engage with

citizens.

can access

content

created by

using

Urbadeus

How:

Hyperlinks

Embedded in the 3D

Documentary

can access

content

created by

using Air

Graffiti

How:

Hyperlinks

Embedded in the 3D

Documentary

can access

content created

during

Community

Reports

How:

Hyperlinks

Embedded in

the 3D

Documentar

y

Urbadeus and Issy 3D

Urbadeus is

used to

create

geolocalized

content

during the

Urban Game

and

Community

Reports

Issy 3D is

used to

visualized

Urbadeus and

other

applications'

content on a

3D map of

Issy

What: From

Issy 3D and

Urbadeus

users can

access Real

and More

How: through

geolocalized

hyperlinks,

embedded in

Issy 3D and

Urbadeus

What: From

Urbadeus

users can see

geolocalized

content that

has been

produced

using Air

Graffiti

How:

through Air

Graffiti API

Urbadeus

retrieves the

geolocalized

content

produced in

Issy

What:

Urbadeus is

used during

Community

Reports to

produce and

geolocalize

contents

How: already supported

Furthermore

What other

kinds of User

Generated

contents

produced

during

Community

Reports can

be

geolocalized


15

in Urbadeus

How through

geolocalized

hyperlinks

embedded in

Issy 3D

Air Graffi t i Air Graffiti is

used to

produce

geolocalized

content and

access it on

mobile.

What: From

Air Grafitti,

users can see

Videos,

Pictures and

Audio created

with

Urbadeus

How: through a script embedded in Urbadeus

What: From

Air Graffiti

users can see

contents

produced

during

Community

Reports

How: already supported

Community Reports

Community

Reports

Programmes

are used to

train trainers

in Issy and to

involve

citizens in

content

creation.

Real and

More is used

to build

scenarios for

Community

Reports

Urbadeus is

used as a tool

for producing

and

geolocalizing

content

during

Community

Reports

Air Graffiti is

used as a tool

for

geolocalizing

content

during

Community

Reports

Based on their current state, the following adjustments will need to be carried

out on the involved technologies :


16

• Real and More : To give access to other content inside the 3D

reconstruction of the Fort, 3D2+ will have to design a solution to

visualize and access hyperlink from within the 3d environment.

• Urbadeus : To exchange content with other apps in Urbadeus, Navidis will

have to develop a script that will make available daily contents for Air

Graffiti.

• Air Grafitti : To exchange content with other apps, Air Graffiti will use its

API.

• People’s Voice Media : To adapt its Community Report training to a

French audience. This involves translating supporting material into

French before the course (with support from french Llabs) and having a

translator (French LL staff) during the course for those people that have

problems understanding English. People’s Voice Media will also have to

adapt to a new target population and will have to choose appropriate

technologies, avoiding Social Media platforms that are not localized in

French.

3.1.3 User involvement

Citizens of Issy will have a core role. They will be asked to participate as History

Explorer and produce content to enrich the different technologies. At the same

time, they will be asked to express their opinions on the technologies they are

using (as described in “User Research”) and to participate to the city life through

innovative eParticipation technologies. From this point of view, Urbadeus will

have a particularly important role of giving citizens the opportunity to express

their opinion by attributing an emotional tag to the content they create during

the urban game.

The inhabitants of Issy are generally proud of their city’s image as one of the

most advanced cities in France and proud of being an active part of it. Thus, the

majority of Issy’s citizens will voluntarily participate in the experimentation of

new services and products.

All these activities will be broadcasted on the city’s local Web TV channel

(www.issy.tv) and relayed on the city’s local magazine, local social network


17

(www.i-‐folio.fr), and on the city’s Web page (www.issy.com), a cornerstone of

Issy’s national dissemination activities.

Issy-‐les-‐Moulineaux Facebook and Twitter pages will also be used to raise

awareness on the Virtual Museum within the framework of cross-‐border Apollon

experiment.

3.2 Manchester pilot

3.2.1 Use scenario

The Manchester pilot will work in partnership with three existing projects in the

city to test and evaluate the Air Graffiti application from IBBT, the Urbadeus

application from Navidis and the Community Reporters model from Peoples

Voice Media. In relation to this, evaluation of Living Labs processes and

methodologies will also take place.

Below is an overview of the three existing projects that will be used in the

Manchester Pilot:

• Manchester Central Library closed in June 2010 for three years of

refurbishment and renovation and there is a need to keep citizens

informed of the progress of the refurbishment work. The use of QR codes

located at key locations around the renovation site and at other libraries

around the city will be used to link to online resources where citizens can

find out more about the renovation work and leave comments and

questions.

• Manchester Galleries is the Decoding Art project

(http://www.manchestergalleries.org/decodingart/), which has fixed QR

codes to twenty public art works in the city. Each QR code links to a web

page about that art work, and the page will include text about the work

and an MP3 audio clip of the text.

• The Greater Manchester County Records Office is running a ‘Big Society’

project funded by the Museums, Libraries and Archives Council

(http://www.mla.gov.uk/) to explore how to engage with citizens who

would not normally access their local archives. Images from the local

images collection of Manchester (http://images.manchester.gov.uk/) will


18

be printed on posters with QR codes and placed at the locations shown in

each photograph to encourage citizens to leave stories, comments and

other information about that location. Events will take place in local

libraries to make citizens aware of the posters and the QR codes.

Walking tours and evaluation sessions will be set up with expert users and

citizens to test and evaluate how Air Graffiti from IBBT and Urbadeus from

Navidis can be used to engage citizens in both the Central Library refurbishment

and the Decoding Art project.

User testing will evaluate subjects including usability of the applications,

language / cultural differences in the application’s interfaces, ideas for future

development and general feedback.

In addition, evaluation will also take place of the preparation work required to

enable the applications from IBBT and Navidis to be used in Manchester, such as

time required to prepare 3D model data of Manchester Central Library for use in

Urbadeus.

Peoples Voice Media will promote the Central Library, Decoding Art and Big

Society projects through their network of Community Reporters, and will work

with Manchester Central Library staff to clarify if and how the Community

Reporters model can be used by a city administration as alternative channel of

communication to citizens. For example, Community Reports will be given access

to inside the Central Library building and blog about the progress of the

renovation work, carry out interviews with key people involved in the

renovation project which will go online, and similar. Community Reports will

also be invited to be involved in the user testing sessions of the Air Graffiti and

Urbadeus applications.

3.2.2 Pilot set-up

The Manchester pilot will involve three phases:

• Preparation of applications for use in Manchester.

• User testing sessions.

• Evaluation of Living Labs processes and methodologies.


19

3.2.2.1 Preparation of applications for use in Manchester

In order to carry out successful testing of the Air Graffiti and Urbadeus

applications during user testing sessions – e.g. walking tours around public art

sites tagged with QR codes using Air Graffiti – time will need to be spent to

prepare them for use.

In the case of Urbadeus, this will involve gaining access to the 3D City Model of

Manchester produced for the city by Arup. Informal agreements have already

been set with Arup to access the data and Manchester is currently working

through the administration processes to hand over 3D data to Navidis for input

into Urbadeus.

In the case of Air Graffiti, Manchester will liaise with IBBT to ensure that the

application is ready for public use.

Timeline: November – December 2010.

3.2.2.2 User testing sessions.

The user testing sessions will consist of the following:

• Expert reviews. MDDA staff will carry out one-‐to-‐one meetings with local

experts with interests in areas of QR codes, e-‐participation / social media

and 3D applications to carry out half-‐day, in-‐depth evaluations of the Air

Graffiti and Urbadeus tools, feeding back findings from the evaluations to

IBBT and Navidis in order to support development of the applications.

These sessions will also be used to enable IBBT and Navidis to gain

knowledge of local business contacts and networks that may be useful to

expand into new markets. By proxy, MDDA will also test Living Labs

processes for cross-‐border knowledge sharing. The expert reviews will

focus on usability, market readiness, language / localisation / culture

differences, scenarios for testing the products with users in the target

market, software / hardware requirements.

• Testing with citizens. A maximum of three user testing sessions will take

place with members of the public. One user testing session will evaluate

how using QR codes and location aware comments can engage citizens in

the Manchester Central Library renovation work. Air Graffiti will be used


20

as the application to engage with citizens using QR codes and Urbadeus

will be used as the application to engage using location aware comments.

A second user testing session using both Air Graffiti and Urbadeus will be

carried out to evaluate how they can augment and support the public art

tour provided by Manchester City Galleries. A third user testing session

using Air Graffiti and Urbadeus will evaluate how they can support access

to local history archives through the use of QR codes and location aware

comments based around posters showing old images of Manchester at

various locations around the city.

Timeline:

• Expert reviews: M14-‐M15.

• User testing session 1: Manchester Galleries is the Decoding Art project

(Walking tour of public art works provided by Manchester City Galleries).

M16-‐M17, dependent on the weather.

• User testing session 2: Manchester Central Library. M17-‐M20, mainly

dependent on when the 3D city model of Manchester can be integrated

into Urbadeus.

• User testing session 3: Greater Manchester County Records Office ‘Big

Society’ project (local images of Manchester). M15-‐M17. This is

dependent on Greater Manchester County Records Office providing local

images for use on posters.


The following adjustments will need to be carried out in order to test the

products in the Manchester scenario.

• Ensuring that English is provided as a language in the user interfaces of

both Urbadeus and Air Graffiti. Air Graffiti already provides an interface

in English. Work will need to carried out with Navidis and MDDA to test

an English language interface for Urbadeus;

• Ensuring that the 3D city model of Manchester can be integrated into

Urbadeus. Agreements are already in place to access the 3D city model,


21

and Manchester is currently going through administrative exercises to

release the data. The Manchester team will be working closely with Peoples Voice Media and

other local partners to ensure that there are adequate opportunities for feedback

built into this part of the planned local collaborative working to enable any

further adjustments that are required to be made efficiently and effectively.


In order to involve users (experts and citizens), Manchester will carry out the

following activities:

1. Decoding Art public art works tour. To involve citizens, Manchester

Galleries and MDDA will put out a call for interested citizens to take part

in a ‘walking tour’ that will test the Air Graffiti and Urbadeus applications.

The call will go out via the Manchester Galleries website, Twitter and

Facebook pages. Additionally, a call will go out via Peoples Voice Media to

ask emerging and existing Community Reporters to be involved in the

session. Users will be invited to meet at MDDA’s offices, given an

overview of the APOLLON project and the Air Graffiti and Urbadeus

applications, then taken on a walking tour where they will play with

applications. During the tour, comments from citizens will be recorded via

audio and video (if consent is provided) and text. When the tour is

finished the users will be taken back to MDDA’s offices and asked to

complete a simple evaluation form asking for their views on the

applications, ideas on how they could be used in e-‐Participation, any

technical / language / cultural issues that arose, and similar. Following

the session a summary report will be prepared and sent back to IBBT and

Navidis.

2. Manchester Central Library. This session will follow the model of (1)

above, apart from that the call for citizens will go out via Manchester

Libraries website, Twitter and Facebook pages, and also via the printed

Full Volume newsletter. Users will be taken to the Central Library


22

renovation site and asked to evaluate the Air Graffiti and Urbadeus

applications.

3. Greater Manchester County Records Office ‘Big Society’ project.

Again, this session will follow the model of (1) above. As with (2) the call

for users will go out via Manchester Libraries website, Twitter and

Facebook and also via Peoples Voice Media’s website. The user testing

session will start at a local library in a suburb of Manchester, and users

will be taken on a walking tour of locations in the area where posters of

old images of Manchester with QR codes on them have been placed. They

will use Air Graffiti to test leaving memories, comments and other

information about that place. They will also be asked to test how

Urbadeus can support information related to the images.

4. For expert reviewers, MDDA will approach local experts from

organisations and businesses with knowledge and experience of the areas

relating to the Air Graffiti and Urbadeus applications – QR codes, 3D

modelling and social media. These will include organisations such as the

Manchester Digital Laboratory, the Centre for Construction Innovation,

Social Media Manchester and Arup.

5. Additionally, for user testing, MDDA will approach citizens who have been

involved in the DEHEMS home energy monitoring Living Labs project and

ask them to be involved in the user testing session for APOLLON. Those

citizens will have experience of previous Living Labs processes and useful

information may be gathered from them by comparing their experiences

of previous Living Labs processes with current thinking.

3.3 Brussels pilot

3.3.1 Use scenario

3.3.1.1 Concept

The working title for the pilot is “Museum Quest”. Within this pilot we will

implement a serious game to increase the involvement of youth with the exhibits

in a museum. This is done by providing a game-‐based motivation to absorb

information and by steering the player towards noticing certain important


23

aspects of the exposition. The innovation in the presented pilot case lies in the

linking of a real and a virtual environment and in doing this by means of a game

concept. This will be achieved by integrating the existing virtual technology of

3D2+ with the Internet Of Things technology (Air Grafitti) of IBBT.

In terms of added value for a museum, the pilot offers two use cases. The first

one is a group of young people playing a game in a museum. This can be done in

the context of a school visit or a visit with the family. The second use case

involves non-‐player visitors that can access content related to exhibits on their

smartphones. This content is used for both use case one and two. In use case two,

non-‐gaming visitors can also add comments to the exhibits that they visit in the

museum.

Both use cases can be combined, to offer visiting families a more appealing visit

to a museum. The children then play the game, while the parents engage in a

more exhaustive visit along the lines of use case two.

3.3.1.2 Scenario

In this section, both use cases are discussed in a more hands-‐on way, to provide a

concrete impression of how the pilot will operate. This scenario will function as a

guideline for the integration of the various technologies.

Use case 1: the Museum Quest game

• The users have or loan smartphones on which they can see a badly kept

3D museum room in 2030 in which certain exhibits are missing.

• They are told that, strangely, all the stolen exhibits were exposed in the

museum in the year 2011. However, much information has been lost over

the years regarding the stolen objects. The user is asked to help the police

identify the stolen objects by going back in time, meaning to move in the

real museum in which they are currently located, and identify the missing

exhibits.

• Each of the exhibits is marked by a question mark in the 3D world. The

user can walk around in the world and click on the question marks.

Clicking on the question mark produces an overview of the remaining


24

information regarding the exhibit. The users are asked to go find the

matching works in the museum.

• Some or all of the pieces in the real museum have been marked with a QR

code. When scanning the code, the user is presented with content related

to the exhibit and an identification code. If the user believes they have

identified a missing object, they can enter the code at the location of the

missing exhibit in the 3D world. If it matches, they get points.

• A Twitter account is set up for the game. Key events that take place in the

game, like players successfully matching a missing exhibit to a real

exhibit, are logged to this twitter account. The twitter account can then be

embedded in the website of the museum to provide an overview of the

activity of the game. This could intrigue visitors of the website and

motivate them to visit the museum. In addition, the twitter account can be

visualized in the heads up display of the 3D application, to give players an

overview of what other players are doing in the game.

Use case 2: participation of museum visitor in museum exhibition

• Before entering the museum, visitors are made aware of the fact that they

can access exhibit-‐related content using their smartphones. However,

they are told that they need a scanner application to do so. For example

OptiScan for iOS costs 1.99$.

• QR codes have been placed close to exhibits. Scanning the QR code using

the scanner opens a browser window that automatically links to the Air

Grafitti page on which the content can be accessed. This can be text,

images, video or audio.

• Visitors are also offered an interface to post comments on the exhibit

about which they are viewing content. They also have access to a

Facebook “like” button, which will automatically post the exhibit to their

friend network on Facebook. Using the like button increases the exposure

of the museum and its exhibitions to the public.


25

3.3.2 Pilot set-up

3.3.2.1 Motivation for pilot reorientation

As the remainder of this deliverable will elaborate, the pilot that will be carried

out in Brussels is substantially different from the pilot that was described in

deliverable 5.3 and that has been summarized in section 2.3. The reason for this

reorientation is threefold:

A first reason is that there are a number of hindrances related to the

implementation of the campus of the future scenario. The main obstacle was the

slow progress in completing all the foreseen infrastructure to establish a

campus-‐wide living lab representing the university of the future. Especially the

usage of a student-‐card featuring an RFID chip and the linking of different

databases containing student and personnel information proved very

cumbersome. The university is an environment with a flat organizational

structure, in which decisions need to be taken by many instances before they are

actually implemented. Unfortunately, we could not wait for the decision making

process on this to be completed before starting to roll-‐out of the Apollon pilot.

Secondly, discussions with the various involved SME’s made it clear that the

business case for the campus of the future was not (yet) within their current and

future business roadmap. This is why we are switching to a more culture-‐

oriented pilot that features more overlap with the pilots that will be conducted

in Paris and Manchester. The SME’s are interested in this type of application,

making the new pilot definition more of a win-‐win situation for all involved

parties.

A final reason why we adjusted the pilot is the fact that a museum partner was

looking to experiment with a similar technology set-‐up and application as the

type of pilot we are offering. Within the role of the Living Lab as a broker for

innovation between SMEs there was an interesting opportunity for a synergetic

cooperation and by so to offer a possible opportunity for each of the

stakeholders involved.


26

3.3.2.2 Non-technical set-up

The pilot involves collaboration between different stakeholders, each with a

specific responsibility:

• The local Living Lab: The responsibility of this actors does not only

include a good integration of the various technologies, but also that this is

in-‐line with the needs and requirements of the hosting partner, i.e. the

museum. Next to that, they also are responsible for involving the users in

the development and evaluation process

• The transfer Living Lab: as they already have experience with one of the

basic technologies that will be part of the integrated pilot, their role is to

provide the necessary input with regard to user experiences on that

technology, based on the research they have conducted so far.

• Technology providers: The technology providers involved, 3D2+ and

IBBT, are responsible for integrating their technologies. They will work

together in developing the ‘Museum Quest’ concept. They are also

responsible for describing the necessary technology set-‐up and guide

both the Living Lab and museum in the actual set-‐up and deployment.

Next to the integration, they will provide the necessary monitoring

instruments to allow the logging, needed for the evaluation.

• The hosting partner: The application that is the subject of the pilot will be

tested and evaluated in a real life environment. In practice this will be a

museum setting. The museum acts as a full partner. This mean that they

will not only use the application in their daily operations, but also that

they will provide the necessary access to their premises. In addition, this

actors will be responsible for the addition of the exhibit-‐related content

that is relevant to the scenario.


27

3.3.2.3 Technical set-up

Use Case diagram

Figure 2: use case diagram for Brussels pilot

As can be seen in Figure 2, both gamers and non-‐gamer visitors can use the

functionality that will be developed in this pilot. The non-‐gaming museum visitor

will be able to scan QR codes and receive content on the related exhibit. In

addition to this, the gamer will use this content to match the missing exhibits in

the 3D world to the exhibits in the real museum. Also, the gamer will be able to

monitor the progress of the other teams that are playing the game, through a

visualization of the game’s twitter log. A similar visualization will be available on

the museum’s website, in order to draw new players to Museum Quest and thus

to the museum.


28

In terms of user-‐generated content, both the gamer and the non-‐gamer will be

able to use a “like” button to post the exhibit content to their social network in

Facebook. Non-‐gaming visitors will also have the ability to write comments on

the exhibits. Although perfectly feasible from a technical point of view, we do not

believe writing comments on an exhibit is something one would do while playing

the game. This is why we are leaving it out of the use case for the Museum Quest

game.

Someone who has access to the appropriate content and can oversee its quality

will add the content to Air Grafitti. In Figure 2, this role is called the “Museum

content provider”. Most museums already have content on their exhibits for

publication in e.g. printed catalogues and web sites, so adding the content will be

mainly a question of selecting and formatting the appropriate content. In

addition to administering the content, the museum content provider will be able

to print a QR code via Air Grafitti that can be placed close to the exhibit it is

related to. Depending of the time frame of the pilot, these QR codes can be

printed or engraved on a durable medium, or not.

3D2+ will develop the 3D world. The world will look like a museum that has

been badly kept. The world does not have to match the actual museum to match

the game concept. Air Grafitti will be used to provide content on the exhibits, and

generate QR codes that can be placed close to the exhibits in the real museum.


29

Technical architecture

Figure 3: software and hardware deployment in the integrated scenario

As can be seen in Figure 3, this scenario foresees that all the functionality of the

game can be used on one smartphone device and in the same application. The 3D

application, built in the Shiva 3D engine technology3 used by 3D2+, will be

ported to a code base that can be executed on smartphones using either iOS or

Android as operating system.

3D2+ will also integrate a QR scanner in the 3D application, so the user does not

need to leave the application when scanning a QR code and displaying the

resulting content.

3 http://www.stonetrip.com/


30

Once a QR code has been scanned in the application, the content that is hosted on

the Air Grafitti server will be read using the Air Grafitti API. This API can be

called upon through REST GET requests and produces XML data that will need to

be parsed by the application in order to be displayed in the 3D world, or in a

dialogue window on top of the application’s heads up display.

When certain events occur in the 3D application, the Twitter API4 will be invoked

and specific information will be passed in XML format. This Twitter log is then

visualized in the 3D application of other players and on the museum’s web site.

When gamers or non-‐gaming visitors use the “like” button, data is sent in JSON

format to Facebook’s Open Graph API5.

All data streams that will need to pass over internet are requests to web

platforms that produce text, images or videos that are optimized for the web.

Therefore, we expect the application to be able to read and display exhibit

content using a HSDPA connection which is readily available from Belgian mobile

carriers. Should the architectural structure of the museum prove to greatly

impair data transmissions, or should the responsiveness of the application prove

to be too low using HSDPA, we will need to look for alternatives, like installing a

temporary Wi-‐Fi network in the museum.

3.3.2.4 Timing

A number of different tasks will need to be undertaken in order to complete this

pilot. They are the following:

1. Scenario co-‐design: adaptation of the game scenario, together with the

museum and the implementation partners: 3D2+ and IBBT. In this task,

more research needs to be done on interface design and technical aspects,

like the exporting of Shiva 3D applications to mobile operating systems

and the integration in Shiva 3D of a QR scanning function. (M13-‐M15)

2. Implementation & testing: the actual creation of the application,

integration of 3D2+ and IBBT technologies and testing of the resulting

application. (M14-‐M20)

4 http://apiwiki.twitter.com/ 5 http://developers.facebook.com/docs/api


31

3. Experiment & user tests: exposing users to the actual application and

tesing the effect of the application. (M18-‐M24)


3.3.3.1 3D2+

3D2+ will use the KidNet platform for this pilot. KidNet is a virtual community

for children in which each user has its own login. In order to decrease the

budget, we will use existing KidNet graphics in the pilot.. Several existing KidNet

funtionalities will be left out. For example, there is no need in the pilot to talk

with one’s friends, personalize one’s avatar or one’s island. This means the

interface of the program will have to suit the requirements of the pilot.

The following elements need to be developed to suit the pilot’s game design:

o Text explaining the game

o The design of the 3D world in the future with the missing exhibits.

o Support for the game mechanics that are part of the pilot :

§ Players can match real-‐world exhibits with missing exhibits in the virtual world.

§ Each time the player matches a missing exhibit, this exhibit appears in the virtual world.

o Visualisation of Twitter log in heads-‐up display

o Integration of Facebook “like” button in heads-‐up display

o Single signon, allowing the players to log in to the 3D application

and in the same stretch, with Air Graffiti.

3.3.3.2 IBBT

The Air Grafitti web-‐based application will be used by non-‐gaming visitors to

access content on exhibits in the museum, but currently has a distinct look and

feel that does not necessarily match the style of a museum. As the site can be

skinned, a new look and feel should be applied to more closely match the style of

the museum in which the pilot will be deployed.

Also, the second use case described in 3.3.1.2 mentions the possibility to “like”

content related to a certain exhibit. This like button needs to be added to the Air

Grafitti platform.


32

Another thing that needs to be taken care of is the printing of QR codes that are

large enough to be displayed in a museum. In the current version of Air Grafitti,

the generated QR codes are non-‐printable.


The user involvement, within the pilot has the objective to, on the hand, steer the

integration process and, on the other hand, evaluate the pilot. Therefore, we plan

to involve the users during the pilot phase on three levels:

1. Firstly, the end-‐users will be involved in the shaping process. This is a

two-‐step approach. The first step is an observation phase that will

investigate how people are currently visiting the museum and looking at

the various exhibits. This will be done through ethnographic observations

in-‐situ in which we will focus on how people browse the museum, how

they interact with the artworks, the interaction between them,… In a

second phase, the integration of the various technologies into the specific

Museum Quest game will be co-‐designed with users during iterative

sessions.. Here, existing results of user research from the different core-‐

technologies used (eg. 3D technology 3D2+ and Air Grafitti) will act as a

reference point. This will be done during the integration and set-‐up

phase. The co-‐design sessions will be organized with the specific target

groups (students, teachers, families) and will focus on the requirements

of each group. During these sessions, we will start with some key-‐

concepts (based on the existing technology, the initial scenario and the

results of the observation) that will be checked with these users and

transferred in the concrete design of the applications.(M14-‐M15)

2. Secondly, the users will be involved in the evaluation process. During the

deployment phase of the pilot in a real-‐life setting, we foresee a

permanent evaluation exercise with the ‘end-‐users’ of the Museum Quest.

This is done through an ex-‐ante and post-‐ evaluation exercise performed

by every user that will test the application in-‐situ. (M16-‐M20) During the

ex-‐ante phase we will make a thorough profile of the users with regard to

their ICT and game use as well as their museum experiences. During the

evaluation, a permanent logging will keep track of their activities on the


33

application level. At the end of this phase, an ex-‐post questionnaire will be

conducted to evaluate the users’ experience with regard to the

application.

3. Thirdly, we will involve the stakeholders themselves as users within the

general post-‐evaluation process. This will done through some interactive

workshops in which all stakeholders (museum, 3D2+ , AirGrafitti and LL)

are involved. (M20-‐M24) The goal of these workshops is not only to

evaluate the application and service as such but also to investigate the

process of the integration and the cross-‐border Living Lab exercise. Here,

we will use the templates from WP1. Part of this research will also focus

on how this has helped the SMEs in their business development activities.

4. Evaluation tracking The evaluation of the pilots will be carried out following the questionnaires and

templates provided by WP1 in D1.3.

WP5 Living Labs are responsible for collecting the necessary data to evaluate the

pilots accordingly to WP1 requirements. In particular, Living Labs will be

responsible for collecting and analyzing user feedback during the “piloting

activity both in local and cross-‐border settings, and to collect and analyze SME

feedback about the benefits and problems of collaborating with a cross border

network of Living Labs.

4.1 Transfer Living Labs evaluate SME technology based on user feedback in cross-border settings

The transfer living labs will be responsible for evaluating the technologies they

are piloting, according to the feedback they receive from normal users, experts

and local stakeholders. Living Labs will track two kinds of user data: qualitative

and quantitative. Qualitative evaluation will be done through the analysis of

users feedbacks during the piloting activities. Since each pilot foresees different

levels of user engagement, each living lab will have to define its methodology for

collecting these feedbacks. One example is the co-‐innovation workshops adopted

for the French Pilot. Living Labs will exchange best practices to achieve the best


34

coherence in the evaluation method in order to obtain comparable data.

Common templates for collecting user feedback will be defined by the end of

January 2011 (M15).

In terms of quantitative evaluation, each SME will record its technology logs during

the piloting activities in order to monitor how many users use the applications and

how much content they produce. The local living labs will make sure these logs are

active and up-to- date. The transfer living lab is responsible for analyzing the logs for

its pilot along with qualitative data.

These data will be collected during the pilot. The final evaluation will be completed

by the end of July 2011 (M21).

4.2 SME evaluation of the collaboration with Living Labs in the cross-border experiments

In order to collect SME feedback about their collaboration with living labs in

cross-‐border settings, each participating SME will be interviewed two times :

once by the local living lab and once by the transfer living lab. The local living lab

will interview the SME about its cross-‐border experience and how they

collaborated with the transfer living lab. The goal is to identify the problems

encountered and the role of the transfer living lab in supporting the SME. The

transfer living lab will interview the SME about its collaboration with the local

living lab. The goal is to evaluate how the local living lab supported the SMEs in

preparing and conducting the cross border activity.

Furthermore, in order to keep track of new contacts and business opportunities

that SMEs discovered thanks to the cross-‐border activity in APOLLON, each

Living Lab will draw a graph, representing the connections (projects, partners,

users) that local SMEs had at the beginning of the cross-‐border pilot They will

keep this graph updated with new connections during the piloting activity.

apollon - eparticipation re-adjusted pilot techniques and methodology

Documents