d301.1 open-source personalised user interfaces for older ...€¦ · course "smart-home...

Ecosystem infrastructure for smart and personalised inclusion

and PROSPERITY for ALL stakeholders

D301.1 Open-source personalised user

interfaces for older people and people with mild

cognitive impairments

Project Acronym Prosperity4All

Grant Agreement number FP7-610510

Deliverable number D301.1

Work package number WP301

Work package title Communication, Daily Living, Health, and

Accessible Mobility

Authors Alexander Henka, Lukas Smirek, Gottfried

Zimmermann (HDM)

Status Final version with updates from 4th year

(chapter 5)

Dissemination Level Public

Delivery Date 23/10/2017

Number of Pages 40

D301.1 Open-source personalised user interfaces for older people and people with mild cognitive

impairments. www.prosperity4all.eu

Keyword List

Personalised user interfaces, universal remote console, Eclipse Smart Home, cognitive

impairments, elderly users.

Version History

Revision Date Author Organisation Description

1 29/11/2016 Lukas Smirek HDM Table of Contents

2 20/12/2016 Lukas Smirek HDM Contribution to the overall

architecture; Chapter 1

3 12/01/2017 Lukas Smirek HDM Integration of content from

Alexander Henka, Gottfried

Zimmermann

4 13/01/2017 Gottfried

Zimmermann

HDM Version for internal review


Zimmermann

HDM Final version for publication.

Feedback of reviewers (Daniel

Ziegler, Stefan Schürz) integrated.


Zimmermann

HDM Additional explanation of

relationship to other tasks.

7 10/02/2017 Lukas Smirek HDM Completion of 4.1.1


Zimmermann

HDM Final fixes

9 06/09/2017 Alexander Henka HDM Addition of new scenarios (chap. 5)

10 07/09/2017 Gottfried

Zimmermann

HDM Review, some edits

11 22/09/2017 Alexander Henka HDM Added screenshots and images of

the new scenarios and prototypes

12 27/09/2017 Gottfried

Zimmermann

HDM Review, minor edits

13 23/10/2017 Gottfried

Zimmermann

HDM Fixed some image descriptions for

better accessibility

http://www.prosperity4all.eu/



Table of Contents

Executive Summary .......................................................................................................... 1

1 Contribution to the global architecture ................................................................... 3

2 Introduction ........................................................................................................... 6

3 Towards mainstream adoption of URC concepts ..................................................... 7

3.1 Scientific background of pluggable user interfaces..................................................... 7

3.2 Universal Remote Console and the Internet of Things ............................................... 8

3.2.1 Market situation ................................................................................................... 8

3.2.2 Integration of Universal Remote Console and Eclipse SmartHome .................. 10

3.2.3 Summary and Integration Roadmap .................................................................. 12

4 Showcases of Personal User Experience .................................................................14

4.1 PUX Lab and Technology Platforms ........................................................................... 14

4.1.1 GPII runtime integration .................................................................................... 16

4.2 From AAL Scenarios to Development Approaches for AAL Solutions ....................... 16

4.2.1 Prototypical user interfaces and smart scenarios for older people and people

with mild cognitive impairments ....................................................................................... 17

4.2.2 Personas of Elderly Persons as a Foundation for an Assistive Solution that

Meets Their Needs ............................................................................................................ 18

4.3 Prototypical AAL Solutions ........................................................................................ 18

4.3.1 NetAtmo ............................................................................................................. 19

4.3.2 Forget-Me-Not ................................................................................................... 20

4.3.3 HELMUT .............................................................................................................. 21

4.3.4 SmarPet .............................................................................................................. 22

4.3.5 SmartCare ........................................................................................................... 23

4.3.6 SmartMedicine ................................................................................................... 23

4.3.7 SMEEP ................................................................................................................. 24

5 Prototypical Scenarios in the 4th Year ....................................................................26

5.1.1 Health Hub.......................................................................................................... 26

5.1.2 Smart Drinking .................................................................................................... 29

5.1.3 RemACT .............................................................................................................. 30




6 Conclusion .............................................................................................................33

Annex I: References .........................................................................................................34

Annex II: Glossary ............................................................................................................36

List of Figures

Figure 1. Overall Picture of Prosperity4All ................................................................................. 4

Figure 2. The PUX Lab at the Stuttgart Media University ........................................................ 14

Figure 3. Screenshot of the user interface of our NetAtmos prototype solution. .................. 19

Figure 4. Screenshot showing the current volume (in decibel) and the source of the noise. . 20

Figure 5. Screenshot of the „Forget-Me-Not“ user interface .................................................. 21


D301.1 Open-source personalised user interfaces for older people and people with mild cognitive impairments.

www.prosperity4all.eu

Executive Summary

The goal of task 301.4 "Pluggable user interfaces for home appliances, home entertainment

and home services (targets)" is to support vendors and third parties in developing smart-

home products that provide personalised user interfaces and thus can adapt to the needs

and preferences of an individual user. This support is two-fold in T301.4: First, through the

provision of ready-to-use code and libraries as open-source to be employed by developers

and manufacturers; second, by showcasing simple user interfaces that are specifically

targeted at older people and people with mild cognitive impairments.

While this goal is still valid, some of the means to achieve it have changed. Since the writing

of Prosperity4All's Description of Work (DoW), the technical landscape regarding smart-

home and AAL platforms has changed significantly. The Universal Remote Console (URC)

technology was once a unique platform with regard to its personalisation and third-party

features. However, with the advent of a multitude of mainstream development and runtime

platforms for smart-home and AAL devices and services, driven by powerful companies and

consortia, opportunities have risen to build personalisation features into mainstream

platforms, thus being promoted to a much larger community than the URC community. After

conducting an analysis of existing platforms, we made a decision to incorporate the main

URC concepts into one selected mainstream development platform (the open-source Eclipse

SmartHome project) rather than continuing our work on the URC technical platform as an

"island solution". This incorporation of concepts should serve as a model for extending other

mainstream platforms with the same concepts – this work will have to be tackled by the

platform communities themselves.

The work of task 301.4 – from August 2014 until now – can be summarized in the following

threads:

• Analysis of the current situation with regard to smart-home and AAL platforms, and –

based on this analysis – development of a strategy with the highest possible impact

on mainstream industry (in cooperation with T203.1 "Development tools for adaptive

interfaces for Mainstream Applications")

• Identification of the most important concepts of the URC technology regarding

personalisation, and planning for incorporation of these concepts into the open-

source Eclipse SmartHome project (in cooperation with T203.1 "Development tools

for adaptive interfaces for Mainstream Applications")

• Investigation of new (persona-driven) development strategies for AAL-related use

cases, aiming for an approach that maximizes the awareness of the developers about

the individual needs and preferences of the user (in collaboration with the master

course "Smart-home lab" at the Stuttgart Media University)


http://www.eclipse.org/smarthome/




• Collection of showcase prototypes illustrating AAL solutions that are tailored to the

needs and preferences of elderly persons and persons with mild cognitive

impairments (currently being set up as part of the "PUX Lab" at the Stuttgart Media

University, and available as short videos on YouTube – see 4.3)

Task 301.4 has started in M6 and will last until the end of Prosperity4All. Therefore, this

deliverable can only give a picture of the work within T301.4 until now (Jan. 2017). In the

remaining fourth project year, we plan to make concrete implementation efforts towards

personalisation features within the Eclipse SmartHome project, and to extend our collection

of showcases by applications that adapt to a range of individual users with different needs

and preferences.




1 Contribution to the global architecture

The results presented in this deliverable are based on work that has been done in the

context of task 301.4 of the Prosperity4All (P4A) project. This work builds upon the smart-

home Lab ("Personal User Experience Lab" or short "PUX Lab) at the Stuttgart Media

University.

Within the Prosperity4All project, task 301.4 is closely connected to task 203.1 -

Development tools for adaptive interfaces for Mainstream Applications. This concerns the

integration of the Universal Remote Console (URC) framework with the Eclipse SmartHome

platform (ESH). The main work related to this integration has been done in task 203.1:

investigating and comparing the two technologies, implementation work on the integration

of the two technologies, and reaching out with the ideas of the Universal Remote Console

and of the Global Public Inclusive Infrastructure to the ESH community.

However, the decision of integrating the two technologies – made after the project start as a

result of our market analysis – had a significant impact on task 301.4. This concerns mainly

the original plan to base the implementation of all scenarios developed in task 301.4 on the

URC reference implementation – the Universal Control Hub (UCH). Due to the new

opportunities that were envisioned by the integration of URC with the ESH project, the

decision was taken to deviate from the original project plan in task 301.4. Thus, the

underlying technology for connecting different devices and user interfaces should no more

be the Universal Control Hub on its own. Instead, the new goal is now to make use of the

new capabilities given by its integration with the Eclipse SmartHome framework. This made

it also necessary to perform some experiments with the Eclipse SmartHome technology in a

first stage, before using it in conjunction with the concepts of the URC in a second stage.

Aside from the close connection to T203.1, Task 301.4 is related to the following other tasks:

• Task 202.4 - Smart Device and Environment Interconnection Modules, in which URC

socket templates for home appliances were developed and published. This has been

reported in deliverable D202.2 (Building blocks report on the set of reusable modules,

delivered on 2016-03-16). However, since the course of task 301.4 is now not directly

based on URC, the importance of the socket templates for our smart-home lab has

decreased. Nevertheless, we will conceptually integrate the socket idea and – in parts –

the templates of task 202.4 in the Eclipse SmartHome project, as part of the work in task

203.1 (see the integration roadmap in 3.2.3).

• Task 202.6 - Web-based Smart Personalization and Interface Adaptation Modules: One of

the outcomes of this task is the Adaptive Web Components (AWC) framework. This

framework is now becoming available as open-source release, and can be used in the

development of adaptive user interface in future projects, running in the PUX Lab.




• Task 203.3 - Runtime Environment: The PUX Lab will serve as a demonstrational space for

the GPII runtime environment (which is part of the Prosperity4All DSpace, as managed by

task 202.1). Initial work on integration of ASTERICS and MyUI into the technical

installations of the PUX Lab has already begun.

• Task 502.2 – Events and other dissemination activities: The PUX Lab is a showcase for

personalized user interfaces, to be disseminated beyond Prosperity4all. Videos of the

work in PUX Lab are shared with other interested communities, e.g. the European

Innovation Partnership on Active and Healthy Ageing.

• Task 503.3 – Standardization and Concertation Activities: The work in the PUX Lab is

supporting the development of standards in the areas of software development

processes in AAL (DIN 92419), and technical requirements for RESTful adaptation

frameworks (ISO/IEC 24572-8).

The overall purpose is to demonstrate the feasibility of personalised and adaptive user

interfaces in the context of smart-home environments. Figure 1 marks the task 301.4 as the

context of this deliverable, the tasks 202.4 and 203.1 as sources for its content, and depicts

how the overall WP301 relates to the other work packages and sub-projects of P4A.

Figure 1. Overall Picture of Prosperity4All




Note 1: The original title of D301.1 was "URC sockets template for home appliances and

open-source pluggable user interfaces for older people and people with mild cognitive

impairments". Due to the above described changes in the goals of task 301.4, the title was

changed to "Open-source personalised user interfaces for older people and people with mild

cognitive impairments".

Note 2: The timeline for T301.4 is M6-M48, i.e. it lasts until the end of Prosperity4All. Due to

the DoW, this deliverable is due in M36, i.e. a year before the end of the task. Therefore, this

deliverable reports about what has been done so far in T301.4, and describes the work that is

planned for the remaining year.




2 Introduction

We are facing an increasing amount of interconnected digital devices in our everyday lives

and our homes are heavily striven by these developments. Usually, the term smart-home is

used for a home equipped with different interconnected devices and services that support

and assist its inhabitants in their everyday lives. The provided functionality can range from

comfort and entertaining features like video/audio distribution, over helpful ones for

everyone like energy management (W3C 2016), to such that can provide a more

independent and self-determined life for the elderly or people with disabilities (Tazari 2010;

Emiliani and Stephanidis 2005). Example applications from this third group are fall detection

or cooking assistance.

Giving people the chance for a longer independent life in their familiar environment is not

only about more self-determinacy and self-esteem for them, but is also gaining importance

when considering the social and economic implications caused by the demographic change

that is taking place in most industrial countries. Giving people the chance to stay at home is

expected to be cheaper than taking care of them in specialised nursing homes, and this is a

way to counteract the increasing costs in the health sector. Hence, accessible smart-homes

that comprise appropriate Ambient Assisted Living (AAL) functionality for everyone is of

general economic and social interest.

However, besides these positive aspects one must be aware that there are some significant

challenges. It must be assured that all groups from our society, independent of age, social

background, disability or cultural background can benefit from these new technologies and

are not hindered by them. One major obstacle are inaccessible user interfaces that might

hinder people to use potentially helpful devices and services. Hence, the idea is to provide

personalised user interfaces that can adapt to the specific needs and preferences to the

user. In the context of task 301.4, "one-size-fits-one user interfaces" are employed to

provide personalised and accessible user interfaces to smart-home environments.




3 Towards mainstream adoption of URC concepts

3.1 Scientific background of pluggable user interfaces

The main idea of pluggable user interfaces is to provide a mechanism that enables an easy

exchange of user interfaces in order to give every user the chance to choose the user

interface fitting best their needs. Choosing the best user interface can be done either by the

user themselves, or – as envisioned by the Global Public Inclusive Infrastructure – (semi-

)automatically by the system.

The Universal Remote Console (ISO/IEC 2014; Zimmermann et al., 2004) realises the concept

of pluggable user interfaces with an approach that incorporates ideas from the Model-View-

Controller pattern (Krasner & Pope, 1988).

In case of the URC framework, the three components Model, View and Controller can be

understood as follows:

• The Controller is the software that runs on a (personal) device controlled by the user.

The software is responsible to execute all tasks that have an impact on a target’s

internal state or that execute a certain functionality.

• The Model is an abstract view on a target‘s operational interfaces. It contains all

information that should be contained in a concrete user interface and that are

important to trigger a target’s state, what kind of commands can be sent from a

remote control to the target and what kind of information can be sent the other way.

In terms of the URC framework, the models are called User Interface Sockets.

• The View is the final user interface that is provided to the user and that is usually

rendered on a remote control.

The idea of separating the three components is that the User Interface Socket Descriptions

serve as a contract for user interface developers. Based on the publicly available information

in a User Interface Socket Description, user interface developers can develop specialised

user interfaces (a.k.a. as "personalised" user interfaces) for different user groups to control a

specific target. In order to make specialised user interfaces globally available, the URC

framework realizes the concept of a resource server. On such a resource server, either

whole user interfaces can be stored or parts of a user interface that can be used for

rendering a user interface on the controller device. Examples for such components are labels

and help texts in different languages, sign language videos and icon sets.

At runtime, a user can discover any device in the network that is compliant to the URC

standard with its personal controller device. According to the user interface, descriptions

that are provided by the target and the user’s preferences, the controller can download an

appropriate user interface or user interface components from the resource server and

render a personalised user interface. Since the User Interface Socket Description is the




common base, the user interface can virtually be plugged into a target’s User Interface

Socket and the controller has all information on how to communicate with the target device

and on how to invoke its API.

In order to apply the concepts of the URC standard to non-standard conformant targets, the

Universal Control Hub (UCH) was developed (Zimmermann & Vanderheiden, 2007). The UCH

is a middleware solution that comprises a mechanism to load software components that can

be used to control any back-end technology (target devices and services). These components

are called Target Adapters.

The User Interfaces Socket Descriptions are now no-more provided by the devices

themselves. Instead, they are uploaded to the resource server by the manufacturers of the

target devices and services, or by third parties. Whenever the UCH discovers a new target it

downloads the related User Interface Socket Description from the resource server and

exposes it via its API.

Doing so, controller devices get a transparent view on all targets connected to the UCH and

can communicate with them, as they were standard conformant targets. Therefore,

controllers connect to the UCH, look up the list of available targets and their User Interface

Socket Descriptions, and render an appropriate user interface for the user that was

downloaded from the resource server.

3.2 Universal Remote Console and the Internet of Things

Smart-homes can be seen as one subdomain of the Internet of Things (IoT). Currently, there

is a huge variety of related platforms available to enable the IoT. The UCH has a unique

feature with regard to providing personalised user interfaces provided by third parties via

the resource server. However, it must be taken into account that, with regard to connecting

different devices and back-end technologies, it has to compete with all other frameworks

that are around.

3.2.1 Market situation

The Internet of Things and its subdomains of smart-homes as well as Ambient Assisted Living

(AAL) are now gaining momentum and various players are entering the market. One

outcome of this development is an extremely scattered landscape of different frameworks

and integration platforms. All in all, three types of platforms/frameworks can be

distinguished:

• Platforms developed by big IT companies that try to set a de-facto standard.

• Integration approaches conducted by alliances and organisations. These are either

central platforms to integrate multiple back-end technologies and devices from




various vendors, or specifications for communication protocols that can be used by

various vendors.

• Smart-home solutions developed by one leading company in order to leverage their

own products.

It must be mentioned that the borders between these categories are not always clear. Some

big companies are active in several of the alliances mentioned in the following list.

Projects that are driven by big IT companies:

• Android Things – formerly "Google Brillo" (Google, 2016) and Thread (Thread Group,

2016)

• Apple Home Kit (Apple, 2016)

• AllJoyn framework (AllSeen Alliance 2016) with Microsoft as one of the major driving

forces

• Open Interconnect Foundation (Open Connectivity Foundation, 2016)

Integration platforms and alliances:

• Eclipse SmartHome / openHAB 2 (Eclipse Foundation, 2016)

• Universal Control Hub (Vanderheiden & Zimmermann, 2007)

• FHEM (German "Freundliche Hausautomatisierung und Energie-Messung") (FHEM,

2017)

• EnOcean (EnOcean Alliance, 2017)

• KNX (KNX Association, 2017)

• Z-Wave (Z-Wave Alliance, 2017)

• ZigBee (ZigBee Alliance, 2017)

Solutions developed by companies:

• Bosch (Robert Bosch Hausgeräte GmbH, 2016)

• Samsung Smart Things (Samsung, 2017)

• Miele@home (Miele, 2017)

• RWE’s Innogy (Innogy, 2017)

• Deutsche Telekom’s Qivicon (Qivicon, 2017)

This is not a complete overview over the smart-home market. Still, from our point of view,

the most significant technologies are mentioned. At this stage, further alliances and

competitors are appearing and disappearing and it cannot yet be foreseen which concept or

concepts will gain the largest market shares. However, it is clear that a project like the

URC/UCH that was mainly of academic interest so far, faces some significant constraints

when competing with other platforms that are developed with the support of industrial

global players and their related network of users and suppliers. Hence it seems to be logical

to transform the most important concepts of the URC framework and integrate them into




the platforms with a larger market share and a well-established supporter community. It is

impossible that P4A and the broader accessibility community can incorporate the URC

concepts into all existing platforms. Rather, we will implement these concepts on a selected

platform as a model, so that other developers (from the mainstream community) can do the

same for their platforms.

3.2.2 Integration of Universal Remote Console and Eclipse SmartHome

The original intention of task 301.4 was to build a smart-home environment based on the

URC framework and its reference implementation UCH. Although the UCH is advanced in its

capabilities to provide pluggable and personalised user interfaces, it has one major

shortcoming – the limited amount of connector modules (Target Adapters) to different back-

end technologies. Hence, the Stuttgart Media University has focused on a transformation

and integration of the URC technology into one of the main current smart-home

development platforms. This was also done in order to connect the URC community with

one of the larger mainstream platforms.

When the decision was taken which platform to choose, the following facts were of special

importance:

• Open-source code

• Development stage / availability of code

• Market impact

• Support for third party contributions

Having these criteria in mind, Stuttgart Media University decided to not use one of the

platforms developed by a specific equipment manufacturer. This is due to the fact that, in

the AAL domain, it is important to have the option to choose from a variety of devices,

services and assistive technologies, possibly provided by multiple vendors, to compose the

best solution for an individual user. Choosing one specific equipment manufacturer’s

platform would limit the variety of available devices and services. Furthermore, most of the

platforms are not open-source.

On the other side of the spectrum, there are the platforms developed by the big IT

companies trying to build a de-facto standard. It is likely that these platforms will take a big

share in the smart-home and Internet of Things market of the future. However, there are

also constraints concerning these technologies.

Starting with Apple Home Kit – at the time of investigation the most advanced platform in

terms of available features and devices. Unfortunately, it is a rather closed system (a.k.a.

"walled garden"). Consequently, the chance to modify the platform to our requirements or

to gain an impact on the developer community seems to be fairly small.




While Apple Home Kit was ready for use, the Google products Brillo and Thread had nothing

more than a very simplistic home page and no code base ready for use yet. Hence, they did

not qualify for our purposes.

The AllJoyn framework developed under the roof of the AllSeen Alliance already had a

prototypical code base. Although the code base was available, there were and there are still

not yet many devices available for usage. Another issue is that manufacturers of Internet of

Things devices that want to use the AllJoyn technology are required to integrate the AllJoyn

code on their devices. This seems to be rather inconvenient for prototyping in the project or

smaller assistive technology companies in general.

Finally, there is the Open Interconnect Foundation (OIF). At the time of investigation, the

foundation already provided about 60 descriptions of device functionalities that had some

similarities to the User Interface Socket Descriptions known from the URC framework.

Unfortunately, there was no code base available at that time.

Summing up, none of the technologies aiming on becoming a de-facto standard nor the

technologies developed by one dedicated manufacturer were suitable for our purposes.

When looking at the large group of integration platforms and Alliances, it was clear that, in

order to build relevant AAL scenarios, an approach was needed that was not limited to the

communication layer only. The decision was taken to investigate the possibilities of

integrating the URC technology with the Eclipse SmartHome project (ESH). The various

reasons for this decision were as follows.

First, the ESH project aims to not only support and integrate different smart-home

technologies in heterogeneous environments, but also to support AAL applications. Next, the

ESH project and the closely related openHAB platform was an already established

community and the code was ready for use. For example, the Deutsche Telekom’s smart-

home technology Qivicon is based on ESH (Eclipse Foundation, 2016).

Furthermore, the UCH and the ESH platform are both Java based systems which greatly

simplifies their integration. Also, the ESH uses a concept of device and service abstraction

similar to the URC framework and the UCH.

The detailed results of the investigation were published as conference paper (Smirek et al.,

2016) at the International Conference on Emerging Ubiquitous Systems and Pervasive

Networks (EUSPN). The summarized result of the investigation is that there are sufficient

similarities between the systems for a future integration, and there are benefits on either

side that are not available on the other side.

The common ground for a future integration of ESH and URC is first, the idea of integrating

different back-end technologies, and second, the provisioning of abstract descriptions for

the connected devices and services. The two projects are complementary since URC can

profit from the well-established community of ESH and its status as an official Eclipse




project. Reaching out to the ESH community brings also the chance to make this community

familiar with the ideas of GPII.

Furthermore, the URC community can benefit from the large amount of back-end

technologies supported by the ESH community. At the same time, ESH can benefit from URC

by adopting the concept of a resource server for user interface resources. So far, ESH

provides a declarative model for device descriptions. However, there are no standardized

descriptions for devices and their functionalities available that would be similar to the URC

User Interface Socket Descriptions. Our intension is to demonstrate the benefits of

standardized device descriptions to the ESH community.

These contributions to the ESH framework would facilitate a platform for flexible and open

user interfaces. This would enable users to download and use user interfaces according to

their personal preferences and needs, and appropriate to the context of use. Furthermore,

third parties would be able to design supplemental user interfaces for specific user groups

with special requirements.

3.2.3 Summary and Integration Roadmap

Due to the competitive market situation for the Internet of Things and smart-home

platforms, a decision in the Prosperity4All project was taken to work towards an integration

of personalisation concepts (such as featured by the URC technology) in the Eclipse

SmartHome project. As a result, the ESH platform is envisioned to be augmented to support

the concept of personalised user interfaces in the spirit of URC and GPII.

Our "integration roadmap" has the following phases and steps, which are planned to be

carried out in the last year of the Prosperity4All project:

Phase I (Feb-Sep 2017): Prototype development

Step 1: Model client implementation with personalisation features

• A client will be implemented as a component that is deployed within the ESH

runtime. It will communicate with the GPII infrastructure and with a user interface

resource server. The client will retrieve needs and preference sets from a GPII

preference server, and collect other context information (including task, equipment

and environment context). Based on this overall context information, the client will

receive personalised user interfaces or user interface components from a user

interface resource server, to be employed in building a personalised user interface

for the user.

Step 2: Exemplary integration with ESH standard channel types

• Based on our URC User Interface Socket Description for the VLC player and the

Wöhlke Websteckdose (remote controllable power outlet), we will integrate




"standard device types" for media players and power outlets in the ESH framework

via bindings. In contrast to most of the currently integrated devices in ESH, the

channel type descriptions will not be included in the bindings of the two targets.

Instead, we will provide a third binding that includes system-wide channel type

descriptions which will serve as "standard device descriptions".

Step 3: Exemplary personalised user interfaces

• We will develop sample user interfaces that look up an ESH instance for available

devices and their channel types. Based on this information, they will connect to the

discovered devices via the ESH REST API. This will be a significant improvement for

ESH since it will allow for automatic discovery of new devices and related user

interface resources for personalisation provided by third parties.

Phase II (Sep 2017 – Jan 2018): Harmonisation and standardisation

Step 4: Workshop with ESH developers

• We will continue the work of the ESH/URC workshop that took place on July 22-23,

2015, looking for ways of combining the strengths of URC and ESH. We will host a

second workshop and invite ESH developers to discuss future strategies of extending

ESH with personalisation features. On this second workshop, we will present our

prototypical work (see steps 1-3 above) to demonstrate the benefits of standardised

channel types and their value for personalisation and adaptation. The outcome of the

workshop will be a joint roadmap for the further development and promotion of

personalisation features within the ESH community, and potentially their

standardisation.

Step 5: Implement joint roadmap

• Based on the joint roadmap (see step 4), we will pursue the overall goal to jointly

work on a common, integrated open-source infrastructure to enable personalisation

in smart-homes and AAL environments.




4 Showcases of Personal User Experience

4.1 PUX Lab and Technology Platforms

The Personal User Experience Laboratory (PUX Lab) at the Stuttgart Media University (HDM)

was established to support research and development of scenario-based and personalised

solutions in a smart environment, combining approaches from the fields of smart-home,

AAL, and IoT (see Fig. 2 below). The PUX Lab has been set up in the context of P4A, and will

be a permanent laboratory at the Stuttgart Media University beyond the lifetime of the

project. It is open for students and other researchers in designing personalised and

accessible user interfaces, building upon the knowledge gained and artifacts developed in

P4A and other projects. Currently (as of Dec. 2016), the PUX Lab is being used intensively by

students for various projects and theses.

Figure 2. The PUX Lab at the Stuttgart Media University

Originally, the PUX Lab was intended to be driven by the Universal Remote Console (URC)

technology and its gateway hub, the Universal Control Hub (UCH) (Zimmermann &

Vanderheiden, 2007). However, there have been significant changes in the technology

landscape of smart-homes, driven by mainstream trends in an area commonly referred to as

the Internet of Things (IoT). These changes have motivated us to rethink our strategy for the

transfer of personalisation technologies combined with an appropriate expertise on the

development of personalised and accessible smart-home systems and its components. With

multiple mainstream development platforms for smart-homes and IoT being available for

free, driven by large industry-led consortia, it does not make sense to only focus on URC,

even though it may have the best selection of personalisation features. If we concentrated




our efforts on the further development of URC, we would end up in building a technology

island that would be nice to demonstrate, but would never be used by the mainstream

market.

See section 3.2 for details and further rationale for the process of investigating current

mainstream platforms and choosing one as our implementation basis. In a nutshell, we

discovered that Eclipse SmartHome (ESH) has some significant similarities to URC,

concerning its concepts and implementation style, but supported by a much larger

community than URC. Hence, our mission now is to participate in the development and

improvement of ESH towards a fully personalisation-enabled smart-home platform. Our task

is to transfer the main principles of URC and GPII into the ESH community.

Prototypical user interfaces for older people and people with mild cognitive impairments, as

highlighted in the following subsections, have been built with the ESH framework as the

underlying software stack. Like URC, ESH can also be installed on small devices (e.g.

Raspberry PI or other embedded computer technology).

The Global Public Inclusive Infrastructure (GPII) provides user interface adaptations based on

a users’ specific needs and preferences. The preferences are stored in so-called "preference

sets", which are machine readable representations of needs and preferences. To transfer a

preference set between contexts, machine learning and rule-based algorithms can be used.

When adapting a specific computer to the needs of a dedicated user, this may involve the

use of several different pieces of software, for instance on-screen magnification, on-screen

keyboard, and a higher volume for speech output. The context in a smart environment

differs in some major points from adaptions for single devices like personal computers.

1. Here, we may have a context where multiple devices are involved in one scenario.

This calls for one “central” component, that utilizes the preference set and is

responsible for the adaptation of multiple devices and/or services.

2. With respect to older people and people with mild cognitive impairments, there may

be situations where devices or services which adapt to the user’s needs, do not

provide a user interface that is controllable by that specific user, but by the nursing

staff. They (the older people) may need assistance in case of an emergency.

3. There may be scenarios where the overall environment needs to adapt to a user's

needs; for instance, in the sense of ambient assistive living (AAL). Therefore, the

environment in its entirety needs to adapt to the needs of its residents.

4. Also, there may be needs or preferences which cannot be directly satisfied by the

adaptation interface components. For instance, somebody can have the need to

control things remotely with by means that fit his needs or to get a reminder for

appointments. Scenarios like these can be common for smart environment.

5. Also, new types of needs and preferences can occur in smart and AAL environments.

Besides user interface and service oriented needs and preferences (e.g., font-size,




contrast, screen-reader support, or magnification), there can be settings for

technology solutions that provide an extended level of assistance in the context of

AAL (e.g., reminder services, social services, remote control service, or assistive

services for fall detection or emergency calls).

4.1.1 GPII runtime integration

One of the major aims is to use the PUX lab as a demonstrator platform for the GPII runtime

environment. The GPII runtime environment was developed by FHG, HDM and UCY, under

task 203.3 as part of the Prosperity4all project (Mettouris & Komodromos & Smirek &

Ziegler, 2016). It combines The AsTeRICS, MyUI and URC frameworks in a way that adaptive

user interfaces for persons with motoric impairments can be supplied. AsTeRICS provides

alternative input capabilities like a head mouse, MyUI an adaptive graphical user interface

and URC/KCH is the connecting element to the available hardware that shall be controlled.

The show cases are based on the use cases described in [D203.1] and enable people with

motor impairments to control a Philips Hue ambient light system, a ventilator and an electric

heater via different ways of head tracking.

The installation of the GPII runtime environment and the related hardware is currently going

on. A first setup is expected to be done at the end of March 2017.

4.2 From AAL Scenarios to Development Approaches for AAL Solutions

“Monika, 72 years old, Stuttgart, Germany. She has a mild cognitive impairment. She is often

disoriented and forget things. She often falls asleep on the couch and has an age-related loss

of hearing and seeing.

A reminder function reminds Monika to take her medicine. The information, which medicine

to take, is specified by a professional care taker or a family member. Monika is informed to

take the medication on several sensorial levels, by using different devices in the smart

environment. An auditory notification is played on the sound system in the living room where

Monika currently is in. The volume of the TV is therefore turned down and a notification is

shown at the television, in an appropriate font-size, contrast ratio and color scheme that

accommodates her needs. Had she fallen asleep, the smart-bracelet at her wrist would have

also started to vibrate.

A colored light at the medication box indicates the medication to take. The box is smart; it

recognizes when Monika has taken the medication and can notify the responsible care taker

or a family member about the status.

In this scenario, the smart environment of Monika’s home adapts to her needs and

preferences. It is aware of different context situations and can make adjustments on how to

remind Monika to take her medication. It can play the auditory notification in the room




where Monika is currently in. If she is sleeping, the smart system may decide to send the

notification signal to her bracelet, which would start to vibrate.”

While scenarios like this conform to the idea of AAL (which is providing an independent and

assistive lifestyle for older people), the question is who is responsible for providing the

information about Monika’s need and preferences and how do all the different devices work

together?

Smart-home middleware technologies like Eclipse SmartHome (ESH) provide developers with

the tools to connect different devices, using communication technologies (e.g., IP, ZigBee,

EnOcean, ZWave, or Bluetooth Low Energy). Besides sub-level communication, they also

provide support for decision making and scenarios, which is: “If a notification should be

delivered to the user, switch on the light in the living room, but only if the user is present in

that room; otherwise, play the notification as an audio message in the kitchen, if the user is

there.”

When dealing with scenarios like that, middleware technologies such as ESH, or, to be more

appellative, the smart environment in its entirety, constitutes a solution in the sense of the

GPII. Here, it is the smart-home middleware that takes input from the GPII system, utilizes

preference sets and adapts the environment accordingly.

Solutions like this do not come out of the box. They need to be researched, conceptualized

and developed. With reference to the scenario above, a smart medicine-reminder-system

(MRS) could be provided by some third-party manufacturer, thus building an ESH-based,

GPII-enabled, smart medicine-reminder-system. Suppose, another person, Peter, is using the

same MRS. system as Monika, but he does not have a dedicated sound system to play

notifications. The MRS. solution could therefore decide to make phone calls and notify Peter

via his landline. It is the same product as Monika’s, but adapted to the preferences of Peter

and his specific context.

Any third-party developer, building applications on basis of GPII, will need to have an

integral understanding of the needs and preferences of the end-users in a smart context to

fully utilize the information that the GPII provides. In working on T301.4, we tackled this

task by focusing on the process of creating such solutions under considerations of user-

centered development approaches to keep track of the needs and preferences. The

following section highlights our developed scenarios for smart-context situations and user

interfaces of older people and people with mild cognitive impairments.

4.2.1 Prototypical user interfaces and smart scenarios for older people and people with mild

cognitive impairments

Instead of providing plain user interfaces for single devices in a smart environment, like for a

thermostat or shutter control, we aim for developing prototypical solutions for elderly users

in smart-home scenarios. This can involve multiple devices or adaptation of the smart




environment as such, without a concrete user interface to be controlled by the user. We

develop prototypical implementations of such solutions.

In the past, we have used tools and tutorials from the Prosperity4All project, like the Socket

Builder (T202.4) and Guidelines for User Interface Sockets (T202.4). As part of our work in

Prosperity4All, we contribute to new standardization approaches to design assistive systems

and products for AAL (T503.3). A new design process approach based on our contributions to

standardization has been used to develop prototypical smart environment solutions. As part

of a master's course, and anchored in the PUX Lab, we have students from the Stuttgart

Media University participate in the development of prototypical solutions.

4.2.2 Personas of Elderly Persons as a Foundation for an Assistive Solution that Meets

Their Needs

As a foundation for our prototypical solutions, we often use the CURE-Personas as a set of

well-defined representatives of the heterogenous group of older users. The CURE-Personas,

introduced by Wöckl et. al (Wöckl, 2012) are a set of 30 elderly personas (age 50+),

developed from the quantitative data of nearly 12,000 people from all across Europe to

provide a grounded set of personas to support researchers and developers of ICT products1.

The CURE-Personas are developed as a design tool to support researchers and other

professionals in the design of AAL related products. We used these personas as the target-

users of our prototypical solutions. Therefore, they served as human-readable definitions of

user needs and preferences.

We argue that providing GPII-enabled AAL solutions (in the context as described above)

requires an understanding of the diversity of impairments and the needs and preferences

linked to that. We believe that only solutions that can overcome the full range of issues and

barriers that individual users encounter can utilize the GPII system in its full potential and

provide adaptions for a wide audience of people (Henka et al., 2016).

4.3 Prototypical AAL Solutions

This section provides brief introductions for the prototypes of T301.4. For each prototype,

we identify the personalisation features that are built into the system. Also, we provide links

to the publicly available source code and to videos documenting the underlying concepts.

Note: The prototypes presented here (as of Dec. 2016) do not yet show a fully GPII-enabled

adaptation ability. However, all of them were designed with a very specific user group in

mind (mainly elderly persons and persons with mild cognitive impairments). This involved

the study of the needs and preferences of the targeted user group from which the

requirements were derived in a special development process. Designing for a particular user

1 They are made publicly available under: http://elderlypersonas.tech-experience.at




group (following a "one-size-fits-one approach") is a first step towards personalised user

interfaces. As a second step, we are aiming for GPII-based adaptation mechanisms that are

built into the applications running in our PUX Lab. This step is planned for the summer

semester 2017, i.e. by M43 of the project.2

4.3.1 NetAtmo

This solution utilizes a weather station with various sensors that can also measure the noise

level and the CO2 concentration in the air. The scenario was designed for a persona named

Mrs. Sabine Wohlfahrt, an older lady from Dortmund, Germany. She has age related hearing

and visual impairments. Therefore, she usually raises the volume of her television and radio

to a high level, which sometimes makes her neighbors complain about this disturbance.

The scenario here is that Mrs. Wohlfahrt feels somewhat embarrassed and awkward that

her neighbors are complaining about her. This prototype is integrated in the Eclipse

SmartHome (ESH) platform. Therefore, the solution can track the noise level in the living

room and can state the source of the noise; whether it is the radio or the television. It can

therefore inform Mrs. Wohlfahrt which specific device may cause a potential disturbance.

Hence, she can turn down the volume.

The user interface runs as web application on a tablet (see Figures 3 and 4).

2 Note that T301.4 is planned to run until Jan. 2018 (M48), according to the DoW.

Figure 3. Screenshot of the user interface of our

NetAtmos prototype solution.




The “NetAtmo project” prototype has the following personalisation features:

• The user interface is designed to match the needs and preferences of a user group

represented by Mrs. Wohlfahrt.

• Reminder functionality for devices with a high noise level, accommodates Mrs.

Wohlfahrt’s personal need of getting the information when the sound level disturbs

the neighbors.

• List with currently available, seasonal fruits and vegetables, according to her personal

diet plan.

• Stress indicator to reduce Mrs. Wohlfahrt’s stress after a heart attack. Thus,

providing information about the current stress level of her surrounding, based on

volume and air quality.

The Source Code of the NetAtmo is project is publicly available in our research repository

under

https://github.com/REMEXLabs/SmartHome_SomSe2015/tree/master/NetAtmoProject.

A video showcasing the prototype is publicly available (including captions) at

https://youtu.be/uF62gFrxT6Y.

4.3.2 Forget-Me-Not

“Forget-Me-Not" is a prototype of a reminder solution to remind a user to take medication,

about upcoming appointments or to water the plants. The solution was developed as an

Android application. The reminder can be displayed either as audio output or visual output,

using the installed audio or light system in one’s (smart) home. This solution also involves

the connection with a sensor that observers the condition of the plants, e.g., the moisture

level.

Figure 4. Screenshot showing the current volume (in decibel) and

the source of the noise.


https://github.com/REMEXLabs/SmartHome_SomSe2015/tree/master/NetAtmoProject

https://youtu.be/uF62gFrxT6Y



Forget-Me-Not has the following personalisation features:

• The reminder can be set to audio or visual output, or both.

• Individually configurable reminders for medication.

• Calendar for individual appointments of the user.

The source code of Forget-Me-Not is publicly available in our research repository at

https://github.com/REMEXLabs/SmartHome_SomSe2015/tree/master/NetAtmoProject.

A video showcasing the prototype (German narration, but English captions provided) is

publicly available at https://youtu.be/oHzs7-2EJxA.

4.3.3 HELMUT

“Helmut” is an old German name and means “guardian” or “protector.” Here, Helmut is a

smart solution for Ingobert Tugend, a CURE persona3 (Wöckl, 2012). Ingobert is 79 years old

and lives alone. He has issues with his memory (cognitive impairment) and rheumatism. He

is also afraid of a third heart attack.

In a nutshell, his needs and preferences are:

• He has an age-related memory loss and lives alone, which could lead to potentially

dangerous situations if he forgets for example to turn off the stove.

• He has rheumatism, which causes pain. He therefore has a preference for controlling

things in his household remotely, with a device that fits his needs.

• Since he lives alone, he is in fear that nobody will come to his aid if he is feeling

badly. He is afraid of a third heart attack; therefore, to die unnoticed.

3 Personas - Copyright © 2011 CURE-Elderly-Personas. All rights reserved. Reproduced under license.

Further copying is prohibited. http://elderlypersonas.cure.at

Figure 5. Screenshot of the „Forget-Me-Not“ user interface


https://github.com/REMEXLabs/SmartHome_SomSe2015/tree/master/NetAtmoProject

https://youtu.be/oHzs7-2EJxA

http://elderlypersonas.tech-experience.at/a4_descriptions/CeP_A4_Descriptions_Ingobert_Tugend.pdf



This AAL solution was implemented using a smartwatch. Via this watch, Ingobert can control

his environment e.g., open or close windows or switch the lights on and off. He can call for

help in an emergency situation by tapping on the watch. By connecting the smartwatch and

its app to the Eclipse SmartHome, the watch reminds him to turn off the stove in cases

where he left the stove turn on after finishing cooking.

HELMUT has the following personalisation features:


represented by Ingobert Tugend. Here, implemented as a smartwatch application.

• A reminder functions that reminds him that the stove, the door or the faucet is still

on.

The source code of HELMUT is publicly available in our research repository at

https://github.com/REMEXLabs/SmartHome_SomSe2016/tree/master/HELMUT.

A video showcasing the prototype is publicly available at https://youtu.be/VVyUB4InzWI.

4.3.4 SmarPet

SmarPet is a solution designed for Luise Insel, a CURE persona4 (Wöckl, 2012). She is 88 years

old and single. She has arthritis, diabetes and feels occasionally dizzy, which is one of her

greatest fears that she falls unnoticed.

For her, it is most important to prevent potential fall risks and, in the event of a fall, call for

immediate help and support. Our prototype accommodates to this need by providing a

detection for dizziness and an automatic emergency call in the case of a fall. This solution

was implemented with the use of a touch sensitive floor (installed in the PUX Lab) that can

detect movement and the characteristic pattern of a toppled person.

The principle is that the floor can detect the typical movement-pattern of Luise if she is

feeling dizzy and staggers around and cases where she falls. In both cases, Luise's care taker

gets a notification and will come to Luise's aid. In addition, it would also be possible to

automatically switch on the lights triggered by Luise's movement; therefore, it will always be

bright when she wakes up in the night and tries to find the way to her toilet.

SmarPet has the following personalisation features:

• Monitoring of personal movement-patterns to detect dizziness and the case of a fall.

• Illumination of the individual way to the toilet if she feels to get up at night.

The source code of SmarPet is publicly available in our research repository at

https://github.com/REMEXLabs/SmartHome_SomSe2016/tree/master/SMARPET.




https://github.com/REMEXLabs/SmartHome_SomSe2016/tree/master/HELMUT

https://youtu.be/VVyUB4InzWI

http://elderlypersonas.tech-experience.at/a4_descriptions/CeP_A4_Descriptions_Luise_Insel.pdf

https://github.com/REMEXLabs/SmartHome_SomSe2016/tree/master/SMARPET



A video showcasing the prototype is available at: https://youtu.be/jiURUDil2Oo.

4.3.5 SmartCare

SmartCare was a solution designed for Ms. Allenson, a CURE persona5 (Wöckl, 2012). She is

88 years old and lives alone. She had a stroke and an age-related memory loss and relies on

a professional care service. She usually stays at home, but she has motoric impairments

which cause pain, especially when she gets up and walks around.

She often forgets to turn the radio off and let the shutters down, which are troublesome

tasks for her since it causes her pain to get up from the bed, lower the shutter and turn the

radio off. She often wakes up at night, when the radio is still playing, and she gets blinded by

the street light in front of her bed room window. This induces a high stress level to her that

prevents her from falling asleep again.

Ms. Allenson's needs are to control things remotely and to sleep well again without being

disturbed by the radio or the streetlight.

Our prototype features a smart watch that can measure her pulse. On the basis of her pulse,

the watch can detect when Ms. Allenson falls asleep and will then automatically switch off

the radio and lower the roller shutters. On the next day, the watch could also detect the

process of waking up and can open the shutters again. Note that the smart watch is just a

concept of something that can be worn at the wrist. In a final product, this could also be a

bracelet or something else, and therefore will not cause any stigmatization.

SmartCare has the following personalisation features:


represented by Ms. Allenson.

• Functionality embedded in a personal device, worn at the body.

The source code of SmartCare is publicly available in our research repository at

https://github.com/REMEXLabs/SmartHome_SomSe2016/tree/master/SmartCare.

A video showcasing the prototype is publicly available at

https://youtu.be/0Mqi4tMqyR4.

4.3.6 SmartMedicine

The SmartMedicine solution is a prototype of a smart pillbox system that reminds a user

when it’s time to take medication and can track the taking of pills. Michael Elend, a CURE




https://youtu.be/jiURUDil2Oo

http://elderlypersonas.tech-experience.at/a4_descriptions/CeP_A4_Descriptions_Ms_Allenson.pdf

https://github.com/REMEXLabs/SmartHome_SomSe2016/tree/master/SmartCare

https://youtu.be/0Mqi4tMqyR4

http://elderlypersonas.tech-experience.at/a4_descriptions/CeP_A4_Descriptions_Michael_Elend.pdf



persona6 (Wöckl, 2012), is 75 years old and lives with his wife. He has diabetes,

hypertension, and a cataract that limits his vision.

Due to age-related memory-loss, he needs to be reminded to take his medication. The times

when to take the medication are stored in a calendar, entered by his wife or children. The

solution uses the lights in his home to notify him to take his medication. The pillbox has

various sensors that can track the interaction with the pillbox; thus, the taking of pills.

A small hall-effect sensor7 in the cap of the box recognizes if the box is open and a tilt sensor

tracks the movement when the pill box is getting tilted in order to extract a pill. In cases of a

successful pill extraction the system turns off the lights on the pillbox and informs the back-

end system (the calendar) that Michael Elend has taken his medication.

SmartMedicine has the following personalisation features:


represented by Michael Elend.

• Individually configurable reminders for taking medication.

• Individually configurable notifications for care takers and family members to notify

that the medication has been taken.

The source code of SmartMedicine is publicly available in our research repository at

https://github.com/REMEXLabs/SmartHome_SomSe2016/tree/master/SmartMediCine.

A video showcasing the prototype is publicly available at https://youtu.be/9SVyQ2ZDJHc.

4.3.7 SMEEP

SMEEP is a prototype for a solution developed for Ingobert Tugend, a CURE persona8 (Wöckl,

2012). Here, we focused on a solution to accommodate Ingobert’s need for a restful sleep.

The SMEEP solution is also based on the touch-sensitive floor and the lighting in a home. The

system supports a calm transition into sleep. If the human organism is exposed to glaring

light, the body’s melatonin is prevented. Melatonin is a hormone that is essential for getting

tired. With this prototype, Ingobert can set a time for when he wants to go to sleep. SMEEP

than starts to dim the lights in his home in order to reduce the illumination, which may favor

the production of melatonin.

If Ingobert feels the need to get up at night, the touch-sensitive floor tracks his movement

and provides an illuminated way, for instance to the toilet or to the kitchen.


Further copying is prohibited. http://elderlypersonas.cure.at 7 A transducer that varies its voltage output in correlation to a magnetic field. 8 Personas - Copyright © 2011 CURE-Elderly-Personas. All rights reserved. Reproduced under license.



https://github.com/REMEXLabs/SmartHome_SomSe2016/tree/master/SmartMediCine

https://youtu.be/9SVyQ2ZDJHc


http://elderlypersonas.cure.at/



SMEEP has the following personalisation features:


represented by Ingobert Tugend.

• Individual configuration for sleeping time, and illumination settings.

The source code of SMEEP is publicly available in our research repository at

https://github.com/REMEXLabs/SmartHome_SomSe2016/tree/master/SMEEP.

A video, describing the prototype, is publicly available at: https://youtu.be/mxZBJpS_8f0.


https://github.com/REMEXLabs/SmartHome_SomSe2016/tree/master/SMEEP

https://youtu.be/mxZBJpS_8f0



5 Prototypical Scenarios in the 4th Year

This section provides an overview on the prototypes of T301.4 that have been developed in

the 4th year in addition to the 3rd years’ prototypes introduced in section 4.

5.1.1 Health Hub

“Health Hub” is a smart solution that provides health tracking and UI adaptations for Karl-

Heinz Ruhend9 and Stefan Vater10 (Wöckl, 2012).

Karl-Heinz is 85 and lives with his wife. They have an adult daughter. Karl-Heinz has an eye-

cataract, arthritis, high blood pressure, and sometimes difficulty in breathing. He often fears

to collapse and to fall down due to his severe conditions. In moments with high blood pressure

and heart rate, he starts feeling dizzy and is often worried that he might have another heart

attack. The resulting panic increases the symptoms even more. As a result, he is constantly

anxious about his conditions, which affects the daily life of him and his wife in a sense where

he prefers to stay at home or to avoid potentially stressful situations in general.

In a nutshell, Karl-Heinz’s needs and preferences are:

• The need of validating his health condition whenever he feels it is necessary to calm

himself – and his wife and daughter - down.

• The need of checking his health in a simple and self-reliant fashion. He doesn’t want

to move into a nursing home or see the doctor’s all the time.

• Simple representation of data in the form of statements as: “Relax, your health

condition is fine!”

• Big icons, images, and texts

Stefan is 63. He has a little overweight and high blood pressure. Unlike Karl-Heinz, he is still

very active with sports and other activity that keeps him mentally and physically in shape. For

him, being active is a way to cope and counter his medical conditions.

In a nutshell, Stefan’s needs and preferences are:

• He hast wants to track his vital data to measure his trainings (sport) progression.

• The need of having a detailed overview of his biometric data.

• The preference of having a continued tracking of his data over time.


Further copying is prohibited. http://elderlypersonas.cure.at 10 Personas - Copyright © 2011 CURE-Elderly-Personas. All rights reserved. Reproduced under license.





Figure 6 The Health HUB pulse overview screen

Using “Health Hub”, the health data can be checked by placing the hand on the “Health Hub-

Scanner”, a device which uses the fingerprint as an authentication mechanism and to load an

adapted user the interface to accommodate the needs and preferences of users. Through

various built-in sensors, “Health Hub” can measure the heart-rate, body temperature and the

stress level. Figure shows the scanner.




Figure 7 The Health HUB scanner

Using Health-Hub, Karl-Heinz gets the information on the status of his health condition in the

form of big smiles, following a traffic light schema, ranging from a green laughing smile –

indicating that everything is okay – to a red smile, indicating a severe condition.

The Health Hub system is integrated into the smart environment, that can utilize the

information provided by the scanner. For instance, Karl-Heinz’s daughter is automatically

informed if his health condition declines. Based on the information provided by Health Hub,

the smart environment can support Karl-Heinz’s well-being by utilizing the sound system

(playing music) and light installations.

Health Hub accommodates Stefan’s needs and preferences by providing a detailed view on

his health data. This includes the precise numbers, time stamps as well as statistics over the

past days, weeks, and months. Here, Stefan can check his data before and after activities to

constantly measure his progress.

The source code of Health Hub is publicly available in our research repository at

https://github.com/REMEXLabs/SmartHome_SomSe2017/tree/master/HealthHUB.

A video showcasing the prototype is publicly available at https://youtu.be/hBb-Izq3hcY

(English audio and subtitles available).


https://github.com/REMEXLabs/SmartHome_SomSe2017/tree/master/HealthHUB

https://youtu.be/hBb-Izq3hcY



5.1.2 Smart Drinking

Smart Drinking is a system that reminds residents to take fluid. Smart Drinking was developed

with the user groups represented by the personas Luise Insel11 (72) and Mr. Traussen12

(Wöckl, 2012) in mind.

Dementia is a disease that causes that one does not recognize well-known objects, people and

circumstances anymore. Forgetting to drink on a regular basis leads to dehydration and severe

medical conditions. The Smart Drinking system is tackling this issue by utilizing sensors in a

smart environment to remind people to drink. Smart Drinking features a pedestal to fit any

customary bottle. The pedestal is equipped with sensors that measure the fluid level of the

bottle. Smart Drinking notifies and motivates the users periodically to drink. The motivation

can be stimulated by playing an audio notification, showing a video featuring a drinking

person, or the “sound of drinking” (a recorded sound sample of a person drinking). The

appropriate notification style depends on the needs and preferences of the user.

Additionally, Smart Dinking can inform relatives or medical care-takers about the user’s

hydration status.

Figure 2 A generic drinking bottle inside the smart drinking pedestal.


Further copying is prohibited. http://elderlypersonas.cure.at 12 Personas - Copyright © 2011 CURE-Elderly-Personas. All rights reserved. Reproduced under license.




http://elderlypersonas.tech-experience.at/a4_descriptions/CeP_A4_Descriptions_Mr_Traussen.pdf



The source code of Smart Drinking publicly available in our research repository at

https://github.com/REMEXLabs/SmartHome_SomSe2017/tree/master/SmartDrinking

A video showcasing the prototype is publicly available at https://youtu.be/6Z0vEWqrmB8

(German audio only, but German and English subtitles available).

5.1.3 RemACT

RemACT stands for remind and act. “Remind” refers to the feature to remind users about

potentially dangerous scenarios like a switched-on stove or an open entrance door. “Act” is

assigned to the feature that if the user does not react to the reminder, the system itself acts

and, for instance, switches off the stove. RemACT does not want to infantilize the users by

doing it automatically all the time, since the elderly user group, represented by Luise Insel13

and Ingobert Tugend14 (Wöckl, 2012), should be supported in active and an self-determined

life style.

Luise Insel is 88 years old, she is oblivious and has a mild form arthritis. She is constant

concerned about her security and safety around the house. Sometimes, when she is already

in her bed, she is not sure if the front door is locked. Getting out of bed and down the stairs is

pain- and stressful for her, due to the arthritis.

In a nutshell, her needs and preferences are:


Further copying is prohibited. http://elderlypersonas.cure.at 14 Personas - Copyright © 2011 CURE-Elderly-Personas. All rights reserved. Reproduced under license Further copying is

prohibited. http://elderlypersonas.cure.at

Figure 8 User interface for Luise Insel


https://github.com/REMEXLabs/SmartHome_SomSe2017/tree/master/SmartDrinking

https://youtu.be/6Z0vEWqrmB8





• The need of feeling secure and safe.

• She is not very savvy towards modern technology, which means she has the need for

an interface that shows only the status of the components but does not provide any

active interaction capabilities.

• The need that the critical devices, like a stove, are turned off automatically if she has

fallen asleep.

• Large Icons

Ingobert Tugend is 78 and, similar to Luise, oblivious. In addition, he fusses over himself every

time when he has forgotten to lock the door or turn off a device. This results in stress and high

blood pressure.

In a nutshell, her needs and preferences are:

• The need of feeling secure and safe.

• The preference to control the devices via his mobile device.

• The preference that the critical devices are turned off he leaves the house.

• Large text and control elements.

RemACT accommodates Luise’s needs and preferences by providing an information panel

that is placed next to the stairs up to her bedroom (Fehler! Verweisquelle konnte nicht

gefunden werden.).

The icons are large so that Luise can spot the current status of her front door and the stove

easily on her way up to her bedroom. The icons are designed in a way that combines an icon,

a text describing the icon, and a visual queue in form of green and red colours. Here, the green

light indicates that everything is “okay” with, for example, the stove – so Louise does not need

to worry.

Figure 9 User interface for Luise Insel




Igobert’s needs and preferences are accommodated by adapting the RemACT’s interface for

his mobile devices, see Fehler! Verweisquelle konnte nicht gefunden werden.. This user

interface provides also the possibility to control the status of the devices remotely, thus

accommodates Ingoberts’ preference to use his mobile device.

The source code of RemACT publicly available in our research repository at

https://github.com/REMEXLabs/SmartHome_SomSe2017/tree/master/RemAct

A video showcasing the prototype is publicly available at https://youtu.be/hfw6ytVeZ0w

(German audio only, but German and English subtitles available).

Figure 10 User interface for Ingobert Tugend


https://github.com/REMEXLabs/SmartHome_SomSe2017/tree/master/RemAct

https://youtu.be/hfw6ytVeZ0w



6 Conclusion

Within task 301.4, we are on track towards the goal of supporting vendors and third parties

in developing smart-home products that provide personalised user interfaces:

• A market analysis was performed regarding the current situation of developing platforms

for smart-home and AAL systems. We have concluded that – due to the recent

competition in this area – it is more promising to build personalisation support into an

existing mainstream platform than to continue the development of URC.

• We have identified relevant personalisation features of URC and GPII, and determined

how these can be built into the Eclipse SmartHome platform. This plan is reflected in a

conference paper (Smirek et al., 2016) and in our integration roadmap (see 3.2.3).

• We expect that – with an implementation and promotion effort inside ESH in the

upcoming fourth year of Prosperity4All – we can reach a significant number of

mainstream developers of the ESH community. For other mainstream platforms, the

relevant communities can take our implementation as a model for personalisation

features, since our code in ESH will be available as open-source to the public.

• We have developed showcases of user interfaces and services in a prototypical AAL

environment that are targeted to elderly users and users with mild cognitive

impairments. These showcases are installed in a prototypical implementation within the

PUX Lab of the Stuttgart Media University. Videos of these showcases are available on

YouTube for the public.

• In a second step, within the fourth year of Prosperity4All, we have extended the set of

showcases with prototypical implementations of scenarios in an AAL environment that

makes use of personalisation concepts of URC and GPII. These implementations support

personalised user interfaces that adapt to the individual needs and preferences of

exemplary users.




Annex I: References

AllSeen Alliance (2016), retrieved from: AllJoyn Framework. https://allseenalliance.org/framework.

Apple. Inc. (2016). Apple HomeKit, The smart-home Just Got Smarter retrieved from:

http://www.apple.com/ios/home/

Eclipse Foundation (2016). The Eclipse SmartHome project retrieved from:

http://www.eclipse.org/smarthome/, accessed October 26, 2016.

Emiliani, Pier Luigi, and Constantine Stephanidis (2005). Universal Access to Ambient Intelligence

Environments: Opportunities and Challenges for People with Disabilities. IBM Systems Journal 44(3):

605–619.

EnOcean Alliance (2017). Building Automation Experts, retrieved from: https://www.enocean-

alliance.org/en/home/, accessed January 11th 2017.

FHEM (2017). Home of FHEM, retrieved from: http://fhem.de/fhem.html, accessed January 11th

2017.

Google Inc. (2016). Google Brillow, retrieved from:

https://developer.android.com/things/index.html.

Henka, A., Smirek, L., & Zimmermann, G. (2016). Personalizing Smart Environments. In Proceedings of

the 6th International Conference on the Internet of Things (pp. 159–160). New York, NY, USA: ACM.

https://doi.org/10.1145/2991561.2998459.

RWE (2017). innogy SmartHome: Die einfach integrierbare, intelligente Haussteuerung für mehr

Sicherheit, Komfort & cleveres Energiemanagement von Innogy retrieved from: https://www.rwe-

smarthome.de/web/cms/de/2768534/home, accessed January 11th 2017.

KNX Association (2017). KNX Association Official website retrieved from: http://www.knx.org,

accessed January 11th 2017.

Christos Mettouris, Marios Komodromos, Lukas Smirek , Daniel Ziegler (2016). Ecosystem

infrastructure for smart and personalised inclusion and PROSPERITY for ALL stakeholders - D203.1

Runtime Environment Final Prototype

Krasner, G. E., & Pope, S. T. (1988). A Cookbook for Using the Model-view Controller User Interface

Paradigm in Smalltalk-80. J. Object Oriented Program., 1(3), 26–49.

Miele (2017). Hausgerätevernetzung retrieved from:

https://www.miele.de/haushalt/hausgeraetevernetzung-1912.htm, accessed January 11th 2017.

Open Connectivity Foundation (2016), retrieved from https://openconnectivity.org/, accessed

October 26, 2016.

QIVICON (2017). Welcome to smart-home with QIVICON - QIVICON retrieved from:

https://www.qivicon.com/en/, accessed January 11th 2017.

Robert Bosch Hausgeräte GmbH (2016). Home Connect, retrieved from http://www.bosch-

home.com/de/home-connect.html#intro.


https://allseenalliance.org/framework

http://www.apple.com/ios/home/


http://fhem.de/fhem.html

https://developer.android.com/things/index.html

https://doi.org/10.1145/2991561.2998459

https://www.rwe-smarthome.de/web/cms/de/2768534/home

https://www.rwe-smarthome.de/web/cms/de/2768534/home

http://www.knx.org/

https://www.miele.de/haushalt/hausgeraetevernetzung-1912.htm

https://openconnectivity.org/

https://www.qivicon.com/en/

http://www.bosch-home.com/de/home-connect.html#intro

http://www.bosch-home.com/de/home-connect.html#intro



Samsung (2017). Smart-home. Intelligent Living | SmartThings, retrieved from:

https://www.smartthings.com/, accessed January 11th 2017.

Smirek, Lukas, Gottfried Zimmermann, and Michael Beigl (2016). Just a smart-home or Your smart-

home – A Framework for Personalised User Interfaces Based on Eclipse SmartHome and Universal

Remote Console. Procedia Computer Science 98. The 7th International Conference on Emerging

Ubiquitous Systems and Pervasive Networks (EUSPN 2016)/The 6th International Conference on

Current and Future Trends of Information and Communication Technologies in Healthcare (ICTH-

2016)/Affiliated Workshops: 107–116. https://doi.org/10.1016/j.procs.2016.09.018.

ISO/IEC (2014). ISO/IEC 24752-1:2014. Information Technology - User Interfaces - Universal Remote

Console - Part 1: Framework. International Organization for Standardization (ISO).

Tazari, Mohammad-Reza (2010). An Open Distributed Framework for Adaptive User Interaction in

Ambient Intelligence. In International Joint Conference on Ambient Intelligence Pp. 227–238.

Springer.

Thread Group (2016). Retrieved from https://www.threadgroup.org/.

W3C WoT group (2016). Use Cases and Requirements for the Web of Things. Use Cases and

Requirements for the Web of Things. retrieved from http://w3c.github.io/wot/wot-ucr.html.

Wöckl B., Yildizoglu U., Buber I., Aparicio Diaz B., Kruijff E., and Tscheligi M. (2012). “Basic Senior

Personas: A Representative Design Tool Covering the Spectrum of European Older Adults,” in

Proceedings of the 14th International ACM SIGACCESS Conference on Computers and Accessibility,

New York, NY, USA, 2012, pp. 25–32.

Z-Wave Alliance (2017). The Internet of Things is powered by Z-Wave. retrieved from: http://z-

wavealliance.org/, accessed January 11th 2017.

ZigBee Alliance (2017), retrieved from: http://www.zigbee.org/, accessed January 11th 2017.

Zimmermann, G., & Vanderheiden, G. (2007). The Universal Control Hub: An Open Platform for

Remote User Interfaces in the Digital Home. Springer LNCS. Human-Computer Interaction. Interaction

Platforms and Techniques., 4551/2007, 1040–1049. https://doi.org/10.1007/978-3-540-73107-8.

Zimmermann, Gottfried, Gregg Vanderheiden, Matthew Ma, et al. (2004). Universal Remote Console

Standard - Toward Natural User Interaction in Ambient Intelligence. In Extended Abstracts of the

2004 Conference on Human Factors and Computing Systems - CHI ’04 P. 1608. New York, New York,

USA: ACM Press. http://dl.acm.org/citation.cfm?id=985921.986169.

Zimmermann, G., & Vanderheiden, G. (2007). The Universal Control Hub: An Open Platform for

Remote User Interfaces in the Digital Home. Springer LNCS. Human-Computer Interaction. Interaction

Platforms and Techniques., 4551/2007, 1040–1049. https://doi.org/10.1007/978-3-540-73107-8.


https://www.smartthings.com/

https://doi.org/10.1016/j.procs.2016.09.018

https://www.threadgroup.org/

http://w3c.github.io/wot/wot-ucr.html

http://z-wavealliance.org/

http://z-wavealliance.org/

http://www.zigbee.org/

https://doi.org/10.1007/978-3-540-73107-8

http://dl.acm.org/citation.cfm?id=985921.986169

https://doi.org/10.1007/978-3-540-73107-8



Annex II: Glossary

Abbreviation Full form

AAL Ambient Assisted Living

API Application Program Interface

AT Assistive Technology

D Deliverable

DoW Description of Work

ESH Eclipse SmartHome

GPII Global Public Inclusive Infrastructure

HDM Stuttgart Media University (German "Hochschule der Medien")

ICT Information and Communications Technology

IDE Integrated Development Environment

IT Information Technology

MRS Medicine Reminder System

P4A Prosperity4All

PUX Personal User Experience

T Task

UCH Universal Control HUB

UI User Interface

URC Universal Remote Console

WP Work Package


d301.1 open-source personalised user interfaces for older ...€¦ · course "smart-home...

Documents