hannu jarvinen presentation

7/31/2019 Hannu Jarvinen Presentation

1/25

Anticipatory Lighting in Smart Building

2nd IEEE International Workshop on ConsumereHealth Platforms, Services and Applications (CeHPSA)

14th January 2011, Las Vegas, Nevada

Satellite Workshop of 9th IEEE Consumer

Communications & Networking Conference (CCNC)

Hannu Jarvinen, Petri Vuorimaa

[email protected], [email protected]

Department of Media Technology

Aalto University School of Science

Espoo, Finland


2/25

Introduction

Assistive living technologies can help the elderly and thepeople with disabilities with their daily life.

The smart building solutions provide means to implementthese systems in homes and can offer valuable applications,for example, an alarming service notifying relatives in case of

irregular behavior. However, different systems are generally not able to

intelligently communicate with each other, decreasing theusability of such technologies.

We propose a general smart building system model aimingfor flexibility and interoperability by offering different parts ofthe system to behave and cooperate intelligently.

We present our implementation of the system with ananticipation method for the lighting control.


3/25

Background

The home automation industry has envisioned smart homesto become reality for over 60 years [1].

Unfortunately, these visions have not become true and asmart home can still today be considered more of anexception than the rule.

However, in recent years the success of the Internet hasbrought us with communication technology that is ubiquitousin almost every modern indoor environment.

The current trend is to keep the end devices communicatingover low level protocols and connect them directly or throughconverters to an integration platform, which then offers a highlevel IP based interface for the enterprise applications.


4/25

Smart Home

A Smart Home is described, for example, asa house or living environment that contains the technology to

allow devices and systems to be controlled automatically [6].

While an environment equipped with independentprogrammable devices and systems could fit into thisdefinition, we extend it with another requirement:

a smart home contains the technology to allow devices andsystems to be controlled by each other.

In practice, this means standardizing the device interfacesand supporting the rule definition or other method for bindingdependencies between the device states.


5/25

Intelligence

To get the smart home to behave in a desired way, one or multiplesystems need to take control for executing actions.

How the control should be implemented? Approaches

simple rule definition, user defines the rules machine learning and fuzzy rules for more automatic and adaptive logic [8] using a neural network to let the environment to learn general patterns in

the inhabitants daily life [1]

an assisting approach where the system is only giving hints to guide theuser to manually perform the actions [7]

Critics of the work needed to be done by the inhabitants. One cannot prepare rules for every possible situation. Also, our dailypatterns can vary as we do not behave the same every day.


6/25

Lighting

Assistive solutions aim for making life easier, interaction with theenvironment pleasant and to satisfy needs of the inhabitants.

Traditionally the main objective for smart home projects has been toimprove energy efficiency and thus receive financial savings.

After the Heating, Ventilating and Air Conditioning (HVAC), thelargest consumption of energy in home is usually caused by thelighting [9].

Different approaches have been proposed. Application of artificial intelligence, more specifically a Genetic Algorithm to

predict the natural lighting levels for using the maximum amount ofavailable daylight for the lighting [10].

A commonly practiced lighting control technique applied, for instance, infactories is to use timers to have lights on only during the working hours.

A more localized approach is to switch on and off the individual lights basedon the presence of the inhabitants or workers in the specific space.

There is a irritating delay when the light goes on.


7/25

System Model

Do not consider devices just as simple sensors andactuators, which are then controlled by the intelligent controlsystem.

Both devices and the control system should be smart. Inpractice, our approach is to have support for the intelligent

control system but also to leave some amount of intelligenceintegrated into the devices.

When the most complex algorithms run on the devices, theuser or the control system can concentrate on simplyconnecting device states for defining rules for the logic.

Support multiple control systems at the same time. This canbe established by co-supporting a standard interface andtreating all the subsystems in an equal manner.


8/25

System Model

Fig. 1. A general smart building system model with distributed intelligence. A standardizedWeb services based interface is used for the communication between the systemcomponents.


9/25

Implementation

A standard Web services based building control systeminterface, open Building Information eXchange (oBIX),

was selected as a high level interface for the

implementation.

A simple RESTful interface and an extendable dataformat to allow devices to define their own specific datacontracts. [11]


10/25

Implementation

The integration platform used forthe implementation was providedby C oBIX Tools (CoT) developedby Andrey Litvinov.

It supports smart devices byoffering a method for devices toconnect to the platform over thehigh level interface.

The virtual television wasimplemented with XForms andJavascript to run in a browser and

play videos from Youtube.


11/25

Implementation

The implementation wasrealized in the mixed-realityspace eXperience InductionMachine (XIM), a spaceequipped with numeroussensors and actuators, andof a size of5.5x5.5m [12].

The space was divided intotwo separate roomsconnected by anapproximately standard sizedoorway.


12/25

Anticipation

The XIM has a marker-free, multi-person tracking system, which was exploited to capturethe users position in the smart home in near real-time.

a Linear Discriminant Analysis (LDA) was used to predict if the user is about to enter theother room.

Algorithm was used as the system needed to distinguish behavior of the user about toenter the room from the user just passing by the doorway.

To obtain the necessary training data for teaching the linear classifier, learning sessionswere conducted and approximately 150 data samples were gathered for the training set.

Factors used for the algorithm included the direction towards which the user is moving(angle) and the turning magnitude (difference between the current and the previous angle).

Anticipation distance was 50 cm. When the LDA prediction yielded a positive result, it meant that the user was probably

about to enter the other room.

Using the high level interface, the module sent a command to switch the lights on in thatroom. A command was not sent to the integration platform but to the rule engine, as it hadbatch rules specifically for the purpose of switching on all the lights in a certain room.


13/25

Rule Engine

A rule engine used for the implementation is a furtherdevelopment result of the Java based open Facility

Management Server (oFMS) [13].

Rules can be created and managed through the oBIXWeb services interface.

In oFMS rule engine, the rules are not structured ascommonly used Event Condition Action (ECA) rules.

While in ECA rules a certain event launches the

evaluation of the condition, in our rule engine the rulesare constantly evaluated (CA).


14/25

Rule Engine

A rule has properties to adjust its functioning, a list ofconditions and a list of actions


15/25

Rules

R1:

if rulestate.has.changed then

if rulestate = true then

lamp1 on

lamp2 on

lamp3 on

else

lamp1 off

lamp2 off

lamp3 off

end if

end if

R3:if user.at.room.1 then

rule1.rulestate onrule2.rulestate off

rule4.active onrule4.statechangestolive 1

end if

R5:

if user.sitting.at.the.tv thentv on

rule6.active onrule6.statechangestolive 1

end if


16/25

Experiments

Thirteen participants in a test laboratory. The purpose of the experiments was toobjectively measure to what extent the anticipation is working and to assurecorrect functioning of the system as a whole.

For natural walking behavior, it was important that the participants did not knowthat the focus was on the prediction of the movement to anticipate switching onthe lighting.

The participants were asked to behave as they were in their home and to movearound the smart home based on the given instructions. They were, e.g., askedto pick up things in the smart home and then put them to another place inanother room. They visited both rooms multiple times, watched the televisionand after the experiment, filled in a questionnaire.

For the later analysis, the system recorded the plain tracking information alwaysfor the duration of the whole experiment. Anticipation module was set toseparately record information about the user positions when the prediction was

commanding the lights to switch on. The rule engine recorded time stampedinformation on the commands it received.


17/25

Results

How much before the anticipation module was sendingthe command to the light switch rules (R1 and R2)

compared to the backup system (R3 and R4)?

Results showed that the average time gained by theanticipation was 1008.42 ms.

Fig. 3. Positions where the prediction algorithm was suggesting that theuser is about to enter the other room. In time, the positions were recordedbeginning from the reference point (a) 0 ms to (b) 133 ms, (c) 341 ms andending to the (d) 502 ms. The circle outlines the anticipation area and theline in the center of the circle marks the doorway.


18/25

Results

For a question, which asked what devices wereconsidered in the experiment, half of the ten participants

did not even mention any tracking devices or cameras.

Fig. 4. Results of two questions in the questionnaire. a: Participants reportedthat the interaction was pleasant. b: A desire to have such technology at homewas on average neutral.


19/25

Results

The behavior of the participants during the experimentsseemed natural.

Fig. 5. A trace from an experiment with one participant. The door is in themiddle and the circle outlines the anticipation area.


20/25

Conclusions

The presented anticipation method provides a simplecontrol technique for smart home lighting system

promoting energy efficiency while preserving the

pleasantness for the user.

Handling its specific tasks, preprocessing theinformation and taking part in the decision making, thetracking system as a smart device suits well into the

smart home architecture with distributed intelligence.


21/25

Conclusions

The implemented system presented a general delay ofaround one second. This meant that when running the

system without the anticipation, the lights went on in a

room approximately one second after the room was

entered. The anticipation module used for the lighting control

managed to remove that delay.


22/25

Conclusions

We believe that in the near future this kind ofarchitecture can provide assistive and energy efficient

solutions and attract users with the pleasant interaction.

Also, new kind of indoor tracking techniques are beingdeveloped which will offer better and multiple ways totrack both the people and the objects. That brings newpossibilities to gather more accurate behavioral data for

activity recognition, anticipation and adaptive control.


23/25

Conclusions

We have shown that the presented system, using a smart deviceand a high level rule engine, is able to reliably anticipate the usersmovement in a simple case of entering rooms.

The solution does not offer one-for-all solution for anticipation insmart home systems. Instead, it demonstrates the cooperation ofthe intelligence implemented on smart devices with the higher level

rule based control system. Further development exploiting emerging indoor tracking

technologies is needed to build a generic movement anticipationmodule. This module could continuously offer the next positionwhere the user is predicted to be.

Then the module would be generalized to execute any actionsneeded to be anticipated, including usage of different types ofdevices from lighting and opening doors to controlling entertainmentsystems like televisions and music as well as localized HVACsystems.


24/25

Questions?

Thank you for your attention.


25/25

References

[1] M. Mozer, The neural network house: An environment that adapts to its inhabitants, inProc. AAAI Spring Symposium on Intelligent Environments, 1998.

[2] T. Maile, M. Fischer et al., The vision of integrated IP-based building systems, Journal ofCorporate Real Estate, vol. 9, no. 2, pp. 125137, 2007.

[3] S. Knauth, R. Kistler et al., SARBAU towards highly self-configuring IP-fieldbus basedbuilding automation networks, in IEEE AINA, 2008, pp. 713 717.

[4] H. Jrvinen, A. Litvinov et al., Integration platform for home and building automationsystems, in Proc. Consumer Communications and Networking Conference (CCNC). IEEE,2011, pp. 292296.

[5] Y. Qiao, K. Zhong et al., Developing event-condition-action rules in real-time activedatabase, in Proc. ACM symposium on Applied computing. ACM, 2007, pp. 511516.

[6] D. Valtchev and I. Frankov, Service gateway architecture for a smart home,Communications Magazine, IEEE, vol. 40, no. 4, pp. 126132, 2002.

[7] S. Intille, Designing a home of the future, IEEE pervasive computing, pp. 7682, 2002.

[8] U. Rutishauser, J. Joller et al., Control and learning of ambience by an intelligentbuilding, Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactionson, vol. 35, no. 1, pp. 121132, 2005.

[9] U.S. DOE, Building Energy Data Book, 2010, table 2.1.6. Residential Energy End-UseSplits.

[10] D. Coley and J. Crabb, An artificial intelligence approach to the prediction of naturallighting levels, Building and environment, vol. 32, no. 2, pp. 8185, 1997.

[11] Open Building Information Exchange (oBIX), OASIS Committee Specification, Rev. 1.0,2006.

[12] U. Bernardet, S. Bermdez i Badia et al., The eXperience Induction Machine: A NewParadigm for Mixed-Reality Interaction Design and Psychological Experimentation, in TheEngineering of Mixed Reality Systems. Springer, 2010, pp. 357379.

[13] H. Jrvinen, An Interoperable Equipment Server for Building Automation Systems,Masters thesis, Helsinki University of Technology, Espoo, Finland, 2007.

hannu jarvinen presentation

Documents