Smart Sensors for Interoperable Smart

Environment

Sara BartoliniDottorato in “Elettronica, Informatica e delle Telecomunicazioni”

XXII CicloTutor: Tullio Salmon Cinotti

Alma Mater Studiorum, Università di Bologna14 Gennaio 2010

2

• Infrastructure Independent Pedestrian Navigation System for Shared and Interoperable Smart Spaces

• Target: – Navigation in public buildings – Guides for museums and archaeological sites– In general multi - vendor, multi - device, multi-

domain applications where user location and tracking in indoor spaces is required (e.g. health and monitoring apps)

OutlineOutline

3

Related workRelated workCho and Park (Univ. Kwangoon): Calibration Techniques for Electronic

Compass

Antsaklis (Univ. of Notre Dame): Wireless Assisted Pedestrian Dead Reckoning System

Ladetto (Lausanne, ENAC): Digital Magnetic Compass and Gyroscope Integration For Pedestrian Navigation

Retscher (Wien, Univ. of Technology): Location Determination in Indoor Environments for Pedestrian Navigation

Bellavista (Univ.of Bologna): Supporting Context Awareness in Smart Environments: a Scalable Approach to Information Interoperability

4

My Contribution in My Contribution in Infrastructure Independent Tracking:Infrastructure Independent Tracking:

Approach, Solutions, ResultsApproach, Solutions, Results

• Requirements Analysis• Multi - Sensor based Compass design with Tilt

Compensation• Multi - Sensor Integration for Positioning• Sensor Calibration Method • Hardware and firmware Design (Multi Sensor Unit)• Power Distribution Design & Management• Self- Configuration for Usability and Testability• Data Modeling for Semantic Platform• Results

5

RequirementsRequirements• Quality Requirements:

– Wearability– Low Power Consumption– Testability– Easy Calibration

• Functional Requirements:– Orientation– Relative Position– Activity Indication– Wireless Communication– Semantic Platform Integration

• Performance:– Azimuth Max Deviation = ±3° when Tilt less than ±30°– Tracking Error <= 3% on a 60 meters walk in a flat area

“Pedestrian Navigation Systems and Location – Based Services” Retscher

6

Dip angle

Hez

Hex

Hey

Heh

He

North

2D Magnetic Sensor As a Compass2D Magnetic Sensor As a Compass

Magnetic Sensor in a Plane Magnetic Sensor in a Plane Parallel to the EarthParallel to the Earth’’s s

SurfaceSurfaceAzimuthAzimuth = arctang HeyHey

HexHex

Azimuth

Hex

Hey

He

North

Azimuth

Hp: Earth’s Magnetic Field is in Horizontal Plane

Earth’s Magnetic Field is NOT in Horizontal Plane

X

Y

7

Dip angle

Hez

Hex

Hey

Heh

He

North

Hez

Hez’

Hey’

Tilted

Magnetic Sensor in a Magnetic Sensor in a Plane Parallel to the Plane Parallel to the EarthEarth’’s s SurfaceSurface

AzimuthAzimuth = arctang HeyHey

HexHex

Magnetic Sensor in a Plane Magnetic Sensor in a Plane NOT Parallel to the NOT Parallel to the

EarthEarth’’s Surfaces Surface

TILT ERROR:TILT ERROR:

(on X- roll)

Azimuth

He’

Hex

Heh’Azimuth’

2D Magnetic Sensor As a Compass2D Magnetic Sensor As a Compass

AzimuthAzimuth’’= arctang HeyHey’’

HexHex≠≠AzimuthAzimuth

8

“A Calibration Technique for a Two-Axis Magnetic Compass in Telematics Devices” Cho, Park

Tilt Compensated AzimuthTilt Compensated Azimuth

AzC=– Hey’ CosΦ + Hez’ sen Φ

Hex’ CosΘ + Hey’ SenΘ SenΦ + Hez’ SenΘ CosΦarctang

Azimuth Tilt CompensationAzimuth Tilt Compensation

Hez’=Sin(DipAngle) + Hex’ SinΘ – Hey’CosΘ CosΦ

CosΘ CosΦ

Vertical Magnetic Component Estimation:Vertical Magnetic Component Estimation:

9

Dip angle

Hez

Hex

Hey

Heh

He

North

Hez

Hez’

Hey’

Tilted

(on X- roll)

Azimuth

He’

Hex

Heh’Azimuth’

2D Magnetic Sensor As a Compass2D Magnetic Sensor As a Compass

Φ

Tilt (Inclination) Angles:Tilt (Inclination) Angles:

Pitch: Θ= Arcsin Axg

Roll: Φ = Arcsin Ayg

Third Magnetic Component Estimation:Third Magnetic Component Estimation:

Hez’=Sin(DipAngle) + Hex’ SinΘ – Hey’CosΘ CosΦ

CosΘ CosΦ

10

“A Calibration Technique for a Two-Axis Magnetic Compass in Telematics Devices” Cho, Park

Tilt Compensated AzimuthTilt Compensated Azimuth

Estimated Dip Angle

Hex’

Θpitch

Φroll

FROM

ACCELEROMETER

FROM MAGNETIC SENSOR

Hey’

Azimuth Tilt Compensation AlgorithmAzimuth Tilt Compensation Algorithm

Hez’

FROM ESTIMATION

AzC=– Hey’ CosΦ + Hez’ sen Φ

Hex’ CosΘ + Hey’ SenΘ SenΦ + Hez’ SenΘ CosΦarctang

11

Y

X

Y

XAzimuth

Φ

θ

λ

1st Step 2nd Step 3th StepHorizontalHorizontal

360360°° RotationRotation

Detection:XmcMax, XmcMinYmcMax, YmcMin

Calculation:

X,YmcMax+X,YmcMin2

BiasX,Y =

X,YmcMax-X,YmcMin2SfX,Y =

Horizontal Azimuth Horizontal Azimuth OrientationOrientation

Detection:Azimuth

Random Inclination Random Inclination (same Azimuth)(same Azimuth)

Detection:PitchRoll

Calculation:

Estimated Dip Angle

Y

X

Azimuth

Typical Compass Calibration ApproachTypical Compass Calibration Approach

“A Calibration Technique for a Two-Axis Magnetic Compass in Telematics Devices”Cho, Park

12

Y

X

Φ

θ

λ

1st Step2nd Step360360°° RotationRotation

Detection:XmcMax, YmcMinXmcMax, YmcMin

Calculation:X,YmcMax+X,YmcMin

2BiasX,Y =

X,YmcMax-X,YmcMin2SfX,Y =

Facing North ApproximatelyFacing North ApproximatelyPitch Inclination Pitch Inclination

Vertical and Horizontal SwingVertical and Horizontal Swing

Detection:

Real Dip Angle Real Dip Angle (not Estimated)(not Estimated)

Y

X

Azimuth

North

Compass Calibration Method:Compass Calibration Method:user assisted selfuser assisted self--calibrationcalibration

easy, fast and anywhere (no instruments required)easy, fast and anywhere (no instruments required)

13

Azimuth 45°

‐25‐20‐15‐10‐505101520

253035404550556065707580859095100105

110115120125130135140145150155160

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80

Pitch (°)

(°)

Compensated Azimuth Non Compensated Azimuth Error from 45°

Results: Azimuth Results: Azimuth vsvs TiltTilt

14

YL sin(AzC)

L cos(AzC)AzC

Direction of Movement (Heading): X

X= Xo + ∑ (L cos (AzC))

Y= Yo + ∑ (L sin (AzC))

L = Step Lengthn = Number of Steps

We need:We need:•• Compensated AzimuthCompensated Azimuth•• Step DetectionStep Detection•• Step LengthStep Length

n

n

Compass Based PositioningCompass Based Positioning

X

15

Gait CycleGait Cycle: The rhythmic alternating movements of the 2 legs which result in the forward movement of the body.

3 Steps

Compass Based PositioningCompass Based PositioningSteps Counter TheorySteps Counter Theory

We need:We need:•• Acceleration ProfilesAcceleration Profiles

“Location Determination in Indoor Environments for Pedestrian Navigation” Retscher

www.pubmed.com

16

Steps Detection and Step Length: User Dependent Approach1. Zero Crossing Detection 2. Flat Zone Detection- Low sensitivity on walking velocity - It depends on walking velocity- For waist sensor pack - Not for waist sensor pack

33. Peaks Detection. Peaks Detection: : �� For waist sensor packFor waist sensor pack�� Derivative of Frontal and Vertical Derivative of Frontal and Vertical

AccelerationAcceleration�� 2 User dependent threshold2 User dependent threshold

1.Frontal Acceleration Double Integration 1.Frontal Acceleration Double Integration 2.Velocity Calculation2.Velocity Calculation3. Adaptive Algorithm3. Adaptive Algorithm

StepStepDetectionDetection

StepStepLengthLength

44. Using only Vertical Acceleration. Using only Vertical AccelerationK: user dependent

Antsaklis4

MinMax AzAzKStepLenght −=

17

Results: TrackingResults: Tracking60 meters Straight Line

•Step Detection: Error 2% •For each walk:

•Average Real Steps: 78,81 steps•Average Detected Steps: 77,31 steps

•Positioning (K=1):•Error: 3,3%•Test used to calibrate the user dependent parameter K:

Open Issue:•Calibration needed for: K, Thl, Thh•During no rectilinear walk:

•Pitch and Roll errors due to centripetal and tangential acceleration

•First and last steps are missed occasionally•Female User to be investigated

pstalTrueStepLength*ToAverageSteistanceTotalTrueDK =

18

Low LevelData

Horizontal Magnetic Field Components

Accelerometer GyroscopeMagnetic Sensor

3 Axes Accelerations 1 Axis Angular

Velocity

Tilt Compensated Azimuth

Step Detection

Tilt Compensated Rotation

Horizontal AzimuthInclination Angles Z – Rotation Angle

Compass Based Positioning and Activity Gyroscope Based Positioning

Smart Space

The Multi Sensor Unit Approach:The Multi Sensor Unit Approach:from low level sensors to the Semantic Spacefrom low level sensors to the Semantic Space

1919

Smart M3

Shared Interoperable Information Space

KP KP

KnowledgeProcessor

(KP)

Digital entity where relevant real-world information is stored and kept up to date

Smart SpaceSmart Space

20

Context Information exchanged with the SS:- Producer- Consumer- Aggregator

Physiological on Body Sensors

Physiological on Body Sensors

EnvironmentalSensors

EnvironmentalSensors

Multi Sensor UnitMulti Sensor Unit

Location(User) Temperature

Humidity(Environment)

Heart Rate (HR)Skin TemperatureRespiration Rate

(User)

INFRASTRUCTURE INDEPENDENT LOCALIZATION SYSTEMINFRASTRUCTURE INDEPENDENT LOCALIZATION SYSTEM

OtherSensors

OtherSensors

Context Information

From the Sensors

KP KP KP KP

KP KP

KPKP

Consumer and AggregatorKPs

Producer KPs

Smart Space

Smart SpaceSmart Space

21

Data Modeling for Semantic PlatformData Modeling for Semantic Platform

Museum

Room1 Room2 Room3ContainsRoom

Cont

ains

Room

ContainsRoom

Map

Painting1 Statue1Painting2 Statue2 Theca

(X,Y)

RefSys1

Sara

MSB5

State (x,y)Heading

Wea

ring

Walking Painting1

Contains

Contains

HasState isNear

Has

Item

HasItem

HasItem

HasItem

HasItem

HasPosition

HasMap

HasRS

HasValue (10,20)Walking 30 40 50° cm

HasValue HasValueHasX HasY

HasRSHasUnit

SmartSpace

RefSys2

HasUnit

22

• “CIMAD - A Framework for the Development of Contest-aware and Multi-channel Cultural Heritage Services” G. Raffa, P. H. Mohr, N. Ryan, D. Manzaroli, M. Pettinari, L. Roffia, S. Bartolini, L. Sklenar, F. Garzotto, P. Paolini, T. Salmon Cinotti - ICHIM07, 2007 Toronto

• “A Stereo Vision based system for advanced Museum services” M. Pettinari, D. Manzaroli, S. Bartolini, L. Roffia, G. Raffa, L. Di Stefano, T. Salmon Cinotti, edited by David Pletinckx, Workshop Proceedings, Page(s): 57 - 68, Ed. ARCHAEOLINGUA

• “Interoperable multimedia mobile services in cultural heritage sites” N. Ryan , P. Mohr, D. Manzaroli, G. Mantovani, S. Bartolini, A. D’Elia, M. Pettinari, L. Roffia, L. Sklenar, F. Garzotto, T. Salmon. EPOCH Conferenceon Open Digital Cultural Heritage Systems (2008)

• “Technology meets Culture: from MUSE to EPOCH”, Tullio Salmon Cinotti, Marina Pettinari, Luca Roffia, Daniele Manzaroli, Sara Bartolini “Atti del Convegno Internazionale Vesuviana. Archeologie a confronto, Bologna, 14-16 gennaio 2008, a cura di Antonella Coralini, Bologna, Edizioni Antequem (in corso di stampa).

• “Personalized Context Based Services for Dynamic User Groups” L. Roffia, L. Lamorte, G. Zamagni, S. Bartolini, A. D’Elia, F. Spadini, D. Manzaroli, C. A. Licciardi, T. Salmon Cinotti, STWiMob'2009, October 12, 2009) in conjunction with WiMob-2009, Marrakech, Morocco

• “Approaching the design of Interoperable Smat Environment Applications” F.Spadini, S. Bartolini, R. Trevisan, G. Zamagni, A. D’Elia, F. Vergari, L. Roffia, T. Salmon Cinotti, 2nD International NoTa Conference, 2009, San Jose, CA, USA

• “Abstracting Knowledge from Physical Parameters in Smart Spaces:A Smart M3 Demonstration” J. Honkola, H.Laine, F. Spadini, F. Vergari, G. Zamagni, S. Bartolini, R. Trevisan A. D’Elia, L. Roffia, D. Manzaroli, C. Lamberti, T. Salmon Cinotti,, 2nD International NoTa Conference, 2009 San Jose, CA, USA

• “First interoperability demo with two indipendent Smart Spaces (aligning was performed by a dedicated KP)”Eurotech, Nokia, VTT, UnBo: P. Azzoni, J. Honkola, T. Salmon Cinotti, J-P Soininen and their teams, 1st ARTEMIS AUTUMN event 2009, Madrid

Publications and Public PresentationsPublications and Public Presentations

23

• “A Smart Space Application to Dynamically Relate Medical and Environmental Information” F. Vergari, S. Bartolini, F.Spadini, A. D’Elia, G. Zamagni, L. Roffia, T. Salmon CinottiSubmitted as paper for presentation at the European Event forElectronic System Design and Test Date2010 (Dresden, Germany, March 8-12, 2010)

• Chapter proposal for“Handbook of Research on Technologies and Cultural Heritage”Edited by Dr. Georgios Styliaras, Dr. Dimitrios Koukopoulos and Dr. Fotis Lazarinis of the University of Ioannina

Submitted and Accepted Paper and Chapter Submitted and Accepted Paper and Chapter

24

• Infrastructure Independent Pedestrian Navigation System for Shared and Interoperable Smart Spaces:• No need for location infrastructure in buildings• Information reusable through the Smart Space • In general multi - vendor, multi - device, multi-

domain applications where user location and tracking in indoor spaces is required (e.g. health and context monitoring apps)

• Future work: • Integration with Path Finding Algorithm • Integration within Smart Objects for Smart Environment Apps• Natural Interface Principles Compliant

Conclusions and Open QuestionsConclusions and Open Questions