wearable mobility monitoring using a multimedia smartphone platform

IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 60, NO. 9, SEPTEMBER 2011 3153

Wearable Mobility Monitoring Using aMultimedia Smartphone Platform

Gaëtanne Haché, Member, IEEE, Edward D. Lemaire, Member, IEEE, and Natalie Baddour

Abstract—Understanding mobility is important for effectiveclinical decision making in the area of physical rehabilitation.Ideally, a person’s mobility profile in a nonclinical setting, suchas the home or community, could be obtained. This profile wouldinclude the environment and context in which the mobility takesplace. This paper introduces a novel wearable mobility monitor-ing system (WMMS) for an objective ubiquitous measurementof mobility. This prototype WMMS was created using a smart-phone-based approach that allowed for an all-in-one WMMS.The wearable system is freely worn on a person’s belt, such asa normal phone. The WMMS was designed to monitor a user’smobility state and to take a photograph when a change of state wasdetected. These photographs were used to identify the context ofmobility events (i.e., using an elevator, walking up/down stairs, andtype of walking surface). Validation of the proposed WMMS wasperformed with five able-bodied subjects performing a structuredsequence of mobility tasks. System performance was evaluated byits ability to detect changes of state and the ability to identifycontext from the photographs. The WMMS demonstrated goodpotential for community mobility monitoring.

Index Terms—Acceleration, cameras, mobility, monitoring,multimedia systems, wearable.

I. INTRODUCTION

MOBILITY can be defined as the ability to move in-dependently from one point to another [1] and is es-

sential for maintaining independence. Mobility is required toperform many activities of daily life, such as, cooking, dressing,shopping, and visiting friends. According to Statistics Canada,mobility problems are one of the issues that affect the greatestnumber of adults [2]. Mobility disabilities can affect an individ-ual’s quality of life, health, productivity, and independence andalso affect the lives of their family and the people around them.

Manuscript received September 5, 2010; revised December 15, 2010;accepted December 16, 2010. Date of publication March 28, 2011; date ofcurrent version August 10, 2011. This work was supported in part by theResearch In Motion, by The Ontario Graduate Scholarships in Science andTechnology Program, and by the Ontario Centers of Excellence. The AssociateEditor coordinating the review process for this paper was Dr. Salvatore Baglio.

G. Haché is with the Ottawa-Carleton Institute for Biomedical Engineer-ing, Department of Mechanical Engineering, University of Ottawa, Ottawa,ON K1N 6N5, Canada (e-mail: [email protected]).

E. D. Lemaire is with the Ottawa-Carleton Institute for Biomedical Engi-neering, Department of Mechanical Engineering, University of Ottawa, Ottawa,ON K1N 6N5, Canada, and also with the Institute for Rehabilitation Re-search and Development, The Ottawa Hospital Rehabilitation Centre, Ottawa,ON K1H 8M2, Canada.

N. Baddour is with the Department of Mechanical Engineering, Universityof Ottawa, Ottawa, ON K1N 6N5, Canada.

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TIM.2011.2122490

Preserving mobility is paramount for staying independent andactive at home and in the community.

Accurate and objective mobility assessment is required fordecision making in rehabilitation medicine. Such assessmentscan be used to determine mobility issues outside a hospi-tal environment, evaluate the progress made during and afterrehabilitation, and enhance clinical decision making about arehabilitation program (i.e., assistive devices, exercises, treat-ment, etc.). Currently, many types of mobility assessments areperformed in a clinical setting and are supervised by the re-habilitation physician. These assessments include clinical tests,quantitative measures, and subjective feedback from client topatient. Although clinical mobility tests have a value, suchassessment tools may not be appropriate for determining thecontributing factors for independent community walking andthe impact of the environment on the individual’s mobility [3],[4]. Monitoring the mobility outside a clinical setting is im-portant because mobility in the real world is typically differentfrom the mobility measured in the clinic [5].

Wearable mobility monitoring systems (WMMSs) are de-signed to be worn on the body and allow mobility monitoringin the person’s home and the community [6]. Many wear-able mobility monitoring studies measure biomechanical and/orlocation parameters [5], [7]–[10], but most lack environmentalor contextual information. In community mobility monitoring,contextual information is important since it provides insight onwhere, how, and on what a person is moving. A camera couldprovide contextual information from a person’s surroundingenvironment.

Some wearable systems that use contextual information, suchas context-aware systems [11] and life logs [12], are not meantfor community mobility monitoring for people with physicaldisabilities. Other context-aware wearable systems use contextinformation to better recognize activities [13]–[15], but theenvironmental characteristics in which activities take place arenot analyzed for their impact on mobility.

We propose a novel WMMS that provides unsupervisedobjective mobility measurements in a cost-effective way, usingsmartphone technology that has already achieved consumeracceptance. In addition to monitoring biomechanical parame-ters, our WMMS also aims to identify mobility tasks andtheir context. This paper uses the smartphone as the centralprocessing hub for data capture, data processing, multimediacapture, outcome storage, and the option for wireless outcomedata transmission. The novelty of this approach is the combina-tion of biomechanical task identification methods and contextidentification via mobile multimedia tools.

0018-9456/$26.00 © 2011 IEEE

3154 IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 60, NO. 9, SEPTEMBER 2011

This WMMS is the first to combine change-of-state detec-tion, sensor-based activity classification, and environmental-image capture into an integrated system designed to improvethe contextual information available for mobility assessment.Photographs provide contextual information that cannot besupplied by inertial sensor systems, such as using an elevator,walking on carpet or grass, or walking in a crowded room. TheWMMS provides information on the context and environmentin which mobility events take place, which will help identifymobility challenges in a person’s own environment. For thispaper, changes of state include starting or stopping an activ-ity, postural changes, walking on stairs, indoor and outdoortransitions, and using transportation. This paper describes theWMMS design from a system perspective, including hardwareintegration, data processing methods, mobility assessment, andcontext identification outcomes.

II. METHODOLOGY

A. WMMS Prototype Development

The BlackBerry smartphone platform was chosen for theWMMS due to capability, acceptance in the health-care sector,and device/platform security. In our preliminary study, theBlackBerry platform demonstrated the multitasking, commu-nication, and processing capabilities required for a WMMShub [16]. BlackBerry smartphones possess the required securityfeatures, built-in global positioning system (GPS), integratedcamera, video recording, Wi-Fi, Bluetooth, data encryp-tion, adequate processing speed, and large storage capacity.Furthermore, newer models provide access to accelerometerraw data that could enable the design of an all-in-one WMMS.A mature Java environment and many secure developmentinterfaces [i.e., application programming interference (API)]are also available with the BlackBerry devices. Based on thelatest available BlackBerry Java development environment andAPI at the time of project inception, BlackBerry Bold 9000was used in this study.

While recently released phones have integrated accelerom-eters and the potential to test ambient light, the BlackberryBold 9000 did not possess these features; therefore, an externalboard with mobility analysis sensors was designed. The ex-ternal sensors were required because BlackBerry smartphoneswith all the required capabilities were not available during theproject development phase (i.e., accelerometer, GPS, Wi-Fi,Bluetooth, and camera). The external board design, integratedinto the phone’s holster (i.e., Smart Holster), provided a flex-ible approach to add other measurement sensors or tools inthe future. The board was connected to the BlackBerry viaBluetooth.

As shown in Fig. 1, a microcontroller CY8C27443(Cypress Semiconductor Corporation, San Jose, CA, USA),Bluetooth Module F2M03GLA (Free2Move AB, Halmstad,Sweden), triaxial accelerometer LIS344alh (STMicroelec-tronics, Geneva, Switzerland), and light sensor APDS-9005(Avago Technologies Limited, San Jose, CA, USA) weremounted on the board. A rechargeable lithium battery pow-ered the board. This external board could continuously run

Fig. 1. Sensors and board for attachment to the holster.

for approximately 14 h on one charge. However, the lengthof time that the WMMS could run without recharging wasapproximately 3 h with the BlackBerry’s Li-Ion 1500 mAhbattery. The sensors were not put to sleep if no activity wasdetected.

To start and stop sensor data sampling, sampling delaycommands were sent from the BlackBerry to the board micro-controller. Bluetooth serial port profile (SPP) protocol was usedfor communication with the external board. The data fromthe accelerometer and the light sensor were stored in a bufferon the microcontroller. At every sampling delay, the last datapacket stored in the buffer was sent to the host (BlackBerry)via SPP. A data packet (21 B) was sent to the host (Black-Berry) every 20 ms (50 Hz). In the data packet, 6 B wererequired for raw acceleration data and 2 B for light sensor data.The data packet-sampling rate equaled the data-sampling rate;therefore, the sampling rate for the accelerometer and lightsensor was 50 Hz.

All signal processing and the state detection algorithms wereimplemented and performed on the BlackBerry. These algo-rithms are subsequently discussed. Following signal process-ing, the outcome data for each record were stored on theBlackBerry’s 8-Gb secure-digital card. The WMMS softwarewas developed using the BlackBerry Java Development Envi-ronment version 4.6.1.

B. Signal Processing

The accelerometer and light sensor sampling rate was 50 Hzsince body-fixed accelerometers placed at the waist must beable to measure acceleration with frequencies up to 20 Hz[17]. The accelerometer was calibrated to remove the direct-current offset [17]. The calibration method involved rotatingthe sensor to known angles, as suggested in the manufacturerdatasheet [18]. For example, under static conditions, the outputfrom an axis pointed toward the center of the earth should beequal to 1 g. If the axis is then rotated by 180◦, its outputshould be equal to −1 g. The sensitivity s and the offset o of a

HACHÉ et al.: WEARABLE MOBILITY MONITORING USING A MULTIMEDIA SMARTPHONE PLATFORM 3155

particular axis of the sensor can be calculated using the follow-ing equations:

s =(umax − umin)

2(1)

o =(umax + umin)

2(2)

where umax and umin are the maximum and minimum acceler-ation measured during the rotation between ±1 g. The output aof one accelerometer can then be expressed as

a =(u − o)

s(3)

where u is the uncalibrated acceleration.On the Blackberry, the calibrated acceleration data were

passed through a median filter (n = 3) to remove spikes [7].The signal was then passed through a low-pass digital filter(0.25 Hz) to separate the static component from the dynamiccomponent [19]. A nonoverlapping sliding window of 1.02 s(51 samples) [7], [20] was used to extract signal features fromthe static and dynamic acceleration components and the averagelight sensor output. These extracted features were used as inputto calculate the parameters that determined the user’s mobilitystate.

C. Mobility Features

Various mobility and activity classification variables wereselected from the literature to help determine mobility changesof state. One of the largest challenges in the development ofthe WMMS is the integration of individual components into acoherent and functional system that meets the overall systemobjectives. Therefore, the decision was made to focus on thesystem-level development and to incorporate previously provenmobility classification variables. Preliminary evaluations, overall activities, with two able-bodied subjects were used to ver-ify these measures and to determine the threshold values foractivity classification.

1) Inclination Angle: Inclination angle was used to helpclassify posture [9], [21]–[23] and identify postural transition[24]. The inclination angle was calculated for every windowperiod using the two-axis method presented by Freescale Semi-conductor [25]. The static components of the acceleration,obtained from low-pass filtering (0.25 Hz), were used to calcu-late the inclination angle Φ using the two-argument arctangentfunction, i.e.,

Φ = arctan(

GAzGAy

)(◦) (4)

where GAz and GAy are the averaged static accelerations of thez-axis (forward) and y-axis (vertical), respectively. An offset of180◦ was then added to the inclination angle to give a range of0 to 360◦. Ideally, an inclination value of 90◦ should be obtainedwhen the person lays on their belly, a value of 180◦ whenthe person stands, and a value of 270◦ when the person layson their back. The averaged inclination angle was comparedwith high and low thresholds (200◦ and 160◦) to determineif the person was in a standing position. If the person was

not standing, the angle was compared with different high andlow thresholds (320◦ and 250◦) to determine if the person waslying on their back. If both conditions were false, the positionwas determined to be somewhere in between. The thresholdvalues were based on the study by Culhane et al. [23] and ourpreliminary observations.

2) Standard Deviation of Vertical Acceleration: Standarddeviation is a well-supported measure for activity classifica-tion [17], [21]–[23], [26]. Since most daily activities can beclassified by changes in vertical axis acceleration, vertical ac-celeration (y-axis) was used to differentiate between static anddynamic states by comparing the standard deviation of the y-axis acceleration with two thresholds (static and dynamic). Thisalgorithm was defined as the double threshold (DT) algorithm.With a DT algorithm, if the initial state is static, the activityclassification remains static until the signal crosses the dynamicthreshold (0.120 g). Then, the state is set to dynamic andstays dynamic until the signal passes below the static threshold(0.075 g). When the person stands still, the standard deviationshould be close to 0 g.

3) Skewness of Vertical Acceleration: The skewness valueof the vertical acceleration is a time-domain feature that wasused by Baek et al. [26] to differentiate walking/running fromgoing up/down stairs. Skewness is a measure of asymmetry ofthe y-axis (vertical) acceleration about the average acceleration(e.g., a skewness value of 0 indicates a symmetric distributionof accelerations). Skewness of the y-axis acceleration wascalculated as

skewness =n

(n − 1)(n − 2)

n∑i=1

(xi − x

σ

)3

(5)

where n is the number of points, xi the y-axis accelerationat point i, and σ and x are the standard deviation and themean of the y-axis acceleration signal, respectively. The DTalgorithm was also applied to the skewness value but onlywhen in a dynamic classification state. Based on preliminaryworks, skewness values greater than 1 were observed for stairdescent [16], [26]. Skewness increased when ascending stairs,but values were less than stair descent. Similar skewness valueswere sometimes observed for both stair ascent and normalwalking, which could result in a false positive change-of-statedetection. High (0.6) and low (0.2) thresholds were chosen todetect stair descent and detect stairs ascent with minimal falsepositive results.

4) SMA of Three-Axis Acceleration: Signal magnitude area(SMA) is another viable activity and mobility measure [7], [9].The SMA normalized to the length of the signal T can becalculated from

SMA =1T

T∫t=0

|ax|dt +

T∫t=0

|ay|dt +

T∫t=0

|az|dt

(6)

where t is the time in seconds and ax, ay , and az are theacceleration of x-, y-, and z-axes, respectively. During pre-liminary testing, peaks occurred in the SMA signal for sit-ting, rising from a chair, and lying down. The SMA also


Fig. 2. User’s state detection algorithm.

helps identify activity intensity changes that could indicate achange of state. Therefore, three thresholds were used, andthree states were determined: no peak with normal intensity,no peak with increased intensity, or a peak. A DT algorithmwas used to determine increases in intensity and peak detection.The “increased intensity” low and high thresholds were 0.100and 0.190 g, respectively. Since an increase in intensity shouldhappen when the person is moving, the algorithm verified thatthe person was in a dynamic state when detecting the “no peakwith increase in intensity” state. The peak detection low andhigh thresholds were 0.100 and 0.320 g, respectively.

5) Light Intensity: The light sensor measured light intensityof the ambient environment. To detect indoor and outdoor statesduring the day, the DT algorithm was applied to the lightintensity feature. Based on preliminary light sensor calibration,a high threshold of 1000 mV and a low threshold of 300 mVwould differentiate outdoor from indoor states during the day.However, during preliminary testing while driving, many falsechanges of state were recorded due to the light intensity changesin the car. To remove those false changes of state, the DT

algorithm was only applied to the light intensity feature whennot moving in a vehicle.

6) GPS Speed: GPS data have been used in mobility mon-itoring to complement motion data, improve activity recogni-tion, and provide contextual data [5], [8], [27]–[29]. Therefore,GPS location coordinates and speed were collected and addedto the WMMS output file when available (outdoors). The GPSdata were extracted from the BlackBerry Bold every 9 s. Forthis WMMS prototype, only the speed was considered forthe change-of-state detection algorithm, to detect whether theperson was in a vehicle. This feature was passed through a DTalgorithm, with the low and high thresholds of 1 and 7 m/s,respectively. When the speed was above the 7 m/s threshold,the state was identified as “in a vehicle.” The state stayed thesame until the GPS speed measured below the low threshold.

D. Determination of State and Change of State

The user’s state was assessed for every data window (1.02 s)(see Fig. 2). A change of state was determined by subtracting


the three previous states from the current state. If the answerwas different from zero for one of the subtractions, a change ofstate occurred. As a result of a change of state, the algorithmdetermined if a picture should be taken. From our cameraperformance test, approximately 0.7 s was required to take apicture, and another 0.9 s was required before another picturecould be taken. Therefore, it was decided to wait at least twowindows before taking another picture (i.e., 3 s later). Pictureencoding was set to Joint Photographers Expert Group, i.e.,640 × 480 pixels, and the quality was set to normal. Thememory size of a picture with this encoding was 10 to 70 KB.

E. Mobility Evaluation

A convenience sample of five able-bodied subjects (threemales and two females; age: 36.6 ± 6.4 years old; height:173.82 ± 13.17 cm; weight: 69.32 ± 16.09 kg) was recruitedfrom the staff at The Ottawa Hospital Rehabilitation Center(TOHRC, Ottawa, Canada) and the community. Informed con-sent was obtained from all the participants. People with injuriesor gait deficits were excluded.

Data collection took place within the TOHRC (hallways,elevator, stairs, and Rehabilitation Technology Laboratory),outside the TOHRC on a paved pathway, and in a car drivingaround the Ottawa Hospital campus.

The subjects were asked to wear the WMMS on their waist,with the holster attached on a belt, on their right hip, with thedevice pointing forward. No additional instructions were givenfor positioning the instrumented holster. The subjects wereasked to follow a predetermined path with a series of mobilitytasks. For every trial, the subjects were filmed with a digitalvideo camera. Three trials per subject were performed. Thedigital camera was synchronized with the WMMS by havingthe subject block the light sensor with their hand for 5 s whenstarting data collection. A digital video was necessary to vali-date the change-of-state detection, to determine the change-of-state timing, and to provide context information for validation.

The predefined mobility tasks were sequentially performed.Moving from one task to another should trigger a change ofstate, providing a total of 38 changes of state per trial. Thegiven sequence of tasks was as follows: 1) stand; 2) walk onlevel ground (25 m); 3) stand-to-sit transition; 4) sit; 5) sit-to-stand transition; 6) walk on level ground (60 m); 7) stand andwait for an elevator; 8) walk to get in the elevator; 9) take theelevator to the second floor; 10) walk out of the elevator andkeep walking on level ground (30 m); 11) stand and wait for theelevator; 12) walk to get in the elevator; 13) take the elevator tothe first floor; 14) walk out of the elevator and keep walking onlevel ground (50 m); 15) ascend stairs (13 steps); 16) walk onstair intermediate landing (level ground for 1.5 m); 17) ascendstairs (13 steps); 18) walk on level ground (30 m); 19) descendstairs (13 steps); 20) walk on stair intermediate landing (levelground for 1.5 m); 21) descend stairs (13 steps); 22) walk onlevel ground (20 m); 23) stand-to-lie transition; 24) lie on back;25) lie-to-stand transition; 26) walk on level ground (45 m);27) ascend and descend a 7◦ angled ramp (5 m); 28) walk onlevel ground (15 m); 29) transition indoor/outdoor and keepwalking outdoors on level ground (60 m); 30) transition out-

door/indoor and keep walking indoors on level ground (40 m);31) transition indoor/outdoor and keep walking outdoors onlevel ground (30 m); 32) stand-to-sit transition to get in a car;33) sit in the car; 34) start car and ride around campus; 35) stopcar ride; 36) sit-to-stand transition; 37) walk on level ground(30 m); 38) transition outdoor/indoor and keep walking indoorson level ground; 39) and finally stand.

Changes-of-state timing from the digital video were com-pared with the WMMS change-of-state timestamps. Each datawindow was analyzed to determine if the change of state was atrue or false positive, or true or false negative. The number oftrue and false positives and true and false negatives were usedto calculate WMMS sensitivity and specificity, i.e.,

Sensitivity =#TruePositives

#TruePositives + #FalseNegatives× 100

(7)

Specificity =#TrueNegatives

#TrueNegatives + #FalsePositives× 100.

(8)

Two research assistants independently evaluated the Black-Berry Bold images. The evaluators were asked to identify thecontext (i.e., stairs, elevator, ramp, floor, outdoor, etc.) fromthe digital images. Only the images taken due to a real changeof state (true positives) were evaluated. The evaluators weregiven a list of context options to choose from. The evaluatorswere not informed of the mobility tasks represented by theimages prior to evaluation. The results from the two evaluatorswere then analyzed to determine if the context was successfullyidentified from the pictures. Table II shows the various contextsthat were identified for each mobility task. For the context “typeof ground,” the evaluators also had to choose between floor,grass, and pavement.

III. RESULTS

An overall sensitivity of 77.7% ± 2.5% and a specificity of96.4% ± 2.2% were obtained across all the activities. Averagedsensitivity of the different change-of-state categories are givenin Table I. Context identification from the photographs had anoverall success rate of 72.5% ± 33%. Table II shows a summaryof the results divided by type of context.

IV. DISCUSSION

Understanding mobility in a nonclinical setting is importantwhen making rehabilitation/health-care-related decisions. Ourresults suggest that smartphones, particularly the newer ver-sions to come, have great potential for community mobilitymonitoring. By using the integrated camera, information on thecontext/environment in which mobility events take place canbe identified. Additionally, the BlackBerry has the necessaryprocessing power to log and process data, run algorithms,collect GPS data, and take pictures, all without data loss.

Our approach of taking a photograph when a change ofstate occurred demonstrated that mobility tasks such as taking


TABLE IPERFORMANCE RESULTS OF THE DIFFERENT CHANGE-OF-STATE CATEGORIES

TABLE IIIMAGES CATEGORIZATION RESULTS FOR THE DIFFERENT TYPES OF CONTEXT

an elevator, going up stairs, and riding in a car could beidentified from the images. The type of walking surface (i.e.,floor and pavement) could also be identified. From the study byShumay-Cook et al. [30], terrain was used to differentiate olderadults with mobility disabilities from people without mobilityissues. Since terrain has been related to fall risk [31], terraincontextual information could be used to help in understandingthe underlying causes of falls and help with fall prevention.

The use of images to capture context and environment inmobility monitoring could also help to monitor activity avoid-ance. Mobility disability has been characterized by a reductionin the number and type of environment challenges [32]. Ac-tivity avoidance could lead to a reduction in mobility, whichcould lead to further deterioration in physical status and socialinteractions.

Since the camera was pointing forward from the pelvis, theWMMS did not provide the downward angle that would berequired for viewing stairs during descent. A wide-angle or

sphere lens could improve context identification by providinga larger field of view. A short video of several seconds, ormultiple pictures of the same context, could help with contextidentification. However, preliminary testing of the BlackBerrycamera showed that multiple still images could not be takenwith the Bold 9000 smartphone camera (i.e., a picture couldonly be taken every 1.6 s). The use of short video clips could beexplored in future research.

Additionally, as with many cell phones, the camera per-formed moderately well under low-light conditions, causingimages to be blurry and dark. The flash was not used for imagecapture since multiple camera flashes would be obtrusive andrapidly drain the battery. The camera location on the front ofthe pelvis could also cause dark images due to subject’s clotheshiding the camera lens, although lens obstruction did not occurin this study.

Our algorithm demonstrated good results at detectingchanges of state caused by walking start/stop (97.4%), postural


change (97.8%), and walking down stairs (100%). These re-sults were obtained using an accelerometer attached to thesmartphone’s holster that was worn “like a usual phone” onthe subject’s waist. The evaluation protocol provided a real-life situation where the mobility tasks were consecutively andfreely performed. It is interesting to note that our approachproduced similar results to other studies that used a differentwearing location [22], several accelerometers [23], or wherethe evaluation was performed in a controlled laboratory settingexecuting discrete mobility tasks [9].

The WMMS poorly performed at detecting the change ofstate due to stair ascent and walking on a ramp. As with stairdescent, skewness was used to detect stair ascent. Since litera-ture was lacking on ramp detection using accelerometer signals,the same method was explored to detect ramp navigation. Thechoice of the skewness feature was based on the work byBaek et al. [26], which obtained a classification rate of 93%for upstairs and 87% for downstairs. Some differences with ourmethods were accelerometer location, where the accelerometerof Baek et al. was worn on the lateral side of the pelvisinstead of the front, their additional signal processing andanalysis features, a larger data window, and the use of a neuralnetwork. More complex algorithms could be explored sincethe current generation of smartphones have greatly enhancedprocessing power. Moreover, in older populations or individualswith mobility disabilities, a slow/almost-stopping movementcould be present before attempting walking up a ramp or stairs;therefore, a picture would be taken to help in identifying themobility task. The healthy young subjects who participated inthis study had a minimal change in forward progression whentransitioning from level ground to stairs or ramps.

A light sensor was added to the WMMS to detect outdoor andindoor conditions. Our approach of selecting outdoor/indoorthresholds did not perform as well as anticipated. A change inlight intensity level could have been a better measure instead ofusing fixed outdoor/indoor thresholds since changes could bedetected on overcast/cloudy days. As seen in the image evalua-tion results, pictures could be used to detect indoors/outdoors.

The GPS speed was used to detect if a person was in avehicle. For the trials where GPS satellites were detected, thechange caused by being in a vehicle was well classified. Whilethe initiation of being in a vehicle can be identified using thecamera images, WMMS classification was delayed by the 9-ssampling interval for GPS speed and the 7 m/s threshold.The main problem with GPS during evaluation was the timerequired to detect satellites and initiate GPS data acquisition.Based on preliminary tests, up to 30 min could be requiredto detect GPS satellites, depending on the exterior conditions.The BlackBerry was set to autonomous mode to detect location,which is slower but more precise than using cell-site mapping.For our WMMS, GPS speed was required for detecting vehicleriding. Cell tower-based location could be investigated sincelocation estimation occurs faster and would work indoors andin cloudy weather, although this method is of lower precision.

Improvement to the change-of-state algorithm is necessaryto enhance detection of stairs ascent, ramp navigation, andindoor/outdoor conditions. Additional signal processing couldbe added offline to improve classification of the raw data and to

compile the results into clinically relevant metrics. Automateddigital images processing to automatically identify context willneed to be considered if the WMMS is to be widely usedin health-care patient monitoring applications. Time-motionanalysis should also be performed in future studies to examinethe time requirements for machine-assisted visual inspectionof the reference images. For example, custom clinical mobilityanalysis software could allow the person analyzing the mobilitydata to quickly jump from image to image and view the pictureand activity classification information or view images groupedby activity classification type (e.g., just walking and just stairs).The acceptable time for a technician to extract meaningfulinformation for clinical use remains unknown.

Implementing the change-of-state algorithm to the new gen-eration of BlackBerry smartphones should be considered sincenew versions provide raw accelerometer data and improvedcamera performance. This will remove the need for the externalboard for the activities evaluated in this paper. However, otherexternal sensors could be integrated into the WMMS using thenew WMMS software and Bluetooth communications, such asfor pressure or electromyography analyses.

Future generations of our WMMS will have to improvethe software to include a power-saving mode. The BlackBerrybattery in this study only lasted 3 h. A larger battery is requiredfor this type of application.

The data from five able-bodied subjects demonstrated thefeasibility of a smartphone-based WMMS for community mo-bility monitoring and therefore will be used to motivate futurework in mobility monitoring. More evaluations are requiredto verify how well the WMMS would perform in real-lifesituations where walking speed, movement obstacles, andenvironments are constantly changing.

Testing on people with mobility disabilities is part of ourobjectives for the next generation of this WMMS. Outcomesfrom people with physical disabilities are important to validatethe signal processing and algorithm methods. Extracting othersignal features may be required, as well as modifying cali-bration methods to adapt the sensitivity of the change-of-statedetection for different levels of mobility (e.g., walking slowerand abnormal gait pattern).

Another population that could possibly benefit from thisWMMS are wheelchair users. Health-care professionals wouldbenefit by knowing how a person uses their wheelchair orscooter in the community, including the proportion of timefor wheeled versus walking mobility. Having images takenwhile using the wheelchair could also highlight the types ofenvironments that challenge the wheelchair user.

V. CONCLUSION

Maintaining independent mobility at home and in the com-munity plays an important role in an individual’s independence,quality of life and health, and in the lives of their family andthe people around them. Nonclinical measurement of mobilityand the context in which mobility events take place can helpwith these roles. Our WMMS approach to respond to theneed for community mobility assessment tools has shown greatpotential. The smartphone approach provides an accessible and


cost-effective option that can easily be implemented in society.Adding the camera to the WMMS suggested that images couldhelp identify mobility tasks such as walking up stairs and takingan elevator, as well as identifying the type of ground and thetype of vehicle. However, the limitations should be addressedto improve performance. Interesting future work exists forsmartphone-based WMMS.

ACKNOWLEDGMENT

The authors would like to thank K. Heggie for his technicalwork on the external sensor board and C. Kendell, S. Doyle,and H. Wu for their assistance with data collection and analysis.The Ottawa Hospital Rehabilitation Centre is acknowledged fortheir facility support.

REFERENCES

[1] A. E. Patla and A. Shumway-Cook, “Dimensions of mobility: Defin-ing the complexity and difficulty associated with community mobility,”J. Aging Phys. Activity, vol. 7, no. 1, pp. 7–19, Jan. 1999.

[2] Statistics Canada, Participation and Activity Limitation Survey 2006:Analytical Report, Minister Industry, Ottawa, Canada, 2007, Accessed:25 Nov. 2009. [Online]. Available: http://dsp-psd.pwgsc.gc.ca/collection_2007/statcan/89-628-X/89-628-XIE2007002.pdf

[3] T. Lam, V. K. Noonan, and J. J. Eng, “A systematic review of functionalambulation outcome measures in spinal cord injury,” Spinal Cord, vol. 46,no. 4, pp. 246–254, Apr. 2008.

[4] R. Corrigan and H. McBurney, “Community ambulation: Environmentalimpacts and assessment inadequacies,” Disabil. Rehabil., vol. 30, no. 19,pp. 1411–1419, 2008.

[5] M. Ermes, J. Pärkka, J. Mantyjarvi, and I. Korhonen, “Detection of dailyactivities and sports with wearable sensors in controlled and uncontrolledconditions,” IEEE Trans. Inf. Technol. Biomed., vol. 12, no. 1, pp. 20–26,Jan. 2008.

[6] C. N. Scanaill, S. Carew, P. Barralon, N. Noury, D. Lyons, andG. M. Lyons, “A review of approaches to mobility telemonitoring of theelderly in their living environment,” Ann. Biomed. Eng., vol. 34, no. 4,pp. 547–563, Apr. 2006.

[7] M. J. Mathie, A. C. F. Coster, N. H. Lovell, and B. G. Celler, “Detectionof daily physical activities using a triaxial accelerometer,” Med. Biol. Eng.Comput., vol. 41, no. 3, pp. 296–301, May 2003.

[8] D. A. Rodríguez, A. L. Brown, and P. J. Troped, “Portable global position-ing units to complement accelerometry-based physical activity monitors,”Med. Sci. Sports Exerc., vol. 37, no. 11 Suppl., pp. S572–S581, Nov. 2005.

[9] D. M. Karantonis, M. R. Narayanan, M. Mathie, N. H. Lovell, andB. G. Celler, “Implementation of a real-time human movement classifierusing a triaxial accelerometer for ambulatory monitoring,” IEEE Trans.Inf. Technol. Biomed., vol. 10, no. 1, pp. 156–167, Jan. 2006.

[10] E. Farella, A. Pieracci, L. Benini, and A. Acquaviva, “A wireless body areasensor network for posture detection,” in Proc. 11th IEEE Symp. Comput.Commun., Cagliari, Sardinia, Jun. 2006, pp. 454–459.

[11] C. Randell and H. Muller, “Context awareness by analysing accelerometerdata,” in Proc. 4th Int. Symp. Wearable Comput., Atlanta, GA, Oct. 2000,pp. 175–176.

[12] Y. Lee and S.-B. Cho, “Extracting meaningful contexts from mobile lifelog,” in Proc. Intell. Data Eng. Automated Learn., Birmingham, U.K.,Dec. 2007, pp. 750–759.

[13] G. H. Jin, S. B. Lee, and T. S. Lee, “Context awareness of human motionstates using accelerometer,” J. Med. Syst., vol. 32, no. 2, pp. 93–100,Apr. 2008.

[14] T. Choudhury, G. Borriello, S. Consolvo, D. Haehnel, B. Harrison,B. Hemingway, J. Hightower, P. Klasnja, K. Koscher, A. LaMarca,J. A. Landay, L. LeGrand, J. Lester, A. Rahimi, A. Rea, and D. Wyatt,“The mobile sensing platform: An embedded activity recognition sys-tem,” IEEE Pervasive Comput., vol. 7, no. 2, pp. 32–41, Apr./Jun. 2008.

[15] U. Maurer, A. Rowe, A. Smailagic, and D. Siewiorek, “Location andactivity recognition using eWatch: A wearable sensor platform,” in Proc.Ambient Intell. Everyday Life, 2006, pp. 86–102.

[16] G. Hache, E. Lemaire, and N. Baddour, “Development of a WearableMobility Monitoring System,” in Proc. Can. Med. Biological Eng. Conf.,Calgary, Canada, May 2009.

[17] C. V. C. Bouten, K. T. M. Koekkoek, M. Verduin, R. Kodde, andJ. D. Janssen, “A triaxial accelerometer and portable data processing unitfor the assessment of daily physical activity,” IEEE Trans. Biomed. Eng.,vol. 44, no. 3, pp. 136–147, Mar. 1997.

[18] STMicroelectronics, MEMS Inertial Sensor—High Performance 3-Axis±2/±6g Ultracompact Linear Accelerometer, LIS344ALH Datasheet,Rev.3, STMicroelectronics, Geneva, Switzerland, 2008.

[19] M. J. Mathie, A. C. F. Coster, N. H. Lovell, and B. G. Celler, “Accelerom-etry: Providing an integrated, practical method for long-term, ambula-tory monitoring of human movement,” Physiol. Meas., vol. 25, no. 2,pp. R1–R20, Apr. 2004.

[20] S. J. Preece, J. Y. Goulermas, L. P. J. Kenney, D. Howard, K. Meijer, andR. Crompton, “Activity identification using body-mounted sensors—Areview of classification techniques,” Physiol. Meas., vol. 30, no. 4,pp. R1–R33, Apr. 2009.

[21] P. H. Veltink, H. B. J. Bussmann, W. De Vries, W. L. J. Martens, andR. C. Van Lummel, “Detection of static and dynamic activities usinguniaxial accelerometers,” IEEE Trans. Rehabil. Eng., vol. 4, no. 4,pp. 375–385, Dec. 1996.

[22] G. M. Lyons, K. M. Culhane, D. Hilton, P. A. Grace, and D. Lyons, “Adescription of an accelerometer-based mobility monitoring technique,”Med. Eng. Phys., vol. 27, no. 6, pp. 497–504, Jul. 2005.

[23] K. M. Culhane, G. M. Lyons, D. Hilton, P. A. Grace, and D. Lyons, “Long-term mobility monitoring of older adults using accelerometers in a clinicalenvironment,” Clin. Rehabil., vol. 18, no. 3, pp. 335–343, Mar. 2004.

[24] B. Najafi, K. Aminian, A. Paraschiv-Ionescu, F. Loew, C. J. Bula, andP. Robert, “Ambulatory system for human motion analysis using a kine-matic sensor: Monitoring of daily physical activity in the elderly,” IEEETrans. Biomed. Eng., vol. 50, no. 6, pp. 711–723, Jun. 2003.

[25] Freescale Semiconductor, Application Note 3461, Rev 2 Tilt Sensingusing Accelerometers Sensors, pp. 2–4.

[26] J. Baek, G. Lee, W. Park, and B. -J. Yun, “Accelerometer signal process-ing for user activity detection,” in Lecture Notes in Computer Science(Including Subseries Lecture Notes in Artificial Intelligence and LectureNotes in Bioinformatics). Berlin, Germany: Springer-Verlag, 2004,pp. 610–617.

[27] J. S. Frank and A. E. Patla, “Balance and mobility challenges in olderadults: Implications for preserving community mobility,” Amer. J. Prev.Med., vol. 25, no. 3 Suppl 2, pp. 157–163, Oct. 2003.

[28] P. J. Troped, M. S. Oliveira, C. E. Matthews, E. K. Cromley, S. J. Melly,and B. A. Craig, “Prediction of activity mode with global positioningsystem and accelerometer data,” Med. Sci. Sports Exerc., vol. 40, no. 5,pp. 972–978, May 2008.

[29] G. MacLellan and L. Baillie, “Development of a location and movementmonitoring system to quantify physical activity,” in Proc. Conf. HumanFactors Comput. Syst., 2008, pp. 2889–2894.

[30] A. Shumway-Cook, A. E. Patla, A. Stewart, L. Ferrucci, M. A. Ciol,and J. M. Guralnik, “Environmental demands associated with communitymobility in older adults with and without mobility disabilities,” Phys.Ther., vol. 82, no. 7, pp. 670–681, Jul. 2002.

[31] J. Stokes and J. Lindsay, “Major causes of death and hospitalization inCanadian seniors,” Chronic Dis. Can., vol. 17, no. 2, pp. 63–73, 1996.

[32] A. Shumway-Cook, A. Patla, A. Stewart, L. Ferrucci, M. A. Ciol, andJ. M. Guralnik, “Environmental components of mobility disability incommunity-living older persons,” J. Amer. Geriatrics Soc., vol. 51, no. 3,pp. 393–398, Mar. 2003.

Gaëtanne Haché (M’08) received the Bachelor ofApplied Science in electrical engineering from Uni-versité de Moncton, Moncton, NB, Canada, in 2001and the M.A.Sc. degree in biomedical engineeringfrom the University of Ottawa, Ottawa, ON, Canadain 2010.

She worked for six years as an Engineer in thehigh-tech sector. She is currently working with theDepartment of Biomedical Engineering, Children’sHospital of Eastern Ontario, Ottawa.


Edward D. Lemaire (M’09) received the B.Sc. de-gree in kinanthropology and the M.Sc. degree in bio-mechanics from the University of Ottawa, Ottawa,ON, Canada, in 1984 and 1988, respectively, andthe Ph.D. degree from the University of Strathclyde,Glasgow, U.K., in 1998.

He is a Clinical Researcher with the Institutefor Rehabilitation Research and Development, TheOttawa Hospital Rehabilitation Centre, Ottawa, andan Associate Professor with the Faculty of Medicine,University of Ottawa. He has extensive experience

with research involving technology to enhance mobility for people with physi-cal disabilities, particularly in the areas of prosthetics and orthotics, computed-aided technologies, and telerehabilitation applications.

Dr. Lemaire is the President of the Canadian National Society of the Inter-national Society for Prosthetics and Orthotics.

Natalie Baddour received the B.Sc. (Physics)degree from the Memorial University of Newfound-land, St. John’s, NL, Canada in 1994, the M.Math.degree from the University of Waterloo, Waterloo,ON, Canada in 1996, and the Ph.D. degree in me-chanical engineering from the University of Toronto,Toronto, ON in 2001.

Following postdoctoral work at the University ofToronto and the University of Bath, Bath, U.K., shejoined the Department of Mechanical Engineering,University of Ottawa, Ottawa, ON, where she is

currently an Associate Professor. Her research interests include mathematicalmethods and algorithms, with applications to dynamics, vibrations, and bio-medical engineering.

wearable mobility monitoring using a multimedia smartphone platform

Documents