ii - web.wpi.edu€¦ · bhawna shiwani worcester polytechnic institute, 2017 abstract bradykinesia...
TRANSCRIPT
ii
ACKNOWLEDGEMENTS
This thesis represents not only my work, but a milestone achieved through the guidance and
mentoring of all people who shared their support in this. I am indebted to many people for making
this endeavor an unforgettable experience.
I gratefully acknowledge my advisor Prof. Michael Gennert for his encouragement,
inspiration and his great efforts to help me surface clear ideas through meaningful discussions. I
also thank my committee members, Prof Joseph Beck for his timely inputs and guidance on the
technical aspects of this research and Prof Serge Roy for helping me to keep in mind the actual
impact of this work and bringing in the perspective of the community that will be benefitted by
this work.
I reserve special thanks for Dr. Joshua Kline whose guidance and encouragement has been
a steady influence throughout my thesis. He has inspired me to become an independent researcher
and helped me realize the power of critical reasoning. His mentoring has been instrumental in
brainstorming the key insights of the thesis as well as in rational thinking and impactful writing.
I am grateful to my colleagues at Delsys Inc, for their assistance and support. It has been a
pleasure to work with such a driven team and to learn from their experiences.
With final words, I deeply thank my family for their continuous support and encouragement.
It’s their unwavering belief in my dreams that kept me motivated throughout my masters.
iii
AUTONOMOUS REAL-TIME TRACKING OF BRADYKINESIA
IN PARKINSON’S DISEASE DURING UNSCRIPTED ACTIVITIES
BHAWNA SHIWANI
Worcester Polytechnic Institute, 2017
ABSTRACT
Bradykinesia is one of the most prevalent yet poorly monitored motor symptoms of
Parkinson’s Disease (PD). Generally defined by slowness of movements, the specific symptoms
and severity of bradykinesia fluctuate throughout the day in PD patients and are therefore difficult
to assess and treat during relatively short duration visits to a clinician. Body-worn sensors and AI
algorithms could provide valuable tool for the clinical assessment of bradykinesia. However, to
date no algorithm has been developed that can solve this problem. Therefore, this thesis aimed to
design a software platform consisting of machine learning algorithms and clinically-informed
metrics to provide real-time tracking of the motor symptoms associated with bradykinesia.
Activity-specific neural-network detection algorithms were designed, trained and tested to classify
sensor data from activities associated with minimal movement, such as non-walking, separately
from those associated with more continuous movement, such as walking. Clinically-informed
metrics were identified through rigorous signal analysis of EMG and IMU sensor data of muscle
activity and limb movement to quantify the motor symptoms used for clinical assessment of
bradykinesia. Real-time processing of more than 2000 minutes of movement data from PD patient
provided the following results: 1) walking from non-walking activities were separated using a
neural-network activity classifier with an accuracy of 99.5%; 2) a second neural network algorithm
trained for walking data provided minute-by-minute detection of bradykinesia with an accuracy of
iv
93.8%; 3) a third neural network algorithm trained for non-walking data provided minute-by-
minute detection of bradykinesia with 97.4% accuracy. In addition, the clinically-informed metrics
successfully quantified changes in motor symptoms of bradykinesia – such as poverty of
movement, reduced limb velocity and reduced range of movement – that occurred before and after
administration of PD medication. Together the detection algorithm and sensor-derived metrics
provide a novel, proof-of-concept framework that establishes the clinical viability of a real-time
tracking system for therapeutic interventions and patient-specific treatment.
v
TABLE OF CONTENTS
Title Page ...................................................................................................................... i
Acknowledgements ....................................................................................................... ii
Abstract ........................................................................................................................ iii
Table of Contents ......................................................................................................... v
List of Tables ............................................................................................................... vi
List of Figures ............................................................................................................... vii
List of Abbreviations .................................................................................................... viii
Introduction ................................................................................................................. 1
Specific Aims ............................................................................................................... 6
Methods ....................................................................................................................... 8
Results ……………...................................................................................................... 29
Discussions ...………….............................................................................................. 37
Future Work …………….............................................................................................. 41
Appendix A .................................................................................................................... 43
Appendix B .................................................................................................................... 44
Bibliography................................................................................................................... 45
vi
LIST OF TABLES
Table 1 Subject Population Characteristics ................................................................. 8
Table 2 Complete List of identified features during non-walking activities …........... 19
Table 3 Complete List of identified features during walking activities ….................. 20
Table 4 Clinically Informed Metrics …………………............................................... 25
Table 5 The classification accuracies of Walking Detector ........................................ 29
Table 6 Bradykinesia classification accuracy using Neural Network Architecture …. 30
Table 7 Bradykinesia classification accuracy using Logistics Regressions …………. 31
vii
LIST OF FIGURES
Figure 1 Manifestations of different symptoms in Parkinson’s Disease ……………… 2
Figure 2 Sensor data captured concurrently from EDS and TA muscles ……………… 9
Figure 3 Sensor data during walking and non-walking activities …………………… 12
Figure 4 Normal Healthy Walking …………………………………………………… 13
Figure 5 Temporal segmentation of signal data ........................................................... 14
Figure 6 Algorithm architecture to capture bradykinesia symptoms during
unscripted activities ....................................................................................... 15
Figure 7 Flow diagram of logistic regression algorithm .............................................. 22
Figure 8 Flow diagram of neural network algorithm .................................................. 23
Figure 9 Voluntary activities highlighted within 2-minute window of gyroscope data 26
Figure 10 Highlighted swing phase during step cycle .................................................. 27
Figure 11 Step detector algorithm results with an example of 10-minute segment of
ADL as captured by gyroscope placed at lower extremity …........................ 30
Figure 12 Plot I of clinically informed metrics during non-walking activities ……...... 32
Figure 13 Plot II of clinically informed metrics during non-walking activities ……...... 33
Figure 14 Plot I of clinically informed metrics during walking activities …………...... 34
Figure 15 Plot II of clinically informed metrics during walking activities …………...... 35
viii
LIST OF ABBREVIATIONS
ACC Accelerometer
ADL Activities of Daily Living
APDA American Parkinson’s Disease Association
EDS Extensor Digitorum Superficialis
EMG Electromyography
GYR Gyroscope
IMU Inertial Measurement Unit
PD Parkinson’s Disease
TA Tibialis Anterior
UPDRS Unified Parkinson’s Disease Rating Scale
VL Vastus Lateralis
WIRB Western Institutional Review Board
1
INTRODUCTION
Parkinson’s Disease (PD) is a neurodegenerative, chronically progressive disease that has affected
mobility and cognitive function of more than 10 million people worldwide [1]. Despite such a
daunting prevalence, there are few options to automatically and accurately monitor its progression
or evaluate treatment efficacy based upon the patient’s motor symptoms [2]. Often the limitations
to track motor symptoms such as tremor, slowness of movements and gait instability, result in an
incomplete picture of the disease [3,4]. However, a more comprehensive view of the disease
symptoms is crucial in determining the appropriate medication regimen. The goal of this research
was to design a framework to autonomously monitor motor symptoms of PD utilizing wearable
sensor technology during unscripted daily activities.
In PD, the cells that are responsible for producing a vital neurotransmitter called dopamine
are progressively depleted. Deficiency of dopamine results in a variety of involuntary movement
disorders as well as a reduced ability to produce normal voluntary movements [5]. Although PD
symptoms may vary throughout the day and manifest differently across patients, the primary motor
symptoms of PD typically include: cyclic shaking movements at rest (Tremor); slowness, paucity,
and hesitancy of movements (Bradykinesia); stiffness in muscles (Rigidity); shortened, shuffling
steps (Parkinsonian Gait); and medication-induced rapid, jerky involuntary movements
(Dyskinesia) (refer to Figure 1). Manifestation and severity of these symptoms can fluctuate
throughout the day depending on individual differences in stage of the disease and the
timing/dosage of dopamine-replacement therapy [6].
Among the symptoms of PD, bradykinesia is one of the most common and early stage
manifestations. Although primarily marked by an overall slowness of movements, body
2
bradykinesia manifests in a variety of symptoms such as hesitation to initiate or stop the voluntary
action, abnormally slow and shortened movements, reduced arm-swing, and poverty of movement.
Traditionally, clinicians assess bradykinesia and other PD symptoms using a protocol known as
the Unified Parkinson’s Disease Rating Scale (UPDRS) [7]. The UPDRS is a clinician-guided
rating tool that is often used to monitor the longitudinal course of PD by conducting periodic
evaluations during clinic visits. It consists of three parts on 1) Mentation, Behavior, and Mood, 2)
Activities of Daily Living (ADL) and 3) Motor Examination. The motor examination scale
assesses PD manifestations based on scripted activities and segments them into 5 different severity
levels based on the clinician’s impression of the magnitude of the impairment(s) during the PD
test. For example, the clinician may ask the patient to tap their fingers together as fast as possible
to rate the amplitude and frequency of the taps on the 0-4 rating scale (Item 23 of Part III Motor
Figure 1. Manifestations of different symptoms in Parkinson’s Disease – Different
symptoms of Parkinson’s Disease are presented along with their characteristic
manifestations. (Image Credit: http://www.afcares.com/blogs/parkinsons-disease-
seeking-in-home-care/)
3
Exam). This approach rests on the assumption that the severity of bradykinesia symptoms during
these coached activities correlates to the same severity levels for unscripted activities in daily
living for all PD patients. However, PD symptoms manifest differently for every patient depending
on their work and daily living habits [8]. UPDRS has also developed a broader guiding scale for
the clinicians to assess “body bradykinesia” (Item 31 of Part III Motor Exam) by considering the
reduced arm swing, slowness, paucity, and reduced range of movements during daily living
activities [7]. In both cases, assessment of body bradykinesia under the observation of a clinician
provides only a momentary picture of disease symptoms and fails to capture the fluctuations of
bradykinesia symptoms between the medication replacements. A more detailed tracking of
fluctuations of bradykinesia symptoms is critical to capture medication response for better disease
management.
Traditional way to track bradykinesia symptoms is to advise PD patients to maintain diaries
to log the disease symptoms daily [9]. However, manual logging methods demonstrate only a
limited efficacy as the objective assessment of highly versatile PD symptoms is difficult for
patients [10,11] and often results in poor compliance and incomplete logs due in part to the fact
that they may suffer from cognitive impairment as a result of PD. In addition to burdening the
patient, the absence of a continuous and more objective measure of PD symptoms greatly
minimizes its effectiveness in monitoring the oftentimes rapid fluctuations in PD symptom severity
throughout the various stages of PD. Proper medication titration for symptom management
therefore requires an autonomous and continuous tracking of PD symptoms in the context of ADL.
Wearable sensors can be leveraged to fill this gap by objectively quantifying the clinically
relevant motor symptoms with precise temporal resolution [12,13,14]. Even though it is not
possible to measure the cognitive symptoms of PD using body-worn sensors, the captured motor
4
symptoms acceptably correlate with the overall disease progression. Initial approaches to track
bradykinesia and other PD symptoms using wearable sensors relied on either accelerometers or
EMG activity [15]. However, the more recent development of inertial sensors, that integrate
gyroscopes to measure angular velocity and accelerometers to measure linear acceleration has
resulted in new studies exploiting the improved measuring capacity of this technology to monitor
motor symptoms [16]. Automated tracking of these symptoms using wearable sensors would be
significantly helpful to relieve patients from a constant pressure of maintaining logs or recalling
symptoms during the clinical visits. Although there have been prior attempts described in the
literature to design autonomous tracking systems capable to operate in patients home environment,
most of these systems have been limited to automating the PD detection during scripted activities
as mentioned earlier. This applies especially to bradykinesia detection [17] which has only been
successfully reported for specific activities such as hand opening and closing, finger tapping
[18,19] or walking [20,21,22]. There is an unfulfilled need to autonomously track and quantify the
symptoms of bradykinesia in such a manner that these clinically informed features can be directly
used by clinicians to inform the presence and severity of bradykinesia during ADL.
This thesis describes the design of a robust algorithm to track bradykinesia during
unscripted activities using wearable sensors that provides information regarding limb movement
and muscle activity. The algorithm is a key requirement to developing a wearable device that will
enable the clinician to obtain objective, minute-by-minute measures of the bradykinesia
manifestations during ADL. The outputs of the algorithm include the identification of occurrences
of bradykinesia and objective metrics to quantify the magnitude of impairments (symptoms)
caused by bradykinesia. The experimental protocol for this research was designed to record several
continuous hours of sensor data to serve as a proof of concept to the intended full day monitoring
5
of the PD patient. The sensor data were captured from combined inertial measurement unit (IMU)
and electromyography (EMG) wireless sensors to capture the limb movement and muscle activity
respectively. Experiments were video recorded and annotated by movement disorder experts to
establish a truth file. The problem of autonomous tracking of bradykinesia incidences was
approached by successfully separating the sensor dataset into walking and non-walking activity
subclasses using neural network. This approach led to capturing the activity-specific symptoms of
bradykinesia more robustly.
After sucessfully separating the unscripted activity data into walking and non-walking
activity sub-classes, activity-related features were developed to capture specific symptoms of
bradykinesia during walking and non-walking. Using these features as inputs, separate
classification algorithms were designed for tracking the symptoms of bradykinesia during walking
and non-walking activities. The algorithms demonstrated a high accuracy in detecting body
bradykinesia on previously unseen test data. In order for clinicians to have a continuous record of
the fluctations in severity of body bradykinesia, quantitative measures such as speed, range and
total time of movements were provided and their clinical usefulness was demonstrated for
individual cases from the data set. Together the detection algorithm and sensor-derived metrics
provide a complete software platform to quantify the quality of voluntary and involuntary
movements in PD. This work demonstrates the first ever algorithm capable of tracking symptoms
of movement impairments in bradykinesia during ADL that equip the clinicians with consistent
real-time tracking to better inform the progression of the disease and provide outcome measures
for therapeutic interventions, ultimately improving their quality of life.
6
SPECIFIC AIMS
To develop an autonomous algorithm for detecting the presence and absence of body bradykinesia
and monitoring the different symptoms of bradykinesia a rigorous signal processing analysis was
performed and the key signal attributes of bradykinesia symptoms were coupled with machine
learning algorithms. The following aims identify key stages of the project development:
Aim 1: Acquire EMG and IMU sensor data during unscripted activities. The target was to
include experimental data from 16 patients to create an adequate sample set capturing a broad
range of regular daily activities from PD subjects with different severity levels and different
presentations of bradykinesia as well as other concomitant PD motor symptoms. An experimental
protocol was designed to collect sensor data while PD subjects engage in ADL in a simulated home
environment.
Aim 2: Autonomously classify unscripted activities into walking and non-walking subclasses:
To identify all characteristic attributes of bradykinesia, the movement data was separated into
walking and non-walking activity subclasses as the symptoms of bradykinesia manifest differently
during these activities. A walking detection algorithm was designed to identify stepping instances
by utilizing the key signal features of leg movement during walking. The identified steps were
used to obtain walking segments, which in turn guided the partitioning of movement data into
walking and non-walking subclasses. As a pre-stage to classifying bradykinesia during ADL, this
approach targeted more specific feature selection of bradykinetic symptoms in isolated walking
and non-walking activity subclasses.
Aim 3: Real-time detection of bradykinesia and tracking of motor symptoms: The walking
and non-walking activity sub-classes were separately targeted to design classification algorithms
to identify bradykinesia. Objective measurements designed through rigorous signal processing of
7
IMU and EMG sensors were employed as inputs to train and test these machine learning
algorithms. The decisions from these activity specific classifications were compiled in a real-time
bradykinesia tracking report throughout the range of unscripted and unscripted activities.
Additionally, a complete profile of clinically informed features including range of movements,
speed of movements, paucity of movements and reduced arm swing was recorded using gyroscope
data. The final deliverable includes the real-time profile of clinically informed features of
bradykinesia symptoms as well as the periods of bradykinesia occurrences identified during
unscripted daily activities.
8
METHODS
Subjects: I worked with a team of clinicians and engineers to contact patients from local
neurological clinics, American Parkinson’s Disease Association (APDA) support groups, and the
Michael J Fox Trial Finder to recruit a total of 16 subjects that volunteered for the study. Prior to
their participation, subjects signed a consent form approved by Western Institutional Review
Board (WIRB) indicating their understanding of the procedures and risks of the study, and
willingness to participate. All the subjects were independently ambulatory and were diagnosed
with mild to moderately severe PD based on the Hoehn and Yahr staging scale (Hoehn–Yahr stages
II–III while “on” and Hoehn–Yahr stages III–IV while “off”) [7]. All patients were on a regimen
of levodopa-replacement parkinsonian medications and reported wearing off with motor
fluctuations between doses. All the patients presented with body bradykinesia, based on the rating
criteria illustrated by the UPDRS (Appendix A), as well as other movement disorders associated
with Parkinson’s disease. Based on their bradykinesia symptoms, the patient data were divided
equally (n=8) into training and testing sets, each consisting of approximately 1000 minutes of
recorded data. A summary of the subject population characteristics is provided in Table 1,
PD Subjects Data Corpus Training Data Corpus Testing
Number n = 8 n = 8
Age (y) 57.5 ± 12.5 63.2 ± 12.1
Men/Women 6/2 6/2
Disease Duration (y) 8.6 ± 5.4 5.4 ± 1.7
Total Data (min) 1000 1000
Bradykinesia Prevalence (%) 58.7% 76.4%
Table 1. Subject Population Characteristics – The bradykinesia prevalence indicates the percentage of
recording period when the PD subjects demonstrated symptoms of bradykinesia.
9
organized according to their use for Training or Testing of the Classification Algorithms. The
subsets were selected such that all symptoms of bradykinesia were covered in both sets.
Study Location: The data were recorded in a simulated home settings in an approximately 900 sf
research laboratory within Delsys Inc. (Natick, MA). The experimental room included household
items such as a refrigerator, a set of table and chairs, a couch and a bed to provide different options
to sit, move around or lie down. Food, drinks, and snacks were also provided during the
experiments. Patients were encouraged to freely engage in activities that they would be doing at
home as a part of their daily routine, and were not coached on how or when those activities should
be performed. Figure 2 demonstrates an example of a subject engaged in daily activities during the
experiment.
Figure 2. Sensor data captured concurrently from EDS and TA muscles – The highlighted locations
indicate the finger extensor muscles of the forearm (EDS) and ankle flexor (TA) muscles of the leg on the
more symptomatic side. Wireless sensors placed on each of these muscle locations captured movement
and muscle activity in 9 channel IMU data comprising of accelerometer, gyroscope and magnetometer
and 1 channel EMG data.
10
Data Acquisition: Trigno™ IM wireless sensors (Delsys, Inc, Natick, MA) were used to record
10 channel IMU (tri-axis accelerometer, tri-axis gyroscope, tri-axis magnetometer) and EMG
(electromyography) data from the subjects. Eight sensors were placed on the skin above the mid-
belly of the following muscles: 1) left and right finger extensor muscles of the forearm (Extensor
Digitorum Superficialis(EDS)); 2) left and right knee extensor muscles of the thigh (Vastus
Lateralis(VL); 3) left and right ankle flexor muscles of the leg (Gastrocnemius; and 4) left and
right ankle extensor muscles of the leg (Tibialis Anterior(TA)). A ninth sensor was placed on the
hip. The number and locations of the sensors used maintained consistency with prior studies [15]
and allowed for analysis of optimal sub-sets of sensor locations to improve usability.
The accelerometer and gyroscope data were sampled at 148.148 Hz, the magnetometer data at
74.074 Hz, and EMG data at 1111.11 Hz. Based on recommendations from prior work [15] the
sensor signals from the EDS and TA muscles from the more symptomatic side reported by PD
patients during their pre-screening interview were selected for the analysis. The selection of these
muscle locations for developing algorithms for monitoring bradykinesia was reinforced by the goal
of utilizing a reduced sensor set for practical ease-of-use by a patient while maintaining sufficient
representations of movement of different limbs. The signal data from the sensors, were acquired
wirelessly by EMGworks Acquisition software (Delsys Inc) in the form of High Performance
Fortran (HPF) files for offline data analysis. Time-synchronized video recordings of the patient
were also captured during the experiment which was later annotated by movement disorder
experts. Data were automatically saved into data files of 10-minute length to minimize asynchrony
between sensor and video data.
Protocol: The sensor data were collected for a total duration of 3 hours while the subjects carried
out unscripted activities in the laboratory to capture the change in bradykinesia symptoms between
11
dopamine replacement dosages as a proof of concept for future applications of all day recordings.
The experiments were preferably timed in the middle of their On-period1 to capture any motor
fluctuations produced by wearing-off of their medication or dyskinesia side-effects that may
follow medication intake. During the recording period, the subjects were free to move about at will
except at the beginning, middle and the end of the overall recording period, when the Part III
standardized UPDRS Motor tasks (Items 23-30) of 10-minute duration were administered as a
possible reference for the video annotator. Throughout the experiment, patients performed a range
of unscripted voluntary activities including walking, sitting, standing, resting, reading, using the
phone, eating, writing, and conversing with researchers and family members. The video camera
operator recorded all the patient activities from a suitable distance which were time synchronized
with the sensor data recordings. The activity type and presence or absence of bradykinesia were
evaluated based upon annotation of video recordings by movement disorder experts using the
standard UPDRS clinical rating scale for Body Bradykinesia (Item 31 of Part III Motor
Examination) (see Appendix A) to identify a truth file for type of activity (sitting, standing,
walking, lying) as well as bradykinesia presence or absence. Those segments of the recordings in
which the presence of bradykinesia overlapped with dyskinesia and freezing were excluded to
avoid possible confounding factors that go beyond the scope of this thesis. The included data had
clear manifestations of bradykinesia as annotated by movement disorder experts.
Manifestation of bradykinesia in unscripted ADL: The impairments caused by bradykinesia
manifest differently by visual observation as well as in the signal data depending on whether there
1 The fluctuating response to levodopa can be broadly divided into "on" and "off" periods. During an "on" period, a
person can move with relative ease often with reduced tremor and stiffness. “Off” periods describe those times when
a person has greater difficulty with movement. A common time for a person with Parkinson's disease to experience
an "off period" is just prior to taking the next dose of levodopa, and this experience is called "wearing off.
12
is a baseline-level of activity present, such as in case of walking, or whether no baseline-level of
activity exists, as is the case for non-walking activities such as sitting or standing. The signals from
lower limb gyroscope (TA muscle) corresponding to walking activities and non-walking activities
are demonstrated in Figure 3. The plots indicated that walking and non-walking signal looked very
different both in presence and absence of bradykinesia. However, on comparing figure 3 (b) and 3
(c) it is observed that due to impairments caused by bradykinesia the signals during walking 3(c)
closely overlap the feature space of normal non-walking activities 3 (b). To eliminate the overlap
of sensor data attributes in feature space, it was pertinent to separate out walking and non-walking
activities.
Figure 3. Sensor data captured during daily activities – Sensor data captured using lower limb
gyroscope during walking and non-walking activities in the presence and absence of bradykinesia.
13
Walking Detection Algorithm Design: The first stage to classify the sensor data into walking and
non-walking subclasses was to accurately detect the individual steps throughout the unscripted
activities. The detection interval of activity type was informed by the time required to capture the
key signal features [23] of the activity subclasses. In the case of step detection, the key feature was
the characteristic swing frequency. It is necessary to capture at least 2 consecutive steps to inform
the swing frequency (Figure 4). As indicated by [24] the average stepping time for PD patients is
1.06 seconds with approximately 2.7% variability.
After analyzing the sensor data during walking activities for different patients, it was
observed that a window with temporal length of 5 second can incorporate 2 consecutive gait cycles
robustly for all severity levels in PD. The detections were provided each second within each 5
second window in which the activity type was determined at the central second and the 2 second
windows on the left and right sides were retained for activity context (Figure 5). The context helped
to correctly capture the transitions between different activity subclasses.
A rigorous signal analysis to detect the stepping activity directed the selection of the
following features: 1. dominant frequency in the axis representative of the step swings (lower
extremity gyrocope); 2. total signal power in the dominant frequency (lower extremity gyroscope);
Figure 4. Normal healthy walking – Sensor data captured using lower limb gyroscope during walking in
the absence of bradykinesia.
14
and 3. total signal power in the voluntary stepping frequency range of 0.3-1.5 Hz in all three axes
(lower extermity accelerometer and gyroscope). These discriminable feature attributes helped to
separate the unscripted activities into walking and non-walking subclasses.
A single-hidden layer Neural Network architecture was designed to classify walking and
non-walking activity subclasses. Input training data for the classifier included feature vectors from
selected training samples incorporating different activity subclasses under the presence and
absence of bradykiensia manifestation. The annotation labels of activity types were used as the
Figure 5: Temporal segmentation of signal data – The first row displays the sensor data captured from
all Trigno sensors concurrently and the window partitions indicate the bradykinesia severity decision
points. For any decision point, the sensor data from 1-min before and 1-min after is evaluated. The second
row displays the segmentation of 2-min sensor data into 5-sec sub windows with 4-sec overlapping and
the third row displays the overlapped sub windows. The walking or non-walking activity decision is made
individually on all these sub windows.
15
truth values for this algorithm. A total of 3000 training samples were used to train the algorithm
and it was tested on unseen 6000 samples.
The trained algorithm was used to determine the activity type within 2-minute windows of
data. This window length was selected to ensure that the slowness characteristics of bradykinesia
can be captured over a long enough duration of time while maintaining sufficient temporal
resolution to track transitions between the different daily activities. Steps detected by the algrotihm
were grouped into walking data and separated from the surrounding non-walking data. If the total
number of seconds of walking activity was greater than 30, then the walking data was used to
assess the presence or absence of bradykinesia during the respective 2-minute window; otherwise
non-walking data was used for the bradykinesia detection.
Bradykinesia Detection Algorithm Design: From the walking and non-walking data, I designed
separate bradykinesia classifiers for walking and non-walking activities that could learn the
relationships between the input features and output clinical annotations of presence/absence of
bradykinesia. The goal of classification was to identify the areas of bradykinesia presence while
Figure 6: Bradykinesia Detection Algorithm Flowchart – Algorithm architecture to capture body
bradykinesia symptoms during unscripted activities of daily living from body worn sensor data.
16
minimizing the misclassifications. Figure 6 demonstrates the overall architecture flow of the
algorithm.
A large set of clinically informed features was designed to capture the movement and muscle
activity characteristics of bradykinesia during the various acivities. Out of these 21 features, 6
were considered most directly related to the objective measures of bradykinesia as defined by the
UPDRS and were therefore also used to provide objective sensor-based measures that can be
used by clinicians to monitor changes in the magnitude of bradykinesia symptoms (please refer to
the section entitled Calculating clinically informed features of bradykinesia manifestations, that
appears later in this thesis). A detail description of all the features is briefly summarized below:
Poverty of Movements: This symptom captures the decrease in the performance of automatic
movements like hand gestures while talking, adjustments in posture while seated, or distinct
temporal sequences in gait. The features designed to capture this symptom include:
Total Active Time Per 2-Minute: Calculated as sum of instances when the absolute value of
low pass filtered gyroscope signal data in any axis is greater than noise threshold value. A low
value of this measurement captured the paucity of movements with in a 2-minute activity
window during non-walking activity.
Degree of Shuffling: Ratio of heel strike and toe-off signal peaks in lower limb GYR data
during walking activities. During normal gait the ratio of toe off peak to heel strike peak is
greater than 1 otherwise it may indicate indistinct toe off and heel strike movements during
walking and thus impoverish movements in the walking during presence of bradykinesia.
Total Activity Peaks: Determines the total number of discrete activity instances above the noise
level. Depending on the total number of peaks present may indicate paucity of movements.
17
This feature is computed for both upper and lower limbs using GYR data during non-walking
activities.
Reduced Velocity of Movements: This symptom captures the inability to produce normal velocity
of voluntary limb movements during ADL.
Mean velocity of voluntary activities in all 3-axis: Identifies voluntary activity peaks in x, y,
z- directions and computes their mean amplitude in GYR data. This feature indicates the speed
of movement with respect to each voluntary activity. Lower value of in this case may indicate
slower average speed of voluntary movements and thus presence of bradykinesia. This feature
is computed both upper and lower limbs for both walking and non-walking activities.
Average toe-off velocity: Captures the average velocity during the toe-off stance of walking.
Lower value of this feature may indicate a slowness during the toe-off stance of gait. This
feature is computed for lower limb using GYR during walking activities.
Dominant arm swing velocity: Identifies the average speed of arm speed during walking
activities and captures the reduced arm swing which is a prevalent symptom of bradykinesia.
This feature is computed for upper limb using GYR during walking activities.
Reduced Range of Movements: This symptom captures the inability to produce the complete
range of voluntary movements in ADL.
The reduced range of movements is captured by integrating the respective velocity features
for both upper and lower limbs.
Hesitancy: This symptom captures the difficulty in initiating a new movement. It is also referred
to as akinesia.
18
Presence of EMG in swing cycle of walking: Captures the EMG signal energy during swing
phase of the walking cycle relative to the energy in toe-off and heel-strike phase. Two distinct
EMG signal peaks during toe-off and heel-strike with relatively lesser energy in the middle for
swing phase may indicate normal gait and absence of bradykinesia. On the other hand, only
one indistinct peak indicates TA activation during the swing phase and thus manifestation of
bradykinesia in terms of lack of coordination during walking.
Power Spectral Density in High Frequency Range: The integrated power spectral density in
frequency range of 7-10 Hz. It provides a measure of overall energy content in high frequency
range which may be used as an indicator of hesitancy. This feature is computed for both upper
and lower limbs using GYR data.
Other Features: These features include objective measurements of overall slowness of movement
during voluntary activities and the measurements capturing the combined effect of the symptoms
described above.
Energy in Dominant Frequency: This feature determines the signal energy of stepping at the
dominant frequency. A low value may indicate (i) consistent but low range stepping (ii) good
range but non-consistent stepping during walking activities.
Dominant Stepping Frequency: Dominant stepping frequency identifies the dominant
frequency value of GYR within the frequency range of walking activities. Lower value of
dominant stepping frequency may indicate inconsistent or arrhythmic walking.
Average zero-crossing of voluntary activity peaks in all directions: Computes the zero-crossing
time of the GYR peaks identified during voluntary activities and provides the average zero
crossing time. Lower value of this feature may indicate faster movements and thus the absence
of bradykinesia. This feature is computed for both upper and lower limbs using GYR data.
19
Ratio of amplitude of peaks to zero crossing distance: This features computes the relative time
of zero crossing with respect to the peak amplitude. It helps to identify the normalized speed
of voluntary movements and presents the speed of movements independent of their range. This
feature is computed for both upper and lower limbs using GYR data.
Total energy in voluntary activity frequency (0.3-1.5 Hz): Computes the total energy in power
spectral density in the range of voluntary activity range in all 3-axis. This feature is computed
for both upper and lower limbs using ACC and GYR data.
To check the robustness of designed features, I identified the discriminability of individual features
using the Bayesian classification that informed the feature selection for next stages of
Table 2. Complete list of identified characteristic features during non-walking activities – The table
represents the list of characteristic features grouped under different symptoms of bradykinesia along with
individual feature discriminability values. The bold red features are the final features selected by
performing the feature selection process.
20
development. Bayesian classification being generative classifiers2 were appropriate choice in this
case to determine individual feature discriminability. The features were ranked with respect to
their individual descriminability. Strating by using the feature demonstrating the highest individual
discreminability as the initial feature, I executed a forward selection process of features. In this
process, the next best discriminative feature was picked iteratively to test the group
discriminability of features. The feature was added to selected list only if it augments the overall
discreminability of group of features. The selection converged when there was no significant
improvement in the overall discreminability of group of features. This method considered the co-
2 Generative classifiers learn a model of the joint probability, p( x, y), of the inputs x and the label y, and make their
predictions by using Bayes rules to calculate p(ylx), and then picking the most likely label y.
Table 3. Complete list of identified characteristic features during walking activities – The table
represents the list of characteristic features grouped under different symptoms of bradykinesia along with
individual feature discriminability values. The bold red features are the final features selected by
performing the feature selection process.
21
dependence in the features and ensured the selection of features that work well together for
classification. This feature selection process was executed for non-walking and walking activities
separately. A complete list of features and their individual separability is presented in Table 2 and
Table 3 respectively.
Classification Algorithms: Once the input features for walking and non-walking classifiers were
finalized, I designed machine learning architectures to robustly classify the presence and absence
of bradykinesia in both walking and non-walking categories. Classification was made on 2-minute
windows of sensor data with 1-minute overlap. Two discriminative models3 were tested: 1)
Logistic Regressions and 2) Neural Networks. The rationale behind choosing these models for
classification was based on the fact that discriminative models learn the (hard or soft) boundary
between classes which made them well suited to discern between presence and absence of
bradykinesia based on different symptoms rather than the generative models which model the
distribution of individual classes and were a better choice to determine individual feature
separability for feature reduction process. Moreover, while generative algorithms make structure
assumptions on the model, the discriminative algorithms make fewer assumptions. For example,
Naive Bayes assumes conditional independence of the features, while logistic regression (the
discriminative "counterpart" of Naive Bayes) does not [25]. As the underlying nature of data is
unknown, wrong modeling assumptions in generative models might have led to higher asymptotic
errors as compared to discriminative models. Therefore, discriminative models were preferred over
the generative models. The motivation behind the choice of Logistic Regressions and Neural
Networks as the starting point was that while former method allows for easier interpretability of
3 Discriminative classifiers model the posterior p(ylx) directly, or learn a direct map from inputs x to the class
labels.
22
model parameters, the latter method allows more flexibility in structuring the model parameters in
terms of the choice of number of hidden layers, hidden units and hyperparameters of the model
[26].
Logistic Regression: Logistic Regressions are discriminative models that allow for easy
interpretability of model parameters. For logistic regression models, it is possible to test the
statistical significance of the model coefficients [26] which can be used to build models
incrementally. Its also easy to update the model to take in new data (using an online gradient
descent method) in logistic regressions. The decision boundary of logistic regressions is linear.
The logistic regression algorithm was implemented as demonstrated in Figure 7.
The mathematical model of logistic regressions can be represented as below.
u = b + x1w1 + x2w2 + … + xmwm
y=eu
1+eu
Here u is the linear regression equation and y is the estimate probability that the given case in a
category. The loss function used in this case was negative log likelihood also known as the
multiclass cross-entropy [27]. Cross-entropy can be used as an error measure when a network's
Figure 7: Flow diagram of logistic regression algorithm
23
outputs can be treated as representing independent hypotheses (i.e. each node stands for a different
concept), and the node activations can be understood as representing the probability (or
confidence) that each hypothesis might be true. In that case, the output vector represents a
probability distribution, and the error measure - cross-entropy - indicates the distance between
network’s belief in the state of bradykinesia manifestation and the true states as annotated by
movement disorder expert.
Neural networks: Neural networks are also discreminiative models considered as non-linear
generalizations of the logistic regression, and thus are theoratically at least as powerful as that
model. However, neural networks are also more flexible than logistic regressions.
One hidden layer neural network was developed (refer to Figure 8)for the classification agorithm
by utilizing ADAM4, an algorithm for first-order gradient-based optimization of stochastic
4 ADAM: an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive
estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has little
memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are
large in terms of data and/or parameters. The method is also appropriate for non-stationary objectives and problems
with very noisy and/or sparse gradients. The hyper-parameters have intuitive interpretations and typically require
little tuning.
Figure 8: Flow diagram of artificial neural network algorithm
24
objective functions, based on adaptive estimates of lower-order moments. The method was inbuilt
in Tensorflow (Google Inc.) and was straightforward to implement, computationally efficient, had
little memory requirements, was invariant to diagonal rescaling of the gradients. The advantage of
using ADAM was that the hyper-parameters required little tuning [28] and it comparesd favorably
to other stochastic optimization methods.
For both classifiers a context layer of classification was implemented using a 5-minute
moving average to obtain more robust results and better capture slower progressions of
bradykinesia . For non-walking activities, the amount of movement had a large variability in per
minute samples which might have led to noisy output detections.The context layer helped in this
case to achieve a consistent classification robust against single 1-minute misclassifications.
Algorithm Training For the training stage, I chose a small set of training data from complete data
set of 8 PD subjects (refer to Table 1) which targeted the differences that could closely discern
between presence and absence of bradykinesia. The training samples were the clear examples of
presence or absence of bradykinesia without any overlap of other PD manifestations. To provide
optimal discrimination of subtle differences in presence and absence of bradyinesia, equal number
of samples were selected to capture each symptom within walking and non-walking activities. The
remaining samples from the same 8 PD patients were used as a part of validation set.
Evaluation Metrics: To evaluate the designed machine learning architectures, discrepancies
between the output of the classification algorithm and expert annotation were evaluated based on
sensitivity (i.e., the ability of the algorithm to correctly identify a movement disorder when it is
present), which can be formulated as
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 =TP
TP + FN
25
and specificity (which describes the ability of the algorithm to correctly identify all instances when
the movement disorder is absent), formulated as
𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦 =TN
TN + FP
The overall accuracy is defined as the as the ratio of the number of correct decisions (true positives
and true negatives) to the total number of decisions, accuracy can alternately be written as
𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦 = (𝐹𝑝 × 𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦) + (𝐹𝑎 × 𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦)
Here, 𝐹𝑝 is the fraction of the total minutes in the database in which the disorder is present,
and 𝐹𝑎 is the fraction of total minutes in the database in which the disorder is absent. From this
formulation, I could observe the dependence of the accuracy on the proportion of disorder present.
Calculating clinically informed features of bradykinesia manifestations: To quantify the clinical
manifestations of body bradykinesia as described in UPDRS guidelines, I calculated 6 clinically
informed features from gyroscope sensor signals (Table 4). Three metrics were used to quantify
Table 4. Clinically informed metrics – The table represents the list of clinically informed metrics to
quantify the symptoms of bradykinesia during walking and non-walking activities.
26
bradykinetic symptoms during non-walking activities and three 3 were used to quantify
bradykinetic symptom during walking activities. Each metric is described below:
1. Total Active Time Per 2-Minute (GAT)
GAT is the total time within a 2-minute window when any type of voluntary activity is
performed. It was calculated as sum of instances when the absolute value of low pass filtered
gyroscope signal data in any axis is greater than noise threshold value. The signal-dependent
noise thresholds were computed as a factor of 2 times the average range of signal values when
no activity is being performed. A low value of this measurement captured the paucity of
movements with in a 2-minute activity window during non-walking activity. Figure 9
demonstrates the instances of voluntary activities within a 2-minute window.
2. Mean Velocity of Voluntary Activities in all 3-axis (GMV)
GMV was computed by first calculating the mean of absolute instantaneous joint velocities of
the voluntary activities in x, y, z- directions using gyroscope signal data. Then an overall mean
of those values was taken as all voluntary movements were treated equally and the assessment
is independent of type of activities within the non-walking subclass. A low value of GMV
indicated low average speed of voluntary movements and hence possibility of bradykinesia
Figure 9: Total active time in 2-minute window – Voluntary activity instances highlighted within 2 -
minute window gyroscope signal. Blue signal corresponds to activity in x-axis, green signal corresponds to
activity in y-axis and Red signal corresponds to activity in z-axis. The total time corresponding to all
highlighted segments is the total activity time in a 2-minute window.
27
occurrence. This measurement was computed for upper-limb and lower limb position during
non-walking activities.
3. Mean Range of Voluntary Activities in all 3-axis (GMR)
GMR was computed by first calculating the integral of absolute instantaneous joint velocities
gyroscope during all voluntary activities in x, y, z-directions and then taking an overall mean
of those values. The integral was computed using trapezoidal method which performs discrete
integration by using the consecutive data points to create trapezoids, so it was well suited to
handling data sets with discontinuities. This method assumed linear behavior between the data
points. GMR captured mean range of voluntary activities during non-walking activities and was
computed for upper-limb and lower limb location.
4. Mean Leg Swing Velocity (GLSV)
GLSV was computed during walking activities by calculating the mean velocity during the
forward swing duration of gait cycle (shaded region in Figure 10) where 10(a) represents a gait
cycle in absence of bradykinesia and 10(b) represents a gait cycle in the presence of
bradykinesia manifestation. Velocity values from gyroscope sensor placed at lower limb were
captured during the swing phase in the dominant axis of forward movement direction with
respect to the knee joint. GLSV was computed as a mean of these values for all the steps taken.
Figure 10: Leg swing during walking – Highlighted swing phase during (a) step cycle in absence of
bradykinesia and (b) bradykinetic step cycle as captured by gyroscope signal.
(a) (b)
28
Depending on the total number of steps taken during walking instances, an average value
indicated the mean leg swing speed. As demonstrated in Figure 10 a high value of GLSV
indicated higher mean leg swing speed as compared to a low value.
5. Mean Leg Swing Range (GLSR)
GLSR was computed by integrating the lower limb gyroscope data during the forward swing
phase of gait cycle during walking activities. The integration was calculated using the
trapezoidal method as used in GMR but only in the swing phase of the dominant axis of forward
movement direction with respect to knee joint. A lower value of GLSR indicated lower range of
movements during walking activities.
6. Mean Arm Swing Range (GASR)
GASR was computed by extracting the upper limb gyroscope signal data during identified
walking instances and then calculating the integral of absolute instantaneous joint velocities
captured by the upper limb gyroscope sensor in dominant axis of movement. The integral is
computed by trapezoidal method as described earlier. A mean of integrated values provides
GASR. A low value of GASR indicates reduced arm swing during walking activities which in turn
indicates the presence of bradykinesia.
29
RESULTS
The test data, including 1000 minutes of unscripted daily activity from 8 patients,
demonstrated the following results: 1) the walking classifier discriminated walking and non-
walking activities from unscripted movements with 99.5% accuracy; 2) the Bradykinesia detection
algorithm detected occurrences of bradykinesia with 96.5% accuracy; and 3) clinically-informed
metrics were able to quantify the motor symptoms associated with bradykinesia to provide subject
specific indications of wearing-off in PD.
Classification of Walking and Non-walking Activities: The neural network architecture
was able to classify walking and non-walking activities with 99.5 % accuracy throughout the
unscripted activity data. The detailed accuracy measures are provided in Table 5. Of the relatively
few false positive and false negatives that did occur, most were comprised of instances in which
the patients turned around in between durations of continuous walking in one direction.
Figure 11, demonstrates the walking detection results on test subject data for a duration of
10 minutes capturing different unscripted ADL. It demonstrates that the walking activitity detector
was successfully able to partition all the activities into walking and non-walking subclasses, even
when frequent transitions occurred or the walking periods were brief.
Table 5. Classification accuracies of Walking Detector – The table represents the Sensitivity, Specificity
and overall Accuracy of Walking Detector.
30
Bradykinesia Detection: A Neural Network architecture and Logistic regression
architecture subclasses were compared. The Neural Networks designed to classify the occurrences
of bradykinesia during walking and non-walking activity performed with an average accuracy of
97.4% for non-walking activities and 93.8 % for walking activities on unseen test data from the 8
PD patients (Table 6). The overall accuracy for all unscripted ADL was 96.5%.
I compared the neural network classification results with those obtained from logistic
regression classification of bradykinesia, and observed the logistic regressions decreased in overall
accuracy by 8.3%. Most of the errors in logistic regressions occurred during walking activities.
The details of results obtained from logistic regressions are demonstrated in Table 7
Figure 11: Walking detector algorithm results – An example of gyroscope data from lower extremity for a 10-minute
segment of ADL. Results of walking and non-walking classifications from the classification algorithm (red bar) are
compared with truth value from annotator (blue bar)
Table 6. Bradykinesia detection accuracies using Neural Network architecture – The table represents
the Sensitivity, Specificity and overall Accuracy of Bradykinesia Detection during walking as well as non-
walking activities using Neural Network architecture.
31
Clinically informed metrics to Track Motor Symptoms of Bradykinesia: Figure 12
demonstrates the classification results for occurrences of bradykinesia and the progression of
clinically informed metrics for one of the subjects tested. It is observed that the classification
results were able to capture the bradykinesia manifestations as annotated by clinician except for a
few false positive detections. The figure also plots the metrics used to quantify changes in symptom
magnitude, including total active time (sec), mean range of movements (deg) and mean velocity
(deg/sec) of movements as a function of time during non-walking activities. At the start of the
experiment, shortly after the subject took PD medication, the clinically informed metrics provided
relatively high values of movement activity, and the neural network correctly detected
bradykinesia was absent. As the experiment proceeded, and the time since the subject last took
medication increased, all three metrics progressively decreased in magnitude and the neural all
three-metrics increased in magnitude and the neural network correctly detected the transition to a
period in which bradykinesia is absent. network correctly detected the presence of bradykinesia.
This response is consistent with the clinical phenomenon of wearing-off. Several minutes after the
administration of PD medication.
Table 7. Bradykinesia detection accuracies using Logistic Regression architecture – The table
represents the Sensitivity, Specificity and overall Accuracy of Bradykinesia Detection during walking as
well as non-walking activities using Logistic Regression architecture.
32
Another example of the performance of our software platform during non-walking data
from a different PD subject is presented in Figure 13. In this example, the neural network algorithm
correctly detects that bradykinesia is present throughout the duration of the experiment. However,
the clinically-informed metrics indicate that at the start of the experiment, approximately one hour
Figure 12: Clinically Informed features during non-walking activities Plot I – Example of clinically
informed metrics during non-walking activities captured by gyroscope placed at lower extremity. The plot
demonstrates instances of bradykinesia presence as well as absence.
33
prior to taking the next dose of medication, there are clear signs of wearing off as Mean Velocity
of Movements and Mean Range of Movements (lower two plots) decrease. These metrics are also
shown to increase soon after the medication dose was taken, indicating that even though the patient
is bradykinetic, they are showing signs of responding favorably to the medication as the magnitude
Figure 13: Clinically Informed features during non-walking activities Plot II – Example of clinically
informed metrics during non-walking activities captured by gyroscope placed at lower extremity. The plot
demonstrates instances of bradykinesia presence.
34
of their bradykinesia symptoms decline. These results demonstrate the same trend of “wearing off”
and improvement following medication dose as displayed in Figure 12 but without the same
changes in the presence or absence of bradykinesia.
Figure 14: Clinically Informed features during walking activities Plot I – Example of clinically
informed metrics during walking activities captured by gyroscope placed at lower extremity. The plot
demonstrates instances of bradykinesia presence as well as absence.
35
Similarly, results for the software platform are provided for walking activities in Figure 14.
As the experiment started the subject demonstrated symptoms of bradykinesia during first 2
instances of walking which were correctly detected by the neural network algorithm (dark pink
shading). The clinically informed metrics including mean leg swing velocity (deg/sec), mean leg
Figure 15: Clinically Informed features during walking activities Plot II – Example of clinically
informed metrics during walking activities captured by gyroscope placed at lower extremity. The plot
demonstrates instances of bradykinesia presence only.
36
swing range (deg) and the range of arm swing (deg) were in relatively similar range for these
instances. In the next walking instance after taking the medicine, all these clinically informed
metrics showed a clear increase and the neural network correctly detected it as absence of
bradykinesia (light pink area).
Another example of objective measures during walking activities is presented in Figure 15.
The figure demonstrates that the patient is identified as bradykinetic throughout the course of
experiment (all the walking periods are dark pink in the figure). However, the clinically informed
metrics gradually decrease from the starting time of the experiment until just after they take their
medication which indicates a wearing off trend. Approximately 50 minutes after taking the
medication all three features show an increase which displays that the patient is responding to their
medication and transitioning to an “on period”. These trends are same as observed in figure 14,
but without the changes in presence and absence detections of bradykinesia.
37
DISCUSSION
The robustness of results achieved, establishes a proof of concept that this framework can
be used to detect the occurrences of bradykinesia and track its symptoms in the form of clinically
informed metrics throughout the day or over the course of several days without interrupting the
daily activities of PD patients. The magnitude of impairments can inform the clinicians regarding
nuanced changes in symptom magnitude which is not captured by the neural networks. It can also
provide information such as which symptoms were more prevalent than others over the monitoring
period and indicate the trends or transitions in symptoms with respect to the medication
replacement schedule, thereby informing the degree of responsiveness to the medication. The
results can be examined more closely to demonstrate the clinical utility of the metrics.
Comparing Figure 12 and Figure 13, it is observed that as per the bradykinesia detection
algorithm, in the first case the subject responded to the medication by reverting to their “On” period
(bradykinesia no longer present) however in second case they didn’t (bradykinesia still present).
Here the clinically informed metrics help to reveal that in fact the subject in Figure 13 did show
some level of responsiveness to the medicine. Moreover, the change in individual metrics
demonstrated that some symptoms of bradykinesia such as range and velocity of movements
responded better to medicine than others such as poverty of movements. Such comprehensive
details of progression trends in bradykinesia symptoms can help the clinicians to identify tailored
medicine for their patients. The comparison of results presented during walking activities in Figure
14 and Figure 15 also support this argument.
With the option to detect occurrences and quantify the underlying symptoms, it is now
possible to see a complete picture of bradykinesia manifestations. This solution is the first attempt
38
in the literature to classify occurrences of bradykinesia during ADL as well as to provide clinically
informed metrics of the symptoms of bradykinesia based entirely on the sensor measurement rather
than trying to use machine learning classifiers to match sensor data to severity scores from
annotators using clinical scales such as the UPDRS [7]. The UPDRS was not designed for
providing continuous metrics for changes in severity and the specific guidelines for classifying
severity into an arbitrary 0-4 classification scale are poorly outlined and susceptible to high
variability due to differences in clinical interpretation. For example, the guidelines for body
bradykinesia are clear in terms of which symptoms may be associated with presence and absence
of bradykinesia but do not provide specific rules for differentiating the different levels of severity.
This limitation is in fact broadly present across all the movement disorder tasks contained within
the UPDRS. The novelty of our approach adopted in this thesis is to use the classifiers to
continuously monitor presence and absence of dyskinesia based upon which symptoms are present,
and then rely on the actual objective metrics from the sensor to monitor changes in the different
symptoms that underlie the disorder.
The ability to successfully provide a software platform for monitoring bradykinesia during
ADL can also be attributed to 1. the algorithm design and 2. the high-fidelity sensors. Strength of
the algorithm comes from the strategy to classify the walking and non-walking activities
separately. These activity categories align themselves with the UPDRS description of bradykinesia
as it specifically identifies a different set of symptoms for the walking and non-walking periods.
This also connects the algorithm interpretations with the expert annotator who also relies on such
clinical impressions to determine when bradykinesia is present or absent. Overall this architecture
provided a dynamic approach to tackle unscripted activities and mitigated the risks of
misclassifications during bradykinesia detection.
39
Rigorous signal analysis supplemented this architecture with highly discriminable features.
The previous work reported in the literature for bradykinesia and other symptoms of PD has mostly
concentrated on using statistical variables of raw signals as features [29,30,31,32,33] where the
current work focuses on activity specific features where each feature is mathematically designed
to capture specific properties of the movement, for example the reduced range and velocity. These
features can characterize the quality of voluntary movements more accurately by capturing the
clinically guided symptoms of impairment directly.
The input training data to these algorithms were intentionally selected to include all
combinations of symptoms of bradykinesia to represent nuanced examples to discern between
presence and absence. All the input samples presented to the algorithm were targeted to tune the
hyperplane close to the detection boundary. Profusion of severe bradykinetic and completely
healthy input samples was carefully avoided as it might have ended up biasing the algorithm.
The selection of neural networks over other machine learning algorithms used in literature
such as support vector machines [31,29,34] was based on the nature of data. Due to inter-patient
variability, the extracted feature data is prone to be noisy which might have resulted in needing a
lot of basis expressions to get separation in SVMs. Neural networks on the other hand are very
flexible on the account of their large number of degrees of freedom and can be well controlled
against overfitting. Choice of neural networks was crucial to guard against the possibility of
misclassifications in the data. When compared against other algorithms such as logistic
regressions, neural networks turned out to perform better because they can learn non-linear
decision boundaries while logistic regressions cannot. In case of bradykinesia detection, the
features demonstrate dependency on each other and sometimes do not have linear discriminability.
40
Clinically informed feature design was not possible without the sensor technology to
accurately capture the body movements. The sensors used for this research provide zero inter-
channel and inter-sensor delay which was critical to capture time synchronized movement and
muscle activity data from upper and lower limbs. The accuracy specifications of these sensors are
provided in Appendix B. Due to their high accuracy these sensors are valid tools to capture the
differences in body movements associated with the disorder and nuanced changes in symptom
magnitude. Gyroscope data was primarily used to compute the clinically informed metrics of
bradykinesia symptoms as it is well suited to capture the body movements in the form of joint
motion. High precision of the clinically informed metrics obtained from gyroscope established the
viability to measure the symptoms of bradykinesia on a quantitative scale to deliver this long-
standing need in the context of unscripted ADL.
These advantages and careful design decisions provided the opportunity to develop the first
software platform capable of tracking symptoms of movement impairments in bradykinesia during
ADL that equip the clinicians with consistent real-time tracking to better inform the progression
of the disease and provide outcome measures for therapeutic interventions, ultimately improving
their quality of life.
41
FUTURE WORK
Future work will be needed on several fronts to achieve a software platform that provides both
comprehensive monitoring capabilities across the full complement of PD movement disorders as
well as sufficient vetting to demonstrate its clinical usefulness. The following items should
command the highest priority:
Expand the relatively short (3 hour) simulated home setting data collections to long-term
(8 – 12 hour) actual home setting recordings. The expanded study will establish the
robustness of these metrics to more accurately represent the longitudinal behavior of
bradykinesia manifestations. It will also validate robustness of clinically informed metrics
under a variety of other activities that were not a part of this study, such as climbing stairs
or preparing meals. With the availability of long-term continuous data, there will also be a
possibility to experiment with the use of time dependent learning algorithms such as
dynamic neural networks, which have the power to increase classification accuracy by
including an added level of information about patterns of change that occur temporally.
Expand the unilateral body sensor data analysis to bilateral recordings: The current study
only explored the clinically informed metrics captured by sensors located unilaterally at
EDS and TA muscle locations on the more symptomatic side of the PD subjects. However,
this approach can be extended to analyze clinically informed metrics from less
symptomatic side which can provide discriminative measures of symptoms of bradykinesia
in both sides. Bilateral recordings also will help in augmenting the algorithms by capturing
other critical features such as the double support time during walking. In addition, because
PD typically progresses from unilateral to bilateral symptoms, this expanded capability
may provide more clinically relevant metrics.
42
Expanding the development of the software platform by monitoring of bradykinesia while
other movement disorders are present: By expanding the sensor data corpus to include
different disorders of PD such as dyskinesia, tremor or freezing, the algorithms will be
tuned to deal with the presence of more than one PD disorder, such as tremor during
bradykinesia.
Expanding the software platform development to include monitoring of other PD
movement disorder classifications and objective metrics. The framework adopted to
classify body bradykinesia in this thesis can be expanded to encompass other PD movement
disorders such as tremor, freezing, and dyskinesia, which are also known to be activity-
dependent and therefore approachable by adopting separate classifiers for different activity
states such as walking and non-walking or sitting vs standing.
These techniques will lead to significant validation and improvements that will help augmenting
the designed algorithms to include broader range of PD symptoms. As a complete solution for
tracking all PD symptoms, the final software platform will be an instrumental tool to guide PD
assessment for enabling better treatment and ultimately improving the quality of life.
43
APPENDIX A
UPDRS
Motor Examination Section
31. Body Bradykinesia and Hypokinesia (Combining slowness, hesitancy, decreased arm
swing, small amplitude, and poverty of movement in general.)
0 = None.
1 = Minimal slowness, giving movement a deliberate character; could be normal for some
persons. Possibly reduced amplitude.
2 = Mild degree of slowness and poverty of movement which is definitely abnormal.
Alternatively, some reduced amplitude.
3 = Moderate slowness, poverty or small amplitude of movement.
4 = Marked slowness, poverty or small amplitude of movement.
44
APPENDIX B
EMG Performance Specifications
Passband Gain = 300 +/- 10%, mean error +/- 1%.
Filter Topology: Butterworth
HP Corner: 20 Hz +/- 5 Hz, -40 dB/dec
LP Corner: 450 Hz +/-50 Hz, -80dB/dec
Noise Baseline: <3mV r.t.i. (rms), mean value ~750 nV r.t.i. (rms)
Accelerometer Performance Specifications
1G error +/-10%, mean error <1%
Offset error, <0.25G
AC RMS error, <0.010 g, mean value ~<0.005g
Gyroscope Performance Specifications
RPM error +/-10%, mean error <1% across 10rpm – 350 rpm.
Max offset +/-7rpm, mean value <1rpm
Max AC RMS error <1 rpm , mean value <0.03 rpm
45
BIBLIOGRAPHY [1] James Parkinson, "An Essay on the Shaking Palsy ," , Sherwood, Neely, and Jones
(London), 1817.
[2] Alan Godfrey, Claudia Mazza, Sue Lord and Lynn Rochester Silvia Del Din, "Free-Living
Monitoring of Parkinson’s Disease: Lessons From the Field," 2016.
[3] Andersen K, Larsen JP, Lolk A, Nielsen H, Kragh Serensen P. Aarsland D, "Risk of
dementia in Parkinson’s disease: A community based prospective study.," Neurology, vol.
56, pp. 730-736, 2001.
[4] Reuben DB., "What’s wrong with ADLs?," J Am Geriatr Soc, vol. 43, pp. 936-937, 1995.
[5] Britne Shabbott, and Juan Camilo Corte´s Pietro Mazzoni, "Motor Control Abnormalities
in Parkinson’s Disease," New York, 2012.
[6] Joseph Jankovic, "Motor Fluctuations and Dyskinesias in Parkinson’s Disease: Clinical
Manifestations," Movement Disorders, vol. 20, pp. S11 - S16, 2005.
[7] Elton RL Fahn S, "UPDRS Development Committee, Unified Parkinson’s Disease Rating
Scale," Recent Developments in Parkinson’s Disease, pp. 153-163, 1987.
[8] Sheridan MR. Flowers KA, "Movement variability and bradykinesia in Parkinson's
disease," Brain: a journal for neurology, pp. 1149-61, Aug 1990.
[9] Grabowski M, Lindvall O, Hagell P Reimer J, "Use and interpretation of on/off diaries in
Parkinson’s disease," J Neurol Neurosurg Psychiatry, vol. 75, pp. 396-400, 2004.
[10] Frisoni GB, Bianchetti A, Zanetti O, Trabucchi M Rozzini R, "Physical performance test
and activities of daily living scales in the assessment of health status in elderly people," J
Am Geriatr Soc, vol. 41, pp. 1109-1113, 1993.
[11] Branch LG, Cummings SR, Curb JD Guralnik JM, "Physical performance measures in
aging research," J Gerontol, vol. 44, 1989.
[12] Angelo Antonini, Carsten Buhmann, Joseph Classen Christiana Ossig, "Wearable sensor-
based objective assessment of motor symptoms in Parkinson’s disease," J Neural Transm,
vol. 123, pp. 57-64, 2015.
[13] Hyung Park, Paolo Bonato Leighton Chan and Mary Rodgers Shyamal Patel, "A review of
wearable sensors and systems with application in rehabilitation," vol. 9, no. 21, 2012.
[14] Elvis Camacho, Mickey E Abraham, Samuel S Bruce, Emmanuel LP Dumont Geoff
Appelboom, "Smart wearable body sensors for patient self-assessment and monitoring,"
New York, 2014.
46
[15] Serge H. Roy, Carlo J. De Luca, and S. Hamid Nawab Bryan T. Cole, "Dynamical
Learning and Tracking of Tremor and Dyskinesia From Wearable Sensors," IEEE
TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, vol.
22, 2014.
[16] Matjaz B. Juric Sebastijan Sprager, "Intertial Sensor-Based Gait Recognition: A Review,"
Sensors, pp. 22089-22127, August 2015.
[17] C.E. Jacobi, J.J. van Hilten R.J.W. Dunnewold, "Quantitative assessment of bradykinesia
in patients with parkinson’s disease," vol. 74, pp. 107-112, 1997.
[18] Kim et. al., "Quantification of bradykinesia during clinical finger taps using a gyrosensor in
patients with Parkinson’s disease," vol. 49, pp. 365-371, Oct 2010.
[19] Haijun Lin and Tim C Lueth Houde Dai, "Quantitative assessment of parkinsonian
bradykinesia based on an inertial measurement unit," 2015.
[20] Danielle E. Filipkowski, David E. Riley, Christina M. Whitney, Benjamin L Dustin A.
Heldman, "Automated Motion Sensor Quantification of Gait and Lower Extremity
Bradykinesia," , San Diego, California USA, 2012.
[21] Malcohm H. Granat Kaiyu Tong, "A practical gait analysis system using gyroscopes," vol.
21, pp. 87-94, 1999.
[22] Rong-Long Wang, De-Jyun Liou, Jin-Siang Shaw Pei-Hao Chen, "Gait Disorders in
Parkinson’s Disease: Assessment and Management," vol. 7, pp. 189-193, 2013.
[23] Frances Huxham, Jennifer McGinley, Karen Dodd, Robert Iansek Meg E. Morris, "The
biomechanics and motor control of gait in Parkinson Disease," vol. 16, pp. 459-470, 2001.
[24] Justine Lowentha, Talia Herman, Leor Gruendlinger, Chava Peretz and Nir Giladi Jeffrey
M. Hausdorff, "Rhythmic auditory stimulation modulates gait variability in Parkinson’s
disease," European Journal of Neuroscience, vol. 26, pp. 2369-2375, 2007.
[25] Michael I. Jordon Andrew Y. Ng, "On Discriminative vs. Generative classifiers: A
comparison of logistic regression and naive Bayes," in Advances in Neural Information
Processing Systems 14, 2001.
[26] Lucila Ohno-Machado Stephan Dreiseitl, "Logigstic regression and artificial neural
network classification models: a methodology review," Journal of Biomedical Informatics,
vol. 35, pp. 352-359, 2003.
[27] Christopher Bishop, Pattern Recognition and Machine Learning., 2006.
[28] Jimmy Lei Ba Diederik P. Kingma, "ADAM: A Method for Stochastic Optimization," in
International Conference on Learning Representations, 2015.
47
[29] M. Pansera, M.T. Arredondo, J.J. Estrada, M. Pastorino, L. Pastor-Sanz, J.L. Villalar J.
Cancela, "A Comprehensive Motor Symptom Monitoring and Management System: The
Bradykinesia Case," in Annual International Conference of the IEEE EMBS, 2010.
[30] A.A. v/d Plas MSc, E.A.H. Wagemans MSc J.I. Hoff MD, "Accelerometric assessment of
levodopa-induced dyskinesias in Parkinson's disease," Movement Disorders, 2001.
[31] Konrad Lorincz, Richard Hughes, Nancy Huggins, John Growdon, David Standaert
Shyamal Patel, "Monitoring Motor Fluctuations in Patients With Parkinson's Disease Using
Wearable Sensors," IEEE TRANSACTIONS ON INFORMATION TECHNOLOGY IN
BIOMEDICINE, vol. 13, November 2009.
[32] Martin W.I.M. Horstink MD, PhD,Stan C.A.M. Gielen PhD Noël L.W. Keijsers MSc,
"Automatic assessment of levodopa-induced dyskinesias in daily life by neural networks,"
Movement Disorders , October 2002.
[33] Tjitske Heida, Member, IEEE, Jeroen P. P. van Vugt, Jan A. G. Geelen, and Peter H.
Veltink, Senior Member, IEEE Daphne G. M. Zwartjes*, "Ambulatory Monitoring of
Activities and Motor Symptoms in Parkinson’s Disease," IEEE TRANSACTIONS ON
BIOMEDICAL ENGINEERING, vol. 57, NOVEMBER 2010.
[34] Member, IEEE, Patrick E. McSharry, Eric J. Hunter, Jennifer Spielman, and Lorraine O.
Ramig Max A. Little∗, "Suitability of Dysphonia Measurements for Telemonitoring of
Parkinson’s Disease," IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, vol. 56,
April 2009.
[35] Resul Das, "A comparison of multiple classification methods for diagnosis of Parkinson
disease," vol. 37, pp. 1568-1572, 2010.
[36] Paolo Bonato, Fatta B. Nahab, Walter Maetzler Alberto J. Espay, "Technology in
Parkinson’s Disease: Challenges and Opportunities," 2016.
[37] Jason A. Chen and Max A. Little Ken J. Kubota, "Machine Learning for Large-Scale
Wearable Sensor Data in Parkinson’s Disease: Concepts, Promises, Pitfalls, and Futures,"
2016.
[38] Kenji Mase and Kiyoshi Kogure Seon-Woo Lee, "Detection of Spatio-Temporal Gait
Parameters by Using Wearable Motion Sensors," vol. 27, 2005.
[39] J Jankovic, "Parkinson’s disease: clinical features and diagnosis," Department of
Neurology, vol. 79, pp. 368-376, 2008.
[40] Markku Kankaanpaa, Mika P. Tarvainen, Vera Novak, Pater Novak, Kun Hu, Brad Manor,
Olavi Airaksinen, and Pasi A. Karjalainen Saara M. Rissanen, "Analysis of EMG and
48
Acceleration signals for Quantifying the Effects of Deep Brain Simulation in Parkinson's
Disease," IEEE Trans Biomed Eng, September 2011.