old motor cortex stroke model and task-specific …
TRANSCRIPT
OLD MOTOR CORTEX STROKE MODEL AND TASK-SPECIFIC
IMAGE PROCESSING ALGORITHM TO DETECT VIDEO
MOTION
by
Chinmayi Bankar
A thesis submitted to Johns Hopkins University in conformity with the requirements for
the degree of Master of Science in Engineering
Baltimore, Maryland
May 2021
© 2021 Chinmayi Bankar
All rights reserved
ii
Abstract
Stroke, depending on the location and the severity, can lead to difficulty in or loss of movement, speech or
vision. Specifically, motor cortex stroke can cause weakness (hemiparesis) or paralysis (hemiplegia) of one
side of the body. To understand the course of recovery and physical treatment options, it is important to
understand the specific effect that motor cortex stroke has on movements. Designing stroke models have
allowed this level of understanding. These animal stroke models, might have a future potential to translate
into human stroke models and aid treatment options for paraplegic or hemiplegic people.
The work in this paper focuses on one such non-human primate stroke model to find out what
specific movements are impacted due to old motor cortex lesion. Specifically, the model1 is designed to look
at “synergies”; a sign of recovery as well as a phase where some patients remain “stuck” during post-stroke
rehabilitation. Post-stroke animal movements are compared with healthy movements using
electromyography (EMG) data to understand and evaluate post-stroke synergies. Chapter 2. Experimental
Details describes this experimental animal model whereas Chapter 3. EMG Analysis and Results and Chapter
4. Kinematic Analysis focus on evaluating the EMG and video data from the model using standard, state-of-
the-art tools and techniques. These evaluations suggest that the specific lesion affects the speed at which an
animal performs a reach with its hand. From the initial data analysis, it is also observed that certain fine-
tuned activities in higher order primates, such as grasping, are impaired due to stroke in the anterior old motor
cortex. Chapter 5. Task Specific Video Analysis Algorithm introduces, describes and validates a new
algorithm that can extract and quantify the observed impairments more accurately than other techniques.
Chapter 6. Discussion revisits and validates the initial observations and links post-stroke synergies to the
grasping impairment. Thus, the results report definite evidence of an impairment in the grasping activity post-
lesion along with recovery after a certain time duration.
First Reader and Advisor: Dr. Reza Shadmehr
Second Reader: Dr. John Krakauer
iii
Acknowledgements
This thesis has been made possible due to the contributions of many incredible scientists and researchers: Dr.
Reza Shadmehr, Dr. Scott Albert, Dr. Stuart Baker, Dr. John Krakauer and Dr. Annie Gott. It is because of
each one of them, with their passionate and invaluable mentorship that has allowed me to contribute to
science as an engineer. I thank Johns Hopkins University for giving me access to such wonderful intellectuals
through the Laboratory of Computational Motor Control, who have shaped me into who I am today.
I cannot thank Reza enough for not only accepting me into his laboratory during my first year as a
naïve master’s student but also for giving me a huge opportunity of being a part of such an exciting and
impactful research project. You have made the impossible; you have made me fall in love with complex
equations and mathematics through your mesmerizing classes and any mathematical work that I do in my
career path would be because of you. You have not only taught me how to be a better researcher but have
also persistently reminded me on how to be a kind human being. I will always remember the values you
imbibed in me. Your motivation, when I was on the verge of giving up, has not only resulted in this work but
has brought me one step closer to my childhood dream of becoming a scientist.
Scott, I do not know where to begin thanking you. Let me begin from the day you introduced me to
the projects in the lab. Your enthusiasm and passion for neuroscience were enough for me to intuitively start
working in the lab. Reflecting back, this was the best decision I took. You have not only been an ultimate
example of a blooming and incredible scientist for me but have also been an amazing, kind and supportive
friend. Not a single page of this thesis would have been possible without you. Your motivational talks have
been the constant voice in my head throughout the rough patches in my master’s journey. Thank you for all
the times you had to painstakingly explain a confusing concept to me. From spoon-feeding me with step-by-
step analytical methods, to making me completely independent, you have been a catalyst in developing my
scientific outlook. Thank you for always supporting me, motivating me, chiming in through all of my
presentations and for taking charge of answering difficult questions. Thank you most importantly, for being
there as a friend and I hope we keep in touch.
iv
I would like to also thank Anna Baines for always helping me out with tons of questions that I had
throughout my thesis work. Without Dr. Annie Gott, the experiment on which this work is based, would not
have been successfully conducted. I also thank all of the people involved in training the monkey and keeping
it healthy. Dr. Stuart Baker and Dr. John Krakauer, I would like to thank you for respectfully challenging my
work and for giving me valuable feedback throughout all of my presentations.
To all of my friends who have shaped me into what I am today and a big thank you to my family
without whom this thesis would not have been possible. Mom, Dad and Tanmay, you have been my backbone
every step along the way. It was because of you that I saw the courage to think differently, dream big and be
bold and fearless in the pursuit of what mattered to me the most. You encouraged me to try out new and
ambitious paths, motivated me throughout my obstacles, believed in me when I questioned myself the most
and most importantly, you gave me my wings; strong wings to fly high. Mom, this thesis is a direct reflection
of the impact that your stories have had on my career path. It also represents my tiny and indirect contribution
to the medical field, something that you so selflessly do every single day and directly impact the lives of
millions of people. Dad, this is a representation of the tenacity, risk-taking nature and values of kindness that
I learnt and adopted from you. Tanmay, you have always been there, supporting me and alleviate my relatable
troubles. All of my success today, is because of you three pillars.
Although due to the pandemic, I could not form a close bond with most of the lab members, I thank
you for sharing your brilliant scientific work in every lab meeting. I learned something new from each one
of you in every single presentation. I will miss the lab and its people very dearly and I hope that our paths
cross again in the future.
v
Dedications
This thesis is dedicated to Ying – the rhesus macaque who was induced with a stroke. I want to thank you
for going through the pain and we promise to make sure that your sacrifice will impact millions of lives in
the future and make a significant difference in the world of neuroscience.
vi
Contents
Abstract ............................................................................................................................... ii
Acknowledgements ............................................................................................................ iii
Dedications ......................................................................................................................... v
List of Tables ..................................................................................................................... ix
List of Figures ..................................................................................................................... x
Chapter 1. Introduction ....................................................................................................... 1
1.1 Stroke ................................................................................................................... 1
1.2 Motor Cortex ........................................................................................................ 1
Chapter 2. Experimental Details ......................................................................................... 3
2.1 Synergies ................................................................................................................... 3
2.2 Implant and Lesion Details ....................................................................................... 4
2.3 Task Setup ................................................................................................................. 5
2.4 Classification of Trials .............................................................................................. 7
2.4.1 Lesion Timeline .................................................................................................. 7
2.4.2 Reach direction ................................................................................................... 7
2.4.3 Arm Support ....................................................................................................... 8
2.5 Recorded Data ........................................................................................................... 8
2.5.1 EMG Data ........................................................................................................... 8
2.5.2 Video Data .......................................................................................................... 9
Chapter 3. EMG Analysis and Results ............................................................................. 10
3.1 EMG data processing .............................................................................................. 10
3.1.1 EMG Data Input ............................................................................................... 10
3.1.2 Amplitude Based Filtering................................................................................ 10
3.1.3 High Pass Filtering ........................................................................................... 10
3.1.4 Butterworth Bandpass Filtering ........................................................................ 11
3.1.5 Full Wave Rectification .................................................................................... 11
3.1.6 Root Mean Squared (RMS) Smoothing............................................................ 12
3.2 Results ..................................................................................................................... 12
vii
3.3 Conclusion and Discussion ..................................................................................... 14
Chapter 4. Kinematic Analysis ......................................................................................... 15
4.1 DeepLabCut ............................................................................................................ 15
4.2 Trajectory ................................................................................................................ 15
4.3 Speeds...................................................................................................................... 16
4.4 Grasp duration ......................................................................................................... 18
4.5 Results ..................................................................................................................... 21
4.6 Limitations .............................................................................................................. 22
Chapter 5. Task Specific Video Analysis Algorithm........................................................ 25
5.1 Sum of Absolute Differences (SAD) ...................................................................... 25
5.2 Motion Energy Analysis (MEA) ............................................................................. 26
5.3 Motion Energy Plot (MEP) ..................................................................................... 27
5.4 Rationale of the task-specific algorithm.................................................................. 28
5.5 Video Processing Pipeline ....................................................................................... 30
5.6 MEP Generation ...................................................................................................... 31
5.7 Cup Opening Detection Algorithm ......................................................................... 31
5.7.1 Moving Average ............................................................................................... 32
5.7.2 Selecting the cup opening peaks ....................................................................... 32
5.8 Grasp Detection Algorithm ..................................................................................... 34
5.9 Algorithm Validation .............................................................................................. 38
5.10 Results of MEA ..................................................................................................... 40
5.10.1 Cup Far Task Combinations ........................................................................... 40
5.10.2 Cup Left Task Combinations .......................................................................... 41
5.10.3 Cup Near Task Combinations ......................................................................... 41
5.10.4 Cup Right Task Combinations ....................................................................... 42
Chapter 6. Discussion ....................................................................................................... 44
6.1 Conclusion ............................................................................................................... 44
6.2 Summary and Future Work ..................................................................................... 45
Appendix ........................................................................................................................... 46
Implanted Muscles ........................................................................................................ 46
EMG Analysis ............................................................................................................... 46
viii
Amplitude Filtering Step ........................................................................................... 46
Fast Fourier Transform Analysis ............................................................................... 48
Artifact at time 0 ........................................................................................................ 50
Interpolation Technique ............................................................................................. 53
Average EMG traces ................................................................................................. 55
Speeds............................................................................................................................ 56
Path length and trajectory analysis ................................................................................ 58
Motion Energy Figures.................................................................................................. 59
Reference List ................................................................................................................... 63
ix
List of Tables
Table 1: Task Marker Names and Codes ............................................................................ 6
Table 2: Tasks classified into 3 different movement groups .............................................. 7
Appendix Table 1: Implanted muscle numbers with their names..................................... 46
Appendix Table 2: Blip: Muscles and time durations ...................................................... 52
x
List of Figures
Figure 1: Flexion synergy1 .................................................................................................. 4
Figure 2: Lesion Details1 .................................................................................................... 4
Figure 3: Reaching Task Steps1 .......................................................................................... 5
Figure 4: Reaching task positions1 ...................................................................................... 5
Figure 5: Top view: Reaching task combinations with names ........................................... 6
Figure 6: Visualization of the 3 different movement groups1 ............................................. 7
Figure 7: Experimental view from the top camera ............................................................. 9
Figure 8: EMG Processing Steps ...................................................................................... 11
Figure 9: Supraspin: Average EMG traces ....................................................................... 13
Figure 10: iEMG reported over 1.5 seconds ..................................................................... 13
Figure 11: iEMG reported over 0.6 seconds ..................................................................... 14
Figure 12: Wrist position tracked with DeepLabCut1 ....................................................... 15
Figure 13: Trend in the trajectories1 ................................................................................. 16
Figure 14: Speed plots for 14 example sessions ............................................................... 17
Figure 15: Speed plots for 4/14 sessions........................................................................... 17
Figure 16: Peak Speeds for the reach (grouped by movement direction) ......................... 18
Figure 17: Speed plots with three subtasks of the experimental task ............................... 19
Figure 18: Cup Near tasks: Grasp duration trend ............................................................. 20
Figure 19: End of first reach and start of second movement ............................................ 21
Figure 20: Grasp duration trend (from speed plots) .......................................................... 22
Figure 21: Sum of absolute differences concept12 ............................................................ 25
Figure 22: Cup ROI for the video frame ........................................................................... 27
Figure 23: MEA concept with Motion Energy Plot .......................................................... 28
Figure 24: Markers depicting parts of Sequence II ........................................................... 30
Figure 25: Video Processing Pipeline ............................................................................... 30
Figure 26: Moving Average of the raw MEP ................................................................... 32
Figure 27: Peak Definitions17............................................................................................ 33
Figure 28: Filtering process for cup-opening peak detection ........................................... 34
Figure 29: Example MEP: Relation between sequence of actions and MEP features ...... 35
Figure 30: MEP with algorithm-detected features and visual estimates........................... 37
Figure 31: Grasp duration detectors plotted on the MEP ................................................. 38
Figure 32: Algorithm validation ....................................................................................... 39
Figure 33: Cup Far tasks: Grasp duration trend ................................................................ 40
Figure 34: Cup Left tasks: Grasp duration trend .............................................................. 41
Figure 35: Cup Near tasks: Grasp duration trend ............................................................. 42
Figure 36: Cup Right tasks: Grasp duration trend ............................................................ 43
Figure 37: Comparison of all grasping duration trends .................................................... 44
xi
Appendix Figure 1: Raw EMG data with high amplitude noise ....................................... 47
Appendix Figure 2: EMG data after amplitude-based filtering ........................................ 48
Appendix Figure 3: Single-Sided Amplitude Spectrum of raw EMG signal ................... 49
Appendix Figure 4: Raw EMG with artifacts (amplitude filtered) ................................... 50
Appendix Figure 5: Result of applying high pass filter .................................................... 50
Appendix Figure 6: Biceps: Prevalence of the blip (average and median) across all trials
........................................................................................................................................... 51
Appendix Figure 7: Triceps: Prevalence of the blip (average and median) across all trials
........................................................................................................................................... 52
Appendix Figure 8: Interpolation results for muscles ....................................................... 54
Appendix Figure 9: BR: Average EMG traces ................................................................. 55
Appendix Figure 10: ECR: Average EMG traces............................................................. 55
Appendix Figure 11: PD: Average EMG traces ............................................................... 56
Appendix Figure 12:Speed plots for the reach (Handle Left Cup Near) .......................... 56
Appendix Figure 13: Speed plots for the reach (Handle Right Cup Near) ....................... 57
Appendix Figure 14: Speed plot outliers .......................................................................... 57
Appendix Figure 15: Speed plot outliers .......................................................................... 58
Appendix Figure 16: Trajectories for the index finger ..................................................... 58
Appendix Figure 17: Grasp duration trend: Cup Far ........................................................ 59
Appendix Figure 18: Grasp duration trend: Cup Left ....................................................... 60
Appendix Figure 19: Grasp duration trend: Cup Near ...................................................... 61
Appendix Figure 20: Grasp duration trend: Cup Right .................................................... 62
Chapter 1. Introduction
1.1 Stroke
A sudden interruption of blood supply to the brain is called an ischemic stroke. Blot clots often cause
blockages that lead to these types of strokes. In some cases, bursting of blood vessels inside the brain can
cause internal hemorrhage resulting in what is termed as a hemorrhagic stroke.
Depending on the type, location and severity of the lesion, stroke can lead to difficulty in or loss of
movement, speech or vision. The function and side that is controlled by the brain region damaged, is impaired
after stroke. Specifically, if stroke occurs in the motor cortex, it affects movements by causing weakness
(hemiparesis) or paralysis (hemiplegia) of one side of the body. Stroke in the left hemisphere will impair the
right side and vice-versa.
The following paper analyzes and describes the effect of motor cortex stroke on voluntary movements
in a non-human primate. This ischemic animal stroke model is developed with a goal of studying post-stroke
voluntary movement patterns.
1.2 Motor Cortex
Motor cortex is the region of the frontal lobe of the brain responsible for the control of voluntary movements.
Primary motor cortex (Broadmann area 4 or M1) is one of the principal brain areas involved in motor
function. The role of the primary motor cortex is to generate neural impulses that control the execution of
movement. Literature shows that M1 can be anatomically subdivided into a region that has direct control
over motor output (new M1) consisting of corticomotoneuronal cells (CM) cells and a separate region that
influences the motor output only indirectly (old M1) through spinal cord mechanisms2. The new M1 region
leads to novel and more refined or finer movements in higher primates3.
Stroke in the M1 area of the brain can cause hemiparesis or hemiplegia depending on the degree of
damage. If the stroke occurs in the new M1 region, more dexterous movements might be affected. Whereas,
if the stroke occurs in the old M1 region which is the central pattern generator or gives motor primitives to
2
generate a wide range of skilled behavior, different movement impairments can be seen. This paper reflects
on some of the changes observed in task specific movement patterns due to stroke in the rostral region or the
old M1 region of the non-human primate’s brain.
3
Chapter 2. Experimental Details
The experiment in this paper was designed and conducted at the Movement Laboratory headed by Dr. Stuart
Baker in Newcastle, England. A non-human primate, a rhesus macaque was trained to perform a reach and
grasp task. After the macaque was fully trained for a month, baseline data was recorded pre-lesion to be
compared with the data acquired post-lesion. More details about the experiment’s goal, lesion, task setup and
about the data are given in the subsequent sections.
2.1 Synergies
Motor cortex stroke often leads to weakness or paralysis but over the following weeks, there is often some
recovery1. During this recovery phase, patients develop “muscle synergies” and get stuck at this phase.
Muscle synergies are positive motor signs, meaning, they involuntary increase the muscle activity,
movements or patterns. Many patients are able to suppress synergies, but a few patients are not able to do so
and remain in this recovery phase. Muscle synergies significantly contribute to motor disabilities in stroke
survivors, thus making it imperative to study these in a stroke model. While many studies have looked at
weakness after motor lesions, very few have looked at synergies. The goal of this experiment was therefore
to design a primate model of post-stroke synergies.
To understand the course of recovery following stroke, it is important to understand the trends in
these synergies in the time period following lesion. A common way of quantifying synergy patterns is with
the help of electromyography (EMG) data as described in Chapter 3. EMG Analysis and Results. One of the
results of stroke is the development of an abnormal shoulder-elbow flexion synergy, where lifting the arm
can cause the elbow, wrist, and finger flexors to involuntarily contract, reducing the ability to reach with the
arm and hand opening (Figure 1).
The experiment design also accounts for the need to analyze differences in synergy patterns if arm
support is provided4. Refer to Section 2.4.3 Arm Support for more details.
4
Figure 1: Flexion synergy1
2.2 Implant and Lesion Details
After the monkey was fully trained to perform the reaching task, 12 stainless steel EMG wires were implanted
in various arm and forearm muscles. Details of the muscles implanted are given in Appendix: Implanted
Muscles. This allowed for recording of the EMG data for the baseline period before stroke. To simulate an
ischemic stroke, the M1 lesion was performed by injecting Endothelin 15. The aim was to lesion the rostral
region of the left hemisphere of the motor cortex surface from ML8 to ML19 (Figure 2).
The trials were resumed 3 days after the lesion and post-stroke data was recorded via the implanted
EMG wires.
Figure 2: Lesion Details1
5
2.3 Task Setup
Each reaching task that the macaque performs, consists of a handle and a locked food cup containing a food
reward. The locked food cup opens after a handle hold period of 1 second. The monkey then performs a reach
from the handle to the food cup containing reward, grasps the food and then makes a second movement to
get the food to its mouth (Figure 3).
Figure 3: Reaching Task Steps1
The handle and the food cup are arranged in the form of a virtual diamond in front of the monkey
(Figure 4). There are 12 different reach and grasp task combinations (4 ways to place the handle and 3 ways
to place the food cup when the handle is positioned = 12 arrangements). These 12 directional tasks allow for
analysis of motor behavioral changes post-stroke in all the possible directions. Each task is named with the
position of the handle and the cup and also has an associated task marker and name defining it (Figure 5).
The task naming system and the corresponding task markers are enlisted in Table 1.
Figure 4: Reaching task positions1
6
Task Marker Number Task Marker Code
0 Handle Near Cup Left
1 Handle Near Cup Far
2 Handle Near Cup Right
3 Handle Left Cup Near
4 Handle Left Cup Far
5 Handle Left Cup Right
6 Handle Far Cup Near
7 Handle Far Cup Left
8 Handle Far Cup Right
9 Handle Right Cup Near
10 Handle Right Cup Left
11 Handle Right Cup Far Table 1: Task Marker Names and Codes
Figure 5: Top view: Reaching task combinations with names
Each task combination is repeated any number of times ranging from 4 to 7 repetitions. Each
repetition is a trial for that particular task combination. Thus, one session consists of any number of trials in
between 48 and 84 trials. Classification of trials is given in the following section.
7
2.4 Classification of Trials
2.4.1 Lesion Timeline
Trials can be classified based on the timeline of the lesion into pre-stroke trials and post-stroke trials. After
the monkey was fully trained for one month to perform the reaching task, pre-stroke data or baseline data
was recorded for up to one week before lesion. The lesion date is 18th of March 2019. Post-stroke data is
acquired for up to 4 months following the lesion date. As compared to full 4 months of post-stroke data, pre-
stroke data is less.
2.4.2 Reach direction
According to the direction of the reach from the handle to the food cup, trials can be grouped into:
1. Movements away from the body (out of synergy)
2. Movements towards the body (flexion synergy)
3. Horizontal movements
Movement group Task Combinations
Movements away from the body Handle Near Cup Far,
Handle Near Cup Left,
Handle Near Cup Right,
Handle Left Cup Far,
Handle Right Cup Far
Movements towards the body Handle Far Cup Near,
Handle Far Cup Left,
Handle Far Cup Right,
Handle Left Cup Near,
Handle Right Cup Near
Horizontal movements Handle Left Cup Right,
Handle Right Cup Left Table 2: Tasks classified into 3 different movement groups
Figure 6: Visualization of the 3 different movement groups1
8
2.4.3 Arm Support
The weight of the arm of the monkey while performing the reaching task could be optionally supported by a
removable table insert. The trials were conducted both with arm support (Table on) and without arm support
(Table off). This accounted for analyzing the effect of using arm support in reaching task performance as
compared to without arm support. Post-stroke rehabilitation strategies usually use arm support. Fatigue drives
increased synergies and therefore it is expected that table off trials will show an increased muscle activity
post-stroke via EMG in at least some of the muscles involved in synergistic groups.
2.5 Recorded Data
Since the left hemisphere of the old motor cortex was lesioned, the monkey was trained to perform the task
with its right hand. The data recorded from the experiment is of two types:
1. Electrophysiology data - EMG
2. Video data
Both the electrophysiology data and video data are available for one weeks preceding the event of lesion
and four months following the lesion date.
2.5.1 EMG Data
The implant of 12 stainless steel EMG wires in 12 arm and forearm muscles (Appendix Table 1) allowed for
one week of pre-stroke baseline EMG data to be collected. Due to recording issues in the Anterior Deltoid
(AD) muscle connector, analysis throughout this paper refers to the remaining 11 muscles. Along with EMG
data, task marker data (Table 1) and the corresponding timing information (start and end of the trial associated
with the task marker) are available. A ground leakage contact sensor informing whether the monkey is
holding the handle or not gives the exact timing when the monkey leaves the handle to make the first reach.
Throughout the course of this paper, time=0 seconds represents the time when the monkey lets go of the
handle to make the first reach from the handle to the food cup. EMG data is continuously recorded in the
entire experiment at a sampling rate of 5000 Hz. The necessary EMG data during task performance can be
extracted using timing information and contact sensor data.
9
2.5.2 Video Data
Apart from EMG data, video recordings for each session are also available. The video data consists of one
week of pre-stroke and four months of post-stroke data. However, unlike the EMG data, video is recorded
intermittently and only during task performance. A video for a particular recording session date would
therefore comprise of a series of individual recordings for respective trials. Each individual recording snippet
consists of a recording of the entire trial i.e., from the time the monkey starts the handle hold period till the
time the food reaches the mouth. Camera frame rate is 100 Hz. The video data is recorded from 3 different
cameras:
1. Top Camera
2. Left Camera
3. Right Camera
The top camera gives an X-Y plane view and is best for quantifying the first reach since the movement
happens in the horizontal plane (X-Y plane) from the handle to the food cup. The left and the right cameras
give a Z plane view and are best for quantifying the second movement (from the food cup to the mouth). The
Z plane view is depicted in Figure 3. The analysis in this paper uses only the top view camera videos as all
the trends and behaviors quantified in the results are best measured in the X-Y plane. Chapter 3. EMG
Analysis and Results describes the methods and results of the analysis of the electrophysiology (EMG) data.
Chapter 4. Kinematic Analysis describes the initial kinematic analysis conducted on the video data and the
observed results. Chapter 5. specifically focuses on the observations from Chapter 4. Kinematic Analysis
and defines a new image processing algorithm and pipeline for video data analysis.
Figure 7: Experimental view from the top camera
10
Chapter 3. EMG Analysis and Results
This section describes the signal processing pipeline developed for processing the raw EMG data from the
experiment. All the sessions (one week of pre-stroke and 4 months of post-stroke electrophysiology) were
processed to identify if EMG amplitudes increased or decreased in each of the 11 muscles after lesion.
3.1 EMG data processing
The EMG data is processed using a standard method of processing electrophysiological signals6.
3.1.1 EMG Data Input
The raw digital data recordings obtained from the experiment are continuous in nature. The EMG data
corresponding only to the task performance period for each trial is extracted using the contact sensor and
timing information. The period of interest for determining trends is the period of the first reach in all
directions. A total of 1.5 seconds of EMG data from 0.5 seconds before the monkey lets go of the handle to
1 second of reach task duration is captured for each trial and for all the sessions in the experiment. Figure 8
shows the signal processing pipeline for filtering the EMG signals. Each step is further explained in detail in
the following subsections.
3.1.2 Amplitude Based Filtering
Amplitude-based filtering is not a typical signal processing step involved in standard EMG analysis. It is an
additional filtering step best suited for filtering this specific experimental data. The amplitude range of EMG
signal is 0-10 mV (+5 to -5 mV) prior to amplification7. Various sources of electrical noise can lead to this
kind of very high amplitude noise observed in the EMG signal. The details of the noise and the need for
introducing this step before digital filtering is explained in the section Appendix:Amplitude Filtering Step
along with examples.
3.1.3 High Pass Filtering
High pass filter is applied to EMG to remove low-frequency noise interference. The general lower cutoff for
applying a high pass filter is 10-30 Hz. However, higher cut-off frequency values are acceptable if frequency
11
estimation is not done on the data8. The details of the noise and the choice of cutoff frequency for the digital
high pass filter is explained in the section Appendix: Fast Fourier Transform Analysis along with examples.
Figure 8: EMG Processing Steps
3.1.4 Butterworth Bandpass Filtering
A butter worth bandpass filter with a frequency range of [70Hz, 1500Hz] is used to smoothen the high pass
filtered data. A lower cutoff of 70 Hz is justified by the frequency cutoff of the high pass filter design in
Section 3.1.3 High Pass Filtering. The lower cutoff of 1500 Hz is applied according to ISEK
recommendations given for intramuscular and needle EMG6. The order of butter worth filter used is fourth
order. The output of this phase is passed on to the full wave rectification phase.
3.1.5 Full Wave Rectification
To get the shape or the envelope of the EMG signal, full wave rectification is performed. The absolute values
of all EMG traces rectify and convert all the negative activations to positive ones. Since the goal of the EMG
analysis is calculating the mean and the max amplitudes reached through amplitude estimation, rectification
is performed. This is necessary as EMG signal is inherently a zero-mean signal with oscillations that swing
12
more or less equally on either side of zero. A “rectify and mean” approach is used to filter the EMG signal
and hence, rectified signal is passed over to the next step.
3.1.6 Root Mean Squared (RMS) Smoothing
RMS Smoothing technique is used to capture the envelope of the EMG signal. The RMS value of the signal
is computed within a time window of 40 seconds. The window slides over the across the signal to capture
the envelope. Since RMS inherently squares the entire signal converting the signal to positive, full wave
rectification is not required but it is still listed as a step as it does not harm the signal. After the EMG traces
pass through the signal processing steps, amplitude estimation is performed. The results and summary are in
described in the consequent sections.
3.2 Results
The digitally filtered EMG traces after passing through the steps above are then analyzed to extract important
metrics determining increases or decreases in muscle activity following lesion. This is done by grouping all
the EMG traces for the trials in these four different categories:
1. Table On Pre-stroke
2. Table On Post-stroke
3. Table Off Pre-stroke
4. Table Off Post-stroke
and then averaging the traces for the four groups separately. These average traces are plotted for all the
individual muscles against time. The average traces are also plotted separately for the 12 reaching directions
(example muscle in Figure 9). Thus, the EMG traces are segregated by task combinations, arm support and
lesion timeline and differences due to changes in the parameters can be summarized.
However, average EMG traces are further scanned for possible artifacts (details in the section
Appendix: Artifact at time 0). The muscles that have the artifact are not considered for the results in this
section. The muscles that are considered are BR, Supraspinatus, ECR and PD.
For certain combinations, average EMG amplitudes for table off (without arm support) are higher
than average EMG amplitudes for table on (with arm support) for the Supraspinatus muscle. This difference
13
is partly consistent with flexor synergy. Results for the average EMG traces for the rest of the considered
muscles are in the section Appendix: Average EMG traces.
Figure 9: Supraspin: Average EMG traces
To compare the average EMG trends in the considered muscles, integral of the average EMG traces
was taken over a time duration of 1.5 seconds (-0.5 seconds before the monkey lets go of the handle and 1
second after the monkey lets go of the handle) and 0.6 seconds (-0.3 seconds before the monkey lets go of
the handle and 0.3 seconds after the monkey lets go of the handle). The iEMG values were then reported for
each of these muscles, grouped by the stroke timeline and arm support conditions (Figure 10 and Figure 11).
Figure 10: iEMG reported over 1.5 seconds
14
Figure 11: iEMG reported over 0.6 seconds
3.3 Conclusion and Discussion
From the iEMG figures reported in the previous section, it is clear that for some task combinations,
Supraspinatus muscle shows an increased iEMG in the table off trials. This is consistent with flexor synergy.
However, it shows a decline in the iEMG levels post-stroke as compared to pre-stroke. For the rest of the
muscles, there is almost no difference between the iEMG for table on and table off trials. However, post-
stroke iEMG is greater than pre-stroke iEMG trials for all of the task combinations in the muscles: BR, ECR,
PD. This is also consistent with increased flexor synergy seen post-stroke.
Further statistical analysis to calculate significant differences between the pre- and post-stroke
scenarios in BR, ECR and PD muscles is required before forming a conclusion. Similarly, statistical analysis
for the difference in the arm support trials for the Supraspinatus muscle is needed. The remainder of the paper
focuses on the kinematics and behavioral aspects of the monkey’s movements by using the video data.
15
Chapter 4. Kinematic Analysis
4.1 DeepLabCut
DeepLabCut software9 was used to track the monkey’s reaching movements. A point on the wrist was labelled
manually in 100-200 frames to create the video training dataset as shown in Figure 12. The network was then
trained to estimate the wrist position throughout the entire video. The analyses in the following sections are
obtained from these wrist estimates of position.
Figure 12: Wrist position tracked with DeepLabCut1
4.2 Trajectory
Estimates of wrist position tracked by the DeepLabCut software in the X-Y plane were converted to physical
distances using known measurements visible in the footage. These trajectories were plotted for a few trials
and for three different reaching directions:
(1) Out of synergy; extension movements – Cup Far movements
(2) Involving flexion synergy – Cup Near movements
(3) Lateral movements – Cup Right movements
The trajectories were also plotted across a varied timeline of the stroke, to give a better idea of how
post-stroke trajectories differ from the pre-stroke ones (Figure 13). It is observed from Figure 13 that the
trajectories do not deviate from their path even after lesion. The trajectories for out of synergy task
16
combination trials, however, do appear to be shorter in length than the pre-stroke ones (where the hand
overshoots).
Figure 13: Trend in the trajectories1
Trajectory analysis also included calculating the length of path traversed during the reach from the
handle to the food cup from these wrist position estimates. Detailed explanation and examples of this analysis
is given in the section Appendix: Path length and trajectory analysis.
4.3 Speeds
The physical distances obtained from the trajectory analysis were then used to create speed plots of the
movements in the task. A speed increase and then a decrease symbolizes one movement. First speed increase
therefore corresponds to the first reach from the handle to the food cup and a second speed increase
corresponds to the second movement from the food cup to the mouth. Figure 14 shows the speed plots of 3
pre-stroke and 11 post-stroke sessions for the first reach.
For the first reach of the Handle Far Cup Near task combination, the average peak speed attained
shows a particular trend over the duration of the experiment. Additional figures for other Cup Near
combinations are attached in the section Appendix: Speeds. Firstly, the mean peak speed of the reaching trials
for sessions just after the lesion is lesser than the mean peak speed for pre-stroke trials. Secondly, as time
17
progresses, post-lesion improvements in average peak speed can be seen. Figure 15 represents 4 out of the
14 sessions in Figure 14 to illustrate a clear trend in the speed.
Figure 14: Speed plots for 14 example sessions
Figure 15: Speed plots for 4/14 sessions
A clearer representation of this phenomenon is shown by the cluster of individual trials in Figure
16. The “Cup Near” combinations, where flexion synergy works, show a decline in the peak speeds for all
trials just after lesion. 56 days post-stroke, these reduced peak speeds recover, or become even better than
18
the baseline. The combinations that are out of synergy, i.e., the “Cup Far” task combinations requiring muscle
extension instead of flexion, the peak speeds show a similar trend. The peak speeds remain unaffected for
lateral movements (“Cup Left” or “Cup Right”).
Figure 16: Peak Speeds for the reach (grouped by movement direction)
4.4 Grasp duration
The speed plots were then extended to account for a greater time duration after the monkey lets go of the
handle. Thus, apart from the peak speeds reached during the first reach, the three different parts of the entire
experimental task, namely:
1) First reach (handle to the food cup)
19
2) Grasp (picking up the food)
3) Second movement (food cup to the mouth)
were also analyzed as illustrated in Figure 17.
Figure 17: Speed plots with three subtasks of the experimental task
The first reach is represented by an increase and then a decrease in the speed of the wrist in the plot.
Similarly, the second movement after the monkey picks up the food to eat, is also represented by an increase
and a decrease in the speed. The grasp duration is then the time between the end of the first reach and the
start of the second movement.
Figure 18 consisting of 3 of the above 14 sessions (one pre-stroke session, 2 post-stroke) shows that
the second movement start is delayed in the session just after the lesion. After some amount of time past the
lesion date, the second movement starts much earlier than in the session just after lesion. Qualitatively, the
lag in the start of the second movement signifies that the monkey spends a longer amount of time grasping
for the food in the food cup right after lesion as compared to a pre-lesion session. An earlier start for the
second movement in a 56-days-post-stroke session compared to an 8-days-post-stroke session suggests some
recovery in the impaired ability to grasp.
20
This trend is more prominent in the “Cup Near” combinations as shown in Figure 18. One might
wonder if the experimental setup of the plexiglass below the monkey’s head and the positioning of the cup
right below the plexiglass makes it difficult for the monkey to get a good visual feedback on the location of
the food, hence making the duration of the grasp longer. To justify the recovery seen 56 days post-stroke, the
monkey just gets better at estimating the position of the food cup and the food itself when there is no visual
feedback. But why would then the monkey be faster at grasping the food before the lesion, when it has way
lesser practice than post-lesion?
Figure 18: Cup Near tasks: Grasp duration trend
The trend surely hints at the underlying impairment and ability of the monkey to grasp the food.
However, a few questions still remain: Is the trend seen only for the “Cup Near” task combinations? If so,
why does the position of the cup play a role in the impairment. Is it due to the flexion movements or due to
the experimental setup (dismissed a while ago)? A detailed analysis of all the existing sessions would be
required to quantify this trend.
Using the speed plots, the grasp duration was calculated mathematically. A speed threshold of 15
cm/sec was set to define the offset of the first reach and the onset of the second reach. A zoomed in version
of Figure 17 marks these offset and onset points on the speed plots for the individual trials (Figure 19).
The time difference between the onset of the second movement and the offset of the first movement
defines the duration of the grasp. The trend in Figure 18 was validated through this speed thresholding-based
calculation (Section 4.5 Results). However, grasp duration calculated via this method is subject to several
heuristic based estimates (Section 4.6 Limitations).
21
Figure 19: End of first reach and start of second movement
4.5 Results
The results of the grasp durations obtained from the threshold-based speed plot analysis for the 14 sessions
are described in this section.
As seen in Figure 20, the grasp duration increases right after stroke and then there is a recovery.
This trend, however, is not just limited to the “Cup Near” task combinations. It is present in almost all of the
combinations. The grasp durations post-lesion, show a decreasing trend signifying the recovery and some
22
grasp durations even go below the baseline time period. 14 sessions are enough to assume that a similar trend
would persist if the entire data were evaluated. But 14 sessions are not enough to conclude that the trend does
not exist for the “Cup Right” combination. This trend obtained from the DeepLabCut speed plot analysis
gives a preliminary overview of the actual trend seen and discussed in detail in the next parts of the paper.
Figure 20: Grasp duration trend (from speed plots)
4.6 Limitations
The shortcomings of calculating a speed threshold-based grasp duration obtained from DeepLabCut speed
plots are listed below:
1. DeepLabCut parameters: The speed plots are a result of the wrist coordinates obtained from
DeepLabCut analysis. However, DeepLabCut’s output depends on variables such as training
parameters used, quality of the videos, visibility of the animal’s body part to be tracked in all the
frames of all videos etc. The trained network described in this paper was trained on 100-200 frames
but did not include all training situations. Differences in training situations such as changes in the
light intensity or changes in the top-view camera’s angle etc., were not fully captured in the training
set. Changes in the camera angle can lead to incorrect representation of speed trajectories if the
23
network is not fully optimized as a difference of even 2-3 pixels can translate into a difference of a
few cm/seconds in terms of speed. Moreover, outputs from DeepLabCut software are an estimate
of the monkey’s wrist coordinates based on the “wrist point” defined by the user. Subjective changes
in the wrist point while manually labelling frames can also lead to erroneous speed plots. Refer to
the Section Appendix: Speeds for examples of such speed plots. Since the speed thresholding method
depends on the speed traces obtained from the wrist position estimates, the ability of the network to
accurately quantify and generalize well is a requirement.
2. Tracking plane: The speed plots give a visual representation of the fact that the second movement
start is delayed after lesion as compared to healthy task reaches. Even if the estimated points from
the DeepLabCut network were close to actual, mathematical quantification of the “exact time
duration spent in grasping the food” still would depend on a number of factors. Since the first
movement was carried out in the X-Y plane, the speeds would be close to an accurate representation
of those in reality. However, since the second movement was carried out in the X-Y-Z plane, the
speed plots for the second movement are a 2D visualization of a 3D movement leading to lower
peak speeds reached. For example, the second movement (food cup to the mouth) is in the Z
direction has a speed of 15 cm/sec but since the tracking is done based on only the top camera
frames, in a 2D plane, the speed will be virtually close to 0. This would lead to a misleading
representation of when the second movement really begins, if quantified by a threshold at a set
quantity. Using all the three cameras for the analysis would help build a proper 3D representation
of the second movement but due to experimental impediments, the 3 cameras are not always synced.
Additionally, the camera frame rate and angle, changes minutely from session to session. Dropping
of frames might lead to sudden increases in speed plots and the change in camera angle might
involve a need for different pixel to physical distance mappings.
3. Speed Thresholds: Using the same threshold for defining onsets and offsets of both the movements
is also inaccurate since both the movements are different in nature (partly due to the 2D
24
representation of the second movement). The speed threshold itself plays a crucial role in
determining starts and ends of the grasping task. 10 cm/sec might be too early to define a movement
onset, but 15 cm/sec might be too late. Here, the shortcomings of quantifying the grasp duration
using a trial-and-error-based or a heuristic-based plot, that is already constructed from a position
“estimate”, gives rise to a need for a more temporally sensitive methodology.
Optimization of the DLC network and syncing of camera frame rates, angles and planes would have led to
an accurate representation of grasp duration, but at the cost of intensive time investment. The next chapter
describes a simple yet very effective way to detect grasp durations using an image processing technique.
25
Chapter 5. Task Specific Video Analysis Algorithm
This section describes an image processing pipeline specifically developed for calculating the time required
for grasping the food in the experimental task.
5.1 Sum of Absolute Differences (SAD)
In digital image processing, SAD is a measure of similarity between two image blocks. In video processing,
SAD can be used to compare two different frames from the same video. Comparison of two different frames
from the same video can give either a measure of similarity or dissimilarity in the positions of the objects in
the video or the objects themselves. It can also be used for object detection and for constructing disparity
maps from stereo images10.
The absolute difference between each pixel in the current video frame and the corresponding pixel
in the consecutive video frame is summed up to give a single scalar quantity. This scalar quantity defines
image similarity between two image blocks which is also referred to as Manhattan distance or
L1 norm11. Figure 21 illustrates the use of SAD in video processing.
Figure 21: Sum of absolute differences concept12
26
5.2 Motion Energy Analysis (MEA)
MEA is based on the SAD algorithm and has been used for quantification of movement from recorded video-
files13. It is based on the concept of frame-differencing. Frame differencing is a computer vision technique
that subtracts an entire video frame from the next frame. If there is a change in the pixel intensities, the
subtraction gives scalar quantities for the pixel positions that define the change and gives zero for the pixel
positions where there is no change. Changing pixels signify a moving object. Hence, frame-differencing
concept is a very simple yet powerful technique to quantify movements in a particular region of interest.
Rather than performing just frame differencing however, MEA calculates the SAD for two
consecutive video frames. SAD is nothing but a specialized version of frame differencing that allows for the
differences in two video frames to be defined using just one scalar quantity. This scalar quantity then changes
as video frames progress and the motion in the video changes. When these scalar quantity changes are plotted
against the time a particular video runs for, specific movement changes can be seen. Since this technique is
already the “velocity” in a particular video or the “energy” in a particular video, it is called motion energy.
Many algorithms have been used to detect movements in video. Some algorithms like optical flow
algorithm14 find the motion estimate or the flow estimate of objects in a video. Since the goal in this paper is
to accurately determine when the grasping movement starts and ends, the timing is more important than a
directional estimate of motion. Estimates of motion can already by tracked by DeepLabCut. Hence, a more
temporally sensitive algorithm determining when a particular change occurs in the video is necessary to
quantify the grasp duration. This motion energy concept has been used in many experiments to characterize
overt movements of animals, for example, overt movement of mice15.
Image processing algorithms can be performed on either the entire frame of the video or certain
specific portions of the frame which are of interest to us. A region of interest (ROI) in image processing
is therefore defined as a portion of an image that we want to perform some operation on, usually created by
setting a binary mask. A binary mask is essentially a binary image that is the same size as the image that
needs processing but with pixels defining the ROI set to 1 and all other pixels set to 0. For the purpose of
analysis in this paper, since we want to know exactly when the monkey starts grasping the food in the food
27
cup and ends the grasping action, the region of interest is the “cup” for all task combinations as shown in
Figure 22. Task combinations having the same cup position will have approximately the same ROI
boundaries subject to minor changes (due to shifts in the camera angle).
Figure 22: Cup ROI for the video frame
5.3 Motion Energy Plot (MEP) Change in the pixel intensities represented by the scalar quantity output of the SAD algorithm for motion
energy denotes some amount of movement in the cup ROI. Thus, the motion energy concept is applied to all
frames of each video to monitor continuous movements in the cup ROI. The motion energy plotted against
the frame number is referred to as the motion energy plot (MEP) in the following sections.
An illustration16 best describes the concept of movement detection using MEA and the MEP itself
(Figure 23).
28
Figure 23: MEA concept with Motion Energy Plot
5.4 Rationale of the task-specific algorithm
The movements in the ROI for this specific task are of the following nature:
1. Monkey enters the ROI with the intent of picking up the food
2. Monkey spends time grasping the food
3. Monkey exits the ROI with the food morsel in hand
Conceptually, large changes in the MEP are going to depict a state transition. A state transition in the
ROI can be of the following types:
1. Leading to sharp increases in the MEP
a. Transition from no movement at all to sudden onset of movement
29
b. Transition from continuous movement to sudden movement stop
2. Leading to small increases in the MEP
a. Transition from little movement to increased movement
b. Transition from increased movement to little movement
Thus, large changes in the MEP are expected when subtask 1 begins and when subtask 3 ends.
Smaller changes in the MEP depict a change from subtask 1 to 2. A large change is expected during transition
from subtask 2 to 3 since the monkey transitions from grasping (very little movement except the fingers) to
sudden onset of movement to leave the ROI.
Apart from the task movements by the monkey, there is a mechanical movement performed by the
lid of the closed food cup. The lid opens to deliver the food to the food cup. The complete sequence (Sequence
I) of actions for the entire task considering the entire video frame as the ROI is as follows:
1. Trial begins
2. Monkey holds the handle for 1 second
3. Lid of the food cup opens
4. Monkey makes the first reach from the handle to the food cup
5. Monkey grasps the food
6. Monkey makes the second movement from the food cup to the mouth
7. Trial ends
From this sequence of actions changes in pixel intensities captured in the cup ROI are represented
by Sequence II:
1. Lid of the food cup opens
2. Monkey enters the ROI with the intent of picking up the food
3. Monkey spends time grasping the food
4. Monkey exits the ROI with the food morsel in hand
Manual, visual estimates of the actions from Sequence II were marked on the MEP. These visual estimates
are shown in Figure 24.
30
Figure 24: Markers depicting parts of Sequence II
The aim of using motion energy concept is to simplify the detection of grasps and accurately
quantify the time spent by the monkey in grasping the food. The representation of grasp duration in the MEP,
is the pattern of peaks between the entry and the exit of the monkey’s hand in the ROI. The grasp thus occurs
somewhere in between the red and the yellow annotations (Figure 24). Entering of the hand in the ROI was
preceded by the cup opening event. The cup opening peaks are denoted by the purple circles in Figure 24.
The background concepts used in the previous subsections and the rationale for the algorithm form the
fundamentals of the video processing pipeline described in the next subsection.
5.5 Video Processing Pipeline
Figure 25: Video Processing Pipeline
The steps in Figure 25 are fundamental to processing each video for calculating grasp duration:
Pipeline: For each task combination in the current video:
1. Extract video frames belonging to that task combination
Extract framesApply a binary
mask for the cup ROI
Calculate the SADs of all
consecutive frames
Plot the MEPCup Opening
Detection Algorithm
Grasp Detection Algorithm
31
2. Apply the binary mask specific for that cup position to extract the ROI
3. For each ROI in each video frame among the chosen video frames:
a. Calculate the SAD of the current frame and the next frame
4. Plot the SADs against the frame numbers to generate MEP (Section 5.6 MEP Generation
describes steps 1-4)
5. Apply the “cup opening” detection algorithm (Section 5.7 Cup Opening Detection Algorithm)
6. Apply the “grasp” detection algorithm (Section 5.8 Grasp Detection Algorithm)
5.6 MEP Generation
Steps 1-4 in the video processing pipeline from Section 5.5 Video Processing Pipeline are described briefly
here.
Extraction of video frames belonging to a particular movement combination is done by manually
watching each video. The start time and total time duration is recorded to generate the MEP for all trials
performed in each task combination. In the next step, specifying the region of interest coordinates was done
using the cup positions in the video frames. Due to camera shifts, each video task was fit to the ROI
boundaries specified for that cup position, validated and modified if necessary due to camera angle shifts.
After extracting the region of interest, SADs were calculated for all consecutive frames and the
motion energy was plotted against the respective frames to get a single MEP per task combination per video.
5.7 Cup Opening Detection Algorithm
Lid opening of the food cup is the start of the sequence of overt movements in the cup ROI (as specified by
Sequence II in Section 5.4 Rationale of the task-specific algorithm). The opening of the lid is a mechanical
movement triggered by the handle hold period. Mechanical movements almost always are comprised of a
typical sequence, rarely deviating, thus producing the same changes in the pixel intensities that account for
the movements. As seen in Figure 24, the spikes that account for the cup openings are almost exactly of the
same nature-visually. They seem to have the same height and width, which is the basis for the cup opening
detection algorithm.
32
1. Smooth the MEP using a moving average window of 11 frames to reduce noise
2. Choose peaks that belong to a certain height range defined by [lowerbound, upperbound]
3. From the chosen peaks, select peaks having:
a. Same or similar peak width
b. Same or similar peak prominence
5.7.1 Moving Average
In statistics, a moving average is an unweighted mean of the previous “m” points. This algorithm uses
“m=11” points to smooth the MEP. The selection of “m” is a heuristic, that removes the noise just enough to
not let the details of the MEP completely vanish. Figure 26 shows the raw MEP and the smoothed MEP
using a rolling average of 11 frames.
Figure 26: Moving Average of the raw MEP
5.7.2 Selecting the cup opening peaks
Since lid opening is a mechanical event, the height range of the cup opening peaks is always within a certain
limit. Ideally, the cup opening peaks should be at the “same” height but instead, they are “similar”, detected
by a limit, defined by a lower-bound and an upper-bound. When the food cup lid opens, the canister shoots
the food into the cup. The color and shape of the food changes with each session. Changes in color also
account for changes in the pixel intensity thus introducing the need for a “bound” instead of a fixed height.
33
Moreover, the lower and upper bounds are different from session to session. This can be partly
attributed to the reason stated above but also to the fact that the camera shifts its position slightly with each
passing session. There may also be differences in the light intensity settings in the experiment’s room leading
to changes in the absolute MEP values.
The selection of the height range is another heuristic defined to account for the heights of the various
cup opening peaks. After the peaks were filtered by their heights, the peak prominence and widths were used
to filter the chosen peaks further. Figure 27 shows the definitions of peak prominence and width measured
at half-prominence.
Figure 27: Peak Definitions17
Peaks having the same width and prominence were chosen as the cup opening peaks. Width and
prominence of the peak both were floating point integers. Thus, floor of the peak width and ceiling of the
peak prominence converted these metrics into integers. The number of cup-opening peaks depended on the
number of trials for that particular task combination. If the number of cup-opening peaks detected using the
“same width and prominence” criteria was less than the total number of trials expected, peaks that had a
similar width and prominence range were selected. This condition was needed because flooring and ceiling
of the width and prominence values respectively may change the actual integer. For example, if the peaks
widths are 29.23, 29.45, 29.9, 29.98, 30.01, and there are a total of 5 reaches, the human brain would classify
34
these values into a particular range and conclude that they belong to the cup-opening peaks. However, the
algorithm, after taking the floor of the peak widths, with the initial criterion of “same peak width” will just
choose the first four peaks with floor of the widths equal to 29,29,29,29 and not consider the last peak even
though it is within the acceptable variance range for a width.
Thus, the second criterion is needed. The second criterion will compare the number of chosen peaks
with the number of trials and look for peaks with widths near the absolute value of 29 and thus include the
peak with width 30 to fulfill for the missing number of trials. This is how the algorithm chooses peaks with
a similar height and width.
If unnecessary peaks with the same height and width are detected as the cup-opening peaks, the
chosen peaks are further filtered using peak prominences as defined by Figure 27. A logic similar to peak
widths is applied to filter peaks with too large or too small prominences.
Thus cup-opening peaks are detected by using three sequential filtering processes according to the
metrics that define a peak. These are given in Figure 28. The peaks detected using this algorithm are
represented by green boxes as given in Figure 31.
Figure 28: Filtering process for cup-opening peak detection
5.8 Grasp Detection Algorithm
Each cup opening peak is followed by a grasp. The detection of grasp involves a more sensitive rationale
regarding MEP that is explained in this section. Sharp changes in the MEP occur when there is a change from
the current state of the ROI to a state different in nature than the current. This is when there is a transition
from:
(1) No or little movement to start of, or increase in movement
(2) Continuous movement to no movement
Filter according to height range
Filter according to peak width
Filter according to peak prominence
35
Both of these state changes lead to changes in MEP. After the cup opens, there is no movement for a
little while, until the monkey enters the ROI. When the hand enters the ROI, there is a state transition given
by (1). Thus, we can see that there is an increase in the absolute value of MEP at that timepoint, given by the
ground truth estimate in Figure 24.
After the monkey has reached the food, the continuous movement state changes to no movement state
with relatively little movement (characterized by only the finger movements while grasping). This leads to a
decrease in the absolute value of MEP as given in (2).
After grasping the food, the monkey starts exiting the ROI, leading to a state transition of nature (1).
Thus, after the grasp is complete, there is a spike in the MEP again, until the monkey exits the ROI (after
which the MEP will again drop as expected in (2). This rationale is better described with illustration given in
Figure 29.
Figure 29: Example MEP: Relation between sequence of actions and MEP features
The algorithm for detecting the grasp duration is based on the above concept.
36
Algorithm for detecting and calculating grasp duration: For each file consisting of trials for an
independent task combination:
For each cup opening peak detected by the “Cup Opening” detection algorithm:
1. Find the next peak just after the respective cup opening peak.
2. Find the minimum value on the MEP between the cup opening peak and the next peak.
3. For the curve after this detected minimum value, find the point where the threshold (T1) is crossed.
This point is the “grasp start”.
4. For the curve after the grasp start, find the point where the curve dips below the defined threshold
(T2). This is the point where the movement in the ROI ends. This is called the “feature end”.
5. Find the last minima between the grasp start and the feature end. This point is the “grasp end”.
6. Calculate the number of frames between “grasp start” and “grasp end”.
7. Divide the grasp duration (in frames) obtained in Step 6. by 100 to get the grasp duration in seconds
(frame rate of the video recording is 100Hz).
Algorithm step 2. is based on the concept of state transition described in (1). At the minima, the monkey
starts entering the ROI (increase in MEP after the green asterisk in Figure 29). However, the grasp occurs at
a later timepoint than when the monkey starts entering the ROI.
Step 3. of the algorithm thus finds the grasp start in the increasing MEP region just after the minima by
finding the point where the set threshold is crossed. This threshold (T1) defines the best heuristic estimate of
when the grasp will occur. It is observed that the grasp starts one or two frames after the monkey enters the
ROI. Thus, the heuristic is set to a scalar quantity which accounts for this delay. For example, if the minimum
point between the cup opening peak and the next peak is at frame number 345, T1 is set to be the value of
the curve at 346 or 347. To check whether the visual observation and the algorithmic interpretation sync with
each other, visual estimates were plotted along with the grasp starts detected by the algorithm. Figure 30
shows that the visual estimates of grasp start (including human reaction time) and the grasp start detected by
the algorithm are close (maximum difference of 2 frames).
37
Step 4. creates an upper bound for the detection of the grasp end. The “feature end” detected using
threshold T2, denotes the exit of the monkey’s hand from the ROI, described in (2). The grasp end has to
therefore be in between the grasp start and the upper bound defined by feature end. After the monkey has
grasped the food, state transition (1) occurs, and the MEP starts increasing again in account of increased
movement marked by the exiting hand out of the ROI. Thus, the timepoint the grasp ends will be the minima
between the grasp start (when the MEP starts dipping slowly) and before the last increase in MEP (when the
hand is exiting) before the feature end. Refer to Figure 29 for more clarity on the MEP increases and
decreases.
Although grasp start is based on a heuristic estimate (maximum error is 2 frames = 0.02 seconds), the
grasp end is not based on a threshold or a heuristic. It is based on a clearly defined minima formed by the
valley between two MEP peaks. Figure 31 shows an example plot with markers depicting the task features
detected by the algorithm.
Figure 30: MEP with algorithm-detected features and visual estimates
Step 6. of the algorithm subtracts the grasp start from the grasp end and calculates the grasp
duration in frames. Step 7. then converts the obtained grasp durations for each trial to seconds. The grasp
38
durations for all trials for all sessions are calculated and segregated into groups based on lesion timeline.
The results are pasted in Section 5.10 Results of MEA.
Figure 31: Grasp duration detectors plotted on the MEP
5.9 Algorithm Validation
The grasp duration algorithm was validated against the speed-threshold based output obtained from the state-
of-the-art DeepLabCut software. The validation was conducted by plotting the trends in the average grasp
duration across a spread lesion timeline for the trials belonging to 14 experimental sessions. Same number of
trials from the same sessions were analyzed using the method in Section 4.4 Grasp duration and using the
algorithm in Section 5.8 Grasp Detection Algorithm. Figure 32 represents this validation.
For the “Handle Far Cup Near” task combinations of 14 videos, the grasp durations obtained by both the
methods have a similar trend. Although the average durations themselves are not exactly the same (owing to
the inherent differences and the heuristics of both the algorithms), the grasp durations for each time-point are
well within a certain range. For example, the average grasp durations for April sessions are ~0.26 seconds
from both the methods. For the other months and pre-lesion sessions, the average grasp durations are within
39
a -0.1 seconds to +0.1 seconds’ range difference in the two methods. The trend is crisp and clear in both the
analyses and this suggests the following two main aspects:
1. Grasp duration prolongation is a sign of the lesion and is likely to reflect when all of the sessions
are analyzed
2. The grasp duration algorithm proposed can be used to further analyze all the sessions
Figure 32: Algorithm validation
40
5.10 Results of MEA
To visualize the trend in the grasp duration due to stroke, the trials are split into groups based on lesion
timeline. Post-stroke data is further divided into a monthly timeline to accurately quantify when a recovery
is seen. The average grasp duration for all of the trials belonging to each sub-group is calculated along with
the SEM. The task combinations are grouped according to the cup positions to verify if the trend is seen while
reaching for the food in any specific direction or if the trend persists throughout.
5.10.1 Cup Far Task Combinations
For the cup far task combinations, the movements of the monkey are “out of flexion synergy”. The monkey
has to perform an “extension” movement instead of flexion to reach to the farthest point of the virtual
diamond. Though the grasp duration increases right after stroke, there is not much difference between the
average pre-stroke grasp duration and the average grasp duration just after lesion. It can also be seen that till
July, the monkey gets faster in grasping the food than the baseline. Figure 33 illustrates this trend.
Figure 33: Cup Far tasks: Grasp duration trend
41
5.10.2 Cup Left Task Combinations
For the cup left task combinations, the movements of the monkey mostly require flexion. The monkey has to
perform a definite “flexion” movement for the Handle Far Cup Left and the Handle Right Cup Left task
combinations. There is a large difference between the average pre-stroke grasp duration and the average grasp
duration just after lesion. It can also be seen that till July, the monkey gets faster in grasping the food than
the baseline. Figure 34 illustrates this trend.
Figure 34: Cup Left tasks: Grasp duration trend
5.10.3 Cup Near Task Combinations
For the cup near task combinations, the movements of the monkey fully consist of “flexion synergy”. The
monkey has to perform a flexion to reach to the nearest point of the virtual diamond. There is a large
difference between the average pre-stroke grasp duration and the average grasp duration just after lesion.
Recovery can be seen in the grasping duration post-lesion although it is hard to conclude whether the
durations drop below the baseline. July trials have slightly increased average grasp duration due to
experimental impediments elaborated in Appendix Figure 19. The increase in the average grasp duration in
the July trials does not have to do anything with the lesion effects. Figure 35 illustrates these observations.
42
Figure 35: Cup Near tasks: Grasp duration trend
5.10.4 Cup Right Task Combinations
For the cup right task combinations, the monkey has to mostly perform an extension except in the Handle
Left Cup Right task. Though the grasp duration increases right after stroke, there is not much difference
between the average pre-stroke grasp duration and the average grasp duration just after lesion. Recovery can
be seen in the grasping duration post-lesion and the durations drop below the baseline. July trials have slightly
increased grasp duration due to experimental impediments elaborated in Appendix Figure 20. The increase
in the average grasp duration in the July trials does not have to do anything with the lesion effects. Figure 36
illustrates these observations.
The average grasp durations for independent task combinations are illustrated in Appendix:Motion
Energy Figures. Chapter 6. Discussion discusses a few key points that state the relation of the trend, synergies
and effect of lesion.
43
Figure 36: Cup Right tasks: Grasp duration trend
44
Chapter 6. Discussion
6.1 Conclusion
The result figures for the grasp duration trend are summed up in this section (Figure 37).
Figure 37: Comparison of all grasping duration trends
Except the trials for the “Cup Near” task combination (Figure 37-Subfigure D)), the grasp duration
post-stroke falls below the baseline grasp duration (exclude the trials in July that have experimental
impediments). All of the “cup near” task combinations require flexion movements that are synergistic. Post-
stroke grasp durations recover but do not fall below the baseline for the “cup near” tasks, suggesting that
flexion synergies are driving some part of the inability to grasp quickly in this particular case.
The rest of the cup position tasks (Figure 37-Subfigures A), B) and C)) mostly do not require flexion
movements, although there are a few exceptions like the “Handle Far Cup Left” task combination where the
monkey needs to flex its hand to reach the cup. For these non-cup-near task combinations, on an average, the
monkey is able to recover from the prolonged grasp durations post-lesion and be better than even the baseline
grasp periods (exclude the trials in July that have experimental impediments). A more detailed analysis
45
however can be made by evaluating the independent plots for each of the different task combinations in
Appendix: Motion Energy Figures.
For the task combinations that require purely extension or non-flexion movements, for example, all
of the Cup Far and Cup Right movements (Figure 37-Subfigures A) and C) respectively), it can be seen that
the grasp durations are not too affected due to the lesion when compared to the grasp durations pre-lesion
(~0.03-0.05 seconds of change in the grasp duration from pre-stroke to March end).
Whereas, for the task combinations that require flexion movements, for example the Cup Left and
the Cup Near task positions (Figure 37-Subfigures B) and D) respectively), the grasp durations are very
prolonged as compared to the durations for pre-lesion trials (~0.13-0.22 seconds of change in the grasp
duration from pre-stroke to March end). This suggests that flexion synergies might have a role in the
development of a larger impairment while grasping objects.
6.2 Summary and Future Work
While it is clear that the lesion in the anterior old M1 does affect several aspects of reaching such as:
1. Slower peak speeds for both out of synergy and flexion movements immediately post-lesion
2. Longer grasp duration for all reach directions immediately post-lesion
as compared to the pre-lesion sessions, further investigation into the correlation of synergies and the
impairments needs to be done. In the current scenario, factors such as the “pre-shaping” of the hand, and
optimal practice level for the reaching task are not defined. Entangling the post-lesion practice effect from
the recovery also needs to be done in order to understand the recovery time for impaired activities such as
the grasp.
Furthermore, post-lesion changes in the EMG according to a monthly timeline might help associate
the trends seen in the EMG activity for certain muscles that take part in the flexion synergy and the impaired
movements.
46
Appendix
Implanted Muscles
The list of names of the implanted muscles for the experiment in this paper along with their anatomical full-
forms and numbers is given in Appendix Table 1 below. Muscle numbers are significant and should be
referred to where muscle names are not used for reference in the figures.
Muscle No. Implanted Muscles
(Names used in this
paper)
Implanted Muscles
(Full forms)
Muscle 1 BR Brachioradialis
Muscle 2 Biceps Biceps
Muscle 3 Brachialis Brachialis
Muscle 4 Supraspin Supraspinatus
Muscle 5 PM Pectoralis Major
Muscle 6 EDC Extensor Digitorum Communis
Muscle 7 ECR Extensor Carpi Radialis
Muscle 8 Triceps Triceps
Muscle 9 PD Post Deltoid
Muscle 10 FDS Flexor Digitorum Superficialis
Muscle 11 FCR Flexor Carpi Radialis
Muscle 12 AD
(Not considered)
Anterior Deltoid
Appendix Table 1: Implanted muscle numbers with their names
EMG Analysis
Amplitude Filtering Step
This section extends Section 3.1.2 Amplitude Based Filtering from the paper to describe in detail why
amplitude-based filtering was necessary for processing the raw EMG data. The raw input EMG data consisted
of very high amplitude, low frequency noise in some of the muscle channels for a few sessions. This noise
was observed to exceed 15mV amplitude in the negative and the positive directions. The general amplitude
range of EMG signal is 0-10 mV (+5 to -5 mV) prior to amplification7.
Out of the various sources of electrical noise, “ambient noise” cannot be virtually avoided while
recording EMG as electromagnetic radiation is the source for ambient noise. Such a kind of noise can have
amplitude that is one to three orders of magnitude greater than normal EMG signal. The observed noise fits
47
into this category as its amplitude magnitude goes beyond the (-15mV to 15mV) range which is almost three
orders of magnitude greater than the normal EMG data observed. Hence, this step focuses on removing the
ambient noise with an amplitude cutoff range of [-15mV, 15mV]. Since the trials containing large amplitudes
surpassing 15mV, do not have any good EMG data, these trials are completely removed from consideration
in this analysis.
The raw data containing the unreal noise (high amplitude, low frequency) is shown in an example
pre-stroke session in Appendix Figure 1.
Appendix Figure 1: Raw EMG data with high amplitude noise
Muscle numbers 6 to 11 in Appendix Figure 1 contain the high amplitude low frequency noise. This
raw EMG data was filtered by removing EMG traces that consisted of “very high amplitudes” greater than
or equal to 15 mV or less than or equal to -15 mV. Appendix Figure 2 shows the raw EMG data after this
amplitude filter was applied. The remaining data after amplitude-based filtering resembled normal raw EMG
patterns. Normal signal processing steps (Figure 8) were then used for processing muscle electrophysiology
data.
48
Appendix Figure 2: EMG data after amplitude-based filtering
Fast Fourier Transform Analysis
Apart from “ambient noise”, inherent electronic equipment noise and motion artifacts also play a key role in
affecting the EMG signal. The equipment noise or also called power-line interference frequency deviates
within the range of 49.5–50.5 Hz for 99.5% of the time in Europe18. The type of noise removed by the
amplitude filtering step was also an ambient power-line noise with a frequency in the range given by the
above reference.
This is analyzed by constructing a single sided amplitude spectrum (refer to Appendix Figure 3) of
one of the muscles in the experimental session. As seen, the line noise removed by amplitude filtering in the
first step has a frequency of 49.9933 Hz and its harmonics.
However, this line noise is different in that, it is a stand-alone signal, not interfering with good EMG
data. Moreover, this type of signal is sparsely seen throughout the course of the experiment, with only about
1% of the traces containing only the line noise along with EMG. Hence, amplitude filtering is justified for
the removal of this noise. However, power-line noise and low frequency motion artifacts interfering with the
good EMG signal need to be handled differently since entire traces cannot be removed in that case. A standard
49
digital high pass filter of 70 Hz was applied to all EMG traces to remove line noise coupled electrically or
magnetically with the equipment. The frequencies of such noise can be 50 or 60 Hz and their harmonics,
depending on the source.
To eliminate any chance of including such line noise interferences, a lower cutoff frequency of 70
Hz was used for the designing the digital high pass filter. Appendix Figure 4 and Appendix Figure 5 show
the effect of applying the high pass filtering step.
Appendix Figure 3: Single-Sided Amplitude Spectrum of raw EMG signal
Appendix Figure 4 (specifically muscles 8 and 11) depict the EMG traces for a session that has
passed through the amplitude-based filtering step but still has low frequency line noise and motion artifacts.
Appendix Figure 5 is the output of applying a high pass filter and shows that the low frequency artifacts in
muscles 8 and 11 are not present.
50
Appendix Figure 4: Raw EMG with artifacts (amplitude filtered)
Appendix Figure 5: Result of applying high pass filter
Artifact at time 0
After applying the signal processing steps mentioned in Figure 8, average EMG traces were plotted against
movement time by distinguishing between pre- and post-stroke data, Table on and Table off data. A few of
the muscles, however, showed signs of an unnatural and sudden increase and then decrease in the average
EMG amplitude at time-point 0. This sudden increase, referred to as the “blip” in this section, is a time-
specific artifact mostly arising due to the nature of the experiment conducted.
51
The fact that the blip was significantly seen even in the average traces for all the trials in the 4-
month period, suggested that the trend was seen in most of the trials for these muscles. To quantify the
prevalence of the artifact, median traces were plotted along with the mean (Appendix Figure 6 and Appendix
Figure 7). Since both median and mean representation of the EMG traces showed a similar trend at time 0,
it was concluded that most of the data had this artifact for the given muscles. For the muscles with the blip,
data in blip’s time duration was not analyzed and was interpolated only for cosmetic purposes.
The blip can be attributed to the following explanation: At timepoint 0 in every trial, the monkey
lets go of the handle and makes the first reach from the handle to the food cup. All trials being continuous in
nature, wet residues of food on the monkey’s hand from the previous trial are sometimes carried forward to
the handle hold event of the next trial. Due to some electromagnetic interference of the monkey’s wet hand
and contact sensor on the handle, the artifact is recorded.
Appendix Figure 6: Biceps: Prevalence of the blip (average and median) across all trials
The duration of the blip is different for different muscles. The blip is not present for 4 muscles out
of 11 and for each of the remaining 7 muscles where the artifact exists, the blip region is defined specifically
and differently. A maximum blip time duration for all trials belonging to that muscle was calculated. These
times are reported in Appendix Table 2. Overall, the blip exists in the time range [30 msec before time 0 and
52
50 msec after time 0] i.e., 0.03 seconds before the monkey lets go of the handle (consistent with the notion
of making a movement) and 0.05 seconds after the monkey lets go of the handle.
Appendix Figure 7: Triceps: Prevalence of the blip (average and median) across all trials
Muscle Start of the Artifact relative to 0 End of the Artifact relative to 0
BR No artifact, just an increase No artifact, just an increase
Biceps 30 msec before 50 msec after
Brachialis 20 msec before 50 msec after
Supraspinatus No artifact, just an increase No artifact, just an increase
PM 20 msec before 50 msec after
EDC 20 msec before 50 msec after
ECR No artifact, just an increase No artifact, just an increase
Triceps 30 msec before 50 msec after
PD No artifact, just an increase No artifact, just an increase
FDS 20 msec before 40 msec after
FCR 20 msec before 50 msec after
Appendix Table 2: Blip: Muscles and time durations
53
Interpolation Technique
The average EMG traces for certain muscles as mentioned in Section 3.2 Results contain the artifact described
in Section Artifact at time 0. For comparison and cosmetic purposes, the region with the artifact is interpolated
for muscles given in Appendix Table 2. Cubic spline interpolation technique was applied in the noted time
durations using MATLAB R2019a to result in a smoother plot with smaller error19. The average EMG traces
before and after interpolation for these muscles are reported in Appendix Figure 8.
54
Appendix Figure 8: Interpolation results for muscles
55
Average EMG traces
The figures for the average EMG traces for the considered muscles in section 3.2 Results are reported in this
section.
Appendix Figure 9: BR: Average EMG traces
Appendix Figure 10: ECR: Average EMG traces
56
Appendix Figure 11: PD: Average EMG traces
Speeds
This section reports the average speed plots for the first reach for a few “Cup Near” task combinations
(extension of Section 4.3 Speeds).
Appendix Figure 12:Speed plots for the reach (Handle Left Cup Near)
57
Appendix Figure 13: Speed plots for the reach (Handle Right Cup Near)
Some examples where the tracking quality of DeepLabCut results in erroneous and haphazard speed plots
are also pasted (reference in Section 4.6 Limitations).
Appendix Figure 14: Speed plot outliers
58
Appendix Figure 15: Speed plot outliers
Path length and trajectory analysis
This trajectory analysis was conducted by labelling the index finger of the monkey rather than the wrist. The
position estimates of the index finger are then plotted as trajectories. The index finger is a better body part to
label in DeepLabCut since it is seen throughout the horizontal reaches and has a lesser potential of changing
subjectively.
Appendix Figure 16: Trajectories for the index finger
59
Motion Energy Figures
Pre-stroke trial bar plot has a very high variability in the averages and very high SEM due to high variability
in pre-stroke grasping behavior and lesser willingness to persistently perform the task.
Appendix Figure 17: Grasp duration trend: Cup Far
60
Appendix Figure 18: Grasp duration trend: Cup Left
61
Appendix Figure 19: Grasp duration trend: Cup Near
62
Appendix Figure 20: Grasp duration trend: Cup Right
63
Reference List
1. Society for Neuroscience, 2019, “Investigating Flexion Synergies post-CST Lesion in a Non-
Human Primate”
2. Rathelot, Jean-Alban, and Peter L. Strick. "Subdivisions of primary motor cortex based on cortico-
motoneuronal cells." Proceedings of the National Academy of Sciences 106, no. 3 (2009): 918-
923.
3. Isa, Tadashi, Masaharu Kinoshita, and Yukio Nishimura. "Role of direct vs. indirect pathways
from the motor cortex to spinal motoneurons in the control of hand dexterity." Frontiers in
neurology 4 (2013): 191.
4. Coscia, Martina, Vincent CK Cheung, Peppino Tropea, Alexander Koenig, Vito Monaco,
Caoimhe Bennis, Silvestro Micera, and Paolo Bonato. "The effect of arm weight support on upper
limb muscle synergies during reaching movements." Journal of neuroengineering and
rehabilitation 11, no. 1 (2014): 1-15.
5. Herbert, Wendy J., Kimerly Powell, and John A. Buford. "Evidence for a role of the reticulospinal
system in recovery of skilled reaching after cortical stroke: initial results from a model of ischemic
cortical injury." Experimental brain research 233, no. 11 (2015): 3231-3251.
6. Rose, William. "KAAP686 Mathematics and Signal Processing for Biomechanics
Electromyogram analysis." (2011).
7. Reaz, Mamun Bin Ibne, M. Sazzad Hussain, and Faisal Mohd-Yasin. "Techniques of EMG signal
analysis: detection, processing, classification and applications." Biological procedures online 8,
no. 1 (2006): 11-35.
8. “EMG Processing with Basic Functions of the EMG Amplitude Estimation Toolbox.” Accessed
May 6, 2021.
https://users.wpi.edu/~ted/emg0_08/basic.html#:~:text=The%20six%20stages%20are%20(1,provi
des%20processing%20for%20each%20stage.
64
9. Mathis, Alexander, Pranav Mamidanna, Kevin M. Cury, Taiga Abe, Venkatesh N. Murthy,
Mackenzie Weygandt Mathis, and Matthias Bethge. "DeepLabCut: markerless pose estimation of
user-defined body parts with deep learning." Nature neuroscience 21, no. 9 (2018): 1281-1289.
10. Wikipedia contributors, "Sum of absolute differences," Wikipedia, The Free
Encyclopedia, https://en.wikipedia.org/w/index.php?title=Sum_of_absolute_differences&oldid=10
21802130 (accessed May 6, 2021).
11. Wikipedia contributors, "Taxicab geometry," Wikipedia, The Free
Encyclopedia, https://en.wikipedia.org/w/index.php?title=Taxicab_geometry&oldid=1007398321
(accessed May 6, 2021).
12. Song, Yang, and Ali Akoglu. "Bit-by-Bit pipelined and hybrid-grained 2D architecture for motion
estimation of H. 264/AVC." Journal of Signal Processing Systems 68, no. 1 (2012): 49-62.
13. Ramseyer, Fabian T. "Motion energy analysis (MEA): A primer on the assessment of motion from
video." Journal of counseling psychology 67, no. 4 (2020): 536.
14. Gibson, James J. "The perception of the visual world." (1950).
15. Steinmetz, Nicholas A., Peter Zatka-Haas, Matteo Carandini, and Kenneth D. Harris. "Distributed
coding of choice, action and engagement across the mouse brain." Nature 576, no. 7786 (2019):
266-273.
16. Fujiwara, Ken, and Ikuo Daibo. "Affect as an antecedent of synchrony: A spectrum analysis with
wavelet transform." Quarterly Journal of Experimental Psychology 71, no. 12 (2018): 2520-2530.
17. MATLAB 2019a, The MathWorks, Inc., Natick, Massachusetts, United States.
18. Roland, Theresa, Sebastian Amsuess, Michael F. Russold, and Werner Baumgartner. "Ultra-low-
power digital filtering for insulated EMG sensing." Sensors 19, no. 4 (2019): 959.
19. Wikiversity contributors, "Cubic Spline
Interpolation," Wikiversity, https://en.wikiversity.org/w/index.php?title=Cubic_Spline_Interpolatio
n&oldid=2270598 (accessed March 25, 2021).