closing the sensorimotor loop: haptic feedback facilitates decoding of arm movement imagery
DESCRIPTION
1 MPI for Biological Cybernetics 2 Stanford University 3 Brain-Computer Interface Laboratory, Wadsworth Center 3 Werner Reichardt Centre for Integrative Neuroscience Eberhard Karls University Tuebingen. Closing the Sensorimotor Loop: Haptic Feedback Facilitates Decoding of Arm Movement Imagery. - PowerPoint PPT PresentationTRANSCRIPT
1
1MPI for Biological Cybernetics 2Stanford University 3Brain-Computer Interface Laboratory, Wadsworth Center
3Werner Reichardt Centre for Integrative NeuroscienceEberhard Karls University Tuebingen
Closing the Sensorimotor Loop: Haptic Feedback Facilitates Decoding of Arm
Movement Imagery M. Gomez-Rodriguez1,2 J. Peters1 J. Hill3
B. Schölkopf1 A. Gharabaghi4 M. Grosse-Wentrup1
SMC Workshop in Shared-Control for BMI, October 2010
2
BCI + robot-assisted therapy
Brain Computer Interface (BCI) + robot-assisted physical therapy for neurorehabilitation of:
Hemiparetic syndromes due to brain damage
may outperform traditional therapy
Traditional rehabilitation
Sensorimotor loop is broken
They do not help for severe motor impairment
We close the sensorimotor loop
Synchronize subject’s attempt and robot arm
Our approach to rehabilitation
3
Stand-alone BCI and robot-assisted therapy
It has been shown that
Robot-assisted physical therapy
Motor imagery
are beneficial for rehabilitation as stand-alone therapies [2,3]but loop is still broken!
Next logical step is to combine both in an integrated rehabilitation to close the loop
4
Hebbian plasticity: Why closing the loop?
Closing artificially the sensorimotor loop islikely to result in increased cortical plasticity
because we induce Hebbian plasticity.
Instantaneous feedback: Delays in the order of ms.
Requirements
High accuracy: On-line decoding of arm movement intention.High specificity: Focus on motor and sensorimotor cortex.
Hebbian plasticity [1]“A positive feedback-mediated plasticity in which synapses between presynaptic and postsynaptic neurons that are coincidently active are strengthened.”
5
BCI decoding: Effect of closing the loop
Our work builds on analyzing the effect of artificially closing the sensorimotor loop on BCI-decoding.
Previous studies: Passive and active movements induce patterns in the brain similar to those induced by motor imagery [2, 3].
Random haptic feedback has been shown to be beneficial for BCI-decoding [4].
In our work:We show how haptic feedback (closing the sensorimotor loop) influences BCI-decoding.
Combining BCI and robot-assisted physical therapy opens many research questions.
6
Outline
1. Experimental Design:Human subjects, recording and task & feedback conditions.
2. Methods:Signal processing, on-line decoding and conditions comparison.
3. Results:Analysis of the haptic feedback effect on decoding performance and spatial/frequency features.
4. Conclusions
7
Subjects and recordings
Human subjects: 6 right-handed healthy subjects between 22 and 32 years old.
Recording: 35 EEG channels250 Hz sampling rateQuickamp with built-in CARBCI2000 + BCPy2000
Pre-motor, primary motor and somatosensory cortex are covered
8
Task and haptic feedback conditions
Subject’s task: Think about moving the right arm forward (extension) or backward (flexion) in the same way the robot does.
Condition TrainingTest
+
+
+
+Condition I
Condition II
Condition III
Condition IV
9
Haptic feedback conditions
Training(25s per condition)
+
Test(consecutively after training)
Arrow in a screen + robot moves-stops
according to classifier while motor imagery
Robot moves while motor imagery
Arrow in a screen moves-stops
according to classifier while motor imagery
X
Trial durationRest: 3sMI: 5s
Rest: 3sMI: min(5s, robot hits border)
10
Outline
1. Experimental Design:Human subjects, recording and task & feedback conditions.
2. Methods:Signal processing, on-line decoding and conditions comparison.
3. Results:Analysis of the haptic feedback effect on decoding performance and spatial/frequency features.
4. Conclusions
11
Signal Processing
Power spectral densities over 2Hz frequency bins for each electrode are used as features.
Welch’s method over overlapping incrementally bigger time segments each 5-s movement or 3-s resting periods.
Surface Laplacian FilterPreprocessing
Band-pass filtering (2-115Hz)Notch filtering (50 Hz)
Features Computation
Larger segments → Less noise and more reliable estimates.Shorter segments → Necessary for on-line feedback.
12
On-line Decoding
During the test periods, on-line classification between movement and resting using spectral features:
Every 300 ms,• One classifier output.• Visual on-line feedback and depending on the condition also
haptic feedback is updated.
A linear support vector machine (SVM) is generated each run on-line after the training period and its outputs are mapped to probabilistic outputs by fitting a sigmoid.
+
13
Conditions comparison
To discover how haptic feedback influences the BCI
1. Two-way analysis of variance (ANOVA) over probabilistic outputs in each condition.
2. Average accuracy per condition.
3. Area under the receiving operating characteristic (AUC) per condition
We expect all three to support the same conclusions to strengthen the empirical evidence.
we compare the BCI performance for each condition of haptic feedback by computing:
14
ANOVA and AUC
1. For each condition, we group M probabilistic outputs from all subjects.
2. Compute ANOVA at significant level α = 0.05 with Bonferroni multiple-comparison correction.
3. ANOVA tell us if we can reject the hypothesis that the probabilistic outputs means are equal between conditions.
1. For each subject and condition, we have N probabilistic outputs.
2. We sweep over different thresholds in (0, 1) to classify mov/rest and compute the accuracy for each.
3. The area under the curve threshold versus accuracy is our AUC.
ANOVA AUC
15
Outline
1. Experimental Design:Human subjects, recording and task & feedback conditions.
2. Methods:Signal processing, on-line decoding and conditions comparison.
3. Results:Analysis of the haptic feedback effect on decoding performance and spatial/frequency features.
4. Conclusions
16
ANOVA
ANOVA confidence intervals
Average probabilistic on-line (every 300ms) output
++
Training Test
Cond
ition
s +
+
Condition I outperforms the rest, very clearly condition IV!
17
Average accuracy
Conditions
++
Trai
ning
Test
+
+
1. In group average, Condition I outperforms the rest.
2. Condition I outperforms Conditions III and IV for all subjects, and it outperforms II for all subjects except two.
Aver
age
accu
racy
The results are coherent with ANOVA!
18
AUC
Conditions
++
Trai
ning
Test
+
+
1. In group average, Condition I outperforms the rest.
2. Condition I outperforms the rest for all subjects.
AUC
The results are coherent with ANOVA and average accuracy!
19
Spatial and spectral features
Average classifiers weights for each electrode over the frequency band (2 – 40 Hz)
Condition I and II(Robot moves during training)
Condition III and IV(Robot does not move during training)
When the robot moves, we have higher weights in the motor/somatosensory area
20
Outline
1. Experimental Design:Human subjects, recording and task & feedback conditions.
2. Methods:Signal processing, on-line decoding and conditions comparison.
3. Results:Analysis of the haptic feedback effect on decoding performance and spatial/frequency features.
4. Conclusions
21
Conclusions
Artificially closing the sensorimotor feedback loop facilitates decoding of movement intention in healthy subjects.
Our results indicate the feasibility of future integrated rehabilitation therapy that combines robot-assisted physical therapy with decoding of movement intention by a BCI.
We assume that the results presented here with healthy subjects can be transferred to stroke patients.
We speculate that haptic feedback support subjects in initiating a voluntary modulation of their SMR.
In a shared-control scenario in BMIs, we may improve performance by means of haptic feedback.