BCI Research at the Wadsworth BCI Research at the Wadsworth

Center: Developments in Noninvasive Center: Developments in Noninvasive

Communication and ControlCommunication and ControlDean J. Dean J. KrusienskiKrusienski

Assistant ProfessorAssistant Professor

Electrical EngineeringElectrical Engineering

University of North FloridaUniversity of North Florida

The Wadsworth BCI LabThe Wadsworth BCI Lab

�� Located in Albany, NYLocated in Albany, NY

�� Part of the New York State Dept. of HealthPart of the New York State Dept. of Health

�� Lab Chief: Jonathan Lab Chief: Jonathan WolpawWolpaw

�� ~20 Research Scientists, ~20 Research Scientists, PostdocsPostdocs, , Technicians, & StudentsTechnicians, & Students

�� Mission: To research and develop BCI Mission: To research and develop BCI technology for helping severely disabled technology for helping severely disabled individuals communicate and interact with individuals communicate and interact with their environments.their environments.

Wadsworth BCI MilestonesWadsworth BCI Milestones

�� 1986: Preliminary BCI studies supported by IBM1986: Preliminary BCI studies supported by IBM

�� 1991: First demonstration of EEG1991: First demonstration of EEG--based 1d cursor based 1d cursor control in humanscontrol in humans

�� 1994: First demonstration of EEG1994: First demonstration of EEG--based 2d cursor based 2d cursor control in humanscontrol in humans

�� 2004: First demonstration of an 2004: First demonstration of an ECoGECoG--based BCI in based BCI in humanshumans

�� 2004: Demonstration of refined 2d cursor control in 2004: Demonstration of refined 2d cursor control in humanshumans

�� 2006: Home System 2006: Home System –– first demonstration of longfirst demonstration of long--term, term, everyday BCI use by a disabled individual in his homeeveryday BCI use by a disabled individual in his home

�� 2009: First demonstration of EEG2009: First demonstration of EEG--based 3d cursor based 3d cursor control in humans (to appear)control in humans (to appear)

�� Present: LargePresent: Large--scale BCI testing and dissemination to scale BCI testing and dissemination to disabled populationsdisabled populations

WadsworthWadsworth’’s Approachs Approach

�� SensorimotorSensorimotor Rhythms (SMR)Rhythms (SMR)

–– Users can be trained to accurately modulate Users can be trained to accurately modulate

amplitudes of rhythms for realamplitudes of rhythms for real--time, continuous time, continuous

dimensional controldimensional control

–– Applications: continuous cursor control, navigation, Applications: continuous cursor control, navigation, robotic/prosthesis controlrobotic/prosthesis control

�� P300 Evoked PotentialsP300 Evoked Potentials

–– Special interface designed to exploit discrete responses Special interface designed to exploit discrete responses

for symbol selectionfor symbol selection

–– Innate, no training requiredInnate, no training required

–– Low SNR: Typically require averaging of multiple Low SNR: Typically require averaging of multiple responsesresponses

–– Applications: discrete symbol selection for word Applications: discrete symbol selection for word processing or environmental controlprocessing or environmental control

P300 Matrix SpellerP300 Matrix Speller

P300 Evoked PotentialP300 Evoked Potential

P300 ProcessingP300 Processing

Krusienski et al, J Neuro Meth 2008

100

80

60

40

20

Mean Accuracy

# Flashes

Offline Online

1515 105105

P300 Processing

� 800ms Responses

� Smooth and Decimate to 20Hz

� Stepwise Linear Discriminant Analysis

P300 Home SystemP300 Home System

Scott Mackler MD, PhD

University of Pennsylvania

School of Medicine

SMR: 2D Cursor ControlSMR: 2D Cursor Control

SensorimotorSensorimotor RhythmsRhythms

SMR ProcessingSMR Processing

SMR Processing

� Large LaplacianSpatial Filter

� 16th Order Autoregressive Spectrum Estimation

� LMS Adaptation of Feature Weights

++=∆ ∑∑ bAwAwK

RL f

f

R

f

R

f

f

L

f

Lxy

Mouse Emulation

McFarland et al, J Neural Eng 2008

Robotic ArmRobotic Arm

Present and FuturePresent and Future

�� Refine P300 & SMR ControlRefine P300 & SMR Control

�� Home SystemHome System–– Practical Considerations: Software & Practical Considerations: Software & MaintenanceMaintenance

�� 33--D/ND/N--D EEGD EEG--Based ControlBased Control

�� ECoGECoG

–– AttentionAttention

–– SpeechSpeech

–– Motor ActivityMotor Activity

AcknowledgmentsAcknowledgments

�� Wadsworth CenterWadsworth Center–– Jonathan Jonathan WolpawWolpaw

–– Dennis McFarlandDennis McFarland

–– Theresa VaughanTheresa Vaughan

–– GerwinGerwin SchalkSchalk

–– Eric Sellers (ETSU)Eric Sellers (ETSU)

–– Lynn Lynn McCaneMcCane

–– Bill Bill SarnackiSarnacki

–– Peter BrunnerPeter Brunner

–– Chad Chad BoulayBoulay

–– Steve Steve CarmackCarmack

–– Brandon Brandon LapaloLapalo

–– AsyeAsye GunduzGunduz

–– Adam WilsonAdam Wilson

–– XoamaiXoamai PeiPei

–– Joseph MacJoseph Mac

�� Collaborators (Past & Present)Collaborators (Past & Present)

–– NeilsNeils BirbaumerBirbaumer ((TuebingenTuebingen))

–– Andrea Andrea KueblerKuebler ((WurzburgWurzburg))

–– Melody MooreMelody Moore--Jackson (Jackson (GATechGATech))

–– GertGert PfurtschellerPfurtscheller, Christa , Christa NeuperNeuper, & , &

Brendan Allison (Graz)Brendan Allison (Graz)

–– George Townsend (Algoma)George Townsend (Algoma)

–– Terry Terry HeimanHeiman--Patterson (Drexel)Patterson (Drexel)

–– Debra Debra ZeitlindZeitlind (Helen Hayes)(Helen Hayes)

–– Dan Moran & Eric Dan Moran & Eric LeuthardtLeuthardt (St. Louis)(St. Louis)

–– Justin Williams (Wisconsin)Justin Williams (Wisconsin)

–– Kai Miller & Jeff Kai Miller & Jeff OjemannOjemann (Washington)(Washington)

–– ChristophChristoph GugerGuger ((gTECgTEC) )

�� SupportSupport

–– NCMRR, NICHD, NIH (HD30146)NCMRR, NICHD, NIH (HD30146)

–– NIBIB and NINDS, NIH (EB00856)NIBIB and NINDS, NIH (EB00856)

–– ALS Hope FoundationALS Hope Foundation

–– James S. McDonnell FoundationJames S. McDonnell Foundation

–– ALTRAN FoundationALTRAN Foundation

–– NSF (0905468)NSF (0905468)