p09027: upper extremity motion capture system -...
TRANSCRIPT
1
Abstract— This paper discusses the development and
implementation of a wireless motion capture system for
the human arm. The system is comprised of a mechanical
brace with integrated rotary potentiometers. It also
contains surface electrodes which acquire
electromyographic signals through the CleveMed
BioRadio device. The brace interfaces with a software
subsystem which displays real time data signals. The
software includes a 3D arm model which imitates the
actual movement of a subject’s arm when under testing.
Index Terms — BioRadio, graphical user interface (GUI), Qt,
surface electromyography (sEMG)
I. INTRODUCTION
The goal of this project is to design a wireless
biomechanical system which will be used for rehabilitation
robotic research projects in the Biomechatronic Learning
Laboratory (BLL) at the Rochester Institute of Technology.
The system is being developed to study motion as it relates to
the human robot interaction issues in rehabilitation.
The specifications of the device were identified by the
customer for the project, Dr. Edward Brown, the director of
the BLL. Five main requirements were identified by him in the
initial stages of the design of the project. These were:
• Four muscle groups required for measuring EMG
activity
• Velocity measurement of limb movements
• Elbow angle, shoulder angle and angle between
arm and body to be measurable
• Device should have 3 degrees of freedom
• Visual display of real time data with graphs
The entire system is comprised of three functional blocks:
mechanical, electrical, and software. The brace subassembly
and platform subassembly form the mechanical block of the
system. The brace is designed such that the electrical
subsystem is integrated with it. The motion sensors identified
by the team during the concept selection phase were rotary
potentiometers which are integrated at the shoulder and elbow
joints of the brace. Electric wires are run through the brace in
an aesthetic manner which, connect to a printed circuit board
(PCB). The CleveMed BioRadio device for measuring surface
electromyographic (sEMG) signals connects to surface
electrodes which are routed in a similar fashion as those
associated with the rotary potentiometers and the PCB. A
casing was selected and is placed on the platform subassembly
at a specific location. The BioRadio is clipped on one side of
the casing while the PCB and batteries are contained inside it.
The software subsystem addresses the customer’s
requirement for a visual display. Data is acquired from the
BioRadio through a network interface and real time data is
then displayed on a user friendly GUI.
The complete system with a test subject strapped in is
displayed below:
Fig. 1: Complete System Depiction
II. ELECTROMYOGRAPHY AND CONCEPT SELECTION
A. Electromyography (EMG)
The evaluation and recording of muscle activity is
accomplished through a technique known as
electromyography (EMG). In order to quantify EMG, it is
imperative to understand the concept of a motor unit and the
associated impulse or action potential generated when the
motor unit fires.
A motor unit is composed of a motor neuron and the muscle
fibers it further stimulates. Once the motor unit fires, the
action potential is transmitted across the neuromuscular
junction upon which an action potential is generated in each of
the muscle fibers connected with the motor unit. A motor unit
action potential (MUAP) is the sum of this electrical activity.
Fig. 2 below depicts a motor unit. EMG enables the study of
electrophysiological activity from a number of motor units
collectively.
P09027: Upper Extremity Motion Capture
System
Melissa Gilbert, J.J. Guerrette, Dan Chapman, Alan Smith, Adey Gebregiorgis, Pooja Nanda
2
Figure 2: Visual Illustration of Motor Unit and Motor
Neuron
Source: www.abcbodybuilding.com
Two different types of EMG are commonly used:
intramuscular and surface. Intramuscular EMG is invasive and
for this particular system, surface EMG was more suitable.
Surface electrodes provide a general picture of muscle
activation which is more useful for physiotherapy.
A number of different muscle groups [2] were researched to
understand which would be the most important for us to target
in accordance with the goals and objectives of the project.
Upper arm movement is governed by the deltoid while the
primary muscles for extension and flexion of the lower arm
are the triceps and biceps. Lastly, the brachioradialis was
chosen to be monitored as it assists the biceps and triceps
muscles for movement of the forearm. A visual illustration of
these four muscle groups is provided in Fig. 3 below.
The left upper corner figure depicts the biceps brachii while
the right upper corner figure depicts the triceps brachii. The
left lower corner shows the deltoid muscles and the right
lower corner is the brachioradialis.
Since another customer requirement was the capability of
velocity and time measurements of different limb movements,
we had to consider a number of motion sensors and choose the
most appropriate one.
Figure 3: Visual Illustration of the biceps brachii,
triceps brachii, brachioradialis and deltoid muscles in
clockwise order
B. Motion Sensors
We looked at several different motion sensors such as
accelerometers, gyroscopes, optical encoders, potentiometers.
A concept selection matrix was developed to scope the
advantages and disadvantages of each. The selection criteria
we used were:
• Signal Processing
• Mechanical Integration
• Power Requirement
• Position Accuracy
• System Overhead
• Cost
Each criterion had a different weight of importance
associated with it. Considering our specific goals, signal
processing, mechanical integration and position accuracy were
our most important criteria. We rated each motion sensor
based on information we researched and the sensor with the
highest score was chosen. As per our selection matrix,
potentiometers ranked the highest.
Once we narrowed down our motion sensor to
potentiometers, the next step was to decide which type of a
potentiometer would be used. For this, we ordered samples of
different types of potentiometers and performed some
engineering analysis. In essence we wanted to understand the
error in measurement from each potentiometer once integrated
with the brace. From this analysis, we came to the conclusion
that rotary potentiometers would be the best way to go for our
project. The rotary potentiometers we are using are single-turn
conductive plastic with a resistance of 10k and +/-1% linearity
tolerance with a life expectancy of 10 million cycles [3]. The
price of these potentiometers is economical and they are easily
available such that a replacement can be done if necessary.
III. DESIGN AND DEVELOPMENT
A. Electrical Circuit
The motion sensors require a constant voltage supply at all
times which is accomplished using two voltage regulators. A
9V battery powers the circuit and the voltage regulators reduce
the 9V signal to 2V and 4V signals [4]. The CleveMed
BioRadio takes in differential inputs limited to a range of 2V
in the positive or negative direction [5]. In order to fully
utilize the input ranges of the BioRadio, the potentiometers are
powered by 4V allowing for the wipers of each potentiometer
to swing from 0 to 4V. One differential input of the BioRadio
is then connected to the wiper of the potentiometer, while the
other input is connected to a virtual ground which is created
by the 2V offset coming from the second voltage regulator.
This setup ensures that as long as the potentiometer’s wiper
varies between 0 to 4V as desired, the input of the BioRadio
will read anywhere between -2V and +2V.
The designed circuit schematic is shown in Fig. 4. From this
circuit schematic, a PCB layout was created and the
dimensions of the board were determined to be 1” x 1”. This
layout is shown in Fig. 5.
3
B. Mechanical Brace
The brace subassembly is designed to fit the 95th
percentile
of both the female and male population [1]. The basic
components of the brace are the lower arm and upper arm
sections with the hinged elbow joint to lock them together, the
hand grip attached to the lower arm section, foam padding on
the lower arm which was added for safety and comfort of the
test subject, and lastly, the 2-axis shoulder joint. The three
rotary potentiometers are integrated such that one fits at the
hinged elbow joint, while the other two fit at the shoulder
joint. A Solid Works model of the brace exhibiting the
integration of the rotary potentiometers is shown in Fig. 6
below.
The assembly of the entire brace is complex and can be
broken down into smaller subassemblies. The first step is to
put together the handle subassembly or the hand grip. There is
a main hand back plate with side plates on both ends
supporting the hand of the subject. A handle is incorporated
through the side plates and essentially, a subject will be
holding on to this handle when using the brace. The handle
has the provision for attaching ringed weights to it as the
customer desired to have the possibility of performing
experiments with the user carrying weights as well.
The next step for the brace development is the elbow joint
assembly. Two plates have been designed for the lower arm,
one which has the channel insert and the other which can be
inserted. Similarly there are two plates for the upper arm. This
type of design allows for length adjustability which is a main
criterion for the brace as it should fit 95th
percentile of the
population as mentioned earlier. The upper arm plate with the
channel insert and the lower arm plate without the insert attach
to form the elbow joint. At the elbow joint area, a
potentiometer bracket is attached and this is where the
potentiometer gets integrated with the brace.
The next assembly is the lower shoulder joint assembly.
The free end of the upper arm plate without the channel insert
is connected to the lower shoulder side plate and a
potentiometer bracket is attached here as well. This is the
second location for potentiometer integration. Lastly, the
upper shoulder joint assembly is considered. Two side plates
connect to a brace beam, which is a square tube designed as
the connection to the platform. On the side of the brace beam,
the third potentiometer bracket is attached and the open ends
of the side plates are inserted into the lower shoulder joint
assembly.
C. Mechanical Platform
An additional requirement of our customer was that a
subject could undergo testing while sitting or standing. In
order to have these conditions fulfilled, a platform was
suggested to which the brace could be attached. During the
concept development phase, some basic requirements of the
platform were identified to be height adjustability, portability,
compatibility with a wheelchair as well as providing left and
right hand compatibility. The actual designed platform has all
the above mentioned requirements along with three linear
adjustments and appropriate shoulder pivot height range for
both the standing and sitting situations. The Solid Works
model created for the platform is illustrated below in Fig. 7.
Figure 4: Circuit Schematic depicting all three
potentiometers, the two voltage regulators and the
BioRadio connections
Figure 5: PCB Layout
Figure 6: Brace subassembly highlighting the major
components
The H-shaped metallic base of the platform is 66”x38.5”
which is big enough to accommodate a wheelchair as desired.
The diagonal flat tube connections in the lower half of the
platform ensure the stability and sturdiness of the platform.
Standard sized bolts, nuts and washers are used to connect the
different components of the platform together. For height
adjustability, the poles attached to the base have hollow
channels in them. On each side a second pole tube is inserted
in the base poles and with a handle, the desired height can be
set.
4
The two horizontal bars which connect the pole inserts
accomplish the dual functions of sitting and standing. If the
test subject is sitting, the brace is connected through the brace
beam to a brace integration T which is set on the lower
horizontal bar. Another brace integration T is set on the upper
horizontal bar. For an experiment where the test subject is to
be standing, the brace along with the beam just has to be
shifted and attached to the T on the upper bar.
The platform also has wheels on the bottom for smooth
transportation and movement if necessary. The total
assembling time of the platform is roughly an hour.
Figure 7: Platform Subassembly Model
D. Software Subsystem
The main elements of the software subsystem are the
hardware interface between the BioRadio and the computer
software, the graph modules which graph the data output from
the BioRadio, the mathematical arm model which will be a
visual rendering of the actual human arm under test and lastly
the Graphical User Interface (GUI) which displays all the
different graphs as well as the different menu options for the
user to process or display data the way he or she desires. All
the software coding has been done using C++ and the
development has been done in a cross-platform application
framework called Qt. Fig. 8 below shows the class diagram
for the entire software subsystem.
Figure 8: Software Class Diagram
The development of the software was done in four different
stages. The first stage was to create a software – hardware
interface which essentially is provided by CleveMed through
the libraries in the software development kit. The next stage
was to learn how to acquire data from the BioRadio and
display the data in graphical form. This required the creation
of graph modules. Originally, graph modules were created in
another platform called Open GL. However, when integration
with Qt was attempted, the integration was not successful. In
order to overcome this obstacle, the graph modules were
recreated in Qt.
One additional requirement from the user was to see the
corresponding angle of the test subject’s movement on the
GUI. Hence, the third stage was to develop an arm model
which could render to a 3-D matrix array to depict the angles
of the brace movement. The 3-D design of the human arm is
obtained using a mathematical arm model consisting of
rotation transformation matrices. The model uses the different
angles measured by the potentiometers as its input variables
and outputs the elbow and wrist coordinates in Cartesian form.
Fig. 9 below shows the brace arm model with the
corresponding joint angles:
Figure 9: Illustration of brace arm model with Angles
converted to 3-D Arm Model in the GUI
The original coordinates of the elbow are:
[ ]0 0 upper arm lengthT
− (1)
while the original coordinates of the wrist are:
[ ]0 0 upper arm length+lower arm lengthT
− (2)
The transformation matrices used for the elbow and wrist
respectively are:
(3)
(4)
5
Using these matrices and coordinates, an algorithm for
calculating the new coordinates is incorporated in the software
code.
The final stage for the software development was to
program the actual GUI. Using Qt, all the menus, the different
buttons and the data displays are developed. The components
of the menu bar are as follows:
• New Session – allows the user to create a new
session
• Open Session – allowing the user to open an existing
session
• Channel Window – user gets to select the channels to
be overlaid on the desired channel window
• Animation – allows the user to see the animation of
the arm model
• Filters – allows the user to select a filter to process
the raw data
Also, the GUI has buttons which are described as follows:
• Change Capture – replay data saved before
• Start – enables user to start collecting data from the
channels being graphed
• Stop – ends recording of data
• Mark – allows the user to set a marker at any location
on the data
The graphs are integrated into the GUI such that a user can
view data collected from all 8 channels of the BioRadio. These
graphs can be overlaid if someone wants to see the motion and
muscle data together.
IV. TESTING AND EVALUATION
To ensure that we had met all the requirements and
specifications of the project, each system was tested
independently first to evaluate its functionality. Once the
performance of the individual systems was verified,
comprehensive system testing was done to ensure there were
no hiccups upon integration.
A. Mechanical Testing
After the brace was assembled, the first thing tested was the
smoothness of the joints. A key aspect for accurate motion
capture was that the joints of the brace did not exhibit
jerkiness. When the brace was designed, bearing spacers were
included for the purpose of ensuring smooth joint operation.
The brace was fitted to everyone on the team and then tested
with several flexing and extending movements to verify the
smooth operation of the joints. The motion data being
captured and displayed on the GUI was monitored and no
blips in the signals were observed.
Another important specification for the brace was comfort
when a subject was wearing it for a considerable amount of
time. Three members of the team underwent comprehensive
testing for an hour at a stretch with the brace strapped to their
arm and there was no issue of pinching or tightness of straps.
The only point that came up during this testing was the
thickness of the foam pads. Ideally, the foam pads should be
thinner to provide more comfort to a test subject and this will
be addressed in a future redesign.
Since the brace was designed to fit a large sample
population, we needed to test the length adjustability of the
brace at least on the 6 of us within the team and another 4
individuals outside the team to ensure the brace was
compatible with different arm lengths and sizes. As per our
testing the brace fit a number of people with different arm
lengths and sizes; signifying the brace was designed within the
specified range.
Another important test was to see if the platform was stable
enough. When a subject was undergoing tests, we needed to
evaluate if the subject’s movement was affecting the stability
of the platform. After the platform was assembled, it was
vigorously shaken to observe if it bounced around a lot. Our
preliminary results were that it did not bounce too much and
was stable enough to ensure accurate operation of the system.
We also tested stability by observing the movement of the
platform, if any, when a test subject was strapped in and
moved his/her arms around vigorously and in all directions.
Once again, there was no corruption of data or movement of
the platform along with the test subject.
The final testing done on the platform was a weight test. The
customer stated at the beginning of the project that the system
should be portable and mobile. This implied that the platform
designed had to be easily disassembled and then transported
from one place to another. The design of the platform is such
that only 4 bolts need to be removed to divide it into two
easily transportable pieces; the base with wheels and the upper
assembly which can be carried. The team tested this out by
transporting the platform from the lower level of the building
to the third level and within ten minutes the platform was
taken apart and the two pieces were easily carried to an
elevator and transported.
B. Software Testing
The main testing for the software was in regards to the GUI
and the code linking the hardware to the software. The first
test conducted was verifying that the BioConnect code (a code
provided by CleveMed systems) executed without errors and
that data was being received from the bioinstrumentation
device. The BioRadio was set-up with the channel tester and
all eight channels were programmed to be on. The code was
then executed and as expected, data was received from the
BioRadio.
Another important test was the display speed of the data. A
test subject was strapped in the brace and main program was
executed. The test subject was asked to move his arm in
different directions and no delay between the subject’s
movement and data display in the GUI was noticeable.
In order to test the GUI, the first step was to ensure that
when the main program was executed, no error was obtained
and the program would not quit until the user decided to close
it. Initially there were some issues with the execution of the
main program. After a number of changes and refinement of
code, the main program execution errors were addressed and
the program now runs smoothly without any issues.
The GUI was designed to provide the user with several
options for processing data, saving it, calibrating the system,
using the help menu and so on. Some options are a specific
component of the menu bar while others are front window
buttons on the GUI. Each component on the menu bar has
been tested thoroughly by clicking several times and making
sure that the user is directed to the correct link.
6
Simultaneously, each button on the GUI has been clicked as
well and proper actions are completed upon every selection.
It was also needed to ensure that the graphs are being
displayed with the correct data from the appropriate channel.
In depth testing has been done to verify that the window data
displays accurately and is properly resized. The GUI has the
ability to overlay data from different channels as per the user’s
requirements.
One last test for the software that was undertaken was to
ensure that the mathematical arm model replicated the actual
movement of the brace and was at the same angle as the brace.
C. Electrical Testing
A circuit test was done to verify that the printed circuit
board (PCB) was operating as expected. Voltages were
measured at a number of different vias and the measured data
was within the acceptable range of the expected values. The
PCB had been designed and built correctly.
Another important test was the dead-zone test. Essentially
each rotary potentiometer has a dead-zone and it was
necessary for accurate system operation that the dead-zones
would be in a region where no measurements would be taken.
This required extremely accurate machining during
mechanical manufacture of the brace. To test the location of
the dead-zones, the brace was set-up and each joint was
rotated to the extremes and the voltage at that point was noted
using the GUI. It was verified that the voltage at each
extremity was either below 2V or above -2V and hence the
location of the dead-zones was accurate.
D. System Testing
A life span test was also conducted to measure the battery
life of the system. The original specification was that the
system should be kept on for more than 2 hours and there
should be no change of batteries or crashing of the system. In
order to verify this, the system was kept on for more than 4
hours. No battery change was required and the software did
not crash either.
The most important system test was the angle/calibration
test. Each joint had to be calibrated using the calibrate
function in the GUI and a goniometer had to be used to
manually measure the angles to be fed into the software.
Three test subjects were used for system calibration and the
process was done using 2 points first and then 3 points. The
angles measured are compared with those given by the
software. Once calibration is done, the test subject is required
to move the joint under consideration at different angles and
the angles given by the software are recorded along with the
manually measured angles. The results obtained for the
calibration testing are summarized in the Results section.
V. RESULTS
Overall, the entire system is functional and each individual
system performs as desired. Table I gives a summary of the
desired and actual values of different specifications for the
entire mechanical system.
Table II gives a summary of the software system
specifications while Fig. 10 shows a screen shot capture of the
functioning GUI demonstrating overlaying of angle data on
EMG data in all three channel windows. We verified that the
mathematical arm model moves in accordance with the brace
and is accurate as far as angular position is concerned which
was a requirement of the customer. Fig. 11 depicts a screen
shot of the mathematical arm model.
Description Desired Actual
Shoulder height
range (standing)
528 – 674
mm 47.5 - 58.5 in
Hand width range 81 – 110 mm 5 in
Shoulder rotation range
Transverse plane -60° to 140° << -60° to 53°
Vertical plane -70° to 190° - 38° to 140°
Elbow rotation range 0° to 160° 0° to 163°
Weight of brace 5 lb 4.4 lb
Comfortable padding Yes yes
Table I: Summary of Mechanical System Specifications
Description Desired Actual
Time delay from actual
movement to movement
in software 100 ms Less than 100 ms
OS compatibility WinXP All
Disk space 2 GB 500 MB
Processor speed 2 GHz 3GHz
Signals displayed 9 21
Animation Yes No
Table II: Summary of Software System Specifications
Fig. 10: Screen Shot of GUI Demonstrating Angle Data
Overlaid on EMG data
7
Fig. 11: Screen shot of mathematical arm model
animation
As mentioned earlier, the testing done regarding the PCB
was successful and Table III below gives a comparison of the
expected and actual values. Table IV summarizes the
specification of the electrical system.
Expected
(V) Actual (V) % Error
Battery 9.000 8.814 2.07%
U1 Output 2.000 1.978 1.10%
U2 Output 4.000 4.013 0.31%
Virtual
GND 2.000 1.996 0.20%
Table III: Expected and Actual Voltage Values for PCB
Testing
In order to summarize the testing conducted for system
calibration, the average error for each joint for each test
subject was plotted for both the 2-point and 3-point calibration
processes. Figures 12 – 14 below depict the results of the
calibration testing.
Fig. 12: Plot of Elbow Joint Calibration Testing
Fig. 13: Plot of Shoulder 1 Joint Calibration Testing
Fig. 14: Plot of Shoulder 2 Joint Calibration Testing
Description Desired Actual
Accuracy of angular position
Elbow Less than ±1° ±5°
Shoulder Less than ±1° ±5°
Accuracy of angular velocity
Elbow Less than
±2.5°/sec DNE
Shoulder Less than
±2.5°/sec DNE
Data acquisition
frequency 960 Hz 960 Hz
No. muscle groups
monitored At least 4 4
Life cycle of motion
sensors Greater than 1
million 10 million
Table IV: Summary of Electrical System Specifications
8
VI. DISCUSSIONS AND CONCLUSIONS
On the whole, the product delivered to the customer was
almost on par with his requirements and specifications. The
team had issues meeting his specification regarding the arm
model integration with software due to time constraints.
Besides that hiccup, the system is robust and functional and
the entire project was well under the budget which was a
major success milestone.
A future generation of the project could most definitely be
conceived and one major improvement to the system would be
to design a bioinstrumentation device instead of using the
device from CleveMed systems. This would make the system
completely independent.
VII. REFERENCES
[1] A. Tilley and H. Dreyfuss, “The Measure of Man and
Woman: Human Factors in Design”, Wiley Publications,
2002
[2] A. Adams. “The Muscular System.” Human Body
Systems. Michael Windeslpecjt. Series Editor. Greewood
Press, Westport, Connecticut, London. 2004.
[3] P3 America, Inc: Single-Turn Conductive Plastic
Potentiometer Series – MP20, 21
[4] National Semiconductors. LP2989LV Micropower 500
mA Low Noise Low Dropout Regulator for Output Voltages.
June 2004. http://cache.national.com/ds/LP/LP2985LV.pdf.
[5] BioRadio 150 User’s Guide, Cleveland Medical Devices
[6] K. Itoh, H. Miwa, Y. Nukariya et al., “Development of a
Bioinstrumentation System in the Interaction between a
Human and a Robot”. Proceedings of the 2006 IEEE/RSJ Intl.
Conf. on Intelligent Robots and Systems, Oct 9 -15, 2006,
Beijing, China