systems for tracking minimally invasive surgical instruments

PLEASE SCROLL DOWN FOR ARTICLE

This article was downloaded by: [TU University of Technology Delft]On: 15 September 2008Access details: Access Details: [subscription number 732450797]Publisher Informa HealthcareInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House,37-41 Mortimer Street, London W1T 3JH, UK

Minimally Invasive Therapy and Allied TechnologiesPublication details, including instructions for authors and subscription information:http://www.informaworld.com/smpp/title~content=t713683124

Systems for tracking minimally invasive surgical instrumentsM. K. Chmarra a; C. A. Grimbergen ab; J. Dankelman a

a Department of BioMechanical Engineering, Delft University of Technology, Delft, The Netherlands b

Department of Medical Physics, University of Amsterdam, The Netherlands

First Published on: 17 October 2007

To cite this Article Chmarra, M. K., Grimbergen, C. A. and Dankelman, J.(2007)'Systems for tracking minimally invasive surgicalinstruments',Minimally Invasive Therapy and Allied Technologies,16:6,328 — 340

To link to this Article: DOI: 10.1080/13645700701702135

URL: http://dx.doi.org/10.1080/13645700701702135

Full terms and conditions of use: http://www.informaworld.com/terms-and-conditions-of-access.pdf

This article may be used for research, teaching and private study purposes. Any substantial orsystematic reproduction, re-distribution, re-selling, loan or sub-licensing, systematic supply ordistribution in any form to anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation that the contentswill be complete or accurate or up to date. The accuracy of any instructions, formulae and drug dosesshould be independently verified with primary sources. The publisher shall not be liable for any loss,actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directlyor indirectly in connection with or arising out of the use of this material.

http://www.informaworld.com/smpp/title~content=t713683124

http://dx.doi.org/10.1080/13645700701702135

http://www.informaworld.com/terms-and-conditions-of-access.pdf

REVIEW ARTICLE

Systems for tracking minimally invasive surgical instruments

M. K. CHMARRA1, C. A. GRIMBERGEN1,2 & J. DANKELMAN1

1Department of BioMechanical Engineering, Delft University of Technology, Delft, The Netherlands, and 2Department of

Medical Physics, University of Amsterdam, The Netherlands

AbstractMinimally invasive surgery (e.g. laparoscopy) requires special surgical skills, which should be objectively assessed. Severalstudies have shown that motion analysis is a valuable assessment tool of basic surgical skills in laparoscopy. However, to usemotion analysis as the assessment tool, it is necessary to track and record the motions of laparoscopic instruments. Thisarticle describes the state of the art in research on tracking systems for laparoscopy. It gives an overview on existing systems,on how these systems work, their advantages, and their shortcomings. Although various approaches have been used, none ofthe tracking systems to date comes out as clearly superior. A great number of systems can be used in training environmentonly, most systems do not allow the use of real laparoscopic instruments, and only a small number of systems provide forcefeedback.

Key words: Minimally invasive surgery, tracking system, motion analysis, objective assessment, skills

Introduction

Minimally invasive surgery (e.g. laparoscopy) is a

technique that requires special surgical skills (1).

Traditionally, resident surgeons start their surgical

education observing experienced surgeons in the

operating room (OR). Afterwards, they are allowed

to contribute to the operation (e.g. they perform a

number of basic techniques). Finally, they become

primary surgeons; however, this generally occurs

without an objective assessment of their skills.

Since the skill and the level of experience of the

surgeons are not exactly known, the current

method of training is potentially unsafe for the

patient (2).

Since the emphasis on medical safety and the

complexity of laparoscopic techniques and equip-

ment continuously increases, it is essential to

develop new, safe training and assessment methods

for laparoscopic skills. Training methods, such as

box trainers and virtual reality (VR) trainers, have

already been developed to learn basic laparoscopic

skills outside the operating room (3–6). However,

the objective assessment of residents’ skills still

remains a challenge (7).To date, there is no one

widely used automatic assessment method of the

basic minimally invasive surgical skills. Commonly,

assessment relies heavily on the expert surgeons and,

therefore, is not always objective (8–10). An

objective assessment of skills is a very important

factor in developing an effective curriculum, since it

motivates residents to actively engage with training

of his/her skills and provides valuable feedback as to

whether that engagement translates into gaining

experience.

In the literature, it has been demonstrated that

basic psychomotor laparoscopic skills can be

assessed by analysing motions of the instrument

(7,10). Several measures have been proposed (e.g.

the length of the curve described by the tip of the

instrument, the total distance travelled by the

instrument along its axis, economy of the move-

ment, changes in instrument velocity over time)

(4,7,11–13). In order to use motion analysis as the

assessment tool, a system is needed to track and

record these motions. In VR trainers such tracking

systems are inherently present. Tracking systems

that can track real laparoscopic instruments during

surgery or in the box trainer are still in their

infancy.

Correspondence: M. K. Chmarra, Department of BioMechanical Engineering, Faculty of Mechanical, Maritime and Materials Engineering (3mE), Delft

University of Technology, Mekelweg 2, 2628 CD Delft, The Netherlands. Fax: 31-15-278 4717. E-mail: [email protected]

Minimally Invasive Therapy. 2007; 16:6; 328–340

ISSN 1364-5706 print/ISSN 1365-2931 online # 2007 Taylor & Francis

DOI: 10.1080/13645700701702135

Downloaded By: [TU University of Technology Delft] At: 12:18 15 September 2008

In general, one may consider the tracking system

as an independent component of a training system,

not directly related to the choice of exercises and

scoring measures. The sole task of the tracking

system is to measure the position (x, y, and z

coordinates) and the orientation (yaw, pitch, and

roll) of the laparoscopic instrument with respect to a

fixed reference frame.

Presently, there is a number of (commercially)

available devices for tracking movements of laparo-

scopic instruments. These devices are specific for

different environments: Box trainers, virtual reality

trainers, and operating rooms. Our study of the

literature showed that no overview of the systems

used for tracking motions of the laparoscopic

instruments has been published. Since there is a

number of such systems already available, and since

new systems are being developed, it is difficult for a

potential customer to find appropriate information

and to choose a proper tracking system. To produce

a complete and structured overview of tracking

systems, a large number of journals and proceedings

available via PubMed, Scholar Google, and patents

have been studied. In this article, special care was

taken to ensure that presented information is correct

and up-to-date; for each approach it was attempted

to answer the following questions:

N How does it work?

N Where can it be used?

N What are its chief advantages?

N What are its shortcomings?

For this reason, nine features were used to evaluate

the systems: the kind of system (active or passive),

the mechanics, the number of degrees of freedom,

possibility of using real laparoscopic instruments,

environment (box trainer, VR trainer, and/or oper-

ating room), haptic feedback, portability, reported

accuracy, and commercial availability.

Tracking systems

Aspects of general tracking systems

Tracking systems are intended as an interface

between humans and computers. In terms of hard-

ware, three components required in such systems

can be distinguished: A source that generates a

signal, a sensor that receives the signal, and a data

acquisition system, which processes the signal and

communicates with the computer (14). Depending

on the technology, either the source or the sensor is

attached to the object, while the other serves as a

reference point and is located at a fixed point in the

environment. Many of the currently used tracking

systems are active; the sensor, which measures the

actual movement, is attached to the target to be

tracked. Passive tracking systems localise (from a

distance) markers/transmitters that have been placed

on instruments (or objects) to be tracked in the field.

Current tracking devices are based on mechanical,

optical, acoustic, or electromagnetic technologies.

Mechanical tracking devices, typically taking the

form of small volume mechanical arms, use a direct

mechanical connection between a reference point

and a target (angles formed by each joint) to

measure the position and orientation of the object

in the environment. Optical position trackers work

one of two ways:

N One or several cameras are connected to the

object, and a set of light emitting diodes (LEDs)

is placed at the fixed reference points, or

N the cameras are mounted at fixed points, while a

set of LEDs is placed on the object.

Acoustic tracking devices employ high frequency

(20 kHz or greater) ultrasonic sound waves in the

form of time-to-flight transducers/sensors or phase-

referent systems. The latter class of systems relies on

comparing the phase of a reference signal to that of a

sensed emitted signal. In a time-to-flight system, the

duration of the travel of an emitted signal is

correlated with the distance travelled in the air at a

given temperature. Electromagnetic tracking is

based on the movement of a number of small sensor

units, each housing three small (orthogonally posi-

tioned) wire coils, within a low frequency electro-

magnetic field generated by a second three-coil

source or transmitter.

Aspects of tracking systems for minimally invasive

surgery

Minimally invasive surgery requires unique psycho-

motor skills that are different from those needed to

perform open surgery. The surgeon’s hand move-

ments are transmitted through the incision point via

a trocar (instrument shaft) to the tip of the

instrument. This limits the range of motions from

six to four degrees of freedom (DOFs): translation of

the instrument along its axis (z coordinate – 1st

DOF), rotation of the instrument around its axis

(roll – 2nd DOF), left–right and forward-backward

rotations of the instrument around the incision point

(yaw – 3rd DOF, and pitch – 4th DOF, respectively).

Information about instrument motions in all four

DOFs provides valuable information, which can be

used during the assessment of basic laparoscopic

skills. Therefore, tracking systems should track

motions in all four DOFs.

Tracking minimally invasive instruments 329


A great number of active systems contain a

mechanical part that mimics the incision (pivoting

point). Many of these systems use a gimbal

mechanism, which allows the control and measure-

ment of the object’s rotations in three-dimensional

Euclidean space. Usually, the gimbal consists of a set

of two or three rings, mounted on axes at right angles

(15). These rings provide a stable reference to the

position and attitude in all three dimensions. Since it

is necessary to define a reference point (position,

space) in the environment in order to measure the

movements of the object, it seems reasonable to use

the pivoting point as a reference point in active

systems.

A wide variety of different instruments is available

for use in minimally invasive surgery. Instruments

are made by various companies and have different

purposes, consequently, the length, the diameter,

the handgrip, and the tip of real laparoscopic

instruments can differ largely. The variation in

instruments has an influence on the design of a

tracking system, which should ideally be suitable for

any laparoscopic instrument.

Motions of the laparoscopic instruments can be

tracked and recorded in three environments: In the

operating room (during an operation), in a box

trainer, or a virtual reality trainer (during training).

The operating room is the most realistic environ-

ment, where a surgeon uses real laparoscopic

instruments, which provide natural instrument-

tissue interaction with realistic force feedback. The

box trainer also presents a realistic environment

where force feedback is obtained due to the use of

real laparoscopic instruments. The virtual reality

trainer presents a less realistic (virtual) environment

for the training of laparoscopic skills, where simu-

lated laparoscopic instruments are often used. Force

feedback in VR trainers (if present) is far from that

provided by real laparoscopic instruments. The use

of simulated laparoscopic instruments in a VR

trainer, however, results in simplified tracking in a

VR trainer. Designing a system that can track real

laparoscopic instruments in the OR seems to be the

most difficult, since factors such as patient safety and

ergonomics in the OR play a critical role. In box

trainers, tracking systems have to track real laparo-

scopic instruments. Ideally, a tracking system should

be designed in such a way that it is possible to use it

in all three environments. Besides, the system should

be small in order to be portable and easy to place at a

desired position.

Surgeons execute laparoscopy using hand-held

instruments that provide little haptic information

(16–18). It is still unknown how important haptic

feedback is during a laparoscopic task and, therefore,

it is very difficult to say whether force feedback is

needed during training of basic laparoscopic skills.

Precision, a degree to which measurements show

the same or similar results, is a very important

characteristic of each tracking system. The precision

with which instrument movements can be measured

depends on the resolution and accuracy of the

system (15). The resolution is defined by the

smallest change detected by the sensor, and is fixed

for a given system. The accuracy of the system is

defined by the range within which a measured

position is correct.

A number of tracking systems is provided only in

combination with VR trainers, while other systems

can be acquired separately. In the paragraphs that

follow, we present both tracking systems that are

currently commercially available and prototypes, i.e.

systems that are still being developed. The practical

use of these tracking systems will also be described.

Passive tracking systems

In the ProMIS simulator (Haptica Inc., Boston,

USA, www.haptica.com), the measurement of the

instrument movements is taken using a passive

tracking system (Figure 1) (19). Three separate

cameras capture the video images of the internal

movement of the laparoscopic instrument from three

different angles (20). This design allows for mea-

Figure 1. In the ProMIS surgical simulator from Haptica

(courtesy of Haptica Inc.), the MIS instrument movements are

tracked using a passive system. Laparoscopic instruments are

covered with two stripes of yellow tape (a marker). Internal

movements of the instruments are captured with three separate

cameras.

330 M. K. Chmarra et al.


surement of the motions in the x, y, and z directions.

Standard laparoscopic instruments are covered with

two strips of yellow tape – markers for the camera

tracking system. The tracking system is situated in a

large mannequin, thus not easily portable. The

system is commercially available as a combination

of a real and a virtual environment. It cannot be used

during operation (in the OR), however. The ProMIS

simulator provides force feedback.

Sokollik et al. used a 3-D ultrasound measure-

ment system to record the motion along the

trajectories of the instruments (Zebris Medial

GmbH, Isny, Germany, www.zebris.de) (21). The

system (Figure 2) determines the spatial coordinates

(x, y, and z together with rotation) of miniature

ultrasound transmitters placed on the instruments

by means of the relative position of these transmit-

ters to a fixed system of three microphones. The

system is portable and commercially available.

Natural haptic feedback is obtained due to the use

of real laparoscopic instruments. Since ultrasound

transmitters can be sterilised, the 3-D ultrasound

measurement system can be used in the operating

room as well as in a box trainer and VR trainers.

An ultrasound wireless positioning system is being

developed at Delft University of Technology (Delft

University of Technology, Delft, the Netherlands,

wwwetis.et.tudelft.nl). This system is intended to be

used in the operating room to detect the exact 3-D

location and orientation of the instrument in the

patient (22). An array of ultrasound receivers detects

the positions of the two markers/transmitters placed

on each of the instruments, outside the patient’s

body (Figure 3). This allows readings with an

accuracy of 40 mm at a distance of about 1 m

between transmitter and receiver. The resolution of

the system is about 5 mm (23). This prototype

measures movements in one DOF. The size of the

system is rather large and, therefore, the system is

not easily portable. Natural haptic feedback is

obtained due to the use of real laparoscopic

instruments.

Active tracking systems

Laparoscopic Surgical Workstation, Virtual

Laparoscopic Interface, and Laparoscopic Impulse

Engine are the best known hardware interfaces

designed for laparoscopic virtual simulations offered

by Immersion Inc. (Immersion Inc. Gaithersburg,

USA, http://www.immersion.com). The Laparos-

copic Surgical Workstation and the Virtual

Laparoscopic Interface offer two fully instrumented

tools (Figure 4). The movements of the instrument

in four DOFs are measured and recorded using four

electromechanical transducers mounted in the gim-

bal mechanism (24–28). The Laparoscopic Surgical

Workstation provides force feedback. The sensor

resolution is 8 mm for translation, 0.03˚ for roll

(rotation), and 0.01˚ for pitch and yaw. The

Laparoscopic Surgical Workstation is rather big

(300 mm6340 mm) and, therefore, not easily

portable.

Figure 2. The Zebris ultrasound system adapted for laparoscopic

surgery application (courtesy of C. Sokollik). In this passive

tracking system, ultrasound transmitters are placed on the

laparoscopic instruments. Spatial coordinates are determined by

means of relative positions of the transmitters to a fixed system of

three microphones.

Figure 3. An ultrasound wireless positioning system that is being

developed at Delft University of Technology (courtesy of F.

Tatar). In this passive tracking system, an array of ultrasound

receivers detects the positions of the two markers/transmitters

placed on each of the instruments.



The Virtual Laparoscopic Interface does not

provide force feedback. The sensor resolution is

22 mm for translation, 0.26˚ for roll (rotation), and

0.064˚ for pitch and yaw. The Virtual Laparoscopic

Interface is easily portable. Both the Laparoscopic

Surgical Workstation and the Virtual Laparoscopic

Interface are commercially available. The

Laparoscopic Impulse Engine is a tool-based force

feedback device that uses servo-motor actuators

(Figure 5). Laparoscopic Impulse Engine allows

movements in four DOFs. A variety of surgical tools

(or tool handles) can be fitted in the device. The

Immersion devices cannot be used in the operating

room.

The CELTS (Computer Enhanced Laparoscopic

Training System) is a prototype simulator (Figure 6)

developed at the Center for Integration of Medicine

and Innovative Technology (CIMIT, Boston, USA,

www.cimit.org). CELTS is a modified Virtual

Laparoscopic Interface from Immersion Inc.; tool

handles and main shafts from the Virtual

Laparoscopic Interface were replaced with a system

that allows for the use of real laparoscopic instru-

ments to detect the trajectories of these instruments

in a simulator (7,29,30). Pitch, yaw, roll, and

translation are measured with the original Virtual

Laparoscopic Interface sensors. CELTS can be used

in a box trainer as well as in a virtual reality

environment. It is not possible to use this system in

the OR during operations. The system is easily

transportable, just like the Virtual Laparoscopic

Interface. The possibility of using real laparoscopic

instruments in the CELTS results in a natural haptic

feedback.

The ADEPT (Advanced Dundee Endoscopic

Psychomotor Tester) was developed at the

University of Dundee (University of Dundee,

Dundee Tayside, Scotland). The ADEPT consists

of a gimbal mechanism that accepts real endoscopic

instruments (Figure 7) (31,32). This active system

tracks and records positions of the instrument in 3-D

space, and detects rotational movements. The

measurements are taken using potentiometers

mounted in the gimbal mechanism; this allows

readings to within a millimetre. The accuracy of

the x, y, and z coordinates (at a distance of 300 mm

from the pivoting point) is within 0.5 mm. The

smallest angle recognised by the ADEPT is approxi-

mately 0.005 .̊ Real laparoscopic instruments can be

used in the ADEPT; nevertheless, it is not possible

to use this system to record instrument movements

during an operation in the OR. The commercially

used ADEPT provides natural haptic feedback.

The Simendo (Figure 8) is a virtual reality

simulator for minimally invasive surgery

(DelltaTech, Delft, the Netherlands, www.dellta-

tech.nl) (33). DelltaTech manufactures the

Simendo, which comprises both the instrument

interface and the virtual reality training software.

The tracking system in the instrument interface

consists of a gimbal mechanism. Translation and

rotation of the simulated laparoscopic instruments

are measured by an optical sensor, while pitch and

yaw are measured by optical encoders. This combi-

nation allows measuring the movements of the

Figure 4. The Laparoscopic Surgical Workstation (left) and

Virtual Laparoscopic Interface (right) from Immersion (www.im-

mersion.com). These active tracking systems measure movements

of laparoscopic instruments using four electromechanical trans-

ducers mounted in the gimbal mechanism.

Figure 5. The Laparoscopic Impulse Engine from Immersion

(www.immersion.com). This active tracking system is a tool-

based force feedback device that uses servo-motor actuators.



instrument in four DOFs. As the Simendo was

especially developed for application in virtual reality

simulation, it does not accept real laparoscopic

instruments and does not provide force feedback.

The instrument interface of the Simendo is small

and light and, therefore, easy to transport. The

Simendo is commercially available.

The BlueDRAGON tracking system was devel-

oped at the University of Washington (University of

Washington, Seattle, USA, http://brl.ee.washington.

edu). This system consists of two four-bar passive

mechanisms that are connected to the instruments

(Figure 9) (34,35). The measurements of the

instrument positions, orientations, and translation

are taken with multi turn potentiometers integrated

into four joints of the mechanism. The system is

rather large, and thus not easily portable. The

Figure 6. The CELTS interface device from Center for Integration of Medicine and Innovative Technology (courtesy of N. Stylopoulos).

This tracking system is a modified Virtual Laparoscopic Interface from Immersion Inc. The main shafts and tool handles are replaced with a

system that allows the use of real laparoscopic instruments.

Figure 7. The ADEPT system from the University of Dundee

(courtesy of M. Schijven). This active tracking system measures

the position of the laparoscopic instrument using potentiometers

mounted in the gimbal mechanism.

Figure 8. The Simendo from DelltaTech (courtesy of

DelltaTech). In this active tracking system, translation and

rotation of laparoscopic instrument are measured with an optical

sensor, while pitch and yaw are measured with encoders.

Figure 9. The BlueDRAGON from the University of Washington

(courtesy of J. Rosen). This active tracking system consists of two

four-bar passive mechanisms that are connected to the instru-

ment. Multi turn potentiometers are integrated into four joints of

the mechanism in order to measure movements of laparoscopic

instrument.



BlueDRAGON allows the use of real laparoscopic

instruments and can be used in the OR during

operation.

The Patriot is a dual sensor tracking system

produced by Polhemus (Polhemus, Colchester,

USA, www.polhemus.com/). The Patriot consists

of an electromagnetic transmitter and receiver. The

transmitter serves as the system’s reference frame for

receiver measurements; the receiver detects mag-

netic fields emitted by the transmitter (Figure 10)

(36). In laparoscopic training setups (e.g.

SimSurgery), the receiver is placed on the laparo-

scopic instrument. The Patriot has a resolution of

0.038 mm for the x, y, and z positions, and 0.1˚ for

the receiver orientation. Static accuracy of the

system is 2.54 mm, and 0.75 .̊ Since the Patriot

can be used to track real laparoscopic instruments, it

is possible to use the Patriot in training setups (VR

and box trainers). The Patriot system is not certified

for medical or bio-medical use; therefore, it should

not be used in the OR.

Xitact ITP (Instrument Tracking Port) and Xitact

IHP (Instrument Haptic Port) are virtual reality

simulation platforms produced by Xitact S.A.

(Xitact S.A. Morges, Switzerland, www.xitact.com).

Both systems (Figure 11) measure movements of the

instruments in four DOFs. Both Xitact systems

consist of the PantoScope (Hybrid Parallel Serial

drive), and the LinRot (Linear and Rotational

drive). The longitudinal and angular positions of

the instruments are measured using optical sensors

placed in the LinRot (37). The yaw and pitch are

measured using two optical encoders situated in the

PantoScope. The sensor resolution is 0.057 mm for

translation, 0.58˚ for roll, and 0.03˚ for both pitch

and yaw. The footprint and the weight of the Xitact

ITP are smaller than those of the Xitact IHP. Both

systems are easy to transport. Contrary to the Xitect

ITP, the Xitect IHP provides force feedback.

The IOMaster5D and the IOMaster7D are pro-

totype hardware interfaces designed at the Research

Centre Karlsruhe (Karlsruhe, Germany, www-

kismet.iai.fzk.de). The mechanical construction of

these systems is based on cable drives and a

pantograph mechanism (Figure 12) (38–40). This

combination allows movements in four DOFs in the

IOMaster5D and six DOFs in the IOMaster7D.

Additionally, the IOMaster7D captures the move-

ments of the trocar. The yaw, pitch, and angular

position of the simulated laparoscopic instrument

are measured by optical encoders; the translation is

measured with a Hall-sensor. The IOMaster5D and

the IOMaster7D have been designed for laparo-

scopic virtual simulation and are not able to track

real laparoscopic instruments. Both systems provide

force feedback (38–40). The IOMaster7D has a

large working space (600 mm (W)6600 mm

(H)6300 mm (D)) and, therefore, is not easily

portable.

The TrEndo (Delft University of Technology,

Delft, the Netherlands, www.3me.tudelft.nl) is a

prototype tracking system, which consists of a two-

axis gimbal mechanism with three optical sensors

Figure 10. The Patriot, a dual sensor tracking system produced

by Polhemus (www.polhemus.com). Patriot is an active tracking

system, which uses electromagnetic transmitter and receiver to

measure the movements of laparoscopic instrument.

Figure 11. The Xitact ITP (left) and Xitact IHP (right) from

Xitact S.A. (www.xitact.com). In both Xitact interfaces, transla-

tion and angular position of the instrument are measured with

optical sensors placed in the LinRot (Linear and Rotational drive),

while the yaw and pitch are measured using optical encoders

placed in the PantoScope (Hybrid Parallel Serial drive).



(Figure 13) (41). The gimbal guides an instrument,

while optical sensors measure the movements of the

instrument in four DOFs. Natural haptic feedback is

obtained due to the use of real laparoscopic

instruments. The TrEndo is small and light and,

therefore, easy to transport. The TrEndo can be

mounted on a box or on a VR trainer. It is not

possible to use the TrEndo during an operation. The

smallest movement that can be recognised by the

TrEndo is 0.06 mm for translation and 1.27˚ for

rotation of the laparoscopic instrument around its

axis. The smallest recognised angle for rotation

around the incision point is 0.23 .̊ The relative error

of the TrEndo is v 5%. The relative error was

defined as the difference between the real and

measured travelled distance divided by the real

travelled distance. The accuracy of the TrEndo is,

hence, higher than 95% (41).

Application of tracking systems in trainers

Tracking systems are provided either only in

combination with trainers (ProMIS, CELTS,

ADEPT, Simendo) or separately (ultrasound mea-

surement system, ultrasound wireless positioning

system, Laparoscopic Surgical Workstation, Virtual

Laparoscopic Interface, Laparoscopic Impulse

Engine, BlueDRAGON, Patriot, Xitact ITP, Xitact

IHP, IOMaster5D, IOMaster7D, TrEndo).

Tracking systems that track real laparoscopic instru-

ments are mostly used in research environments to

develop new scoring methods (7,21,42–45).

Presently, most of the companies that produce VR

trainers for laparoscopy focus on the development of

the software and they use commercially available

tracking devices. For example, LapSim and MIST-

VR trainers include the Virtual Laparoscopic

Interface, the LAP Mentor trainer includes the

Xitact IHP, and the SimSurgery includes the

Patriot tracking system. A number of articles on

training systems for laparoscopy has already been

written (46–50). These articles provide information

about the features, assessment, advantages and the

shortcomings of these systems. Since this informa-

tion about training systems has already been

published, the focus was placed on the technical

features of the tracking systems.

Presently, the most often used objective perfor-

mance measures in trainers are those based on the

kinematics analysis theory:

N time – total time taken to complete the task

N path length – the length of the curve described by

the tip of the instrument

N movement economy – ‘‘ideal path length’’ divided

by the actual path length, where the ideal path

Figure 12. Prototypes of the haptic devices for laparoscopic surgery applications from Karlsruhe (courtesy of H. Maass). Mechanical

construction of the IOMaster7D (left) and the IOMaster5D (right) is based on cable drives and pantograph mechanism. Yaw, pitch, and

rotation of the laparoscopic instrument around its axis are measured using optical encoders. The translation of the instrument is measured

with a Hall-sensor.

Figure 13. The TrEndo tracking system from Delft University of

Technology. In this active tracking system, movements of the

laparoscopic instrument are measured by three optical sensors

mounted on the gimbal mechanism.



length is the straight-line distance between two

targets

N deviation from the ideal path – the sum of the

differences between the actual and ideal path

N depth perception – total distance travelled by the

instrument along its axis

N rotational orientation – the amount of rotation of

the instrument around its axis

N motion smoothness – parameter which represents

a change of the acceleration, etc. (7,12,13,42,43).

All these measures are based on the time-dependent

three-dimensional representation of the tip of the

instrument and rotation of the instrument around its

axis (which represent four DOFs). Some of the

measurement parameters, for example time and path

length, do not require a very precise and accurate

measurement of the movements. Other parameters,

such as motion smoothness, do depend on the

precision and the accuracy of the measurement.

Performance measures depend on the kind of

exercise that is performed (e.g. clip application,

suturing). For different exercises different motion

characteristics are optimal. It is essential to realise

that the tracking system is a device which can be

seen as an independent component of the training

system. It does not automatically assess the perfor-

mance of the laparoscopic task and it is not related to

the choice of exercises. Tracking systems consist of

hardware and software that are only used to measure

the position of the laparoscopic instruments at each

time point. From these measured positions and their

changes in time, additional software can derive

performance measures.

Generally, tracking systems need a computer

program that collects measured data and evaluates

the performance of the user. Theoretically, it is

possible to use one universal program to collect and

analyze data from each particular tracking system

mentioned above. Such a universal program would

allow for easy application of a standard scoring

system independent of the tracking system.

Discussion

Sixteen tracking systems for minimally invasive

surgery developed over the past decade have been

presented (Table I). The overview shows that

various approaches have been used. These

approaches have their own advantages and disad-

vantages. Nine of the described tracking systems can

be used in one environment only (in most cases in a

virtual environment). There are only a few systems

that can be used either in the operating room

(however, still not very successfully) or in a box

trainer.

Based on the technology used, the systems

described in this paper have been divided into

passive and active systems. Passive systems benefit

Table I. Overview of tracking systems*

Name System Mechanics DOF

Lap.

instr. Environment Portability Feedback Accuracy Availability

ProMIS P 2 3 + box, VR 2 + +UMS P 2 4 + box, VR, OR 2 + +UWPS P 2 1 + OR 2 + v40 mm 2

LSW A gimbal 4 2 VR 2 + +VLI A gimbal 4 2 VR + 2 +LIE A joystick 4 2 VR + + +CELTS A gimbal 4 + box, VR + + 2

ADEPT A gimbal 4 + box + ¡0.5 mm +Simendo A gimbal 4 2 VR + 2 +BlueDRAGON A bar passive 4 + box, OR 2 +Patriot A 2 4 + box, VR + + 2.54 mm,

0.75˚+

Xitact ITP A PantoScope/

LinRot

4 2 VR + 2 +

Xitact IHP A PantoScope/

LinRot

4 2 VR + + +

IOMaster5D A cable drive/

pantograph

4 2 VR + + 2

IOMaster7D A cable drive/

pantograph

7 2 VR 2 + 2

TrEndo A gimbal 4 + box, VR + + w95 % 2

* DOF – degrees of freedom; UMS – ultrasound measurement system; UWPS – ultrasound wireless positioning system; LSW –

laparoscopic surgical workstation; VLI – virtual laparoscopic interface; LIE - laparoscopic impulse engine; P – passive; A – active; OR –

operating room; VR – virtual reality trainer; box – box trainer.



from not requiring any type of cables wiring attached

to the instrument. For that reason, this approach is

to be favoured from the user’s perspective. However,

there is a number of factors that limit the passive

systems (for tracking hand-held objects). The most

significant is the need for line-of-sight. Objects held

in hand are, by definition, partially hidden behind a

person’s hand, and totally hidden behind a person’s

body from certain perspectives. A solution to this

problem can be to add extra trackers (localisers).

However, this will introduce additional cost and

computational complexity. Therefore, many of the

currently used tracking systems for minimally

invasive surgical instruments are active. Most of

the active systems use gimbal mechanisms to guide

and measure the movements of the instrument in

four degrees of freedom. Such a mechanism benefits

from its simplicity; a gimbal is an easy and

inexpensive-to-produce mechanism. On the other

hand, it would be very complex to use gimbal

mechanisms to guide the motions of the laparoscopic

instrument during real surgery.

In laparoscopy, the surgeon’s hand movements

are transmitted through the incision point to the tip

of the instrument. This results in a reduced number

of degrees of freedom from six to four. Since

information of these four DOFs provides valuable

information that can be used to assess basic

laparoscopic skills, the focus of this article was only

on these DOFs. Nevertheless, laparoscopic instru-

ments also provide a fifth DOF: The opening and

closing of the instrument handles. Therefore, a

number of tracking systems (mainly for VR trainers)

tracks and records also this fifth DOF.

Precision and accuracy are important character-

istics of each tracking device. Ideally, a measurement

device (or system) is both precise and accurate. In

laparoscopy, small movements in the incision point

can result in large movements of the tip of the

instrument. Therefore, sensors that record the

instrument’s movements at the incision point should

be able to recognise small movements. The literature

survey showed that there is no study which defines

how precise and accurate the measurements of the

laparoscopic instrument motions should be.

Therefore, it is difficult to say whether the precision

and accuracy of the present tracking systems are high

enough to be used to track motions of instruments.

Some systems have to be calibrated before use in

order to make measurements more reliable and

accurate. Often, such calibration is done by posi-

tioning the laparoscopic instrument in a predefined

start position. Since all measurements start at the

same position, it is easy to compare the results of the

measurements.

Presently, a single method of establishing and

presenting the accuracy of tracking systems for

laparoscopy is lacking. Usually, the precision and

accuracy are established by the manufacturer by

repeatedly measuring some traceable reference

standard. In this manner, the accuracy can be

calculated in different ways, depending on what is

important for the manufacturer or designer.

Additionally, a number of manufacturers do not

provide any information about the accuracy of their

systems. This can be confusing for customers.

In laparoscopy, the assessment of the basic

laparoscopic skills is preferably done based on the

movements of the instrument tip; therefore, this

study focuses only on systems that track the tip of

laparoscopic instruments. Nevertheless, there is a

number of tracking systems that can track various

hand-held objects (e.g. PHANTOM (51)). Some of

these systems can be easily adapted and used for

tracking motions of laparoscopic instruments.

Additionally, systems have been developed for

tracking various parts of the human body (e.g. arms,

legs, head). These systems can also be modified and

used for tracking surgical movements. Examples of

such systems are: The Imperial College Surgical

Assessment Device (ICSAD) (52,53), which tracks

the movements of the hands, and the ultrasound

measurement system, which was adapted to track

the movements of the laparoscopic instrument (21).

There are also robotic systems, such as da Vinci

(54,55), RobIn Heart (56,57), and a spherical

mechanism (58) which contain tracking systems.

These systems can be used to assess the motions

when performing minimally invasive surgery with a

robot (59,60).

Motion analysis is purely used for objective

assessment of basic technical skills. However, even

a simple task in laparoscopy requires the use of

several different skills at the same time (e.g.

anatomical knowledge, protocol knowledge, instru-

ment use, eye-hand coordination) (2). Furthermore,

a competent surgeon must possess not only basic

technical skills, but also cognitive and clinical skills,

which require other assessment methods, e.g. multi-

ple choice questionnaires, assessment of patient

management problems, observation and assessment

by a skilled trainer, and video analysis (61–64).

Motion analysis alone fails to demonstrate the actual

surgical competence of the individual.

This study does not focus on the costs of the

presented tracking systems. However, the accep-

tance of a system strongly depends on its cost.

Currently, there is a great variation between costs of

the systems; the cheapest system costs around 1.000

Euro, while the most expensive systems cost more



than 50.000 Euro. Nevertheless, it is still very

difficult to compare the costs of the tracking systems,

since a number of these systems is at a prototype

stage, and the systems available on the market are

often sold with additional products (e.g. software, a

VR trainer). In order to make training and assess-

ment of the basic laparoscopic skills feasible in every

surgical education, it seems to be necessary to

reduce the costs of both training and assessment.

The introduction of new competitive tracking

systems and new technologies that can be used

in these systems will hopefully lead to decreased

costs.

The possibility of recording the movements of

standard laparoscopic instruments with tracking

systems is an important issue; residents must learn

how to use various laparoscopic instruments for

different purposes. They should also get a sense of

what they actually can feel when performing laparo-

scopic tasks. Standard laparoscopic instruments

provide exactly the same haptic feedback as during

surgery. Some of the current tracking systems for

virtual reality also offer force feedback. However,

feedback in these systems is still distant from the

feedback provided by the real laparoscopic instru-

ment during an operation. In order to understand

the way the sensory cues can be delivered or

simulated, the development and use of haptic

feedback in laparoscopy should be further investi-

gated. Since a high level of fidelity of sensory cues

can be very expensive and may not always be

advantageous, it is very important to investigate

when, how, and how accurate sensory cues should be

used during training of basic laparoscopic skills.

Conclusions

Tracking both the position and orientation of

minimally invasive surgical instruments is an impor-

tant challenge when creating a scoring system based

on motion analysis for the performance of trainees.

Although various approaches to this challenge have

been studied, none has come out as clearly superior;

each system has some advantages and some dis-

advantages. There is no tracking system that could

easily be used in all three environments: The

operating room, the pelvi trainer, and the virtual

reality trainer. A great number of systems do not

allow use of real laparoscopic instruments, and only

a small number of systems provide force feedback.

Additionally, the cost of the systems currently

present on the market is rather high. There is still

much work to be done in order to develop simple

and affordable tracking systems for the various

training environments.

References

1. Arregui ME, Fitzgibbons RJ, Kathouda N, McKernan JB,

et al. Principles of laparoscopic surgery: basic and advanced

techniques. Springer-Verlag: New York Inc.; 1995.

2. Dankelman J, Chmarra MK, Verdaasdonk EGG,

Stassen LPS, et al. Fundamental aspects of learning

minimally invasive surgical skills. Minim Invasiv Ther.

2005;14:247–56.

3. Aggarwal R, Moorthy K, Darzi A. Laparoscopic skills training

and assessment. Br J Surg. 2004;91:1549–58.

4. Gallagher AG, Lederman AB, McGlade K, Satava RM, et al.

Discriminative validity of the minimally invasive surgical

trainer in virtual reality (MIST-VR) using criteria levels based

on expert performance. Surg Endosc. 2004;18:660–5.

5. McClusky DAM, van Sickle K, Gallagher AG. Relationship

between motion analysis, time, accuracy, and errors during

performance of a laparoscopic suturing task on an augmented

reality simulator. 12th International Congress of the European

Association for Endoscopic Surgery, Barcelona, June 2004:

O207.

6. Munz Y, Kumar BD, Moorthy K, Bann S, et al. Laparoscopic

virtual reality and box trainers: is one superior to the other?

Surg Endosc. 2004;18:485–94.

7. Cotin S, Stylopoulos N, Ottensmeyer M, Neumann P, et al.

Metrics for Laparoscopic Skills Trainers: The Weakest Link!

LNCS. 2002;2488:35–43.

8. Darzi A, Smith S, Taffinder N. Assessing operative skill.

Needs to become more objective. BMJ. 1999;318:887–8.

9. Martin JA, Regehr G, Reznick R, MacRae H, et al. Objective

structured assessment of technical skills (OSATS) for surgical

residents. Brit J Surg. 1997;84:273–8.

10. Moorthy K, Munz Y, Sarker SK, Darzi A. Objective

assessment of technical skills in surgery. BMJ. 2003;327:

1032–7.

11. Van Sickle KR, McKlusky DA, Gallagher AG, Smith CD.

Construct validation of the ProMIS simulator using a novel

laparoscopic suturing task. Surg Endosc. 2005;19:1227–31.

12. Acosta E, Temkin B. Haptic laparoscopic skills trainer with

practical user evaluation metrics. Stud Health Techn Inform.

MMVR. 2005;111:8–11.

13. Cavallo F, Megali G, Sinigaglia S, Tonet O, et al. A

biomechanical analysis of surgeon’s gesture in a laparoscopic

virtual scenario. Stud Health Techn Inform. MMVR. 2006;

119:79–84.

14. Baratoff G, Blanksteen S. Tracking Devices. The

Encyclopedia of Virtual Environments, Human Interface

Technology Laboratory, University of Washington. http://

www.hitl.washington.edu/scivw/EVE/, 1993.

15. ‘‘Wikipedia, The Free Encyclopedia,’’ http://www.wikipedia.

org/.

16. Heijnsdijk EAM, Pasdeloup A, van der Pijl AJ, Dankelman J,

et al. The influence of force feedback and visual feedback in

grasping tissue laparoscopically. Surg Endosc. 2004;18:

980–5.

17. Bholat OS, Haluck RS, Kutz RH, Gorman PJ, et al. Defining

the role of haptic feedback in minimally invasive surgery. Stud

Health Technol Inform. 1999;62:62–6.

18. Bholat OS, Haluck RS, Murray WB, Gorman PJ, et al.

Tactile feedback is present during minimally invasive surgery.

J Am Coll Surg. 1999;189:349–55.

19. Lacey G. A Surgical Training Simulator. Patent Number

IE20030351.

20. van Sickle KR, McClusky DA, Gallagher AG, Smith CD.

Construct validation of the ProMIS simulator using a novel

laparoscopic suturing task. Surg Endosc. 2005;19:1227–31.



21. Sokollik C, Gross J, Buess G. New model for skills assessment

and training progress in minimally invasive surgery. Surg

Endosc. 2004;18:495–500.

22. Tatar F, Mollinger JR, Den Dulk RC, van Duyl WA, et al.

Ultrasonic sensor for measuring position and orientation of

laparoscopic instruments in minimally invasive surgery. 2nd

Annual International IEEE-EMBS Special Topic Conference

on Microtechnologies in Medicine & Biology, Madison,

Wisconsin USA, 2002.

23. Tatar F, Bastemeijer J, Mollinger JR, Bossche A. Two-

frequency method for measuring the position of surgical tools

with mm precision. Eurosensors XIX Conference, Barcelona,

Spain, 2005.

24. Rosenberg LB. Computer interface or control input device for

laparoscopic surgical instrument and other elongated

mechanical objects. Patent Number 5,623,582.

25. Jacobus CJ, Griffin JL. Method and system for simulating

medical procedures including virtual reality and control

method system for use therein. Patent Number 5,769,640.

26. Jacobus CJ, Riggs AJ, Taylor MJ. Force feedback system.

Patent Number 5,629,594.

27. Martin KM, Levin MD, Rosenberg LB. Mechanical interface

having multiple grounded actuators. Patent Number

5,828,197.

28. Rosenberg LB. Physically realistic computer simulation on

medical procedures. Patent Number 6,654,000.

29. Stylopoulos N, Cotin S, Dawson S, Ottensmeyer M, et al.

CELTS: a clinically-based computer enhanced laparoscopic

training system. Stud Health Technol Inform. 2003;94:

336–42.

30. Stylopoulos N, Cotin S, Maithel SK, Ottensmeyer M, et al.

Computer-enhanced laparoscopic training system (CELTS):

bridging the gap. Surg Endosc. 2004;18:782–9.

31. Hanna GB, Drew T, Clinch P, Hunter B, et al. A

microprocessor-controlled psychomotor tester for minimal

access surgery. Surg Endosc. 1996;10:965–9.

32. Hanna GB, Drew T, Clinch P, Hunter B, et al. Computer-

controlled endoscopic performance assessment system. Surg

Endosc. 1998;12:997–1000.

33. Demirtas D, Post JJ, Dankelman J. Joystick. Patent Number

WO2005088419.

34. Rosen J, Brown JD, Chang L, Barreca M, et al. The

BlueDRAGON – a system for measuring the kinematics

and the dynamics of minimally invasive surgical tools in-vivo.

Proc. IEEE Intl. Conf. on Robotics and Automation ICRA,

Washington, DC, 2002.

35. Rosen J, Brown JD, Barreca M, Chang L, et al. The Blue

DRAGON - A system for Monitoring the Kinematics and

Dynamics of Endoscopic Tools in Minimally Invasive Surgery

for Objective Laparoscopic Skill Assessment. Stud Health

Techn Inform. MMVR. 2002;85:412–8.

36. Polhemus, ‘‘Patriot brochure,’’ www.polhemus.com/

PATRIOT/PATRIOT_brochure.pdf.

37. Vecerina I, Zoethout J, Betrisay S. Device for determining the

longitudinal and angular position of a rotationally symme-

trical apparatus. Patent Number 20050075558.

38. Maass H, Chantier BBA, Cakmak HK, Kuhnapfel UG. How

to Add Force Feedback to a Surgery Simulator. LNCS.

2003;2673:165–74.

39. Kuhnapfel U, Cakmak HK, Chantier B, Maass H, et al.

HapticIO: Haptic Interface-Systems for Virtual-Reality

Training in Minimally-Invasive Surgery. Proceedings

Internationale Statustagung ‘‘Virtuelle und Erweiterte

Realitat, 2004.

40. Maass H, Cakmak HK, Kuhnapfel UG, Trantakis C, et al.

Providing More Possibilities for Haptic Devices in Surgery

Simulation. CARS, International Congress Series. 2005;

1281:725–9.

41. Chmarra MK, Bakker NH, Grimbergen CA, Dankelman J.

TrEndo, a device for tracking minimally invasive surgical

instruments in training set-ups. Sensor Actuat A-Phys.

2006;126:328–34.

42. Strom P, Kjellin A, Hedman L, Johnson E, et al. Training in

tasks with different visual-spatial components does not

improve virtual arthroscopy performance. Surg Endosc.

2004;18:115–20.

43. Smith CD, Farrell TM, McNatt SS, Metrveli RE. Assessing

laparoscopic manipulative skills. Am J Surg. 2001;181:

547–50.

44. Rosen J, Brown JD, Chang L, Sinanan M, et al. Generalized

Approach for Modeling Minimally Invasive Surgery as a

Stochastic Process Using a Discrete Markov Model.

IEEE Transactions on Biomedical Engineering. 2006;53:

399–413.

45. Chmarra MK, Jansen FW, Wentink M, Grimbergen CA, et al.

Performance during distinct phases in laparoscopic surgery.

10th World Congress of Endoscopic Surgery, 2006:112.

46. Schijven M, Jakimowicz J. Virtual reality surgical laparoscopic

simulators. How to choose. Surg Endosc. 2003;17:1943–50.

47. Halvorsen FH, Elle OJ, Fosse E. Simulators in surgery.

Minim Invasiv Ther. 2005;14:214–223.

48. Cosman PH, Cregan PC, Martin CJ, Cartmill JA. Virtual

reality simulators: Current status in acquisition and assess-

ment of surgical skills. ANZ J Surg. 2002;72:30–4.

49. Sutherland LM, Middleton PF, Anthony A, Hamdorf J, et al.

Surgical simulation. A systematical review. Ann Surg. 2006;

243:291–300.

50. Dunkin B, Adrales GL, Apelgren K, Mellinger JD. Surgical

simulation: a current review. Surg Endosc. 2007;21:357–66.

51. Massie TH, Salisbury JK. The PHANTOM Haptic Interface:

A Device for Probing Virtual Objects. Proceedings of the

ASME winter annual meeting, symposium on haptic inter-

faces for virtual environments and teleoperator systems,

Chicago, IL, 1994.

52. Datta V, Mackay S, Mandalia M, Darzi A. The use of

electromagnetic motion tracking analysis to objectively

measure open surgical skill in the laboratory-based model. J

Am Coll Surg. 2001;193:479–85.

53. Moorthy K, Munz Y, Dossis A, Bello F, et al. Motion analysis

in the training and assessment of minimally invasive surgery.

Min Invas Ther & Allied Technol. 2003;12:137–42.

54. Binder J, Kramer W. Point of technique. Robotically-assisted

laparoscopic radical prostatectomy. BJU International. 2001;

87:408–10.

55. Lanfranco AR, Castellanos AE, Desai JP, Meyers WC.

Robotic surgery. A current perspective. Ann Surg. 2004;239:

14–21.

56. Nawrat Z, Malota Z, Kostka P, Religa Z. The polish project

of telemanipulator for cardiac minimal invasive surgery with

preplanning and advisory system - assumptions and actual

state of art of RobIn Heart. 8th IEEE International

Conference on Methods and Models in Automation and

Robotics, Szczecin, Poland, 2002.

57. Podsedkowski L, Mianowski K, Wroblewski P, Nawrat Z.

Kinematic aspects of selected manipulators for cardiac

surgery. 8th IEEE International Conference on Methods

and Models in Automation and Robotics, Szczecin, Poland,

2002.

58. Rosen J, Lum M, Trimble D, Hannaford B, et al. Spherical

mechanism analysis of a surgical robot for minimally invasive

surgery – analytical and experimental approaches. MMVR.

2005;13:422–8.



59. Prasad SM, Prasad SM, Maniar HS, Chu C, et al. Surgical

Robotics: Impact of Motion Scaling on Task Performance.

Am J Surg. 2004;199:863–8.

60. Lin HC, Shafran I, Murphy TE, Okamura AM, et al.

Automatic Detection and Segmentation of Robot-Assisted

Surgical Motions. MICCAI, LNCS. 2005;3749:802–10.

61. Satava RM, Cuschieri A, Hamdorf J. Metrics for objective

assessment. Preliminary summary of the Surgical Skills

Workshop. Surg Endosc. 2003;17:220–6.

62. Satava RM, Gallagher AG, Pellegrini CA. Surgical compe-

tence and surgical proficiency: definitions, taxonomy, and

metrics. J Am Coll Surg. 2003;196:933–7.

63. Yule S, Flin R, Paterson-Brown S, Maran N. Non-technical

skills for surgeons in the operating room: A review of the

literature. Surgery. 2006;139:140–9.

64. Healey AN, Undre S, Vincent CA. Developing observational

measures of performance in surgical teams. Qual Saf Health

Care. 2004;13:33–40.



systems for tracking minimally invasive surgical instruments

Documents