Multi-Modality Communication Systems for Robotic Telesurgery

Syed Saqib Hussain Shah 1 and Jackrit Suthakorn

2

1, 2 Center for Biomedical and Robotics Technology (BART LAB), Department of Biomedical Engineering,

Faculty of Engineering, Mahidol University.

Abstract. Remote surgery combines elements of robotics, cutting edge communication technology such as

high-speed data connections and elements of management information systems. Several network and systems

option provide different Quality of Service (QoS) depending on several limitations and difficulties, such as,

transmission delays, bandwidth constraint, and connection rupture packet lost. In this paper an attempt has

been made to configure the best possible mode of communication to mock-up system which provides faster

data transmission with minimum time latency. Here, a concept of tele-surgery via real-time multi-modality

telecommunication networks and systems is presented. To test the effectivity of the proposed systems LAN,

WAN, the 3rd generation network and broad band communication system while short distance and long

distance internet communications test were examined, as result longer distance face little discomfort whereas

short distance does not have any such issues. The survey was conducted in consultation with

anesthesiologists and surgeons at Ramathibodi hospital, Thailand.

Keywords: Telesurgery, Latency, Robotics assisted surgery, Telecommunication surgery, Internet based

surgery.

1. Introduction

In today’s globalised technological scenario one is able to get the whole world under one’s grip by a

single click of the mouse. Invention of computers has made everything possible. Today we find its use in

almost all the fields changing the humans’ life style whether it at work force or personal. One such area in

the official is the field of medicine. Technology has evolved the communication from mechanical to

electronic broadcast using optic wires, radio signals and the internet making the world into a close knit

family. Tele-surgery or Remote surgery is the performance of robotics over a distance with a communication

link enabling the extension of the surgeon’s surgical expertise to rural area community cutting down the

transport expenses. The telesurgery conducted is a composition of a master and slave where the master site is

operated by human and slave site follows it. Zeus Robotic system which consisted of two

telecommunications computers are the first one to be introduced in this area where each computer is

connected to local site. The advantages of this system was that it had a robust error checking and fail safe

performance of standard Zeus even while operating in a tele-operative mode. Further it could be easily

switched into a tele-surgical Zeus-TC [1].

The first remote human telesurgery was successfully accomplished by operation Lindberg where the

operation surgeon was shifted to a distant location from the surgical room to perform a minimally invasive

surgery on a patient. The integrated telecommunication system enabled the surgeons from the master site to

collaborate with assistant surgeons from slave site. Telephones were used as voice communication, fax

machines as enable written communication, video call enabling visual interaction and robotics enabling

physical interaction over a distance [2]. The first telesurgery was performed by the National Aeronautics and

Space Administration in 1972 and since then we could witness a lot of development. M.Anvari.et.al. [3]

Corresponding author. Tel.: +66-2-889-2138 Ext 6446; fax: +662-441-4254.

E-mail address: jackrit.sut@mahidol.ac.th.

International Proceedings of Chemical, Biological and Environmental Engineering, Vol. 89 (2015)

DOI: 10.7763/IPCBEE. 2015. V89. 12

72

explains the importance of communication for tele-surgery using commercial fiber optics. Tele-surgery using

research based laparoscopic robot was developed and brought to application by S.S.Sastry.et.al. [4]. .

Expert Site/Master Site Surgical Site/Slave Site

Communication

Fig. 1: BART Lab Telesurgical system

Because of telesurgery’s limitations such as absence of haptic information from the target area, use of

two dimensional video images by surgeons to observe patient and time delay on telecommunication and lack

of quality service due to internet to establish robotic technology as a standard treatment becomes difficult.

Hence Communication plays a very important role here [5]. Though it has many advantages it has

disadvantages too. Transmitting time delays, bandwidth constraint and connection rupture packet loss are its

major challenges. This paper discusses the advantages in general and the major challenges in specific and

tries to overcome the challenges and provide a solution for it. Various studies have shown that data loss is a

main challenge to determine the quality of service in time latency sensitive multimedia application is quite

challenging [5]. Understanding that the internet is the best way for transmitting data between human system

interface and the remote teleoperation [6] at the same time determining the quality of service in time latency

sensitive multimedia application is quite challenging an attempt has been made to configure the best possible

mode of communication to mock-up system which provides faster data transmission with minimum time

latency. Based on BART Lab Telesurgical system as shown in Fig. 1 [7] a setup of a mock up master and

slave system was established [8], [9]. Initially a pick and pace task was conducted. Two commercial robots

namely Phantom Omni and Motoman HP3NX 100 were connected to evaluate the multi-modality

communication system. To test the effectivity of the multi-modality communication systems LAN,

WAN ,the 3rd generation network and broad band communication system while short distance and long

distance internet communications were examined.

This paper is organized as follows. In Section 2, the workspace analysis were sensible phantom Omini

workspace and motorman HPS workspace were explained. Further, in this section, the control algorithm for

the position control is also explained. Then, the effect of multimodality system was analysed experimentally

and the results are discussed in Section 3 and 4, this is the main contribution of this paper. Finally, this paper

is summarized in Section 5.

z

y

xϴ4

{BASE}

z

y

xϴ1

z

y

xϴ2

z

y

xϴ3

z

y

xϴ5

z

y

xϴ6

{EE}

H1

'1

H2

'2

H3

'3

H4

'4

H5

'5

H6

'6

Fig. 2: Phantom Omni haptic device Fig. 3: Phantom Omni MATLAB workspace

2. System Modelling Workspace Analysis

73

2.1. Sensable Phantom Omni Workspace

For the Tele-surgery system, sensible phantom Omni was used as the master site surgical console. The

workspace and frame relation between each joints of Omni are shown in Fig. 2 and Fig. 3.

Fig. 3 Show the relationship between homogeneous transformation matrix of HEEBASE . It is computed to

show the position and orientation of the end effector with respect to the base frame as (1).

THTHTHTHTHTHHHEEBASE

1

1

1

1

1

1

1

1

1

1

1

167

6

6

56

5

5

45

4

4

34

3

3

23

2

2

12

1

1

17 (1)

2.2. Motoman HP 3 Workspace

For our experiment, we used Motoman HP3RX 100 as a surgical Robot. Fig. 4. Shows the relationship

of frame of motions for each joint and the workspace measurements were well explained in Fig. 5. The

equation of motion for the Motoman HP 3 is shown in equation (1). It shows the position and orientation of

the end effector with respect to the base frame. Fig. 6 shows the mock-up system for teleoperation using

haptic device as master and motoman HP 3 manipulator as slave.

x

Y

z

{EE}

ϴ3

ϴ1

ϴ2

ϴ4

ϴ5

ϴ6

L1

L2

L3L4

L5

{BASE}

x

Y

z

x

Y

z

x

Y

z

x

Y

z

x

Y

z

H1

'1

H2

'2

H3

'3

H4

'4

H5

'5

H6

'6

Fig. 4: Kinametics of Motoman HP3 RX 100 Fig. 5: Motoman HP3 RX 100 workspace

Fig. 6: Mock up system for Tele-Communication

2.3. Workspace Comparison with Control Algorithm

For comparison between Motoman and Omni workspace, we used position to position command

algorithm, as shown in Table 1.

Table 1: Comparison of Motoman and Omni workspace (Numbers show in degree) Degree of

Freedom

Omni Motoman

#1st DOF ±55 ±170

#2nd DOF +3/+104 -45/+150

#3rd DOF +49/+103 -142/+210

#4th DOF ±138 ±190

#5th DOF ± 65 ±125

#6th DOF ± 146 ± 360

In the experiment, author have used position to position command equation, as shown in (2)

74

Here, master robot workspace position is given by [Ym Zm Xm] where, K is the diagonal matrix (Ky KZ Kx)

of constant scaling terms, and [XS YS ZS] is the slave robot work space position. For Tele-operation system,

slave robot follows the master robot, which is described by the equation (3).

kuP (3)

where P is the Position Vector [XS YS ZS]T; k is position vector of K matrix and u is position vector of [YS ZS

XS]T

3. Experimental Setup

Multi-modality Communication Test was conducted to show the importance of modality and how it

would affect the Transfer of data during telesurgey and the experimental platform which is shown in Fig. 7.

Motoman NX 100 arm robot was used as a slave located in where Mahidol under the control of the Master

robot located in Ramathibodi Hospital. A pick and place task test was performed in order to check the

precision and accuracy of the work done by robots where surgeons were asked to move the five colour

blocks .i.e., red, black and white bocks from position A to position B and then again set it back to its original

position as shown in Fig.8.

This was designed as to hone the ability of the surgeon to understand motor movements in two

dimensions .The data was sent as surgical data expert surgeon visual data and voice communication from the

master site while the slave site included surgical robot data, endoscope camera, patient and room over view,

voice communication date and anesthesiologist data as shown in Fig. 9. As the performance and the success

of telesurgery depends on the well classified communication. LAN, WAN, Third generation Network and

broadband Internet communication systems a multi -modality communication test was conducted as

explained in Fig. 10.

The master site was set up at Ramathibodi Hospital, Dept. of Neurology and 24 km away from the slave

site in the BART Lab, Faculty of Engineering, Mahidol University, Salaya. Testing multi-modality

communication system is one of most important factor, for this method where we can find out each modality

bandwidth and latency on each modality, while understanding each modality bandwidth and latency on the

system, while performing surgery author get tested CISCO VPN Switcher, which apply two different

communication system on network, while one communication is fail to perform CISCO switch from backup

line to not let it fail the system, switching one line to another this CISCO required between 2-5 second, Fig.

11. shows the backup system for both expert site and surgical site.

Master Site Slave Site

Multi-Modality

Communication

Fig. 7: Experimental Platform

4. Results and Discussion

Five modality tests were done on the same systems where WIRESHARK the commercial network

analyzer software was used to capture network ability while performing telesurgey shown in Fig. 12. Each

data was tested 5 times between 10-30 minutes. Significant difference between each modality revealed. i.e.

between the LAN and Wireless LAN setup where master and slave sites are connected to each other in the

same network, Wireless LAN found an average delay of 5ms which is a smaller difference than the LAN

network. Both networks experience unnoticeable delay while controlling the robot. While using the other 3

modalities the results reveal much higher latency in each network. The five modalities LAN, Wireless LAN,

m

m

m

x

z

y

s

s

s

X

Z

Y

K

K

K

Z

Y

X

00

00

00

(2)

75

Internet, 3G and long distance were tested in the same rook, whereas long distance tests were performed

between a Master and Slave site, which were 24 km apart.

Fig. 8: Slave side pick and place task Fig. 9: Multi-modality communication system

Fig. 10: Multi-modality communication system individual communication network

76

For communication in short distance again trails where conducted for this average time was 700

milliseconds. Maximum time recorded for task was 4.2 seconds and minimum time recorded was 120

milliseconds. In the Fig. 13 and Fig. 14 it shows that some of the experimental subjects took little long time

to understand 2D direction movements of the experimental task which results in the increase of average time

little more than expected. Anvari et.al [3] uses the fibre optic communication system in between Master and

slave side and expensive method too.

0

0.1

0.2

0.3

0.4

0.5

0.6

Ist 2nd 3rd 4th 5th

Tim

e D

ura

tio

n (

In M

illis

eco

nd

)

Number of Test of each modality

Communication Modality

LAN

WiFi

Internet

3G

LD

Fig. 11: Communication backup system for safety Fig. 12: Different modality of communication

0

0.5

1

1.5

2

2.5

3

3.5

4

0 2 4 6 8 10 12 14 16 18 20

Tim

e D

ura

tio

n in

Min

ute

Number of Trial by different Users

Pick and Place Task at Long Distance

0

0.5

1

1.5

2

2.5

3

3.5

4

0 2 4 6 8 10 12 14 16 18 20

Tim

e D

ura

tio

n in

Min

ute

Number of Trial by different Users

Pick and Place Task at Short Distance

Fig. 13: Long Distance Pick and Place object Test Fig. 14: Short Distance Pick and Place Test

5. Conclusion

We understand that invention of telesurgery is a boon to all those whole lot who are not in reach of

medical treatment. We understand that only when the doctors are in coordination with the robot with the help

of Internet the telesurgery can be successful. The Graphic User Interface designed here to display vital and

other physiological science for anaesthesiologist monitoring is the first of its kind in the field of telesurgery

itself. The impact of this field is very high because of its real time patient monitoring during telesurgery as it

not only improves the quality of surgical procedure but also helps to reduce any possibility of human error.

From the above experiment we can infer that telesurgery can be performed on various modalities of

communication like the 3G and Broadband Internet even if the modality fails surgery need not be a familiar

as there is a possibility of the control to switch to another modality without any impediments are losses of

data. When compared long distance and short distance performance we notice that longer distance face little

discomfort whereas short distance does not have any such issues. Success depends on the training and

adaptation to the handling of the instruments before performing any real time telesurgery no matter from

whatever distances the surgeon performs robotic surgery.

6. Acknowledgement

77

This project is supported by the National Research University Grant through Mahidol University. The

first author would like to thank Ramathibodi Hospital’s surgical staff. The first author would like to thank

Mr. Sakol Nakdhamabhorn for his great help of tele-communication control, Ms. Nantida Nillahoot for her

great help of unilateral tele-operation and also his fellow BART LAB members for their technical support.

7. References

[1] Taylor, R.H., Stoianovici, D., “Medical Robotics in Computer-Integrated Surgery,” IEEE Trans. on Robotics and

Automation, 19(5), Oct. 2003.

[2] Marscausx J., Leroy, J., Rubino, F., Smith, M., Vix, M., Simone, M., Mutter, D., “Transcontinental Robot-

assisted Remote Tele-surgery: Feasibility and Potential applications” Ann. Surg., 235(4), 2002, pp. 487-492.

[3] Anvari, M., McKinley, C., Sein, H., “Establishment of the World’s First Telerobotic Remote Surgical Service for

provision of Advanced Laparoscopic Surgery in a Rural Community” Ann. Surg., 241(3), March 2005.

[4] Sastry, S. S., et al., “A Laparoscopic Telesurgical Workstation”, IEEE Transactions on Robotics and Automation,

Vol. 15, No. 4, August 1999.

[5] Moustris, G., et al., Evolution of autonomous and semi‐autonomous robotic surgical systems: a review of the

literature. The International Journal of Medical Robotics and Computer Assisted Surgery, 2011. 7(4): p. 375-392.

[6] Kang, Y., et al., "Control Design for Tele-Operation System with Time-Varying and Stochastic Communication

Delay" International Journal of Innovative Computing, Information and Control, Volume 8, Number 1(A), January

2012.

[7] Suthakorn, J., “A Concept on Cooperative Tele-Surgical System Based on Image Guiding and Robotic

Technology,” Proc. 2012 Pan American Health Care Exchange (PAHCE 2012), USA, March 2012.

[8] Itsarachaiyot, Y., Pochanakorn, R., Nillahoot, N., Suthakorn, J., “Force Acquisition on Surgical Instruments for

Virtual Reality Surgical Training System,” Proc. 2011 Int’l Con. on Computer Control and Automation (ICCCA

2011), Korea, May 1-3, 2011.

[9] Direkwatana, C., Suthakorn, J., “On the Design and Development of A Novel 4-DOF Wire-Driven Laparoscopic

Surgical Robotic System, “MU-LAPAROBOT”,” Proc. The 7th Asian Conference on Computer-Aided Surgery

(ACCAS 2011), Bangkok, Thailand, August 26-27, 2011.

78