integration of internet and real time control systems in telerobotics

Integration of Internet and real time control systems

in telerobotics

Introduction

Many devices are today accessible through Internet:

Web cams

Washing machines, coffee pots and other appliances

Robots

The term “control” is referred to the possibility of sending a list of commands to the remote device

Introduction

Note:

The control loop is not closed across the network

The remote equipment executes the commands under the supervision of its own local controller

Why?

The unpredictable performance does not allow the realization of an Internet-based, real-time control system, in which the control loop is closed across the network

Introduction

Question:

If the characteristics of the Internet connection in use are known, is it possible to implement a real-time, closed-loop control system, integrating that connection?

Answer:

JBIT – Java Based Interface for Telerobotics: a telerobotics equipment for real-time teleoperation over Internet, with visual and force feedback

Introduction

Local system

Internet

InternetRemotesystem

Internet

Closing the loop across Internet…

Summary

Internet modeling

Delays, losses and bandwidth

Control of dynamic systems with variable time-delay

Solutions available in literature

Packet loss handling

The JBIT project

Internet modeling

Internet can be considered as a strongly connected network of computers, communicating with each other using packet switched protocols

A

B

DC

To B

To D

To B

To BTo B

Internet modeling

From source to destination, each packet traverses several nodes

Each node has different throughput, routing policy, and buffering and queues management

Each node handles many other data flows, this resulting in random service time

Internet modeling

When the data flow between end users exceeds the available bandwidth, congestion occurs

The available (or marginal) bandwidth for a single user depends on the data flow generated by other users, sharing the same physical connection

Internet modeling

End-to-end connection modeled by:Average delayAvailable throughputDelay statistics/variationsPacket loss statistics

Simple experiments to get the modelICMP packet injection and Round Trip Time (RTT) measurement

Internet modeling

RTT vs. Time:

Short term Long Term (1 week)

Internet modeling

RTT distribution:Depends on the number of nodes traversed:

10000 km connection30 km connection

1 arianna.dei.unipd.it 2 dei-cisc.dei.unipd.it 3 147.162.6.254 4 pd4000.unipd.it 5 cnaf-gw-2.infn.it 6 cnaf-gw1.infn.it 7 131.154.8.1 8 mix-serial3-4.Washington.mci.net 9 core1-fddi-0.Washington.mci.net 10 border1-fddi-0.Bloomington.mci.net 11 border1-fddi-0.Bloomington.mci.net 12 csunet-losnettos.Bloomington.mci.net 13 ISI-SWRL-GW.LN.NET 14 acg-isi.ln.net 15 jpl-acg.ln.net 16 192.138.85.65 17 b198-fddi.jpl.nasa.gov 18 helios.jpl.nasa.gov

1 arianna.dei.unipd.it 2 dei-cisc.dei.unipd.it 3 147.162.6.254 4 pd4000.unipd.it 5 147.162.5.254 6 berica2.gest.unipd.it

17 hop

s5

hops

Internet modeling

RTT distribution:

Exponential-like Gaussian-like

30 km10000 km

Internet modeling

RTT phase plots (RTTn+1 vs. RTTn)

Non-congested Congested

rttn+k+1 = rttn+k + P/ -

Note: P is the packet length, is the channel throughput, is the period of the probing packet

Internet modeling

Available bandwidth can be obtained by injecting a data flow that brings the connection close to saturation.

The closer to saturation, the higher the packet loss rate

Host name

Average RTT [msec]

Std. Dev. [msec]

Packet Loss rate

Berica 10 8.44 6.25 5.97 Helios 10 319.0 16.70 51.13 Berica 100 8.10 5.35 0.08 Helios 100 326.3 27.20 41.36

Internet modeling - Summary

Available bandwidth, average delay, delay deviation and loss rate can be easily obtained with simple injection of probing packets

Saturation can be avoided by setting the upper limit of the application throughput below the available BW

Control over Internet

Local system

Variable delay 2

Variable delay 1

Remotesystem

Internet

Closing the loop across Internet


Few solutions available in literature for time-varying delay

All algorithms are designed on the basis of maximum delay and its max. derivative

Structural constraints:With Internet, short interruptions may occur Decentralized control is preferredIn case of interruptions, it must be allowed to have reduced perfomance


Other solution: provide a pair of queues to even out the delay jitter

In this case, all constant-delay techniques can be applied (e.g. Smith predictor)


Which protocol is best suited for Internet-based control?

TCP: Acknowledge mechanism interrupts control loop

RTP: Oriented to audio-video streaming

UDP: No detection of missing packets

Enhanced UDP:

Packet loss detection and time stamping


Other problem: random packet losses

Losses may be single or burst (more than one packet lost sequentially)

Burst losses occur if the data rate is over the maximum available BW

Data flow must be adapted to operating conditions

Monitoring of available bandwidth through RTT on-line measurement


Single losses can be handled with a predictor

The prediction is used instead of the missing data

This works in conjunction with the buffering mechanism and a packet loss detection procedure

Based on E-UDP features

Control over Internet - Summary

Not all applications are suitable for IP-based control

The application must “survive” even in case of long interruptions:

Decentralized control must be allowed

Monitoring of connection characteristics and adaptation of the controller to operating conditions

Data recovery must be provided

Case study: Telerobotics

Bi-directional data exchange

Case study: Telerobotics

This application is well suited for IP-based control:

Decentralized structureSeveral control algorithms for constant time delay are available

s

Telerobotics at the Industrial Electronics Lab.

Research started in 1995Biorobotics Lab. – UW – SeattleTelerobotics Group – Jet Propulsion LaboratoryLVR - Università di Orleans-Bourges

Design and realization of an Internet-based telerobotic equipment with force feedback

Enanched-UDPJBITP@dus - OTELO

Web-based telerobotic systems

Several systems are available

None allow real time feedback and control

CGI interface and script-like commands

Connection is open when the query is sent and closed upon reply arrival

no continuous video streaming

e-mail or updated environment picture as feedback

JBIT: Java-Based Interface for Telerobotics

Targets:

to enable any Internet user to access a remote robot, with real time video, VR and (possibly) force feedback

low cost (overall!)

no special tools (SW & HW)

JBIT solutions - 1Cost:

Freeware

Low cost camera (Quickcam)

Commercial force display (Microsoft Side Winder Pro) as master

Direct-drive, low cost robot as slave (cannibalized from HDDs)

Sensorless force feedback

JBIT solutions - 2

Real-time visual and force feedback:

Java servlets for fast server response

compressed video streaming (H.263) for video

VR-based interface to improve the visual feedback performance

coordinating force feedback + queuing + data recovery to cope with IP delays and losses

Structure of JBIT system

JBIT: Client-server structure

Server (implemented with Java servlets)

Waiting for client requestsAccess control, timeout Data exchange between client and robot (no direct access form client to robot)Video encoding (H.263) and streamingMonitoring of available bandwidthAdaptation of video and control output rate


Client (implements user interface)Communication with servletsSelection of operating modeRadio button and active joystick interfaceVR interfaceVideo decoding and playing (Java Media Framework - JMF)


Client interface

JBIT - Force feedback

•Up to 200 Hz sampling rate on a LAN, 50 Hz with a 14.4 kbps modem

JBIT - Force feedbackPerception of the remote environment

Depends on the connection delay

Remote tracking of a square object

Conclusions

The realization of an IP-based closed loop control system is not a trivial task.

Precise knowledge of the connection’s characteristics is needed

Such characteristics must be constantly monitored

Reconstruction of lost data should be ensured

Need for some smart strategies to deal with long term interruptions

Few applications are suitable for IP-based control

Several issues still to be addressed in order to achieve a general solution for IP-based control

Conclusions

Some help may arrive from new network technologies:

Hard real-time connections (e.g. Tenet) New version of Internet (IPv6)RSVP (Resource Reservation Protocol)New video compression (MPEG4)

It will be possible to have connections with prescribed Quality of Service (QoS), in terms of throughput, losses and delays

Reliable control of remote equipment using Internet could be implemented more easily.

Conclusions

On-going project: P@dus

COMMUNICATIONCHANNEL

PATIENT STATION (SLAVE)

EXPERT STATION (MASTER)

HAPTIC MASTEREXPERT PC

ECHO PROBE

PCECHOGRAPHSLAVEPATIENT

Tele-echographies

integration of internet and real time control systems in telerobotics

Documents