national university of singapore dept of mechanical engineering singapore 119260 gintic institute of...

National University of SingaporeDept of Mechanical EngineeringSingapore 119260

Gintic Institute of Manufacturing Technology

1

Robotic Telepresence

Marcelo H Ang Jr, mpeangh@nus,edu.sg & Wong Hong Yang

Dept of Mechanical Engineering, National University of Singapore

Lim Ser Yong

Automation Division, Gintic Institute of Manufacturing Technology

5 Jan 2001



2

Background

• Robots as “super” tools for humans– vs complete autonomy

• Tasks unpleasant/unsuitable for humans– hazardous environments– tedious, low-level and high-level skills

• Tasks where robots can improve performance– better accuracy and repeatabilty



3

The Teleoperator



4



5

Challenges

• Telepresence– as if operator is at remote site– all five senses are available

• sight, touch, sound, smell, taste

– real-time feedback of sensation• via Internet?

• Robotic manipulator with human dexterity



6

• What are the limits in terms of tasks– that can be operated “telepresently”?



7

Technologies

Position tracking

3-D position tracking:Magnetic, Ultrasonic

or optical sensors

Visual feedback

Head-MountedDevices: LCD or CRT,

CAVE systems

Aural feedback 3-D sound systems

Touch feedback Pneumatic orvibrotactile systems

Force feedback Force reflectingexoskeletons, gloves



8

Haptic Devices*

*Phantom devices, Sensable Technologies



9



10

System Overview



11



12



13

Robot ArmHEM

HumanOperator

Master Environment

Slave Environment



14

Position Tracking using Polhemus Position Tracking using Polhemus FASTRAK Magnetic Position SensorFASTRAK Magnetic Position Sensor

FASTRAK receiver in

HMD

FASTRAK receiver on

velcro

FASTRAK transmitter



15

• 3 translation and 3 rotation data (Euler angles) are obtained (at receiver from transmitter).



16

Head-Eye Module (HEM, V 1)Head-Eye Module (HEM, V 1)• Separated into 2 parts: the Eye Module and the Head

Module.• Motion provided by servo motors, directly coupled to

drive each joint.• Rotation is provided in 2 orientation frame axes (x,y)

and 1 reference frame axis (Z)• Motion control provided by a 3 axes motion control

ISA bus PC card with user programming library



17

Eye Module

Leadscrew

Camera holder

Linear guides

“Carriage”



18

Head ModuleCoincidenceof rotating

axes



19

Robot (Slave) Arm and GripperRobot (Slave) Arm and Gripper• Movement is done in tool mode.

(Cartesian)• Orientation is defined by a series of 6

numbers (orientation and approach vectors)

• A data record of 9 numbers (3 position + 6 orientation values) is passed to the robot controller through RS232 interface from PC.

+x

+y

+z



20

System In OperationSystem In Operation



21

Robotic Telepresence System, V2



22



23

Telepresence with Haptics



24

SummarySummary• Successful integration of hardware and

software components. Only a short period of time is needed for operators to feel immersed and gain skill in operator the robot arm.

• Possible applications : The Armed Forces, Underwater dredging and salvage operations, Telesurgery, Education



25

Future Work

• Improved Head Eye Module– vergence control– compact

• Inherent delay in Internet



26

But are the existing Internet Protocols enough for communications between machines?• Remote Monitoring and Control

• Machines as Web Controllable Objects

• http? ftp? telnet?



27

RequirementsRequirements• a system for different machines to be controlled anda system for different machines to be controlled and monitored on different platforms monitored on different platforms

• a system that links up the various software packagesa system that links up the various software packages available now available now

• an open standardan open standard

• an object (component) oriented system, OOSan object (component) oriented system, OOS

Machines Accessed Via InternetMachines Accessed Via Internet

architecturearchitecture

Off the shelf components working together (Plug and Play for software and hardware interoperability)



28



SolutionSolution• 2 new layers between objects in the system2 new layers between objects in the system Machine Command Abstraction (MCA) and Machine Command Abstraction (MCA) and Message Communication Protocol (MCP) Message Communication Protocol (MCP)

• MCA bridges the differences between the functional languageMCA bridges the differences between the functional language of different machines (device independent) of different machines (device independent)

• MCP defines a standard way of communicating betweenMCP defines a standard way of communicating between different objects using text messages (platform independent) different objects using text messages (platform independent)

• Open Object Oriented StandardOpen Object Oriented Standard



29



Parallel/AnalogyParallel/Analogy• A MAVI compliant system is just like an FTP systemA MAVI compliant system is just like an FTP system

• Any FTP client can connect to any FTP server and send Any FTP client can connect to any FTP server and send commands and receive feedbacks from the servercommands and receive feedbacks from the server

• Similarly, any device connected to a MAVI compliant server Similarly, any device connected to a MAVI compliant server can be controlled and monitored by any MAVI clientcan be controlled and monitored by any MAVI client

• User Interface will be independent of device and platformUser Interface will be independent of device and platform

• Can be expanded for data transfer (example between smart Can be expanded for data transfer (example between smart agents or feedback sensors)agents or feedback sensors)



30



MCAMCA• The Machine Command Abstraction translate different The Machine Command Abstraction translate different machine commands that are vendor specific to a standard set of machine commands that are vendor specific to a standard set of commandscommands

• Free programmers from having to rewrite their User Interface Free programmers from having to rewrite their User Interface for different devices (one UI many devices)for different devices (one UI many devices)

• Allows different UIs to be used on a single device (one device Allows different UIs to be used on a single device (one device many UIs)many UIs)



31

MCA - example commands for WCOs• On/Off

• axis label 5 m

• power label 90%

• start/stop

• get status

• etc.



32



MCPMCP

• The Message Communication Protocol defines a standard The Message Communication Protocol defines a standard language to be used in the TCP/IP pipe language to be used in the TCP/IP pipe

• Follows the widely accepted XML formatFollows the widely accepted XML format

• By formatting all communication messages in the TCP/IP pipe in By formatting all communication messages in the TCP/IP pipe in textual XML format, it makes it easy to understand and textual XML format, it makes it easy to understand and implement, platform and programming language independentimplement, platform and programming language independent

• Secondary communication pipes can Secondary communication pipes can be established in real time through be established in real time through MCP for binary communication and MCP for binary communication and encrypted messagesencrypted messages



33

MAVI Server

An ExampleAn Example

InterfaceInterfaceDriverDriver

UserUserInterfaceInterface

Vendor BVendor BDeviceDevice

Vendor AVendor ADeviceDevice

MachineMachineDriverDriver

MachineMachineDriverDriver

MAVI Client

MAVI ServerMCP

MCP

MCA

MCA



34



FeaturesFeatures• Designed for the Internet and Intranet.Designed for the Internet and Intranet.

• Based on TCP/IPBased on TCP/IP

• Simple but expandable and scalableSimple but expandable and scalable

• Independent ofIndependent of

• Machine VendorsMachine Vendors

• Operating SystemOperating System

• Programming LanguageProgramming Language



35

With MAVI…..

• Plug and Play WCO s

• Pervasive

• Home/Factory Use

• No programming experience

• Vendor/Hardware/Software independent



36



ScenariosScenarios• User designs a prototype at home. Uses a shareware MAVI User designs a prototype at home. Uses a shareware MAVI client to log into an Internet Rapid Prototyping shop. Remotely client to log into an Internet Rapid Prototyping shop. Remotely control, monitor and produces his prototype. Product delivered control, monitor and produces his prototype. Product delivered to him.to him.

• Factory supervisor on leave. Uses the Internet and a MAVI Factory supervisor on leave. Uses the Internet and a MAVI client to log into his factory machines while overseas. Check on client to log into his factory machines while overseas. Check on the machine up and down time and number of products the machine up and down time and number of products produced.produced.



37

Case Study 1 - TelerobotCase Study 1 - Telerobot

http://telemfg.eng.nus.edu.sgFeatures:Features:

• Control and monitor robot from anywhere in the worldControl and monitor robot from anywhere in the world

• Virtual Model for faster feedbackVirtual Model for faster feedback

• Easily adapted to any other robotEasily adapted to any other robot

• User Interface can be changed easilyUser Interface can be changed easily example force feedback joystick example force feedback joystick

• Additional components can be upgraded or addedAdditional components can be upgraded or added examples Security, Encryption, Image Processing, examples Security, Encryption, Image Processing, Object Avoidance etc Object Avoidance etc



38



39

Case Study 2 – i2hCase Study 2 – i2hhttp://mavi.mpe.nus.edu.sg/http://mavi.mpe.nus.edu.sg/

Internet Integrated HomeInternet Integrated Home

• Joint project with NCB to connect the Singapore One HomeJoint project with NCB to connect the Singapore One Home to the Internet (completed in 2000) to the Internet (completed in 2000)

• Objective – To implement the control and monitoring of homeObjective – To implement the control and monitoring of home devices via the Internet. The implementation must be simple devices via the Internet. The implementation must be simple and secured so to encourage market adoption. and secured so to encourage market adoption.

• Phase 1 – Implementation of some Web Controllable Objects,Phase 1 – Implementation of some Web Controllable Objects, (WCO) like Pan/Tilt WebCam and Table Lamps. (WCO) like Pan/Tilt WebCam and Table Lamps.



40



41

Other work…..• Jini

– architecture based on Java for federating services in a distributed system

– requires Java virtual machine, language dependent

• CORBA– Common Object Request Broker defines an Interface Definition

Language (IDL) to create interfaces to objects– Requires language dependent ORB to work

• Microsoft’s (UPnP) Universal Plug N Play (newest)– Windows software in any device….

national university of singapore dept of mechanical engineering singapore 119260 gintic institute of...

Documents