medical robotics virtual fixtures - uniroma1.itvenditt/didattica/mr/04_virtualfix.pdf · motion...

$: Medical Robotics Virtual Fixtures - uniroma1.itvenditt/didattica/mr/04_VirtualFix.pdf · motion along a curve in space; n= 2: motion on a plane . . . ) { de ne the projection operator$
Universita di Roma “La Sapienza”

Medical Robotics

Virtual Fixtures

Marilena Vendittelli

March 8, 2017

Allison M. Okamura • The Johns Hopkins University • Workshop on Computer-Integrated Surgery & Interventional Robotics • ICRA 2007

Haptic Assistance: Virtual Fixtures

Perceptual overlays designed

to enhance performance

• introduced in 1992 by Rosenberg (US Air Force) in the context of telerobotic manip-ulation to study “the beneficial effects of corrupting the link between operator andremote environment by introducing abstract perceptual information into the interfacecalled virtual fixtures”

• an alternative to force feedback from the environment providing useful physical con-straints for improving safety, accuracy, and speed of robot-assisted manipulation tasks

• can be applied to both cooperative manipulation and telemanipulation systems

• two main types

– Guidance Virtual Fixtures (GVFs) which assist the user in moving the manipulatoralong the desired paths or surfaces in the workspace

– Forbidden-Region Virtual Fixtures (FRVFs) which prevent the manipulator fromentering into the forbidden regions of the workspace

• force/motion relationships can be of either admittance or impedence type for bothcategories of VF (see later)

M. Vendittelli Medical Robotics (Universita di Roma “La Sapienza”) – Virtual Fixtures 2

admittance-type GVFs: V (s) = Y (s)F (s) (linear device)

• in general, the admittance Y (s) is chosen differently in different Cartesian directions

• by filtering the applied force components in the non-preferred directions, a passiveguidance is created along the preferred direction of motion

• varying response in non-preferred force components creates different guidance levels

– hard guidance: none or almost none of the non-preferred force component is per-mitted, leaving the user with no or little freedom to deviate from the preferred(planned) path

– soft guidance: give the user the freedom to move away from the path by allowingsome motion in the non-preferred directions

experiments in microsurgical environments with an admittance-controlled cooper-ative robot have demonstrated that operators will perform best in uncertain envi-ronments with a medium-level GVF because the VF planner is not perfect

impedance-type GVFs: F (s) = Z(s)V (s) (linear device)

• in general, the impedance Z(s) is chosen differently in different Cartesian directions

• act as potential fields, actively influencing the motion of the robotic manipulator


admittance-type FRVFs

• FRVFs are actually a subclass of GVFs for admittance-controlled cooperative robots(see the Acrobot control)

• in telemanipulation, we are principally concerned with the penetration of the slavemanipulator into the forbidden region; penetration of the master device into the cor-responding region of its workspace has no consequences

impedance-type FRVFs

• can be implemented on telemanipulators by overlaying a penalty-based virtual wall onthe existing telemanipulation controller

• it is possible to implement the virtual wall on either the master or the slave side (orsimultaneously on both); both have the effect of reducing movement of the slave intothe forbidden region

• each presents a different haptic experience for the user, depending on the underlyingtelemanipulation controller


Example 1: a simple linear system

control based on admittance and impedance type virtual fixtures for a mass m translatingon the plane under the action of the control force F and the guiding force FG (e.g., forcedue to the interaction with a surgeon) without friction

mF

FG

x, v

x1

x2

x = (x1, x2)T

v = (x1, x2)T

mv = F + FG

• admittance-based control

– from an admittance-type GVF

V (s) = Y (s)FG(s)

– choosing- the desired mass velocity V d(s) = V (s)

- the control input F = mvd +KD(vd − v) +KP(xd − x)

– the system will move according to the designed admittance Y (s)


Example 1: a simple linear system (cont’d)

mF

FG

x, v

x1

x2

x = (x1, x2)T

v = (x1, x2)T

mv = F + FG

• impedance-based control

– in this case the control F must be designed so that the mass m, under the actionof the guiding force FG, matches the behavior of the impedance model provided bythe impedance type GVF

FG(s) = Z(s)V (s)

– the impedance model may be characterized by a desired, apparent, mass md, desireddamping KD > 0 and stiffness KP > 0 with respect to a motion reference xd

md(v − vd) +KD(v − vd) +KP(x− xd) = FG

– this is obtained through the control

F =m

md

(vd +KD(vd − v) +KP(xd − x)) +

(m

md

− 1

)FG

where a measure of FG is not needed if md = m (compliance control)


control schemes

admittance-based

V (s) = Y (s)FG(s)

mF

FG

x, vlow level controller

admittance model

Vd

impedance-based

FG(s) = Z(s)V (s)

mF x, v

impedancemodel

Vd

md

FG

in both cases

m(v − vd) +KD(v − vd) +KP(x− xd) = FG

but with different design options and different implementations


Example 2: GVF-based control of cooperative systems for retinal surgery

• some retinal pathologies could benefit from the insertion of a needle into a vein on thesurface of the retina as a path for drug delivery; vein cannulation is a procedure whichis not safe to perform clinically today, but would have great clinical utility if it couldbe done reliably

• the outer diameter of the blood vessel is on the order of 100 µm (root mean squareamplitude of the tremor of a surgeon has been measured to be 182 µm!!!)

• the difficulties related to manipulation on a microscopic scale can be reduced with theuse of cooperative systems


• the John Hopkins University Steady-Hand Robot (SHR) was designed to cooperativelyshare control of a surgical tool with the surgeon while meeting the performance, ac-curacy, and safety requirements of microsurgery

needed DOFs in tool positioning for eye surgery

• approach: at least 3 DOFs• insertion: 3 DOFs (one translation, two

rotations)• retinal surgery: 4 DOFs (three rotations,

one translation)

in addition, the retinal surgery phase requirestool motions to be constrained by an insertionpoint ⇒ an RCM is needed


VF-based control of the SHR

• given sensed handle forces and torques f ∈ IR(6) exerted by the operator, a tool velocityscrew is computed as

v = Gf

the matrix G ∈ IR(6 × 6) determines the relative gains of the sensed forces, and thuspermits shaping of the motion response to force inputs

• the virtual RCM point is achieved by minimizing the cost function

||Jh(q)∆q −Gf ||subject to the constraint

||P cl + J cl(q)∆q − P o|| ≤ εP cl: a point on the robot tool closest to the virtual RCM P o; Jh(q), J cl(q): themanipulator Jacobians resolved respectively at the handle and at P cl


for illustration, two main types of GVFs are considered

Allison M. Okamura • The Johns Hopkins University • Workshop on Computer-Integrated Surgery & Interventional Robotics • ICRA 2007

Design of Guidance Virtual Fixtures

Two main types:

Combine to create

compound tasks

Bettini, et al. 2004 (JHU)

they can be combined for more complex tasks

GVF-based control of the first prototype of the SHR

• cartesian positioning device with “virtual contact” between the robot tool tip and theenvironment modeled as

v = c f

which is an isotropic admittance-type device

• a VF generalizes this model to include anisotropic admittances


• GVF on tool position

– given a 3 × n, 0 < n < 3, time-varying matrix δ(t) with column rank n whichrepresents the instantaneous “preferred” directions of motion for the tool tip (n = 1:motion along a curve in space; n = 2: motion on a plane . . . )

– define the projection operator Dδ ≡ δ(δTδ)−1δT on the subspace generated by δ

– decompose f

f δ ≡ Dδf f τ ≡ f − f δ⇒ fTτ f δ = 0

– the “virtual contact” model becomes

v = c(f δ + f τ)

– the introduction of a new admittance cτ ∈ [0,1] that attenuates the non-preferredcomponent of the force input f τ results in

v = c(f δ + cτf τ) = c(Dδ + cτ(I −Dδ))f = G(c, cτ , δ)f (1)

– the final control law is in the general form of an admittance control with a time-varying gain

– the value of cτ determines the robot stiffness in non-preferred directions

∗ cτ = 0⇒ hard VF

∗ 0 < cτ < 1⇒ soft VF

∗ cτ = 1⇒ isotropic admittance


• reference target: the desired path is the segment joining the current tool position tothe target point; the reference direction is computed as

δt(pa) = pt − pawith pa position of the tip in IR3

the output velocity is computed from (1) by setting δ = δt

• reference curve: given the parametric expression of the reference curve

p(s) ≡ (x(s) y(s) z(s))T , s ∈ [0,1]

– determined the point on the curve s(pa) which is closest to the “tool tip”

‖p(s(pa))− pa‖ = mins∈[0,1]

‖p(s)− pa‖

– the reference direction δp is given by the tangent to the curve in s(pa)

t(pa) =d

dsp(s)|s=s(p

a)

δp(pa) =t(pa)

‖t(pa)‖in this case (1) does not apply directly with δ = δp because an initial offset would notbe recovered ⇒ introduce a correction term for the cartesian error e(pa) = p(s(pa))−pathe new direction δc is defined as

δc(pa) = sign(f · δp(pa))δp(pa) + kde(pa)


• using the assumptions that δ is full rank and the human input force is continuous, theadmittance control law (1) is guaranteed to converge asymptotically

• in the reference target case, the rank constraint on δ is violated at the target (illconditioned near the target) ⇒ it is convenient to define a spherical switching S(ρ)with center at the target point and radius ρ: outside S(ρ) the path following is active,inside S(ρ) a different control scheme is activated

– fix the direction and use that reference direction as the basis for a VF

– switch to an autonomous motion (e.g., proportional control v = kc(pt − pa))

• the velocity discontinuities on the switching surface can be avoided by resorting to aVF-based acceleration control scheme

• it is useful for practical applications to extend this approach to confine the motionwithin volumes (tube around the reference path, cone with vertex at the referencepoint)


Bibliography

“The Use of Virtual Fixtures As Perceptual Overlays to Enhance Operator Performance inRemote Environments,” L. B. Rosenberg, Technical Report AL-TR-0089, USAF ArmstrongLaboratory, Wright-Patterson AFB OH, 1992

“Virtual fixtures: Perceptual tools for telerobotic manipulation,” L. B. Rosenberg, IEEEVirtual Reality Annual International Symposium, pp. 76–82, 1993.

“Vision-assisted control for manipulation using virtual fixtures,” A. Bettini, P. Marayong,S. Lang, A. M. Okamura, G. D. Hager, IEEE Transactions on Robotics, vol. 20, no. 6,pp. 953–966, 2004.

“Development and Application of a New Steady-Hand Manipulator for Retinal Surgery,”B. Mitchell, J. Koo, I. Iordachita, P. Kazanzides, A. Kapoor, J. Handa, G. Hager, R. Taylor,IEEE International Conference on Robotics and Automation, pp. 623–629, 2007.

“Spatial motion constraints using virtual fixtures generated by anatomy” M. Li, J. Koo,M. Ishii, R.H. Taylor, IEEE Transations on Robotics, vol. 23, no. 1, pp. 4–19, 2007.

“Surgical and interventional robotics: Part III,” G. D. Hager, A. M. Okamura, P. Kazanzides,L. L. Whitcomb, G. Fichtinger, R. H. Taylor, IEEE Robotics and Automation Magazine,vol. 15, no. 4, pp. 84–93, 2008.

Steady-Hand Eye robot web site:

https://ciis.lcsr.jhu.edu/dokuwiki/doku.php?id=research.eyerobots


medical robotics virtual fixtures - uniroma1.itvenditt/didattica/mr/04_virtualfix.pdf · motion...

Documents