Augmenting spatial

awareness with the

haptic radar

Augmenting spatial

awareness with the

haptic radar

I shikawa Namiki KomuroLaboratoryParallel Processing for Sensory informationI shikawa Namiki KomuroLaboratoryParallel Processing for Sensory informationI shikawa Namiki KomuroLaboratoryParallel Processing for Sensory informationI shikawa Namiki KomuroLaboratoryParallel Processing for Sensory information

Álvaro Cassinelli, Carson Reynolds & Masatoshi IshikawaThe University of Tokyo

Concept & Motivation Concept & Motivation

• Antennae, hairs and cilia precede eyes in evolutionary development

• Efficient for short-range spatial awareness (unambiguous, computationally inexpensive)

• Robust (insensitive to illumination & background)

• Easily configurable (hairs at strategic locations) and potentially omni-directional

+• Today’s MOEMS technology enables mass produced, tiny opto-mechanical devices...

Time to (re)grow antennas on people (and machines)?

An opto-mechanical hair? An opto-mechanical hair?

• Hair shaft is a steerable beam of light (a laser-radar).

• Local, range-to-tactile stimulation

• Active scanning of the surrounding:

– Proprioception-based

– Automatic sweeping of the surroundings to extract important features (inspired by animal whiskers’ motion, two-point touch technique, etc)

• Modular, but interconnected structure (artificial skin)

… but also Human-Machine interface technology: – “Hairy electronics”: versatile human- computer interface

– Sensitive spaces: human-aware “hairy” architecture

– Display: laser-based vectorial graphics, laser annotation on

surrounding (augmented reality, attentional cues)

Possible applicationsPossible applications

Augmented spatial awareness & sensing – Electronic Travel Aid for the visually impaired .

– Augmented spatial awareness for motorcycle drivers

and workers in hazardous environments.

– Collision avoidance (robotic limbs, vehicles, etc).

– Augmented sensing & tele-sensing (texture, speed).

Input…

… and output!

(*) “The Smart Laser Scanner”, SIGCHI 2005

Laser-based module: feasability*Laser-based module: feasability*

• MEMS galvano-mirrors

• Smart Laser Tracking (*) principle (can work as an antenna sweeping mode)

Concept (“hairy electronics”)…

… opto-mechanical implementation

( )

The haptic radar as a travel aidThe haptic radar as a travel aid

A few fundamental questions:

– New sensorial modality: how easy to appropriate? (Would it be like re-exercising an atrophied one?)

– reflex reaction to range-to-tactile translation?

– Is the brain capable of intuitive integration of data from eyes on the back, the front, the sides…?

Prototype characteristics and limitations:

– Configuration studied: headband with few modules.

– Limitation: non-mobile beam

– Two prototypes built: one without range-finders (simulated maze exploration), another with range-finders (but short range).

(a) Haptic Radar Simulator(a) Haptic Radar Simulator

• Six actuators & LED indicators

• No range-sensors (controlled virtual space)

• Adjustable horizon of visibility

• Perception modalities:• proximity• open-space

Q: How participants deal with 360 of spatial awareness without previous training?

Q: How participants deal with 360 of spatial awareness without previous training?

Simulator features

(a) Simulator demo(a) Simulator demo

Simulator discussionSimulator discussion

• orientation is rapidly lost => add compass?

• Interactive horizon of visibility is a necessary feature

• “proximity feel” mode disturbing if many actuators vibrate at the same time

=> compute center of gravity

• “open-space” perception mode interesting, but counterintuitive (needs

training).

• continuous range-to-vibration function not easy to interpret = > discretize

levels (3 or 4 levels).

• Too few actuators/sensors (annoying jumping effect)

• vibrators need to be calibrated to produce same perceived effect (motors

characteristics differ, as well as sensitivity on each site)

Prototype features:

• Six sensor & vibrators

• Non-steerable “hairs” (infrared sensors)

• Max range 80 cm (arm’s range)

(b) Prototype with sensors(b) Prototype with sensors

Q: Can participants avoid unseen object approaching from behind ?

(b) Collision avoidance demo(b) Collision avoidance demo

DiscussionDiscussion

Immediate problems & possible improvements– Range detection too short (1 meter max) [ next prototype will use

utrasound sensors (up to 6 meters), then laser rangefinders]

– Simultaneous stimulus confusing [ only one actuator active at any time, perhaps in the opposite direction (showing direction of clear path)]

– Low spatial resolution of actuators [ more vibrators / different actuators]

– Variable motor characteristics [ individual calibration]

– Range-to-tactile linear function too simplistic [ log scale / discrete]

– Effect of rotation is confusing in the simulator [ head tracking]

– Sense of direction is rapidly lost when there is no “reference background” [use “interactive horizon” technique & add compass cue]

Future Research DirectionsFuture Research Directions

• Ultrasound sensors (more range – up to 6 meters)

• MEMS based steerable laser beams (automatic sweeping)

• Evaluate other tactile actuators (skin stretch?) and tactile signals (ex: tactons).

• More compact MEMS device modules for more density

• Grid network of interconnected modules

• More comprehensive experiments:

– Can participants navigate through crowds?

– Can participants predict if an object will hit them?

– In the long term, is there habituation to vibration stimulus?

• Other interesting research directions:

– Study haptic radar for rehabilitation of hemi-negligent patients.

– Use the optical hair to write/annotate objects in the surrounding

Questions?Questions?

• Laser display

– Visual cues (attentional cueing, augmented reality).

– Screen-less display– Retinal display?

Proof-of-principle: Laser annotationProof-of-principle: Laser annotation

Expected (final) module performanceExpected (final) module performance

• Module size: roughly 3x2 cm2 including laser, micromirrors and microcontroller electronics.

• Sampling rate: kilohertz range

• Range measurement: using just intensity and modulated laser diode, up to one or two meters.

• Angular “sweeping” speed: depends on selected micromirror (ex: 500 rad/s for MEL-ARI devices).

• Power: to study (at least 200mW/module…)

Smart Laser Scanning principleSmart Laser Scanning principle

• laser excursion is intelligently confined to the area of interest

• Simplest laser trajectory for tracking: a circular laser “saccade”.

– Fast! (kHz range).