MagGetzCustomizable Passive Tangible

Controllers on and around Conventional Mobile Devices

Presenter: S.J. HwangPh.D. Student | 6th Semester

MOTIVATION

Physical objects in real life provide users with intuitive tactile clues through its physical artifacts

MOTIVATION

MOTIVATION

Lacks of tactile clues

Occlusion caused by fingers

MOTIVATION

PREVIOUS WORKS

by adding other types of tangible objects

Researchers have attempt to increase this input space …

by expanding the interaction area

PREVIOUS WORKS

Tangible Interactions for Mobile Devices

PREVIOUS WORKSTangible Interactions for Mobile Devices: Using capacitive markers

CapStones and ZebraWidgets(Chan et al., CHI’12)

CapWidgets(Kratzet et al., CHIEA’11)

TUIC(Yu et al., CHI’11)

Tangible Remote Controllers(Jansen et al., CHI’ 12)

Cilp on Gadgets(Yu et al., UIST’11)

PREVIOUS WORKS

Portico(Avrahami et al., UIST’11)

Camera-based Tangible Controller (Kato, mobileHCI‘09)

Vision-based force sensor(Kato, mobileHCI’ 12)

Tangible Joysticks Physical tokens

Tangible Interactions for Mobile Devices: Using cameras

PREVIOUS WORKS

Technique (Author, Year) Sensor used (property) Limitations P/A(tangibles)

Cilp on Gadgets(Yu et al., 2011) Touchscreen (spatial) Confined within the device

Invisible when it is untouched Passive

TUIC(Yu et al., 2011)

Touchscreen(spatial, frequency)

Confined within the touchscreenOcclusion problem

Need battery (active)Invisible when untouched (passive)

Passive/ Active

CapWidgets(Kratz et al., 2011)

Touchscreen (spatial)Occlusion problem

Confined within the touchscreenInvisible when untouched

PassiveTangible Remote Controllers (Jansen et al., 2012)

CapStones(Chan et al., 2012)

Camera-based Tangible Controller (Kato et al., 2009)

Camera (marker) Controllers are fixed and limitedNeed light

PassiveVision based force sensor(Sato et al., 2012)

Portico(Avrahami et al., 2011) Camera (color) Bulky

Need light

Previous techniques to enable tangible interactions on mobile devices

PREVIOUS WORKS

Expanding the interaction area

PREVIOUS WORKS

Cheap; Small; Passive; Permanent;Detectable;

PREVIOUS WORKSMagnetically Driven Input for Mobile Devices

Abracadabra(Harrison et al., 2009)

MagiWrite(Ketabdar et al., 2010)

Digital Music Performance (Ketabdar et al., 2011)

Nenya(Ashbrook et al., 2011)

MagiTact(Ketabdar et al., 2011)

GaussSense(Liang et al., 2012)

PREVIOUS WORKS

Technique (Author, Year) Task Limitation

Abracadabra(Harrison et al., 2009)

accurate selections for small screens No tactile clue

MagiWrite(Ketabdar et al., 2010) writing system No tactile clue

Digital Music Performance(Ketabdar et al., 2011) musical performance No tactile clue

Nenya(Ashbrook et al., 2011) Menu selection Need additional mechanical hardware.

GaussSense(Liang et al., 2012) Pen interaction

Need additional mechanical hardware.(a board with 192 magnetic sensors and USB

connection).

Malfunctioning for ferromagnetic materials.

Magnetically Driven Input Techniques

OUR METHOD : MAGGETZ

+passive magnets

2. expand input area to avoid occlusions.

+a magnetometer

3. using a magnetometer that already installed on current mobile devices.

1. provide richer tactile clues.

Multiple control widgets

OUR METHOD : MAGGETZ

IMPLEMENTATION

IMPLEMENTATION

The type and location of a magnet for each widget

BACKGROUND

The location of a magnet and visualization of magnetic fields for each control widgets.

Push button

ToggleSwitch Slider Joystick Rotational

Wheel

OUR METHOD

Software

- We developed a MagGetz toolkit and 3D visualizer (OpenGL)on the Android 4.0.4. (SHV-E160S, Samsung Galaxy Note).

- The inertial three-axis magnetometer senses the magneticfield at a sampling frequency of 100 Hz.

- Used a fixed size buffer to reduce sampling noise.

This process generates a 3D vector that reflects thevariations of the magnetic field around the device.

OUR METHOD

Software – linear type

OUR METHOD

Software – linear type

Where p1 is a last reference point,p0 is a first reference point, and x’is a vector that new point xprojected onto p0 p1.

x’

OUR METHOD

Software – curved type

c

OUR METHOD

Software – curved type

where pi is ith reference point fromstarting point p0, k is a sequenceof the reference point closest to anew magnetic input point.

pk

OUR METHOD

Software – 2D plane type

center

East West

South

North

OUR METHOD

Software – 2D plane type

Where pc is a center point, pk is a vector that is closest to a new input point x among vectors between a center point pc and directional reference points (p0~pn), pk

n is the last point of vector pk

starting from pk0 (= pc), and x’k is a vector that new

point x projected onto pcpkn. The direction of a

joystick can be obtained by applying k of pk to equation

OUR METHOD

Software – multiple widgets

Button a Button b

OUR METHOD

Software – multiple widgets

Button a Button b

OUR METHOD

Software – multiple widgets

Button a Button b

OUR METHOD

Software – multiple widgets

Button a Button b

OUR METHOD

Software – multiple widgets

where x is a new magnetic point according to the user’s input, pa is a reference point of button a with maximum pressure, pb is a maximum point of button b, and p0 is a reference point of button a and b with minimum pressure.

OUR METHOD

Software – multiple widgets

Two buttons Three buttons Joystick + Button

APPLICATONS

LIMITATION

- The magnetism involved might damage the objects

- Occasionally reports a large offset in the data or a stuck pointing in one direction

However, magnets do not affect flash memories on modern mobile devices or IC chips on credit cards.

This technical glitch, however, will be undoubtedly solved as sensor technology improves.

CONCLUSION

- We have presented a MagGetz toolkit and various applications using it, which provide users with a wider interaction area, higher input expressiveness, and a customizable layout of richer tactile clues.

- MagGetz do not need power, a wireless connection, or the hardware modification of mobile devices.

- We believe that MagGetz toolkit can be successfully integrated for conventional mobile devices.

- For the future work, we plan to design different shapes and forms of input control widgets (e.g., deformable, stretchable, or elastic properties).

THANK YOU