Team Sekkeisha Presents:The

Hybrid Motion Controller

Bringing the core to Kinect

Full Sail University Brian Palmgren - Lawrence Trask - John Vecchitto - Ryan Young

Abstract: The Hybrid Motion Controller (HMC) is a triage of wireless controllers designed to work in conjunction with the Xbox 360 Kinect. Designed to bring precision and responsiveness to the Kinect for the hardcore gaming crowd, the HMC is a device consisting of a glove, a wand, and a multi-touch pad, which are consequently inspired by Nintendo’s Power Glove, Wii Nunchuck, and Apple’s iPhone. The HMC has the capability of enhancing the usability for several game genres, but is focused on more core titles such as first-person shooters, RPGs, and racing simulators.

Introduction and Design Purpose:• Motion controls; they seem to be taking over the video game

industry as of late. Everything from a swift swipe of a finger on an iPhone to swinging a Wii-Mote, or even full-out break dancing on the Xbox 360 Kinect has been seriously considered as the future of human control interface technology (and the sales figures show). The overall attraction to these devices is that the user experiences a level of intuitiveness never before seen in a video game controller. Gamers of all ages and skill levels can now strike the enemy with their fists or avoid obstacles by dashing about in their living room, or they can play a virtual game of tennis or practice a specific yoga stance with ease. This is a phenomenal development in game interaction, and has and will continue to bring forth many ingenious ideas for its implementation.

Introduction and Design Purpose:• These are undoubtedly recognized as popular control schemes

because movement controlled interfaces will always be the king of usability, as they offer the best natural mapping and affordance stemming from the basis of learnability and memorability. But because the users have absolutely nothing in their hands, and the responsiveness of the technology hasn’t quite caught up to the vision, important UI elements such as feedback, consistency, and constraints still lag behind in efficiency, so much to the point that many game developers haven’t succeeded in making a “hardcore” title for many of these devices (such as a frantic FPS, complicated RPG, or a precise and realistic simulator), although several attempts have been made.

• This is where the Hybrid Motion Controller for the Xbox 360 comes in. With this device, we aim to bridge this vision with reality by lessening the gap that lies between the basic functionality of the Kinect and the tactile feel of a responsive and accurate gamepad. Our ultimate goal is to persuade core gamers to embrace motion controller interfaces by proving to them that not every Kinect supported title has to be completely controller free, thus bringing with it the usual feelings of disengagement that these types of experienced gamers have when using it. With the HMC, three intuitive devices combine to produce the most accurate and responsive motion controlled experience to date, all while keeping the principles of usability in the forefront of its design.

Action Glove (Palm)

Worn on the right hand it comes in three sizes (S, M, L).Flex sensors add tension and tracks hand contractions.Constructed of a flexible vibration-damping polymer to contain physical feedback for precise control.Finger sensors gather additional data for motion tracking.

FIGURE 1: ORIGINAL CONCEPT

TO REDUCE CONFUSION BETWEEN DIFFERENT VERSIONS OF THIS CONTROLLER, THE ONLY FEATURES LISTED WILL BE ONES ADAPTED TO THE FINAL ITERATION.

Action Glove (Palm)

A padded area, raised a half inch from the palm, contains a gyroscope, accelerometer, and feedback vibration motor. In addition this pad contains a rechargeable battery charged through a power-mat station.Programmable touch sensor buttons (R1, R2, R3, R4).

FIGURE 2: FIRST ITERATION

R4R3 R2 R1

Action Glove (Palm)

Five programmable touch sensor buttons (Z=purple, A=green, B=red, X=blue, Y=yellow).Fingertips are cased in knitted silver for touchscreen use.Mesh vents allow heat to escape and cool air to enter.

Y

X

B

A

Z

FIGURE 3: FINAL ITERATION

Action Glove (Back of Hand)

Optical sensor functions as a precise cursor.Additional mesh vents.Adjustable strap to keep optical sensor in place.Stability fins add additional support to keep optical sensor in place.Adjustable velcro strap to keep glove in place.

FIGURE 4: FINAL ITERATION

Motion GripJoystick with click button.Internal gyroscope, accelerometer, and vibration motor with rechargeable battery.Thumb button for additional input (O=red).

Three trigger buttons (L1=black, L2=grey, L3=white).Anti-sweat grip to prevent slippage.Held in left hand with an adjustable velcro strap to keep it secured.

L1

L2

L3

FIGURE 6&7: ORIGINAL CONCEPT(RIGHT), FIRST ITERATION (LEFT)

O

FIGURE 5: FINAL ITERATION

Hybrid PadMulti-touch screen.Velcro straps to secure pad to forearm.Physical Xbox guide menu button with associated player number.Scrollable menu #1 that controls which image is displayed on pad screen (map, inventory, start menu).Scrollable menu #2 contains customizable in game quick keys (easy access to spell book, healing potions, weapon swaps, etc…).

FIGURE 8: FIRST AND FINAL ITERATION

Testing Methods• Three phases of proposed Playtesting would be

cost effective for this controller.• Phase 1: Individual participants are assigned a

game based on personal preference; after an hour of testing they go through an interview process.

• Phase 2: Group testing for split screen and online multiplayer to determine emotional responses in group settings; this session is recorded and followed by an electronic survey.

• Phase 3: A two part phase consisting of biometric testing for individuals and then in groups to give a final scientific measurement of emotional response in each setting.

Figure 9: Phase 1 example.

Figure 10: Phase 2 example.

Findings and Results• Based off preliminary testing, several changes were

made to the overall design of the glove and wand (as depicted in previous slides).

• In response to the initial mixed feelings of the glove and wand, several iterations were created until a final design was determined.

• The multi-touch pad on the other hand needed no improvements as it’s design is based on smartphones so most users are already comfortable with the interface.

• Based off the final design schematic, participants were able to clearly identify and perceive the use of all the button mappings on each controller.

• In addition, many participants in this preliminary test displayed feelings of excitement (verbally and physically) over the integration of accurate controlling with the Kinect system.Figures 11-16: Displays the

progression of wand and glove.

The one thing that Kinect needs more than anything right now is a core FPS title to help bring fans of that crowd to the forefront. With the Hybrid Motion Controller we aim to bridge the gap that lies within normal Kinect functionality by focusing on immersion, intuitiveness, and most importantly with the FPS genre, accuracy. Let’s take a quick look at how one of the gaming industry’s most lucrative franchises might be able to adapt to the HMC.

Action Glove:Z (thumb - pressed against side of index) -Fire Weapon

A (index) -Use/Reload

B (middle) -Jump

X (ring) -Switch Weapons

Y (pinky) -Objectives/Menu

Pointing Optical Sensor -Aiming

Overhand Throwing Movement -Lethal Grenade

Underhand Throwing Movement -Tactical Grenade

Quick Lowering of Glove -Crouch/Prone

Quick Punching Motion of Glove -Melee

Motion Grip:

Analog Joystick -Movement/Strafing

Analog Joystick (pressed) -Sprint/Hold Breath

O -Scoreboard

L1 (Black) -Aim Down Sights

L2 (Grey) -Inventory Item #1

L3 (White) -Inventory Item #2

O + L2 -Inventory Item #3

Hybrid Pad: Since the nature of a touch screen device is so versatile and directly relates to the genre of the title that is being played, for an FPS such as Call of Duty, it would only have a few practical uses.

Scoreboard: Instead of pressing O button to bring up the scoreboard during an intense multiplayer match, players may have this visible at all times via the Hybrid Pad and can choose to mute players by simply tapping on their name.

Directing Kill Streaks: For a lot of the kill streaks in Call of Duty (and during the campaign) players are tasked with manually directing some sort of drone, missile, or other lethal device (usually) from a top down view onto their enemies. Now, players can get a chance to relax their right hand with the option to use the Hybrid Pad to control these elements. Perks such as the napalm strike could now be executed more precisely with a swipe of a finger across the mini-map.

Accessing Equipment/Weapons: Players may access a screen that shows every item in their current load-out which can then be quickly selected by tapping them. This may or may not prove to be more efficient for certain players, making it (of course) optional.

RPG Elder Scrolls V: Skyrim

Action Glove:•A Button- Action button

•B Button- Cancel action

•X Button-Sneak Mode

•Y Button-first person/third person

•Z Button- Jump

•Index Finger IR beam- Aiming crosshairs away from center screen if desired

•Hand movement- Fully mimics the users hand motions, allows tothrow weapons not normally meant to be thrown and more special actionsas well as changing how a spell can be effected, making it a projectile or area.

Motion Grip:•Joystick- Movement and selections

•L1 (Black)- Hold to change between using held item and let go for camera control

•L2 (Grey)-Sprinting

•L3 (White)-Draw or put away held Items

•O (Red)-Opens quick slot menu

•Gyroscopic action- Controls camera unless L1 is held down, then uses left hand to mimic the hand movement

Hybrid Pad:Can display and interact with:

•Maps

•Inventory screen

•Equipment management

•Spell lists

•Quests

•Start Button- Brings up menus on screen

Take a seat (or stand up) and get comfortable; get ready to strap the Hybrid Pad to your forearm, make a fist with the Action Glove to start your engines, and shift your way to the top with the Motion Grip. The HMC component’s work in many different ways for racing games of many

different types, with the major functions of each controller of it’s own playing a big role, from searching the menus, camera movements, motion sensors, to the feeling of hitting the NOS button on the steering wheel.

Action Glove

• Hold “A” and L1 to activate the engine• Optical sensor movement for each vehicles / camera view for each

corner, swerve, and spin out. • Press Z against pointer-finger to active nitrous.• Lean left hand in the direction of each corner when the glove begins

to vibrate begin to motion your hand in a “jerking motion” until the Action Glove stops vibrating.

Motion Grip:

• Pressing the L3 button to change view of car. • Inside view; you control the view from the drivers

perception by moving the Motion Grip left / right. • Pressing the “O” button brings up the map on the

Hybrid Pad. • Press Joystick, then move Motion Grip in shifting

position - to gain speed: double click “L2”

Hybrid Pad:

• Slide your finger over the screen to get into the inventory screen or save/load game menu. If you slide two fingers across the screen it will switch over to the garage where you can purchase more accessories from interior to exterior designs and logos, etc…

• Want to see how far your opponent is from you!? Check out the rearview mirror image to see.

• Customize cars faster and easier with simple clicks on the screen. Scroll through different decals, motors, exhaust pipes, seats, etc…

• Customize players/characters through the create your own player option screen.• Double press the E-Brake application during drifting / racing.

Design Method:• Two of the principles of gamenic theory is “Engaging choreography, gradual learning

curve” and “Beyond gaming “. We believe this deeply describes the Hybrid Motion Controller with each feature that’s involved.

• Action Glove has a sensor that can be used for camera movement by simply pointing and moving your middle knuckle around the screen. This frees up the rest of the fingers for other actions while keeping natural “aiming” conventions familiar to most people.

• The most-used face buttons are mapped to the tips of the glove’s fingers to allow quick and intuitive access to many different button combinations.

• Strapping a comfortable remote (Motion Grip) to your left hand, the grip of the controller fits perfect and firm. It allows the use of moving your character, checking inventory, watching your kill count go up, or even other left hand orientated movements (such as drawing a shield).

• A third piece of the puzzle is revealed as the Hybrid Pad, the touchscreen that holds many information from maps, direct kills, review mirrors, scoreboard, quest to complete. This is definitely the most versatile device featured within the HMC.

• Flexibility is very common with many different easy motions and button configurations that can be used or pressed. Just place your index finger onto your palm or against your thumb (pressing the “A” button) to throttle an engine, swings a battle-axe, or reload a weapon, if it feels better with your ring finger, no problem, the HMC was designed with customization in mind.

Action Glove Inspiration:

•Glove controller as basis of the Action Glove

•Difficulty of use to help show us what not to do

•Codes needed for each game for power glove, game is made specifically for the Hybrid Motion Controller

•Helps the player feel more immersed in their experience by mimicking natural movements

Motion Grip Inspiration:

•Nearly perfect controller for the left hand

•Needed more buttons for more advanced games

•Wireless can cause less damage, but wired means no need for built in power supply

•Motion Control allows for more immersive gameplay

Hybrid Pad Inspiration:

•Touch screen adds many different options to games and menus

•Multi touch allows to zoom in and out easily

•Slide from side to side to allow for different windows, menus, views, etc…

•Works very similar to smartphones in many different aspects

•Could easily be mistaken for a smartphone with the Xbox logo on it

Conclusion & Recommendations:• In conclusion, the Hybrid Motion Controller is what hardcore

gamers have yearned for since the Xbox 360 Kinect’s first functionality presentations were shown at the E3 almost two years ago. Finally, the same demographic that keeps breaking sales records year after year by consuming core titles (such as the blockbusting Call of Duty franchise) can finally be brought over to the Kinect, solidifying the device as an integral part of the future of interactive gaming (instead of a gimmick that caters only to the casual crowd). The HMC will be a surefire hit, not only because it is a unique device that will definitely help to bridge this cultural gap, but also because it is a device that has been carefully designed with both functionality and usability theory in mind to make for some of the most interactive and intuitive Kinect experiences to date.

• The designers of the HMC recommend as more titles are designed to work in tandem with the device that more usability testing should be done as needed. For core titles, especially games that feature fast-paced multiplayer battles, intensity testing focusing on accuracy and the actual stability of the hardware are recommended to ensure that the device holds up over pro-longed (and somewhat abusive) gaming sessions. If marketed correctly, the HMC could be a sales juggernaut, especially if it was packed into a Kinect Bundle Pack that featured the camera device, the HMC, and a popular core title.

References:• http://www.xbox.com/en-US/

• http://www.xbox.com/en-US/kinect

• http://www.callofduty.com/blackops/age-gate?referrer=http%3A//www.callofduty.com/blackops

• http://www.elderscrolls.com/skyrim/

• Navarro, A. (2007, September 25). Juiced 2: Hot import nights review. Retrieved January 29, 2011, from http://www.gamespot.com/juiced-2-hot-import-nights/reviews/juiced-2-hot-import-nights-review-6179867/

• Shadowstew. (2011, July 25). motion control: It was always here. Retrieved January 29, 2011, from http://www.destructoid.com/blogs/Shadowstew/motion-control-it-was-always-here-207021.phtml

• Holmes, J. (2011, November 21). Skyward sword should've given us control of its controls. Retrieved January 29, 2011, from http://www.destructoid.com/skyward-sword-should-ve-given-us-control-of-its-controls-216198.phtml

Image Sources:• http://www.gadgetlite.com/wp-content/uploads/2010/01/juiced2hotimportnights-03.j

pg

• http://dsmedia.ign.com/ds/image/article/830/830321/juiced-2-hot-import-nights-20071025020306773.jpg

• http://storeimages.impulsedriven.com/product_gfx/juiced2_ss4.jpg

• http://ps2media.ign.com/ps2/image/article/822/822977/juiced-2-hot-import-nights-20070925054232792.jpg

• http://gunduzgece.free.fr/capslar/jui2pc006.jpg

• http://thegamershub.net/wp-content/uploads/cod-black-ops.png

• http://cdn3.callofduty.com/images/intel/DOM_Hangar18_overhead_map.png

http://www.blackops2game.com/wp-content/uploads/black-ops-2.jpg

http://i.dailymail.co.uk/i/pix/2011/01/27/article-1350709-0CBA01E6000005DC-786_634x427.jpg

http://www.szprice.com/product_images/i/wii_43b__51688.jpg

http://bulk2.destructoid.com/ul/200354-edenheader.jpg

http://www.empireonline.com/images/image_index/580x250/46061.jpg http://www.scenicreflections.com/files/Digital_hand_print_scanner_Wallpaper_m2xxl.jpg