introduction to ar with unity3d

AR with Unity3DIntroduction to Augmented Reality

withUnity3D

Saturday, May 26, 12

26/05/2012 - CatDroid Unity3D II @aquarioverdeAR with Unity3D

Intro

• Examples

• Pantalla Global (CCCB)

• Animac

• Salón de Turisme



Koffa@Animac

http://vimeo.com/37868229





What is AR?

• Virtual content overlayed onto the real word

• 2D / 3D & audio

• Realtime / Live camera view



Detection

• Vision based

• QR codes

• Markers

• Real life objects

• Location based (GPS, Compass)



Vision Based AR

• Tracking

• Frame by frame analysis searching for “trackables”

• Return position and orientation relative to device



AR Apps

• Layar

• http://www.layar.com/

• Junaio

• http://www.junaio.com/

• Wikitude

• http://www.wikitude.com/

• Appunta ;-)

• http://appunta.com/


http://www.layar.com/

http://www.layar.com/

http://www.junaio.com/

http://www.junaio.com/

http://www.wikitude.com/

http://www.wikitude.com/

http://appunta.com/

http://appunta.com/


Use

• Arts & Gaming

• Advertising

• Navigation

• Tourism

• Education

• ... and a lot more



Unity3D Libraries

• String

• www.poweredbystring.com

• Vuforia (Qualcomm)

• https://developer.qualcomm.com/mobile-development/mobile-technologies/augmented-reality


http://www.poweredbystring.com

http://www.poweredbystring.com

https://developer.qualcomm.com/mobile-development/mobile-technologies/augmented-reality





String - Tracking

• Framed Markers

• Multiple marker detection

• <= 10

• Unflexible marker creation

• black border surrounded by a white border necessary



String - Tracking

• Any png

• Full marker has to be in camera (>85%)

• Markers not in scene

• dificult scaling



Vuforia - Tracking

• Image Target

• Frame Markers

• Simple 3D Objects (Boxes)

• More flexible than String on marker creation



Vuforia - Tracking

• Multiple Image Targets or Frame Markers

• <= 5

• Works if only part of the marker is seen

• Markers placed into 3D Scene

• easy scaling

• Virtual Buttons



Target Management

• Online

• ImageTargets

• MultiTargets

• jpg & png

• RGB & B&W

• Scaling (Unity units)

Rich in detail

High in contrast

No repetitive patterns!



Performance

• String a little bit faster in detection and tracking

• String is more stable



Platform support

• Vuforia

• Android & iOS

• String

• iOS



String - Setup

• Download library

• Unity3d

• iOS

• Add to Unity3D project

• Add to XCode project



Vuforia - Setup

• Download Vuforia library

• available in Assets Store

• Add to Unity3D Project



Testing

• String:

• In-Editor testing using webcam

• Vuforia:

• No image tracking nor life camera

• Device Only Testing

Unity Remote!



Licensing

• Unity / Unity iOS / Unity Android

• https://store.unity3d.com/

• String

• Expensive!

• http://www.poweredbystring.com/licensing

• Vuforia

• Free!!!


https://store.unity3d.com/

https://store.unity3d.com/

http://www.poweredbystring.com/licensing

http://www.poweredbystring.com/licensing


Decision ???


Vuforia



Requirements iOS

• Mac OS Snow Leopard & Lion

• XCode

• https://ar.qualcomm.at/qdevnet/sdk/ios


https://ar.qualcomm.at/qdevnet/sdk/ios

https://ar.qualcomm.at/qdevnet/sdk/ios


Requirements Android

• MS Windows 7 & XP

• Linux & Mac OS posssible but not documented

• JDK + Eclipse + ADT

• Android SDK + NDK

• https://ar.qualcomm.at/qdevnet/sdk/android


https://ar.qualcomm.at/qdevnet/sdk/android

https://ar.qualcomm.at/qdevnet/sdk/android


Installation

• Download pagackage vuforia-unity-OS-Version.zip and install

• base extension & samples

• Asset Store



Simple Project• New project

• Add Vuforia

• Add Dataset (markers)

• Add ARCamera Prefab to scene

• Add ImageTarget Prefab to scene

• Select Dataset and ImageTarget

• Add 3D objects + Light

• Set Load Dataset

• Deploy



Unity Folder Structure



Hands-On

• DroidBalls

• Floating spheres in space

• Touch to dissappear



Raycasting

• Detect 3D object touched on screen

• Ray from touched screen point to camera

• Detect object collisions



Tracking

• ITrackableEventHandler

• OnTrackableStateChanged

• DefaultTrackableEventHandler

• enable/disable rendering

• ITrackerEventHandler

• OnTrackablesUpdated



Splash Screen

• Display when no marker is detected

• Modify ITrackableEventHandler



Virtual Buttons

• Hotspots on an ImageTarget

• IVirtualButtonEventHandler

• OnVirtualButtonPressed

• OnVirtualButtonReleased

• Can be created at runtime



Problems

• Perfomance

• Image Recognition

• symmetry / similar elements

• contrast / illumination

• Real world sizing



Problems

• Fixed camera vs fixed space

• Physics!!!

• Multiple markers

• center???



Tips & Advices

• 3D Models

• < 2K polygons overall

• < 200 polygons/object (draw calls)

• <= 3 materials/object

• Animations

• < 30 bones

• Use texture sequences

Device dependent!



Tips & Advices

• Attach markers to cardboard

• Good lightning but avoid reflections

• Disable objects that are not in view

• Test using mouse instead of touch



Tips & Advices

• Scale correctly from the beginning

• Create simple reference objects to map to real world

• e.g. cubes



Resources

• Unity3D

• http://forum.unity3d.com

• http://answers.unity3d.com

• Vuforia Forums

• https://ar.qualcomm.at/qdevnet/forums

• String ???


http://forum.unity3d.com/

http://forum.unity3d.com/

http://answers.unity3d.com/

http://answers.unity3d.com/

https://ar.qualcomm.at/qdevnet/forums

https://ar.qualcomm.at/qdevnet/forums

Thank you!Andreas Blick @aquarioverde


introduction to ar with unity3d

Technology

12 vuforia

qualcomm

www