report on augmented reality 2015

Augmented Reality | Avishekh Bharati

1 IntroductionThis chapter consists of general introduction, current scenario and history of augmented reality. It also identifies the major keywords used in the report.

1.1 General IntroductionAugmented Reality is a term used for interactive computer graphics which overlays a view of the physical world. The trick is done by capturing the current view with a camera, and inserting computer generated content into the scene, so the user sees both the real world and the extra content: an augmented view of reality. (McGrath, 2014)

AR allows the user to see the real world, with virtual objects superimposed upon or composited with the real world. Therefore, AR supplements reality, rather than totally replacing it. Ideally, it would appear to the user that the virtual and real objects coexisted in the same space. (Azuma, 1997)

1.2 Current ScenarioAugmented Reality started to boom and came under lime light soon after the introduction to smart phones. Marxent’s Visual Commerce is cross-platform SaaS (Software as a Service) designed to manage infinite 3D products on multiple platforms. (Marxent Labs LLC, 2014)

Google’s project tango and magic leap, Microsoft’s HoloLens indicate the involvement in the AR sector. Recently, Apple has also acquired augmented reality company: Metaio. (Constine & Miller, 2015)

Wikitude, layar are the AR based browser, providing location based experiences and assisting to find various items based upon AR technology. (CUTLER, 2009)

Automobile giant Volkswagen uses AR app called MARTA (Mobile Augmented Reality Technical Assistance) aimed at repair technician and car dealers (Zibreg, 2013).

The Umpire Decision Review System (UDRS) is an AR based system used in cricket for reviewing the controversial decisions made by the on-field umpires. (espncricinfo.com, 2015)

1

Figure 1.1 Microsoft's HoloLens (Hushen, 2015)


1.3 Key Terms used in Report AR: Augmented reality, VR: Virtual Reality, Magic Leap, HoloLens

2

Figure 1.2 UDRS on Cricket (ESPN SPORTS MEDIA LTD., n.d.)


1.4 History The timeline below highlights the major achievements and milestones in the history of Augmented Reality.

(Fritz Nelson, 2014)

3

1968Ivan Sutherland invented first head mounted display system

1975Myron Krueger creates Videoplace that allows users to interact with virtual objects for the first time

1981Dan Reitan brings AR to tv by displaying virtual reality earth maps and abstract symbols for television weather broadcasts.

1990Term AR was coined by Boeing reasercher Tom Caudell

1992Louis Rosenberg develops virtual fixtures, one of the first functioning AR system for the air force.

2000Bruce H. Thomas develops AR Quake, the first outdoor mobile AR game.

2004 German reserchers brought see through AR to cell phones.

2009ARToolKit brings AR to Web browsers.

2011A team at University of Washington created a contact lens assembled with functional circuity and LED's.

2012Project Glass by Google.

2013Volkswagen’s MARTA(Mobile Augmented reality Technical Assistance) app assists technician in car repairing.

2014Wearable AR makes headlines, mostly Google Glass.Innovega introduces smart contact lenses.


2 BackgroundAugmented Reality (AR) is a variation of Virtual Environments (VE), or Virtual Reality as it is more commonly called. Virtual Reality completely immerse a user in a synthetic environment. While immersed, the user cannot see the real world around him. (Azuma, 1997)

The further section under background consists of working mechanism of AR, its types, key components used and the application areas.

2.1 Working Mechanism of Augmented RealityThe figure below provides information on how AR technology works in one of the AR integrated Hardware i.e. HMD (Head mounted display).

Figure 2.3 Optical see-through HMD conceptual diagram. (Azuma, 1997)

The video see-through HMDs work by combining a closed-view HMD with one or two head-mounted video cameras. The video cameras provide the user's view of the real world. Video from these cameras is combined with the graphic images created by the scene generator, blending the real and virtual. The result is sent to the monitors in front of the user's eyes in the closed-view HMD. Figure 2.1 shows a conceptual diagram of a video see-through HMD.

(Azuma, 1997)

4


2.2 Types of Augmented Reality There are two primary types of AR implementations: Marker Based and Markerless. Markerless may also be referred to as Location-based or Position-based AR. (Dartmouth College Library, 2015)

The brief working mechanism of these two types of Augmented Reality has been described below.

i. Marker-based Markers work by having software recognize a particular pattern, such as a barcode or QR/2D code, when a camera points at it, and overlaying a digital image at that point on the screen. If the image is (3D) three-dimensional or animated, the effect is of a digital experience unfolding on the surface upon which the pattern is printed. (Ooi, 2014)

Figure 2.2 Marker based Augmented reality. (Ooi, 2014)

ii. MarkerlessMarkerless/Location-based applications use the ability of a particular device to record its position in the world and then offer data that’s relevant to that location: finding your way around a city, naming the mountains, monuments around you or the stars in the sky. (Ooi, 2014)

Figure 2.3 Location based Augmented Reality. ( Dice UK, 2014)

5


2.3 Key Components of Augmented RealityThe list below gives the core components required for building AR systems for both fixed and mobile environments:

Hardware• Computer of some form factor, either a PC or a mobile device

• Display screen

• Camera

• Tracking and sensing systems (GPS, compass, accelerometer)

• Network infrastructure

• Marker

Software• An app or program running locally

• Web services

• A content server

(Kipper & Rampolla, 2012)

6


2.4 Application AreasFew among many application areas have been discussed under this topic.

i. Advertising

Lego stores use an AR system to provide an animated version of the completed Lego set inside the box they are holding. (AppFutura.com , 2015)

ii. Task Support

AR for Maintenance and Repair (ARMAR) as being viable in the execution of procedural tasks in the maintenance and repair domain. (Padzensky, 2014)

Figure 2.5 Demonstration of viability of AR for Maintenance and Repair tasks. (Padzensky, 2014)

iii. Navigation

7

Figure 2.4 Animated version of complete lego set. (AppFutura.com , 2015)


City guides such as Yelp helps people find places to eat, drink, and shop have AR capabilities that give a user real-time visual directions to the destinations.

iv. Education

With AR students can manipulate virtual objects or representations of real objects that would otherwise be impossible to hold as well as learn tasks and skills.

v. Translation

An update of Google Translate also integrates Word Lens, which instantaneously translates written text. (Pachal, 2015)

8

Figure 2.6 Yelp app on iOS. (Mind Vomit, 2013)

Figure 2.8 Google translate (Pachal, 2015)

Figure 2.7 Mobile AR app for education. (Santana, 2012)


3 AnalysisThis chapter analyzes Augmented Reality technology from different perspective. This includes its advantages, disadvantages and also the analysis based on various graphs, charts and stats.

3.1 Advantages of Augmented RealityUnder this topic advantages of Augmented reality has been identified and briefly elaborated.

i. Better Navigation

Wikitude Navigation helps drivers and pedestrian in navigation with turn-by-turn GPS based guidance using a unique augmented reality display. It adds convenience and safety benefits of never having to take their eyes off where they’re going. (Wikitude, 2015)

ii. Improves Education and Educational Sector

The 3D object viewers from Augmented Reality Development Lab (ARDL) increases student’s involvement making teaching and learning process more interactive. Teacher can used AR along with VR to show the bully, how harmful his activities are from the perspective of victim. (Kluwe, 2014)

iii. Aid those with disabilities

AR assists people with disabilities. LookTel Recognizer is an iOS apps that aid visually impaired or blind people to recognize everyday objects. (LookTel, 2015)

iv. Being multi-modal increase Productivity

Technology like Google Glass responds to voice commands, taps, gestures or head movements. This means that anyone from any field could use the glass for increased productivity. (Galer, 2014)

v. Helps in sports for better decision making and achievement tracking

Coaches from the German Bundesliga football club TSG 1899 Hoffenheim have tested Google Glass to gain insights on player performance during practice helping to make better decisions. (Galer, 2014)

9


vi. Provides advantage with diminished reality

Technical University of Ilmenau in Germany have developed software that can remove objects from live videos. For example: Architects can imagine what a site will look like after a building has been demolished. (Khanna & Khanna, 2011)

3.2 Disadvantages of Augmented RealityThis topic discusses the disadvantages of augmented reality.

i. Privacy

Maintaining privacy would be much harder. Using AR facial recognition, a digital profile of an individual can be pulled up on anyone that can be seen.

ii. Augmented Bullying and Harassment

Bully may paste a marker on the back of the victim which on viewing through AR tech shows “Kick Me” sign. Similarly, someone can find the nude pictures of the others who are walking by simply looking at them through AR tech. (Gordon, 2010)

iii. Augmented Terrorism

High-profile targets could be tagged and then tracked through a crowd of any size; densely populated areas could be monitored and shown exactly when they have reached a peak density for maximum devastation. (Gordon, 2010)

iv. Disadvantage with diminished reality

It could be destructive tool for spreading apathy in society. If you don’t want to see any homeless people on the street, you can program the AR glass to do so, and the software will keep deleting the poor lying on the sidewalks and you will never feel guilty of not helping those. (Khanna & Khanna, 2011)

10


3.3 Augmented reality innovations patentsThe increasing number of patent registration indicates the innovations in the field of Augmented Reality.

Figure 3.4 AR and Headmounted display US patent holders. (Lee, 2015)

The Sony tops the list owing over 600 patents, followed by Samsung electronics which owns 550 US patents related to AR and HMD technologies. Samsung is followed by IBM(550), Microsoft(476) and Google(392). The smaller companies in the list among the Tech giants: MicroVision and Kopin also holds the significant amount of US patents in this area with 289 and 206 patents respectively.

Hence, it shows that augmented reality is getting bigger and tech companies have been actively innovating in this field.

11


3.4 Augmented Reality Vs Virtual Reality Device and Revenue Forecast

With major tech giants like Microsoft, Google, Sony, Facebook, Samsung etc. all involved in the latest AR and VR revolution, it is perhaps not surprising that the market is set to explode in popularity.

Figure 3.5 AR and VR forecast by CSS Insight. (Lamkin, 2015)

According to chart above, AR and VR devices sales will rise from 2.5 million this year (2015), to a whopping 24 million by 2018. It forecasts that the market will be worth more than $4billion.

The hardware shipments of VR devices alone will increase from $2.2M in 2015 to $20M in 2018. Similarly shipments of AR devices will increase 16-fold i.e. $300,000 to $4M by 2018. (Lamkin, 2015)

12


3.5 Developer Investment in ARThe graph below shows the new market forecast from ABI research on Developer investment in AR.

Figure 3.6 Developer investment in AR forecast . (Extension Media, 2013)

Developer investment in AR applications was total US$670 million in 2013. The annual amount was expected to exceed US$2.5 billion in 2018. This application segment accounts for 68% of the developer spend in 2013, with product enhancement and standalone apps contributing 22% and 10% of the total, respectively. (Extension Media, 2013)

13


3.6 Installed Base of Mobile Augmented Reality AppsThe chart is published by Tractica, shows the installed base of actively used mobile AR apps in 2015 to be 292 million which will grow to 2.2 billion by 2019. This will represent a compound annual growth rate of 76%. (Tractica LLC., 2015)

The coloring of bars in figure 3.4 depicts the install base of AR apps in specific areas.

Figure 3.7 Install base of AR apps by application type. (Tractica LLC., 2015)

14


3.7 Google trends for AR and VR

Figure 3.8 Google trend for AR (blue) and VR (red). (Google.com, 2015)

The above chart shows the google trend for AR (blue) and VR (red). It could be seen that the popularity of the AR surpassed VR form 2009 and continued until mid-2014. It is particularly due to the Google glass. The rise of Oculus rift and downfall of Google glass was the major factor for the boost in VR popularity and decrement in AR interest.

The figure 3.6 shows the google trends for Google glass (blue) and Oculus Rift (red) which supports the statement mentioned above.

Figure 3.9 Google trend for Oculus rift (red) and Google glass (blue) (Google.com, 2015)

15


3.8 Augmented Reality in Marketing: Compelling StatisticsHere below are the few concreate example of how augmented reality is making an impact across various industries.

i. 8.5 Million Downloads

The IKEA in 2014 launched AR based mobile app for its product catalog, which quickly gained popularity and has since yielded over 8.5 million downloads. (Oser, 2014)

ii. 152% sales boost

With the launch of AR based mobile app for street guide, The Sunshine Aquarium in Tokyo yielded a 152% boost in ticket sales entirely as a result of the application. (Oser, 2014)

iii. Sales Rise $50 Million

Sales reps uses in Mitsubishi Electric uses the AR based mobile app to let customers visualize air conditioning units in various locations of their homes. This has reported a 50 million dollar boost in sales in response to the success of their mobile AR application. (Oser, 2014)

16


4 ConclusionThis chapter concludes the report with critical evaluation and future escalation of the topic.

4.1 Critical evaluationBased on the graphs and charts presented in the analysis chapter, it is expected AR, VR market would heavily grow by 2020. AR being one of the most promising technologies is gaining its importance in various fields of application with its multi modal support. Whereas, VR currently seems to be constrained within Gaming and entertainment.

From the perspective of current giants, there are pluses and minuses. Facebook placed an early bet on Oculus, which might win VR but not address the larger AR market. Google learned from Glass, and had the foresight to invest in Magic Leap. HoloLens could allow Microsoft to regain the glory it lost to Apple in the last decade (DIGI-CAPITAL, 2015). Apple acquiring Metaio owning 171 patents worldwide (Eadicicco, 2015) and hiring of an audio engineer form Microsoft’s HoloLens gives the clear insight that Apple’s starting to build out its AR initiative (Sullivan, 2015).

Since the development of first head mounted display in 1968 to the failure of google glass, AR have continuously making the headlines. Different companies stepping up in AR, VR tech are competing each other to make the better product. In the context of Nepal, particularly youths are keenly interested to get hands on experience with Oculus Rift and HoloLens. Hence, this technology has able to grab attention of people from all over the world.

Even though AR may assist in different areas but it could also become handy for notorious activities like privacy breaching, harassment etc. This could be the major challenge which needs to be address sensitively.

4.2 Future escalationIt is very obvious that AR is a big business and growing every year. The future for AR is immense.

It is expected that by 2016 the revenue for AR technology will be more than 600 billion dollar and 2.5 billion AR apps will be downloaded onto smartphones and tablets by 2017. ( the digital marketingbureau , 2015)

Similarly by 2020, 103 million automobiles will contain AR technology. ( the digital marketing bureau ,2015)

Microsoft on collaboration with NASA have develop software called OnSight, enabling scientists to work virtually on Mars using HoloLens. (NASA/JPL-Caltech, 2015)

Facebook’s CEO Zuckerberg has also confirmed that the instead of working only in VR, they will also engage themselves with AR technology. (Amber, 2015)

17


Niantic is preparing to launch location based AR game: Pokémon Go which has secured $20M funding from Google, Nintendo and The Pokémon Company.

5 ReferencesDice UK, 2014. Tech News: 4 Great Location-Based Mobile Apps in Homage to our GIS Experts. [Online] Available at: http://uk.dice.com/technews/4-great-location-based-mobile-apps-homage-gis-experts/[Accessed 17 10 2015].

the digital marketing bureau , 2015. Category: Augmented Reality. [Online] Available at: http://www.thedigitalmarketingbureau.com/category/augmented-reality/[Accessed 13 11 2015].

Amber, 2015. augmented reality : facebook. [Online] Available at: http://www.augmentedrealitytrends.com/augmented-reality/facebook.html[Accessed 26 11 2015].

AppFutura.com , 2015. What’s Augmented Reality and how does it work?. [Online] Available at: http://www.appfutura.com/blog/whats-augmented-reality-and-how-does-it-work/[Accessed 28 10 2015].

Azuma, R., 1997. A Survey of Augmented Reality, Malibu: Hughes Research Laboratories.

Constine, J. & Miller, R., 2015. Apple Acquires Augmented Reality Company Metaio. [Online] Available at: http://techcrunch.com/2015/05/28/apple-metaio/#.msj9c3:GWup[Accessed 27 09 2015].

CUTLER, K.-M., 2009. Layar, Wikitude duel to become augmented reality browser of choice. [Online] Available at: http://venturebeat.com/2009/07/08/layar-wikitude-duel-to-become-augmented-reality-browser-of-choice/[Accessed 27 09 2015].

Dartmouth College Library, 2015. Augmented Reality: About AR. [Online] Available at: http://researchguides.dartmouth.edu/AR[Accessed 12 10 2015].

DIGI-CAPITAL, 2015. Augmented reality. [Online] Available at: http://digitallynumb.com/post/399172973/augmented-reality[Accessed 13 11 2015].

Eadicicco, L., 2015. Apple working on augmented reality technology. [Online] Available at: http://www.businessinsider.com/apple-working-on-augmented-reality-technology-2015-8[Accessed 16 11 2015].

ESPN SPORTS MEDIA LTD., n.d. RENAULT INNOVATION ZONE. [Online] Available at: http://www.espncricinfo.com/ci/content/zones/renault_new/index.html[Accessed 1 11 2015].

18


espncricinfo.com, 2015. Renault Innovation Zone. [Online] Available at: http://www.espncricinfo.com/ci/content/zones/renault_new/index.html[Accessed 29 09 2015].

Extension Media, 2013. Consumer electronics: Developers to Invest $2.5 Billion in Augmented Reality in 2018. [Online] Available at: http://eecatalog.com/consumerelectronics/2013/07/10/developers-to-invest-2-5-billion-in-augmented-reality-in-2018-look-for-enterprise-to-drive-smart-glasses/[Accessed 26 11 2015].

Fritz Nelson, M. Y., 2014. Reviews: The Past, Present, And Future Of VR And AR: The Pioneers Speak. [Online] Available at: http://www.tomshardware.com/reviews/ar-vr-technology-discussion,3811-3.html[Accessed 2 10 2015].

Galer, S., 2014. Technologies: 7 Ways To Benefit From Augmented Reality Devices. [Online] Available at: http://www.digitalistmag.com/technologies/mobile-applications/7-ways-to-benefit-business-01244412[Accessed 25 10 2015].

Google.com, 2015. Google Trends. [Online] Available at: https://www.google.com/trends/explore#q=augmented%20reality%2C%20virtual%20reality&cmpt=q&tz=Etc%2FGMT-5%3A45[Accessed 01 12 2015].

Gordon, R., 2010. Augmented Reality: The Good, The Bad and The Ugly (Part Three: The Ugly). [Online] Available at: http://www.futureconscience.com/augmented-reality-the-good-the-bad-and-the-ugly-part-three-the-ugly/[Accessed 2 11 2015].

Hushen, T., 2015. Microsoft hololens googleglass. [Online] Available at: http://www.wearables.com/microsoft-hololens-google-glass/[Accessed 15 11 2015].

Khanna, P. & Khanna, A., 2011. The Pleasure and Danger of Augmented Reality. [Online] Available at: http://bigthink.com/hybrid-reality/the-pleasure-and-danger-of-augmented-reality[Accessed 26 10 2015].

Kipper, G. & Rampolla, J., 2012. Augmented Reality: An Emerging Technologies Guide to AR. 1st ed. Waltham: Syngress.

Kluwe, C., 2014. talks: How augmented reality will change sports and build empathy. [Online] Available at: https://www.ted.com/talks/chris_kluwe_how_augmented_reality_will_change_sports_and_build_empathy?language=en[Accessed 01 11 2015].

Lamkin, P., 2015. Tech: Virtual Reality Devices: $4 Billion+ Business By 2018. [Online] Available at: http://www.forbes.com/sites/paullamkin/2015/06/18/virtual-reality-devices-4-billion-

19


business-by-2018/[Accessed 12 11 2015].

Lee, A., 2015. PATENTS SHOWS WIDESPREAD AUGMENTED REALITY INNOVATION. [Online] Available at: http://patentvue.com/2015/05/26/patents-shows-widespread-augmented-reality-innovation/[Accessed 5 11 2015].

Lee, C., 2013. New Audi app uses augmented reality to help you find stuff on your car. [Online] Available at: http://www.idownloadblog.com/2013/08/13/audi-iphone-app-augmented/[Accessed 28 09 2015].

LookTel, 2015. LookTel: Recognizer. [Online] Available at: http://www.looktel.com/recognizer[Accessed 1 11 2015].

Marxent Labs LLC, 2014. ar-solution: visual-commerce/. [Online] Available at: http://www.marxentlabs.com/ar-solution/visual-commerce/[Accessed 27 09 2015].

Matt, 2015. Augmented Reality In Sports. [Online] Available at: http://www.peelinteractive.co.uk/augmented-reality-in-sport/[Accessed 2 11 2015].

McGrath, R., 2014. PART I. BACKGROUND: AUGMENTED REALITY. [Online] Available at: https://robertmcgrath.wordpress.com/the-next-big-thing/aloha-from-the-big-island-of-hawaii-ideas-for-augmented-reality-and-a-modest-proposal/part-i-background-augmented-reality/[Accessed 10 10 2015].

Metz, R., 2015. News: Why NASA Wants Microsoft’s HoloLens in Space. [Online] Available at: http://www.technologyreview.com/news/541126/why-nasa-wants-microsofts-hololens-in-space/[Accessed 25 11 205].

Mind Vomit, 2013. Nokia City Lens And The Trouble With AR Location Apps. [Online] Available at: http://carlfranzen.net/post/36808285728/nokia-city-lens-and-the-trouble-with-ar-location[Accessed 19 10 2015].

NASA/JPL-Caltech, 2015. News: NASA, Microsoft Collaboration Will Allow Scientists to 'Work on Mars'. [Online] Available at: http://www.jpl.nasa.gov/news/news.php?feature=4451[Accessed 11 11 2015].

Ooi, A., 2014. How does Augmented Reality Work?. [Online] Available at: http://playmear.com/blog/how-does-augmented-reality-work/[Accessed 16 10 2015].

Oser, A., 2014. pulse: Augmented Reality in Marketing: 5 Compelling Statistics. [Online] Available at: https://www.linkedin.com/pulse/20141209155630-28910946-augmented-reality-in-

20


marketing-5-compelling-statistics[Accessed 14 11 2015].

Pachal, P., 2015. Google Translate gets smarter with language detection, Word Lens. [Online] Available at: http://mashable.com/2015/01/14/google-translate-word-lens/#W330INCslSqi[Accessed 26 11 2015].

Padzensky, R., 2014. 04: ORIGINS AND EVOLUTION. [Online] Available at: http://augmera.com/?p=65[Accessed 1 11 2015].

Santana, P., 2012. Students for Smarter Planet. [Online] Available at: http://asmarterplanet.com/studentsfor/blog/2012/10/preliminary-evaluation-of-a-mobile-ar-app-for-education.html[Accessed 17 10 2015].

Shu, C., 2015. TecCrunch: MagicLeap Demo. [Online] Available at: http://techcrunch.com/2015/10/21/magic-leap-demo/#.msj9c3:EpwR[Accessed 12 11 2015].

Sullivan, M., 2015. Business: Apple poaches key augmented reality engineer from Microsoft HoloLens team. [Online] Available at: http://venturebeat.com/2015/08/31/apple-poaches-key-augmented-reality-engineer-from-microsoft-hololens-team/[Accessed 16 11 2015].

Tractica LLC., 2015. Press Release: Installed Base of Mobile Augmented Reality Apps to Reach 2.2 Billion by 2019. [Online] Available at: https://www.tractica.com/newsroom/press-releases/installed-base-of-mobile-augmented-reality-apps-to-reach-2-2-billion-by-2019/[Accessed 30 11 2015].

Wikitude, 2015. Wikitude Navigation. [Online] Available at: http://www.wikitude.com/showcase/wikitude-navigation/[Accessed 29 10 2015].

Zibreg, C., 2013. Volkswagen shows off augmented reality iPad app for repair technicians. [Online] Available at: http://www.idownloadblog.com/2013/09/30/volkswagen-marta-augmented-reality-ipad-app/[Accessed 28 09 2015].

21