next-generation matrix 3d - amddeveloper.amd.com/wordpress/media/2013/06/1729_final.pdf ·...

NEXT-GENERATION MATRIX 3D IMMERSIVE USER INTERFACE [ M3D-IUI ] H Raghavendra Swamy AMD Senior Software Engineer

3 | Next-Generation Matrix 3D Immersive User Interface | June 2011

Quick Keywords Abstract and Scope Introduction Current User Interface [ UI ] and its limitations Current trends in 3D display space and compute Proposed Idea

– The Matrix 3D Immersive User Interface [ M3D-IUI ], pronounced as " I-Uee " – The M3D-IUI compute engine The swarm algorithm

Essence of swarm engine in M3D-IUI

Role of OpenCLTM and AMD APU [ Accelerated processing Unit ]

SESSION AGENDA


Sample applications – The 3D icons / objects floating in 3D space – The M3D-IUI desktop – The context based windowless Matrix 3D-IUI – The 2D canvas of Microsoft® Paint expanded into 3D – The FILE browser as the MATRIX FILE VAULT – The WinampTM visualization in true 3D – The system tools and utilities – Google search bar moved floating into 3D space – The 3D web browser and 3D Markup language – The true 3D video chatting – Co-existence of existing 2D application windows

SESSION AGENDA


Stack overview for 3D compute M3D-IUI software stack overview Conclusion Q and A

SESSION AGENDA


QUICK KEYWORDS


APU : Accelerated Processing Unit OpenCLTM

3DMLW : 3D Markup Language for Web 3D Display Unit UI : User Interface

QUICK KEYWORDS


ABSTRACT AND SCOPE


A context based windowless Matrix 3D Immersive User Interface [ M3D-IUI ] built on swarm engine using OpenCLTM on Accelerated Processing Unit [ APU ] based platform Here scope is to show or highlight the essence of next generation M3D-IUI

Note :

– Matrix 3D Immersive User Interface [ M3D-IUI ] is pronounced as “ M3D I-Uee “ – Third person camera view immersive experience.

ABSTRACT AND SCOPE


INTRODUCTION


The UI plays an important part in interfacing the compute unit with the outside world It has evolved from

– Simple switching / toggle lights – Character based command consoles – 2D Graphical Interfaces – 3D Graphical Interfaces

INTRODUCTION


CURRENT UI AND ITS LIMITATIONS


Historical trend: The rectangular, resizable graphical window object was introduced to align with the 2D display unit available at that time.

The current user interfaces are 2D in nature; the interfaces are made to look 3D, but they are mapped onto 2D window display units like conventional flat panel display units. Hence, they are not truly 3D.

The 3D graphical objects mapped onto the 2D display unit form the limitations.

CURRENT UI AND ITS LIMITATIONS


CURRENT TRENDS IN 3D DISPLAY AND COMPUTE


Date: CES 2011 [ Consumer Electronics Show 2011 ] – Toshiba's glass-free 3D laptop prototype Link : http://ces.cnet.com/8301-32254_1-20026948-283.html

– Sony makes a push for 3D Link : http://ces.cnet.com/8301-32254_1-20027515-283.html

– AMD Fusion APU series debuts

CURRENT TRENDS IN 3D DISPLAY AND COMPUTE

http://ces.cnet.com/8301-32254_1-20026948-283.html

http://ces.cnet.com/8301-32254_1-20027515-283.html


THE MATRIX 3D IMMERSIVE USER INTERFACE

[M3D-IUI]


All Applications currently are not truly 3D in nature as they get mapped onto conventional 2D display units.

The M3D-IUI incorporates an out-of-the-box approach by moving into windowless 3D environment, which is similar to real world.

The idea is to break free from the window-bounded region and create a true 3D interface.

Note : – The term “window” is currently used as a generic representation of a rectangular resizable

bounded graphical object, and should not be confused with or interpreted as the Microsoft Windows® operating system “NAME” series.

The inspiration being the real world, where objects of all shapes, sizes, colors… seamlessly integrate into the same environment without the concept of window, except for cases like the paper on which this idea was written or the window on the wall…

PROPOSED IDEA | The Matrix 3D Immersive User Interface [ M3D-IUI ]


THE M3D-IUI COMPUTE ENGINE


The M3D-IUI is based on the following – The swarm algorithm Example: Bee, Ants, ….

– Essence of swarm engine in 3D-IUI Example: 3D Agents

– Role of OpenCL TM and AMD APU [ Accelerated Processing Unit ]

M3D-IUI compute engine :

– Harness APU’s array of SIMD computing power using OpenCL TM by mapping low level 3D computing agents to synthesize complex 3D objects and environment using swarm based algorithm.

THE M3D-IUI COMPUTE ENGINE


SAMPLE APPLICATIONS


The 3D icons / objects floating in 3D space

The M3D-IUI desktop

The context-based windowless Matrix 3D-IUI

The 2D canvas of Microsoft Paint expanded into 3D

The FILE browser as the MATRIX FILE vault

The Winamp visualization in true 3D

The system tools and utilities

The Google search bar moved floating into 3D space

The 3D web browser and 3D Markup language

The true 3D video chatting

The co-existence of existing 2D application windows

LIST OF SAMPLE APPLICATIONS THAT MAY BE MOVED INTO M3D-IUI


SAMPLE APPLICATIONS | The 3D Icons / Objects Floating in 3D Space

The matrix 3D-IUI proposes the 3D icon which gets mapped into the matrix 3D-IUI space

It can be viewed 360o and moved across in 3D space

The notion of full 3D feature is achievable: i.e., even grabbing and moving 3D objects, which gives the user more flexibility in interacting and manipulating the 3D content in true 3D style

These graphical objects can also be made governed by the physical laws to give a more realistic feel to the end user

Current scenario 3D Icon in 2D display device

Proposed 3D icon in true 3D space


SAMPLE APPLICATIONS | The M3D-IUI Desktop

Matrix 3D-IUI desktop objects with content tagging

P

XYZ

a


Existing solution: The BumTop Desktop – The sample 3D desktop mapped to 2D display can be seen in the BumpTop desktop by

“Dynamics Graphics Projects” now part of Google Inc. Here bumptop provides a varied 3D-based command interface like pile, fish layout view, etc...

– It has shortfalls: The desktop is not fully 3D, but made to look 3D on 2D space, by having an inclined spatial position

To have full-fledged 3D objects on desktop and not just 3D documents and images

– Source: http://www.ted.com/talks/anand_agarawala_demos_his_bumptop_desktop.html

Proposed solution: The M3D-IUI Room – The M3D-IUI room provides true 3D interface with third-person camera view – It provides full-fledged 3D objects in 3D space – Content tags can be added to give easier identification of document and arrangement – This can be extended to audio and video tag pop-ups

SAMPLE APPLICATIONS | The M3D-IUI Desktop…Opps!!! Now “ROOM”

http://www.ted.com/talks/anand_agarawala_demos_his_bumptop_desktop.html


SAMPLE APPLICATIONS | The Context Based Windowless Matrix 3D-IUI

The M3D-IUI part is equipped with the context sensitivity / awareness, which can change the behavior of the UI based on the context.

As a simple example explained below, the graphics objects are selectively computed rather than repeated computing depending on the context.

Example 1: The current user interface in PowerPoint® has different kinds of menus, icon, other GUI components that are common to other applications like Firefox, Internet Explorer, etc...gets re-generated or computed when switched back and forth.

Here each GUI component or object is bound to the individual application window. These common menu items, icons and other GUI components can be moved outside the window so that the same components can be reused by changing the behavior based on the context thereby increasing efficiency.

Common graphical objects moved outside the bounded window

region into M3D-IUI space

Firefox

Internet Explorer


The current Paint software has a 2D canvas space where 3D objects gets mapped.

The 2D canvas space can be expanded to 3D space canvas where the 3D objects are moved, created and manipulated using 3D objects/tools located in 3D space and not in the current 2D layout inside the Paint application.

This can be a boon for the 3D animation field and 3D movie post-production, as users will be viewing in true 3D and manipulating in a true 3D environment.

SAMPLE APPLICATIONS | The 2D Canvas of Microsoft Paint Expanded Into 3D


SAMPLE APPLICATIONS | The FILE Browser as the MATRIX FILE Vault

The matrix file vault M3D-IUI file management tool has the following features:

– The 3D vault – Racks / rows – 3D arrangement of files – Content management

The 3D vault is the 3D storage compartment which is the equivalent of a drive partition.

The racks or rows are equivalent of folder or directory structure arrangement; the files are arranged in 3D layout on racks.

The content management UI provides facilities to tag the file and file contents in hierarchical 3D space.

C:Vault

Matrix File vault 3D-IUI

Current File Browsers Matrix File Vault M3D-IUI

x: Drive partition X: 3D vault

folder / directory Racks/rows similar to book racks

Files 3D arrangement of file on racks


Winamp player’s audio synchronized visualization display, which is currently 3D on 2D displays, becomes full 3D giving the user a new kind of experience.

The 3D audio equalization settings and the 3D Winamp skins/interfaces will give an ultimate new way to the user to interact with the audio player.

This can be also extended to other audio players.

SAMPLE APPLICATIONS | The WINAMP VISUALIZATION in True 3D


The system tools and utilities can be given a true 3D touch with some of the examples listed below:

Example A – The hard disk management utility can now show hard disk in full 3D view for easier management.

Example B – The matrix file vault 3D can be used to view and search old files and folders in a better way, so that

they can be moved to part of the hard disk where access time is slow, and move the active part of the content where the access/seek time is fast.

The network admin utility with the 3D touch provides an complete 3D view of network configuration for easier management.

SAMPLE APPLICATIONS | The System Tools and Utilities


The Google search bar gets suspended in 3D space and can be rotated 360 degrees. It can be made to always point to the user, using third-person camera effect. The search result can fill up the 3D space.

SAMPLE APPLICATIONS | Google Search Bar Moved Floating Into 3D Space

Google search bar in 3D space

Google


The simple 2D graphics manipulation inside the browsers is controlled through the most popular scripting language “HTML”. These scripts have tags which describe 2D.

The main intention was to reduce the load of graphics data on the network, but still have a graphics experience at the client end.

Along the same lines, 3D Markup Language for Web [ 3DMLW ] can be used to minimize the 3D graphics load. This is an open source scripting language for 3D computations and manipulation on 3D-enabled browsers.

Most of the popular browsers currently support the 3DMLW.

This 3DMLW is a major advancement for easier transition into true 3D web interfaces and 3D user experiences.

Since internet applications are gearing up for an increase in 3D support adoption, it would be ideal if the M3D-IUI is adopted so that websites and online stores can be searched in true 3D.

SAMPLE APPLICATIONS | The 3D Web Browser and 3D Markup Language for Web


Adding the 3D touch to current video chatting will take user experience to the next level.

SAMPLE APPLICATIONS | True 3D Video Chatting


The topic of discussion until now was all about 3D spaces. So what happens to the existing 2D-window based UI applications?

Hmmm....so these applications can be layered on page layouts and can be folded, twisted, bent on the lines of "Dynamics Graphics Projects”.

It will be floating paper in 3D space, and hence the existing 2D window applications can still be used without any changes maintaining backward compatibility.

SAMPLE APPLICATIONS | Co-existence of Existing 2D Application Windows


STACK OVERVIEW FOR 3D COMPUTE


STACK OVERVIEW FOR 3D COMPUTE

The top module is the M3D-IUI desktop or application or 3D-enabled browser which is compliant with 3D mark-up language for web natively through the support of 3D mark-up language plug-ins.

The next layer forms the M3D-IUI swarm engine based API libraries.

The next layer forms the OpenCL APIs and runtime layer, which finally gets mapped to the APU.

3D-Enabled Browser / Desktop

M3D-IUI Swarm Engine OS APIs /

Library

OpenCL APIs and Runtime Layer

CPU and GPU Component of APU

overview stack for 3D compute


MATRIX M3D-IUI SOFTWARE STACK OVERVIEW


MATRIX M3D-IUI SOFTWARE STACK OVERVIEW

Accelerated M3D-IUI software stack on APU

True 3D-Based Application Space

3DML Browser

3DML Plug-in

Low level APIs for stereo matrix 3D primitives implemented through OpenCL based swarm algorithm

OpenCL runtime platform

OS APIs

Low Level drivers GPU, Mouse, USB, Touch pad, etc…

APU (CPU and GPU) Platform Hardware

3D UI component APIS like 3D Icons, 3D floating text, etc…

3D-UI Room 3D Vault 3D Audio Player Other 2D mapped applications


CONCLUSION


CONCLUSION

The idea here focuses on harnessing APU processing power using OpenCL to provide the user next-level 3D immersive experience by “Matrix 3D-IUI”, which is built on context-based swarm engine using open technologies like 3DMLW by leveraging the 3D display units in the space of laptop, netbook, desktop segments operating system user interface and also the internet applications which are moving towards the 3D space.

QUESTIONS


Disclaimer & Attribution The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. There is no obligation to update or otherwise correct or revise this information. However, we reserve the right to revise this information and to make changes from time to time to the content hereof without obligation to notify any person of such revisions or changes. NO REPRESENTATIONS OR WARRANTIES ARE MADE WITH RESPECT TO THE CONTENTS HEREOF AND NO RESPONSIBILITY IS ASSUMED FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION. ALL IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE ARE EXPRESSLY DISCLAIMED. IN NO EVENT WILL ANY LIABILITY TO ANY PERSON BE INCURRED FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. AMD, the AMD arrow logo, and combinations thereof are trademarks of Advanced Micro Devices, Inc. All other names used in this presentation are for informational purposes only and may be trademarks of their respective owners. © 2011 Advanced Micro Devices, Inc. All rights reserved. Microsoft, Windows, PowerPoint, and DirectX are registered trademarks of Microsoft Corporation in the United States and/or other jurisdictions. Google, BumpTop are the registered trademarks of Google Inc. in the United States and/or other jurisdictions.


Disclaimer & Attribution Toshiba is the registered trademarks of Toshiba Corporation in the Japan and/or other jurisdictions. Sony is the registered trademarks of Sony Corporation in the Japan and/or other jurisdictions. Winamp is the registered trademarks of NULL soft / AOL Inc. in the United States and/or other jurisdictions 3DMLW is developed and distributed under GNU GPL V3 license by the 3D technologies R & D.

next-generation matrix 3d - amddeveloper.amd.com/wordpress/media/2013/06/1729_final.pdf ·...

Documents