comp 4026 lecture4: processing and advanced interface technology
TRANSCRIPT
![Page 1: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/1.jpg)
LECTURE 4: PROCESSING AND ADVANCED INTERFACES
COMP 4026 – Advanced HCI Semester 5 - 2016
Mark Billinghurst University of South Australia
August 18th 2016
![Page 2: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/2.jpg)
RECAP
![Page 3: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/3.jpg)
App Inventor • http://appinventor.mit.edu/ • http://appinventor.org/ • Visual Programming for Android Apps • Features
• Access to Android Sensors • Multimedia output
• Drag and drop web based interface • Designer view – app layout • Blocks view – program logic/control
![Page 4: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/4.jpg)
App Inventor Designer View
![Page 5: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/5.jpg)
App Inventor Blocks View
![Page 6: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/6.jpg)
Interactive Coding - Processing ▪ Programming tool for Artists/Designers ▪ http://processing.org ▪ Easy to code, Free, Open source, Java based ▪ 2D, 3D, audio/video support
▪ Processing For Android ▪ http://wiki.processing.org/w/Android ▪ Strong Android support, builds .apk file
![Page 7: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/7.jpg)
![Page 8: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/8.jpg)
Buchenau, M., & Suri, J. F. (2000, August). Experience prototyping. In Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques (pp. 424-433). ACM.
![Page 9: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/9.jpg)
Experience Prototyping The experience of even simple artifacts does not exist in a vacuum but, rather, in dynamic relationship with other people, places and objects. Additionally, the quality of people’s experience changes over time as it is influenced by variations in these multiple contextual factors.
![Page 10: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/10.jpg)
Role Playing
![Page 11: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/11.jpg)
littleBits - http://littlebits.cc/
• Plug and play hardware components • Sensors, input, output
• Rapid design with hardware
![Page 12: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/12.jpg)
Little Bits Demo
https://www.youtube.com/watch?v=wDa3dOERxvA
![Page 13: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/13.jpg)
Interaction Design Process
Evaluate
(Re)Design
Identify needs/ establish
requirements
Build an interactive version
Final Product
![Page 14: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/14.jpg)
When to evaluate? • Once the product has been developed
• pros : rapid development, small evaluation cost • cons : rectifying problems
• During design and development • pros : find and rectify problems early • cons : higher evaluation cost, longer development
design implementation evaluation redesign & reimplementation
design implementation
![Page 15: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/15.jpg)
Four evaluation paradigms
• ‘quick and dirty’ • usability testing (lab studies) • field studies • predictive evaluation
![Page 16: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/16.jpg)
Characteristics of approaches
Usability testing
Field studies
Predictive
Users do task natural not involved
Location controlled natural anywhere When prototype early prototype
Data quantitative qualitative problems Feed back measures &
errors descriptions problems
Type applied naturalistic expert
![Page 17: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/17.jpg)
Evaluation approaches and methods Method Usability
testing Field studies
Predictive
Observing x x
Asking users
x x
Asking experts
x x
Testing x
Modeling x
![Page 18: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/18.jpg)
CASE STUDY
![Page 19: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/19.jpg)
Interaction Design Process
Evaluate
(Re)Design
Identify needs/ establish
requirements
Build an interactive version
Final Product
![Page 20: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/20.jpg)
MOBILE AUGMENTED REALITY FOR SPATIAL
NAVIGATION Sharon Brosnan
0651869 Bachelor of Science in Digital Media Design
![Page 21: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/21.jpg)
STORYBOARD
![Page 22: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/22.jpg)
INITIAL SKETCHES Pros:• Goodforideagenera,on• Cheap• ConceptsseemfeasibleCons:• Notgreatfeedbackgained• Photoshopnotfastenoughformakingchanges
![Page 23: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/23.jpg)
POST IT PROTOTYPING
FirstDra.
CameraViewwith3D
SecondDra. ThirdDra.
• Selec,onhighlightedinblue
• Homebu<onaddedforeasynaviga,ontomainmenu
![Page 24: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/24.jpg)
POWERPOINT PROTOTYPING Benefits• UsedforUserTes,ng• Interac,ve• Func,onali,esworkwhenfollowingthestoryofScenario1• Quick• EasyarrangementofslidesUserTes<ng• Par,cipantsfound• 15minutesessionsscreencaptured• ‘TalkAllowed’techniqueused
• Notestaken
• Post-Interview
![Page 25: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/25.jpg)
WIKITUDE • Popular augmented reality browser for
mobile devices
• Mapping
• Point of Interest abilities
• Multiplatform
• Shows the points of interest of Bunratty Folk Park • Markers can be selected in and an
information pop-up appears
![Page 26: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/26.jpg)
VIDEO PROTOTYPE ! Flexibletoolforcapturingtheuseofaninterface
! Elaboratesimula,onofhowthenaviga,onalaidwillwork
! Doesnotneedtoberealis,cineverydetail
! Givesagoodideaofhowthefinishedsystemwillwork
![Page 27: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/27.jpg)
MORE PROCESSING
![Page 28: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/28.jpg)
![Page 29: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/29.jpg)
![Page 30: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/30.jpg)
Processing - Notes • Language of Interaction
• Physical Manipulation • Input using code • Mouse Manipulation • Presence, location, image • Haptic interfaces and multi-touch • Gesture • Voice and Speech
![Page 31: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/31.jpg)
![Page 32: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/32.jpg)
Basic Parts of a Sketch /* Notes comment */!//set up global variables!float moveX = 50;!!//Initialize the Sketch!void setup (){!}!!//draw every frame!void draw(){!}!
![Page 33: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/33.jpg)
Sample Drawing int m = 0;!float s = 0;!!void setup(){! size(512,512);! background(255);!}!!void draw (){! fill(255,0,0);! ellipse(mouseX,mouseY,s,s);!}!!void mouseMoved(){! s = 40 + 20*sin(++m/10.0f);!}!
![Page 34: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/34.jpg)
![Page 35: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/35.jpg)
Drawing • draw() gets called as fast as possible, unless a frameRate is specified • stroke() sets color of drawing outline • fill() sets inside color of drawing • mousePressed is true if mouse is down • mouseX, mouseY - mouse position
!void draw() { !!stroke(255); !!if(mousePressed) {!! !line(mouseX, mouseY, pmouseX, pmouseY);!! !}!!}!
![Page 36: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/36.jpg)
Processing and Drawing • Basic Shapes rect(x, y, width, height)!ellipse(x, y, width, height)!line(x1, y1, x2, y2), line(x1, y1, x2, y2, z1, z2)!
• Filling shapes - fill( ) fill(int gray), fill(color color), fill(color color, int alpha)!
• Curve • Draws curved lines
• Vertex • Creates shapes (beginShape, endShape)
![Page 37: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/37.jpg)
Vertex Demo void setup(){! size(400,400);!}!!void draw(){! background(255);! fill(0);! beginShape();! vertex(0,0);! vertex(400,400);! vertex(mouseX,mouseY);! endShape();!}!
![Page 38: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/38.jpg)
Curve Demo void setup(){! size(400,400);!}!!void draw(){! background(255);! fill(0);! ! int xVal = mouseX*3-100;! int yVal = mouseY*3-100;! ! curve(xVal, yVal, 100, 100, 100, 300, xVal, yVal);! curve(xVal, yVal, 100, 300, 300, 300, xVal, yVal);! curve(xVal, yVal, 300, 300, 300, 100, xVal, yVal);! curve(xVal, yVal, 300, 100, 100, 100, xVal, yVal);! !}!
![Page 39: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/39.jpg)
Class and Objects
• see http://processing.org/learning/objects/ • Object
• grouping of multiple related properties and functions
• Objects are defined by Object classes • Eg Car object
• Data • colour, location, speed
• Functions • drive(), draw()
![Page 40: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/40.jpg)
Classes
• four elements: name, data, constructor, and methods. • Name class myName { }!
• Data • collection of class variables
• Constructor • run when object created
• Methods • class functions
![Page 41: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/41.jpg)
Car Class
![Page 42: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/42.jpg)
![Page 43: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/43.jpg)
Class Usage // Step 1. Declare an object.!Car myCar;!!void setup() { ! // Step 2. Initialize object.! myCar = new Car(); !} !!void draw() { ! background(255); ! // Step 3. Call methods on the object. ! myCar.drive(); ! myCar.display(); !}!
![Page 44: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/44.jpg)
![Page 45: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/45.jpg)
Constructing Objects • One Car Car myCar= new Car(); !
• Two Cars !// Creating two car objects !!Car myCar1 = new Car(); !!Car myCar2 = new Car(); !
• One car with initial values Car myCar = new Car(color(255,0,0),0,100,2); !
![Page 46: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/46.jpg)
Modifying Constructor Car(color tempC, float tempXpos, float tempYpos, float tempXspeed) !
{ ! c = tempC; ! xpos = tempXpos; ! ypos = tempYpos; ! xspeed = tempXspeed; !}!
![Page 47: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/47.jpg)
Mouse Interaction • Mouse position
• mouseX, mouseY variables
• Mouse Interaction • mousePressed() • mouseReleased() • mouseDragged()
• Add in own code void mouseDragged(){!line(pmouseX, pmouseY, mouseX, mouseY);!}!
![Page 48: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/48.jpg)
Keyboard Interaction
• Check keyPressed variable in draw() method !void draw(){!! !if(keyPressed){!! ! !print(" you pressed " +key);!! !}!}!
• Use keyPressed() method !void keyPressed(){!! !print(" you're pressing "+key);!!}!
![Page 49: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/49.jpg)
![Page 50: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/50.jpg)
Importing Libraries • Can add functionality by Importing Libraries
• java archives - .jar files
• Include import code import processing.opengl.*;!
• Popular Libraries • Minim - audio library • OCD - 3D camera views • Physics - physics engine • bluetoothDesktop - bluetooth networking
![Page 51: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/51.jpg)
http://toxiclibs.org/
![Page 52: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/52.jpg)
Graphical Controls
• Use ControlP5 Library • http://www.sojamo.de/libraries/controlP5/
• Add graphical controls • Buttons, sliders, etc • Support for OSC (Open Sound Controller)
• Use ControlP5 class import controlP5.*;!addButton(name, value, x, y, width, height);!
• Event Handing
![Page 53: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/53.jpg)
Interface Elements
• Interfascia • http://www.superstable.net/interfascia/
• GUI Library for Processing • Buttons • Check boxes • Textfields • Progress bar
![Page 54: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/54.jpg)
![Page 55: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/55.jpg)
Graphical Controls • Use ControlP5 Library
• http://www.sojamo.de/libraries/controlP5/
• Add graphical controls • Buttons, sliders, etc • Support for OSC (Open Sound Controller)
• Use ControlP5 class import controlP5.*;!addButton(name, value, x, y, width, height);!
• Event Handing
![Page 56: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/56.jpg)
P5 Example Controls
![Page 57: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/57.jpg)
Programming Graphics
• Transformations • Creating motion and animation • Bitmaps and pixels • Textures
![Page 58: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/58.jpg)
3D Graphics
• Two options • P3D Library • OpenGL Library
• P3D • Simple, integrated directly into processing • Lightweight 3D
• OpenGL • Full graphics support • Complex
![Page 59: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/59.jpg)
Shapes • beginShape(SHAPE);
• Define Vertices • SHAPES: QUADS, QUAD_STRIP, TRIANGLE_FAN
• endShape(); • Eg: Quads !beginShape(QUADS);! !fill(0, 1, 1); vertex(-1, 1, 1);! !fill(1, 1, 1); vertex( 1, 1, 1);! !fill(1, 0, 1); vertex( 1, -1, 1);! !fill(0, 0, 1); vertex(-1, -1, 1);!!endShape();!
![Page 60: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/60.jpg)
P3D Scene size(640, 360, P3D); !background(0);!lights();!!noStroke();!pushMatrix();!!translate(130, height/2, 0);!!rotateY(1.25);!!rotateX(-0.4);!!box(100);!popMatrix();!!noFill();!stroke(255);!pushMatrix();!!translate(500, height*0.35, -200);!!sphere(280);!popMatrix();!
![Page 61: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/61.jpg)
Vertices Demo
• RGB Cube • Vetices and vertex fills
• VertexDemo • Different types of quad strips • User defined drawing function
![Page 62: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/62.jpg)
Transformations • Rotation ! !rotateX(a), rotateY(a * 2.0), rotateZ(a)!
• Translation ! !translate(X,Y); translate(X,Y,Z);!
• Scale • Push and Pop functions
• Push - Saving current coordinate system • Pop – Restores previous coordinate system • Eg: PushPopCubes
![Page 63: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/63.jpg)
OpenGL
• Import opengl library import processing.opengl.*;!
• Create drawing context !void draw(){!! !PGraphicsOpenGL pgl = (PGraphicsOpenGL) g;!! !GL g = pgl.beginGL();!! !//use GL methods!! !pgl.endGL(); //end opengl calls!}!
![Page 64: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/64.jpg)
3D Model Loading • Using saito model loader
• Loads obj files • http://code.google.com/p/saitoobjloader/downloads
• Code fragment import saito.objloader.*;!OBJModel model ;!model = new OBJModel(this, "sa.obj", "absolute", TRIANGLES);!
![Page 65: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/65.jpg)
![Page 66: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/66.jpg)
Other Tools
• GLGraphics • http://glgraphics.sourceforge.net/
• Library that extends the Processing OpenlGL renderer • opengl textures, • image post-processing filters, • 3D Models, and shaders in GLSL, • Cg and CgFX
![Page 67: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/67.jpg)
ADVANCED INTERFACE TECHNOLOGY
![Page 68: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/68.jpg)
Advanced Interface Technology • Wearable Computers • Augmented Reality • Virtual Reality • Invisible Interfaces • Environment Sensing • Physiological Sensing
![Page 69: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/69.jpg)
Class Project 1. Pick Advanced Technology 2. Brainstorm use case 3. Develop conceptual design 4. Prototype interface/experience design 5. Conduct user evaluation 6. Repeat steps 3-5 7. Write report
![Page 70: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/70.jpg)
WEARABLE COMPUTERS
![Page 71: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/71.jpg)
![Page 72: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/72.jpg)
![Page 73: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/73.jpg)
![Page 74: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/74.jpg)
Major changes in computing
![Page 75: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/75.jpg)
![Page 76: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/76.jpg)
Wearable Computing ▪ Computer on the body that is: ▪ Always on ▪ Always accessible ▪ Always connected
▪ Other attributes ▪ Augmenting user actions ▪ Aware of user and surroundings
![Page 77: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/77.jpg)
Wearable Attributes
▪ fafds
![Page 78: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/78.jpg)
Google Glass
![Page 79: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/79.jpg)
![Page 80: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/80.jpg)
View Through Google Glass
![Page 81: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/81.jpg)
Research Problems • Hardware
• Power, networking, display
• User Interaction • User input, speech, gesture, gaze, etc • Novel interaction methods
• Social Acceptance • Privacy, social factors
• Novel Applications • Collaboration • Intelligent assistance
![Page 82: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/82.jpg)
AUGMENTED REALITY
![Page 83: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/83.jpg)
1977 – Star Wars
![Page 84: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/84.jpg)
Augmented Reality Definition
• Defining Characteristics [Azuma 97] • Combines Real and Virtual Images
• Both can be seen at the same time • Interactive in real-time
• The virtual content can be interacted with • Registered in 3D
• Virtual objects appear fixed in space
Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4), 355-385.
![Page 85: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/85.jpg)
2008 - CNN
![Page 86: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/86.jpg)
• Put AR pictures here
Augmented Reality Examples
![Page 87: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/87.jpg)
Typical Demo Application
https://www.youtube.com/watch?v=UOfN1plW_Hw
![Page 88: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/88.jpg)
Research Problems • Low level hardware/software
• Head mounted displays • Tracking systems
• User Interaction • Gesture based interaction • Multimodal input (speech, gesture)
• Novel Applications • Face to face collaboration • Authoring tools
![Page 89: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/89.jpg)
VIRTUAL REALITY
![Page 90: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/90.jpg)
Virtual Reality
• 1985…
![Page 91: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/91.jpg)
Virtual Reality
• Immersive VR • Head mounted display, gloves • Separation from the real world
![Page 92: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/92.jpg)
Occulus Rift
• $300 USD • 360 degree head tracking • 100 degree field of view
![Page 93: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/93.jpg)
Wearable Virtual Reality
• Samsung Gear VR • See virtual reality on your phone • Using phone display, compass
![Page 94: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/94.jpg)
Gear VR Demo
https://www.youtube.com/watch?v=CjpGnh2PDoU
![Page 95: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/95.jpg)
AR vs VR
![Page 96: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/96.jpg)
Research Problems • Low level
• Wide area tracking • Development tools
• User Interaction • Intuitive input (gesture, controllers) • Avatar control and representation • Techniques for navigation/manipulation
• Novel Applications • Massive multi-user environments • Content capture and sharing
![Page 97: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/97.jpg)
INVISIBLE INTERFACES
![Page 98: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/98.jpg)
Early Examples
• Interaction without devices: • BodySpace [Strachan 2007]: Functions to body position • Abracadabra [Harrison 2007]: Magnets on finger tips • GesturePad [Rekimoto 2001]: Capacitive sensing in clothing
• Palm-based Interaction • Haptic Hand [Kohli 2005]: Using non-dominant hand in VR • Sixth Sense [Mistry 2009]: Projection on hand • Brainy Hand [Tamaki 2009]: Head worn projector/camera
![Page 99: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/99.jpg)
Unobtrusive Input Devices
▪ GesturePad ▪ Capacitive multilayered touchpads ▪ Supports interactive clothing
![Page 100: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/100.jpg)
ImaginaryPhone
• Gustafson, S., Holz, C., & Baudisch, P. [2011]
![Page 101: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/101.jpg)
Imaginary Phone Demo
https://www.youtube.com/watch?v=xtbRen9RYx4
![Page 102: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/102.jpg)
Transfer Learning
![Page 103: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/103.jpg)
Invisible Interfaces – Gestures in Space
• Gustafson, S., Bierwirth, D., & Baudisch, P. [2010] • Using a non-dominant hand stabilized interface.
![Page 104: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/104.jpg)
Imaginary Interfaces
https://www.youtube.com/watch?v=718RDJeISNA
![Page 105: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/105.jpg)
Project Soli
• Using Radar to support free-hand spatial input
![Page 106: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/106.jpg)
Project Soli
https://www.youtube.com/watch?v=0QNiZfSsPc0 https://www.youtube.com/watch?v=jWNebDDmuXc
![Page 107: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/107.jpg)
Research Gaps • Free-hand interfaces using relative input • Combining invisible interface + mobile device • Multimodal interaction
• speech + gesture input • Affordances and discoverability • Interaction frameworks
![Page 108: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/108.jpg)
Research Problems • Hardware
• Power, networking, display
• User Interaction • User input, speech, gesture, gaze, etc • Novel interaction methods
• Social Acceptance • Privacy, social factors
• Novel Applications • Collaboration • Intelligent assistance
![Page 109: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/109.jpg)
ENVIRONMENT SENSING
![Page 110: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/110.jpg)
Environmental Sensor
• New sensors track and capture real environment • Navigation, 3D modeling, user tracking
• Depth Sensors • Microsoft Kinect, Intel RealSense
• Integrated Devices • Google Tango
![Page 111: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/111.jpg)
Google Tango • Tablet based system
• Android OS • Multiple sensors
• RGBD Sensor • IR Structured light • Inertial sensors
• High end graphics • Nvidia tegra chip
![Page 112: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/112.jpg)
Google Tango
![Page 113: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/113.jpg)
Research Problems • Content creation
• Creating better 3D models • Segmenting objects
• User Interaction • Interaction with real world • Interacting with multiple devices
• Novel Applications • AR notes/real world tagging • Social networking
![Page 114: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/114.jpg)
PHYSIOLOGICAL SENSING
![Page 115: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/115.jpg)
Physiological Sensors • Sensing user state
• Body worn devices
• Multiple possible sensors • Physical activity • Eye tracking, gaze • Heart rate • GSR • Breathing • Etc
![Page 116: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/116.jpg)
Tobii Eye Tracker
• Wearable eye tracking system • Natural data capture • Scene camera capture • Recording/streaming eye gaze, 60 Hz sampling
![Page 117: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/117.jpg)
Tobii Demo
• https://www.youtube.com/watch?v=hDG1mRFFusc
![Page 118: COMP 4026 Lecture4: Processing and Advanced Interface Technology](https://reader034.vdocuments.mx/reader034/viewer/2022052606/587082001a28ab57368b6983/html5/thumbnails/118.jpg)
Research Problems • User Interaction
• Implicit vs. Explicit interaction • Measuring cognitive
• Social Acceptance • Privacy, social factors
• Novel Applications • Collaboration • Intelligent assistance