October 30, 2017 Sam Siewert

CS415Human Computer Interaction

Lecture 10 – Advanced HCIUniversal Design & Intro to Cognitive

Models

Summary of Thoughts on Intelligent Transportation Systems

Collective Wisdom of Our Classes (2015, 2016, 2017)– Self driving Cars will become ubiquitous by?

Sam Siewert 2

Year 2020 2025 2030 2035 2040 2045 Beyond Out of

2015 class 1 1 1 0 1 0 0 6

2016 class 0 1 3 0 1 0 0 5

2017 class 0 0 1 1 2 1 2 7

Summary of Thoughts on ITSRevolutionary vs. Evolutionary from Now to Ubiquitous

– Evolutionary [7]“Self-driving cars have to deal with personal acceptance, like the touch screens did, but these cars also have to deal with safety” … “By 2035 the (sic) millenial generation will” … “have a big push on automation and technology” … “the Baby Boomer generation will be getting to a point where they are out numbered”. “evolution of self-driving cars will be slow and steady” … “with many government and safety hoops” … “like manual transmission vehicles, modern vehicles (human driven) will stick around”“It will never become a nationwide reality” … “most people enjoy driving” … “a small percentage of the population will benefit from the technology like people with disabilities”“evolution with each generation going more towards self driving” … “contingent upon whether or not self-driving” … “does not have any large number of (sic) causulties or negative impact”“as the generations shift” … “self-driving will be the last step of this evolutionary process” … “each dealer will eventually come up with their own design”“enough time to develop software and hardware to be deemed safe” … “enough time to be deemed socially acceptable”“younger generations” … “are more comfortable with the thought of self driving” … “The key to this problem is social acceptance”

– Revolutionary [0] - No responses interpreted as revolutionary

Sam Siewert 3

AssignmentsAssignment #4 – Posted, Last Practice Lab (Cameras and OpenCV)

Assignment #5 – Propose Group Project (Groups of 3)

Assignment #6 – Project

Final Oral Exam – Presentation of Project– HCI Design– Experimental Design for Usability Analysis– Prototype or Mockup of Design

Sam Siewert 4

This Week … Chapter 10 – Introduction to Universal Design– Making Computers Available to Everyone– Computer Solutions for Prosthetics– Active Learning Discussion – The Human Cyborg?

Chapter 12 – Cognitive Models– GOMS – Goal, Operators, Methods and Selection– Goal - describing what the user wants to achieve– Operators - basic actions that the user must perform in order to

use the system– Methods - several ways in which a goal can be split into sub-

goals– Selection – choice of method and prediction of which will be

used

Sam Siewert 5

Assignment #4Use of Digital Cameras with Embedded Linux (V4L2/UVC/USB) - OpenCV

Applications in Intelligent Transportation, Augmented Reality, Communications (e.g. Skype, Apple Facetime, WeChat)

Jetson has OpenCV installed (JetPack 3.1)

Sam Siewert 6

Assignment #4 - ObjectivesPlay with the Cameras and OpenCV First– Verify Hardware– Get a Feel for Performance– Capabilities

Learn about Camera Devices and Device Interfaces

Develop a Camera Application

Consider Processing for Display and Interaction

Human Aspects – Color, Frame Rate, Segmentation

Sam Siewert 7

Assignment #4 DemoReal-time Camera Scene Analysis– Uncompressed, Un-encoded Camera Frames (Not MPEG)– Transforms RGB or Graymap XY Pixel Array Into Segments and

Major Components in Frames– Eliminate Backround (Difference of Frames – Current, Historical)– Segments Analyzed (Recognition) – e.g. Gesture (Thumbs Up,

Thumbs Down)

Sam Siewert 8

Segmenting Linear Elements Segmenting Circular Elements

Universal Design and Embedded HCIChapter 10 – Introduction to Universal Design– Making Computers Available to Everyone– Computer Solutions for Prosthetics– Active Learning Discussion – The Human Cyborg?

Popular NewsTed Talk on Color BlindnessTed Talk – We are Already Cyborgs

Multi-modal Systems– Speech– Audio– Touch– Handwriting– Gestures

Sam Siewert 9

Minute PaperHumans have long been using tools to extend or restore physical capability, and instruments to enhance perception of the world, which fits the definition of cyborg (cyber organism).

Sam Siewert 10

https://en.wiktionary.org/wiki/cyborg

Tools and instruments which progress to wearable devices, which can in turn become prosthetics, are a form of personal cybernetics, but can also be used for social networking as well as personal perception. Is this a positive and useful form of HCI? What costs, benefits, and risks do you see related?

Universal DesignDesign for Diversity– People with Sensory, Physical or Cognitive Impairment– Age Groups– Varying Cultures and Backgrounds– Localization – Languages

7 Principles of Universal Design1. Equitable Use – No User is Excluded2. Flexibility – Methods of Use, Pace, Precision, Ability3. Intuitive – Wide Range of Knowledge and Experience4. Perceptible Information – Graphic, Verbal, Text, Touch5. Error Tolerance – Minimize Impact of Mistakes (e.g. Undo)6. Low Physical Effort – Comfortable, Minimal Fatigue7. Size and Space – Sitting or Standing, Body Size, Posture,

Mobility Sam Siewert 11

SpeechGeneration – Text to Speech - SynthesisRecognition – Speech to Text, to Syntax and Semantics

Sam Siewert 12

Speech RecognitionBetter Today, But Still in Limited Use

Google – Demonstration Page

Sam Siewert 13

Speech SynthesisScreen Readers, Web Readers

http://www.bobby-approved.com/

Sam Siewert 14

Uses for Speech Recognition and Synthesis

Where and When to Use

Sam Siewert 15

Haptic InterfacesHaptics - Touch– Cutaneous (skin tactile sensations)– Kinesthetics – movement perception and position

Current Use – Surgical Training, e.g. LapSim

Sam Siewert 16

Handwriting SystemsRecord – e.g. Signatures (FedEx, UPS, DocuSign, …)AOS Swipe in Settings (Hybrid Handwriting/Typing)Recognize Input– No Systems Provide General Cursive Script Recognition– Some Success with Character at a Time Input– Apple Newton (Limited Success)

Touch Screen Typing

Sam Siewert 17

Apple Newton – Stylus PDA

Gesture RecognitionCameras (Active or Passive) with Depth Mapping, Segmentation and TrackingData Gloves – Rotation (Yaw, Pitch, Roll), Translation (X,Y,Z), Rates of Translation/Rotation, and Finger Flexures

Camera Alone, Glove Alone, or Sensor Fusion

Glove is Intrusive, but Improves Accuracy

Challenge – Full Speed American Sign Language Recognition

Sam Siewert 18

3D Active Computational Photometry Concept (Rev-A + TI Kit)

Sam Siewert 19

AnalogCamera #1

RGB(Visible)

Altera FPGACVPU

(Computer Vision Processing Unit)

MobileSensor Network

Processor(TI OMAP, Atom)

Networked Video

Analytics

HD DigitalCamera Port(Snapshot)

USB 2.0, PCIeHost Channels

Flash SD Card

AnalogCamera #2

(Near Infrared)

TI DLP Light-crafter Kithttp://www.ti.com/tool/dlplightcrafter

Depth Map

IR Pattern Projection

https://www.cs.purdue.edu/homes/aliaga/cs635-10/lec-structured-light.pdf

Photo credits and reference:Dr. Daniel Aliaga, Purdue Universityhttps://www.cs.purdue.edu/homes/aliaga/

3D Computer Vision TransformsLong Range ( > 5 meters) Using Passive Binocular Methods– Impractical to Project from a UAV or Long Range Observer– Requires Image Registration– Accurate Camera Intrinsic (Camera Characteristics) & Extrinsic (e.g. Baseline)

Short Range ( < 5 meters), Structured IR Light Projection for RGB-D– Compare to ASUS Xtion and PrimeSense – Off-the-Shelf– Robust Depth Maps with Less Noise– Showing Significant Promise to Improve CV Scene Segmentation and Object

Recognition Compared to 2D– “Change Their Perception”, By Xiaofeng Ren, Dieter Fox, and Kurt Konolige,

IEEE RAS, December 2013.

Sam Siewert 20

Noise in Passive Depth Maps

Robust Active Depth Map“Change Their Perception”, By Xiaofeng Ren, Dieter Fox, and Kurt Konolige, IEEE RAS, December 2013.

Design for Users with DisabilitiesMinority of Market, but Support Required

Visual ImpairmentHearing ImpairmentPhysical ImpairmentSpeech ImpairmentCognitive Impairments – Dyslexia, Autisim Sam Siewert 21

Cultural and Age ConsiderationsElderly, Age 50+ TabletChildren, Kids Tablet – e.g. IPAD Mini 2, Amazon Fire Kids Edition, etc.Language Localization – Fundamental Translation, Keyboard, Graphics for Native Language Support

Cultural Differences – E.g. Nodding or Shaking Head in Agreement

Global Marketplace – High Value Today with Globalization

Sam Siewert 22

Cognitive Models - IntroductionChapter 11 of HCI - User Support Systems (Built-in Help)

Chapter 12 of HCI - Cognitive Models– Hierarchical Models – Task and Goal Structure

GOMS (Goals, Operators, Methods, Selection), p. 421-425Goal - what you want to achieveOperators - basic actionsMethods - splitting goal into many sub-goalsSelection - choice of method for more than one

– Linguistic Models – System Grammar

– Physical and Device Models – Human Motor Skills

– Cognitive Models – Combine All 3 of the Above

Sam Siewert 23