google glass app for colorblind individuals and people with impaired vision

www.tugraz.at

Institute of Information Systems and Computer Media

OmniColorGoogle Glass app for colorblind individuals and

people with impaired vision

December 15th, 2016

Georg Lausegger, BSc.

2 Agenda

1. Motivation

2. Colorblindness

3. Smartglasses

4. OmniColor Prototype

5. Evaluation and Results

6. Outlook

Georg Lausegger, BSc.,December 15th, 2016

2/16

3

Motivation

Motivation

Fig. 1: Raw meat

Fig. 2: Clothes

Fig. 3: Traffic Lights, taken from wikipedia.org1

Fig. 4: Bananas, taken from banabay.com2

(1) https://de.wikipedia.org/wiki/Ampel , (2) http://www.banabay.com/press-room/going-green/


3/16

4

Colorblindness

Colorblindness

Human Eye

Consists of rods and cones

Cones classified in three different types

L (long-) wavelength (Red)M (middle-) wavelength (Green)S (short-) wavelength (Blue)

Colorblindness = lack or disfunction of one or morecone type(s)


4/16

5

Colorblindness

Colorblindness types

Anomalous trichromacy

ProtanomalyDeuteranomalyTritanomaly

Dichromatism

ProtanopiaDeuteranopiaTritanopia

Monochromacy

Fig. 5: The three colorblindness types, taken fromJohaness Ahlmann3

(3) https://www.flickr.com/photos/entirelysubjective/6146852926


5/16

6

Colorblindness

Colorblindness Testing Methods

Color arrangement test

Ishihara color plate test

Fig. 6: Color arrangement test, taken fromcolor-blindness.com4

Fig. 7: Ishihara color plate test, taken fromcolor-blindness.com4

(4) http://www.color-blindness.com/


6/16

7

Colorblindness

Daltonization

Doesn’t affect each pixel

Color shifting algorithm

Algorithm consisting of 4 steps:

RGB to LMS color spaceSimulation of the specific color blindness typeCalculate compensationConvert LMS values back to RGB


7/16

8

Smartglasses

Devices

Smartphones

+ computation power

+ distribution

+ costs

- handfree navigation

- availability

Smartglasses

+ handfree navigation

+ availability

- computation power

- distribution

- costs


8/16

9

Smartglasses

Google Glass

Optical Head-mounteddisplay

Released in February2013

Upgraded Model(2GBRAM)

costs ∼1500$

Input via voicecommands, touchpad orhead gestures

Fig. 8: Sergey Brin wearing Google Glass, taken fromAndrew Kelly/Reuters5

(5) http://www.computerworld.com/article/2495675/personal-technology/google-s-sergey-brin-rips-smartphones–shows-off-glass.html


9/16

10

OmniColor Prototype

OmniColor Prototype

Non-real time

All computations performed on Glass

No network connection required

Uses OpenCV library

OmniColor Flow Design


10/16

images/omnicolor_flowdesign.png

11

Evaluation and Results

Evaluation

Ishihara color plate test

without OmniColorwith OmniColor

Individual appointments

14 Participants

Five colorblind peopleNine people with normalcolor vision

Fig. 9: Ishihara color plate test performedwith by participant with OmniColor


11/16

12

Evaluation and Results

Results (Colorblinds)

Participantage

CVD* Without OmniColor With OmniColor

36 protanomaly 7/17 12/17





Tab. 1: Results of the Ishihara color plate test performed by colorblind participants

*Color Vision Deficiency


12/16

13

Outlook

Outlook

Hardware issues with Google Glass

Google Glass discontinued in January 2015

Great potential of smartglasses

Further work

Other approaches and algorithmsPerformance optimization (offloading)Support for other Android Smartglasses


13/16

14

Thank you for yourattention!


14/16

15 References I

[1] David Flatla and Carl Gutwin. “SSMRecolor: Improving Recoloring Tools with Situation-specificModels of Color Differentiation”. In: Proceedings of the SIGCHI Conference on Human Factors inComputing Systems. CHI ’12. Austin, Texas, USA: ACM, 2012, pp. 2297–2306. ISBN:978-1-4503-1015-4. DOI: 10.1145/2207676.2208388. URL:http://doi.acm.org/10.1145/2207676.2208388.

[2] Jia-Bin Huang et al. “IEEE Int’l Conf on Acoustics, Speech and Signal Processing, (ICASSP2009)”. In: Image Recolorization For The Colorblind. Apr. 2009.

[3] Luke Jefferson and Richard Harvey. “An Interface to Support Color Blind Computer Users”. In:Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI ’07. SanJose, California, USA: ACM, 2007, pp. 1535–1538. ISBN: 978-1-59593-593-9. DOI:10.1145/1240624.1240855. URL: http://doi.acm.org/10.1145/1240624.1240855.

[4] D. S. Khurge and B. Peshwani. “Modifying Image Appearance to Improve Information Content forColor Blind Viewers”. In: Computing Communication Control and Automation (ICCUBEA), 2015International Conference on. Feb. 2015, pp. 611–614. DOI: 10.1109/ICCUBEA.2015.125.

[5] Viet Nguyen and Marco Gruteser. “First Experiences with GOOGLE GLASS in Mobile Research”.In: GetMobile: Mobile Comp. and Comm. 18.4 (Jan. 2015), pp. 44–47. ISSN: 2375-0529. DOI:10.1145/2721914.2721931. URL: http://doi.acm.org/10.1145/2721914.2721931.

[6] Helio M. de Oliveira, J. Ranhel, and R. B. A. Alves. “Simulation of Color Blindness and a Proposalfor Using Google Glass as Color-correcting Tool”. In: CoRR abs/1502.03723 (2015). URL:http://arxiv.org/abs/1502.03723.


15/16

https://doi.org/10.1145/2207676.2208388

http://doi.acm.org/10.1145/2207676.2208388

https://doi.org/10.1145/1240624.1240855

http://doi.acm.org/10.1145/1240624.1240855

https://doi.org/10.1109/ICCUBEA.2015.125

https://doi.org/10.1145/2721914.2721931

http://doi.acm.org/10.1145/2721914.2721931

http://arxiv.org/abs/1502.03723

16 References II

[7] Brooke E. Schefrin. “Diagnosis of Defective Colour Vision, by Jennifer Birch, Oxford UniversityPress, New York, 1993, Paperback, 187 pp., $35.00.” In: Color Research & Application 19.6(1994), pp. 484–484. ISSN: 1520-6378. DOI: 10.1002/col.5080190608. URL:http://dx.doi.org/10.1002/col.5080190608.

[8] N. A. Semary and H. M. Marey. “An evaluation of computer based color vision deficiency test:Egypt as a study case”. In: Engineering and Technology (ICET), 2014 International Conferenceon. Apr. 2014, pp. 1–7. DOI: 10.1109/ICEngTechnol.2014.7016817.


16/16

https://doi.org/10.1002/col.5080190608

http://dx.doi.org/10.1002/col.5080190608

https://doi.org/10.1109/ICEngTechnol.2014.7016817

google glass app for colorblind individuals and people with impaired vision

Education