blueeyes report
TRANSCRIPT
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 1/25
A
SEMINAR
ONBLUE EYES TECHNOLOGY
By:-
Rohan Chugh
1531323107
Department of Information Technology
Guru Tegh Bahadur Institute of Technology
Guru Gobind Singh Indraprastha University
Kashmere Gate,New Delhi
Year 2010-2011
1
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 2/25
ACKNOWLEDGEMENT
I would like to express my great gratitude towards, Mr. Munshi Yadav
(H.O.D CSE/ IT Department) and Mr. Amanpreet Singh who has given
me support and suggestions. Without their help I could not have
presented this dissertation up to the present standard. I also take this
opportunity to thank everyone who helped me directly or indirectly in the
completion of this seminar.
Rohan Chugh
1531323107
2
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 3/25
ABSTRACT
A study has shown that people with personalities that are similar collaborate well. Dryer (1999) has also shown that people view their
computer as having a personality. For these reasons, it is important to
develop computers which can work well with its user. This is where the
blue eyes technology comes into force. With the abilities to gather
information about you and interact with you through special techniques
like facial recognition, speech recognition, etc., it can even understand
your emotions at the touch of the mouse. It verifies your identity, feels
your presence, and starts interacting with you. So the ultimate aim of
BLUEEYES TECHNOLOGY is to create a user friendly system that canincrease the productivity of the user by adapting itself to the user’s mood.
With the eye tracker technology it can reduce the user’s work load. It will
move the cursor directly to the place where the user is staring at. The
additional facility of speech recognition bestows you with the ability to
just dictate commands to the computer while you can engage yourself
with some other work. Using eye tracker technology the computer can
use the SUITOR facility to provide the user additional info about the
topic he is reading/ looking at in a tab. This will for sure reduce his job of
having to search for the information all by himself. So this concept of BLUEEYE TECHNOLOGY IS SURELY A BOON. After all who
would not want to own a computer that:
(1) Can easily adapt to your mood,
(2) Perform tasks the moment you “actually tell” it what you want
(3) Help you by gathering more additional information
(4) Saves you the job of having to move your cursor to a long distance
(5) With further additions, can even help to monitor your health.
3
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 4/25
LIST OF FIGURES
Fig No Figure Name Page
1.1 The BlueEye system with the camera(s) 9
mounted on top and the display integrated
into the table surface.
3.1 Emotional Mouse 11
3.2 Bright and dark pupil images 12
4
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 5/25
CONTENTS
Chapter Page No.
Title Page 1
Acknowledgement 2
Abstract 3
Tables and figures 4
1. Introduction 6
1.1 Aim 6
1.2 Objective 7
2. Definition of BlueEyes Technology 8
3. Technologies used 10
3.1 Emotion Mouse 10
3.2 MAGIC 11
3.3 Artificial Intelligent Speech Recognition 13
3.4 SUITOR 16
4 System Overview 18
4.1 Parts of BlueEye system 18
4.1.1 DAU
4.2.2 CSU
5 Applications and Future Scope 23
7. References 25
5
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 6/25
CHAPTER 1
INTRODUCTION
Is it possible to create a computer which can interact with us as we
interact each other? For example imagine in a fine morning you walk on
to your computer room and switch on your computer, and then it tells you
“Hey friend, good morning you seem to be a bad mood today. And then it
opens your mailbox and shows you some of the mails and tries to cheer
you. It seems to be a fiction, but it will be the life lead by “BLUE EYES”
in the very near future. The basic idea behind this technology is to give
the computer the human power. We all have some perceptual abilities.
That is we can understand each others feelings. For example we can
understand ones emotional state by analyzing his facial expression. If we
add these perceptual abilities of human to computers would enable
computers to work together with human beings as intimate partners. The
“BLUE EYES” technology aims at creating computational machines that
have perceptual and sensory ability like those of human beings.
Blue Eyes uses sensing technology to identify a user's actions and to
extract key information. This information is then analyzed to determine
the user's physical, emotional, or informational state, which in turn can be
used to help make the user more productive by performing expected
actions or by providing expected information. For example, in future a
Blue Eyes-enabled television could become active when the user makes
eye contact, at which point the user could then tell the television to "turn
on". This paper is about the hardware, software, benefits and
interconnection of various parts involved in the “blue eye” technology.
6
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 7/25
1.1 Aim
The aim of this seminar is to study about BlueEyes Technologyand
have a deep insight of latest developments taking place in this field.
1.2 Objectives
• Research current methods to implement BlueEyes technology.
• Research into latest developments in this area.
• To realise the applications of BlueEyes technology.
7
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 8/25
CHAPTER 2
1.WHAT IS BLUEEYES TECHNOLOGY?
Animal survival depends on highly developed sensory abilities. Likewise,
human cognition depends on highly developed abilities to perceive,
integrate, and interpret visual, auditory, and touch information. Without a
doubt, computers would be much more powerful if they had even a small
fraction of the perceptual ability of animals or humans. Adding such
perceptual abilities to computers would enable computers and humans to
work together more as partners. Toward this end, the Blue Eyes aims at
creating computational devices with the sort of perceptual abilities that
people take for granted Blue eyes is being developed by the team of
Poznan University of Technology& Microsoft. It makes use of the “blue
tooth technology “developed by Ericsson.
BlueEyes is the name of a human recognition venture initiated by
IBM to allow people to interact with computers in a more natural manner.
The technology aims to enable devices to recognize and use natural input,
such as facial expressions. The initial developments of this project
include scroll mice and other input devices that sense the user's pulse,
monitor his or her facial expressions, and the movement of
his or her eyelids. BlueEyes technology aims at creating computational
machines that have perceptual and sensory ability.It uses cameras and
microphones to identify user actions and emotions. BLUE in the term
stands for Bluetooth, which enables reliable wireless communication and
EYES has been used because the eye movement enables us to obtain a lot
of interesting and important information. It doesn’t predict nor interfere
with operator’s thoughts nor can it force directly the operator to work.
8
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 9/25
Figure 1.1 The BlueEye systems with the camera(s) mounted on top and
the display integrated into the table surface.
9
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 10/25
CHAPTER 3
2.TECHNOLOGIES USED
To implement BlueEyes technology four main technologies are used:
Emotion Mouse
Manual And Gaze Input Cascaded (MAGIC)
Artificial Intelligent Speech Recognition
Simple User Interest Tracker (SUITOR)
3.1 Emotion Mouse
By matching a person’s emotional state and the context of the
expressed emotion, over a period of time the person’s personality is being
exhibited. Therefore, by giving the computer a longitudinal
understanding of the emotional state of its user, the computer could adapt
a working style which fits with its user’s personality. The result of this
collaboration could increase productivity for the user. One way of gaining
information from a user non-intrusively is by video. Cameras have been
used to detect a person’s emotional state (Johnson, 1999). We have
explored gaining information through touch. One obvious place to put
sensors is on the mouse. Through observing normal computer usage
(creating and editing documents and surfing the web), people spend
approximately 1/3 of their total computer time touching their input
device. Because of the incredible amount of time spent touching an input
device, we will explore the possibility of detecting emotion through
touch.
10
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 11/25
Figure 3.1 Emotional Mouse
3.2 Manual and Gaze Input Cascaded (MAGIC)
This is a way of providing input to the computer through eye gaze.
With such an approach , pointing on screen appears to the user a manual
task, used for fine manipulations and selections. In this a large portion of
the cursor movement is eliminated by warping the cursor to the eye gaze
area, which encompasses the target. This technique has many advantages
including reduced physical effort and fatigue, greater accuracy and
naturalness and faster speed. Here we have limited to the most advanced
technique of MAGIC i.e. the IBM Alamaden eye tracker.
3.2.1 The IBM Almaden Eye Tracker
This system is much more compact and reliable. However, it is still
not robust enough for a variety of people with different eye
characteristics, such as pupil brightness and correction glasses. Available
commercial systems, such as those made by ISCAN Incorporated, LC
Technologies, and Applied Science Laboratories (ASL), rely on a single
light source that is positioned either off the camera axis in the case of the
ISCANETL-400 systems, or on-axis in the case of the LCT and the ASL
11
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 12/25
E504 systems. Illumination from an off-axis source (or ambient
illumination) generates a dark pupil image.
When the light source is placed on-axis with the camera optical
axis, the camera is able to detect the light reflected from the interior of
the eye, and the image of the pupil appears bright
Figure 3.2: Bright (left) and dark (right) pupil images resulting from on-
and off-axis illumination. The glints, or corneal reflections, from the on-
and off-axis light sources can be easily identified as the bright points in
the iris.
Almaden system uses two near infrared (IR) time multiplexed lightsources, composed of two sets of IR LED's, which were synchronized
with the camera frame rate. One light source is placed very close to the
camera's optical axis and is synchronized with the even frames. Odd
frames are synchronized with the second light source, positioned off axis.
The two light sources are calibrated to provide approximately equivalent
whole-scene illumination. Pupil detection is realized by means of
subtracting the dark pupil image from the bright pupil image. After
thresholding the difference, the largest connected component is identified
as the pupil. This technique significantly increases the robustness and
reliability of the eye tracking system.
Once the pupil has been detected, the corneal reflection (the glint reflected
from the surface of the cornea due to one of the light sources) is
determined from the dark pupil image. The reflection is then used to
12
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 13/25
estimate the user's point of gaze in terms of the screen coordinates where
the user is looking at. The estimation of the user's gaze requires an initial
calibration procedure.
3.3
Artificial Intelligent Speech Recognition
It is important to consider the environment in which the speech
recognition system has to work. The grammar used by the speaker and
accepted by the system, noise level, noise type, position of the
microphone, and speed and manner of the user’s speech are some factors
that may affect the quality of speech recognition .When you dial the
telephone number of a big company, you are likely to hear the sonorous
voice of a cultured lady who responds to your call with great courtesy
saying “Welcome to company X. Please give me the extension number
you want”. You pronounce the extension number, your name, and the
name of person you want to contact. If the called person accepts the call,
the connection is given quickly. This is artificial intelligence where an
automatic call- handling system is used without employing any telephone
operator.
3.3.1 The Technology
Artificial intelligence (AI) involves two basic ideas. First, it involves
studying the thought processes of human beings. Second, it deals with
representing those processes via machines (like computers, robots, etc).
AI is behavior of a machine, which, if performed by a human being,
would be called intelligent. It makes machines smarter and more useful,
and is less expensive than natural intelligence. Natural language
processing (NLP) refers to artificial intelligence methods of
communicating with a computer in a natural language like English. Themain objective of a NLP program is to understand input and initiate
13
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 14/25
action. The input words are scanned and matched against internally stored
known words. Identification of a key word causes some action to be
taken. In this way, one can communicate with the computer in one’s
language. No special commands or computer language are required.
There is no need to enter programs in a special language for creating
software.
3.3.2 Speech Recognition
The user speaks to the computer through a microphone with a simple
system that may contain a minimum of three filters. The more the number
of filters used, the higher the probability of accurate recognition.
Presently, switched capacitor digital filters are used because these can be
custom-built in integrated circuit form. These are smaller and cheaper
than active filters using operational amplifiers. The filter output is then
fed to the ADC to translate the analogue signal into digital word. The
ADC samples the filter outputs many times a second. Each sample
represents different amplitude of the signal. Evenly spaced vertical lines
represent the amplitude of the audio filter output at the instant of
sampling. Each value is then converted to a binary number proportional
to the amplitude of the sample. A central processor unit (CPU) controls
the input circuits that are fed by the ADCS. A large RAM (random access
memory) stores all the digital values in a buffer area. This digital
information, representing the spoken word, is now accessed by the CPU
to process it further. The normal speech has a frequency range of 200 Hz
to 7 kHz. Recognizing a telephone call is more difficult as it has
bandwidth limitation of 300 Hz to3.3 kHz.
As explained earlier, the spoken words are processed by the filters
and ADCs. The binary representation of each of these words becomes a
template or standard, against which the future words are compared. These
templates are stored in the memory. Once the storing process is
completed, the system can go into its active mode and is capable of
14
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 15/25
identifying spoken words. As each word is spoken, it is converted into
binary equivalent and stored in RAM. The computer then starts searching
and compares the binary input pattern with the templates. t is to be noted
that even if the same speaker talks the same text, there are always slight
variations in amplitude or loudness of the signal, pitch, frequency
difference, time gap, etc. Due to this reason, there is never a perfect
match between the template and binary input word. The pattern matching
process therefore uses statistical techniques and is designed to look for
the best word.
The values of binary input words are subtracted from the
corresponding values in the templates. If both the values are same, the
difference is zero and there is perfect match. If not, the subtraction
produces some difference or error. The smaller the error, the better the
match. When the best match occurs, the word is identified and displayed
on the screen or used in some other manner. The search process takes a
considerable amount of time, as the CPU has to make many comparisons
before recognition occurs. This necessitates use of very high-speed processors. A large RAM is also required as even though a spoken word
may last only a few hundred milliseconds, but the same is translated into
many thousands of digital words. It is important to note that alignment of
words and templates are to be matched correctly in time, before
computing the similarity score. This process, termed as dynamic time
warping, recognizes that different speakers pronounce the same words at
different speeds as well as elongate different parts of the same word. This
is important for the speaker-independent recognizers.
3.3.3 Applications
One of the main benefits of speech recognition system is that it lets user
do other works simultaneously. The user can concentrate on observation
and manual operations, and still control the machinery by voice input
commands. Another major application of speech processing is in military
15
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 16/25
operations. Voice control of weapons is an example. With reliable speech
recognition equipment, pilots can give commands and information to the
computers by simply speaking into their microphones—they don’t have
to use their hands for this purpose. Another good example is a radiologist
scanning hundreds of X- rays, ultrasonograms, CT scans and
simultaneously dictating conclusions to a speech recognition system
connected to word processors. The radiologist can focus his attention on
the images rather than writing the text. Voice recognition could also be
used on computers for making airline and hotel reservations. A user
requires simply to state his needs, make reservation, cancel a reservation,
or make enquiries about schedule
3.4 The Simple User Interest Tracker (SUITOR)
Computers would have been much more powerful, had they
gained perceptual and sensory abilities of the living beings on the earth.
What needs to be delivered is an intimate relationship between the
computers and the humans. SUITOR is a revolutionary approach in this
direction.
By observing the Webpage the subject is browsing, the SUITOR
can help by fetching more information on the desktop. By Simply
noticing where the user ‘s eye focus on the computer screen , the
SUITOR can be more precise in determining his topic of interest. It can
even deliver relevant information to a handheld device. The success lies
in how much the SUITOR can be intimate to the user. IBM’s BlueEyes
research concentrated on if we can explore non verbal cues tp create more
effective user interface.
One such cue is gaze-the direction in which a person is looking.
IBM’s research has incorporated gaze-tracking technology into two
prototypes. One, called SUITOR, fills a scrolling ticker on a computer
16
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 17/25
screen with information related to users task. SUITOR knows where you
are looking, what applications you are running and what page you may be
browsing. For example if one is reading the headline of a story, it pops up
the story in the browser window. If he reads story the computer adds
related stories to the ticker.
17
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 18/25
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 19/25
connection, buffers incoming sensor data, performs on-line data analysis,
records the conclusions for further exploration and provides visualization
interface.
4.1.2.1 CSU Components
Connection Manager – main task to perform low-level blue tooth
communication.
Data Analysis Module – performs the analysis of the raw sensor
data in order. to obtain information about operator’s physiological
condition
Data Logger Module – provides support for storing the monitored
data.
Visualization Module – provides user interface for the
supervisors.
19
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 20/25
CHAPTER 5
4.APPLICATIONS
1. Computers Can Anticipate User’s Wants
BlueEyes is developing ways for computers to anticipate users'
wants by gathering video data on eye movement and facial
expression. Your gaze might rest on a Web site heading, for example,
and that would prompt your computer to find similar links and to call
them up in a new window. But the first practical use for the research
turns out to be snooping on shoppers.
BlueEyes software makes sense of what the cameras see to answer key questions for retailers, including, How many shoppers ignored a
promotion? How many stopped? How long did they stay? Did their
faces register boredom or delight? How many reached for the item
and put it in their shopping carts? BlueEyes works by tracking pupil,
eyebrow and mouth movement. When monitoring pupils, the system
uses a camera and two infrared light sources placed inside the product
display. One light source is aligned with the camera's focus; the other
is slightly off axis. When the eye looks into the camera-aligned light,
the pupil appears bright to the sensor, and the software registers thecustomer's attention. This is way it captures the person's income and
buying preferences. BlueEyes is actively been incorporated in some
of the leading retail outlets.
2. In Automobile Industry
Another application would be in the automobile industry. By
simply touching a computer input device such as a mouse, the
computer system is designed to be able to determine a person'semotional state. For cars, it could be useful to help with critical
decisions like:
"I know you want to get into the fast lane, but I'm afraid I can't do
that. You are
too upset right now" and therefore assist in driving safely.
3. Face-responsive Displays" and "Perceptive Environments
Current interfaces between computers and humans can present
information vividly, but have no sense of whether that information is
20
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 21/25
ever viewed or understood. In contrast, new real-time computer vision
techniques for perceiving people allows us to create "Face-responsive
Displays" and "Perceptive Environments", which can sense and
respond to users that are viewing them. Using stereo-vision
techniques, we are able to detect, track, and identify users robustlyand in real time. This information can make spoken language
interface more robust, by selecting the acoustic information from a
visually-localized source. Environments can become aware of how
many people are present, what activity is occurring, and therefore
what display or messaging modalities are most appropriate to use in
the current situation. The results of our research will allow the
interface between computers and human users to become more
natural and intuitive.
4. Video Games
We could see its use in video games where, it could give
individual challenges to customers playing video games. Typically
targeting commercial business.
The integration of children's toys, technologies and computers is
enabling new play experiences that were not commercially feasible
until recently. The Intel Play QX3 Computer Microscope, the
Me2Cam with Fun Fair, and the Computer Sound Morpher are
commercially available smart toy products developed by the Intel
Smart Toy Lab in . One theme that is common across these PC-
connected toys is that users interact with them using a combination of
visual, audible and tactile input & output modalities. The presentation
will provide an overview of the interaction design of these products
and pose some unique challenges faced by designers and engineers of
such experiences targeted at novice computer
users, namely young children.
5. Less Explicit Communication
The familiar and useful come from things we recognize. Many of
our favorite things' appearance communicate their use; they show the
change in their value though patina. As technologists we are now
poised to imagine a world where computing objects communicate
with us in-situ; where we are. We use our looks, feelings, and actions
to give the computer the experience it needs to work with us.
Keyboards and mice will not continue to dominate computer user
interfaces. Keyboard input will be replaced in large measure bysystems that know what we want and require less explicit
21
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 22/25
communication. Sensors are gaining fidelity and ubiquity to record
presence and actions; sensors will notice when we enter a space, sit
down, lie down, pump iron, etc. Pervasive infrastructure is recording
it. This talk will cover projects from the Context Aware Computing
Group at MIT Media Lab.
22
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 23/25
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 24/25
8/3/2019 Blueeyes Report
http://slidepdf.com/reader/full/blueeyes-report 25/25
CHAPTER 7
6. REFRENCES
1. www.alphaworks.ibm.com/awap.nsf/alltype/86ECE8569056DDC7
88256AEE007A9702
2. www.cse.ogi.edu/CHCC/Series/text2001.html
3. BlueEyes Technology, Computer Edge, Oct.2002, pages 23-27.
4. BlueEyes Technology and Applications, Business Solutions, Nov
2001,pages 95-99.
5. www.cs.put.poznan.com
6. www.whitepapers.com
7. www.ibmresearchcenter.com
8. www.almaden.ibm.com
9. Carpenter R. H. S., Movements of the eyes, 2nd edition, Pion
Limited, 1988, London.
10. Quinlan J. R., C4.5 programs for machine learning, Morgan
Kaufmann Publishers, 1993, San Mateo, California.
11. http://www.articlesbase.com/networks-articles/blue-eyes-
monitoring-human-operator-system-420263.html
12. Milgram, P. and Kishino, F. (1994) A Taxonomy of Mixed Reality
Visual Displays, IEICE Transactions on Information Systems,
1321-1329.