blue eye technology - 123seminarsonly.com · bluetooth technology provides means for creating a...
TRANSCRIPT
![Page 1: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/1.jpg)
![Page 2: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/2.jpg)
Blue eye technologyBlue eye technology
( blue eyes –Human operator
monitoring system )
BY
![Page 3: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/3.jpg)
AgendaAgenda
n Motivationn What is BlueEye technology ?n What is BlueEyes ?n System designingn System overviewn DAUn CSUn Data securityn IBM researchn Conclusion
![Page 4: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/4.jpg)
MotivationMotivationn Human error - a frequent reason of
catastrophes and ecological disasters
• Lack of complex solution exploiting wireless link technology, database systems and AI algorithms
– temporal indisposition
– weariness – oversight
• Monitoring of consciousbrain involvement
![Page 5: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/5.jpg)
What is blue eye What is blue eye technology ?technology ?n The BLUE EYES technology
aims at creating computational machines that have perceptual and sensory ability like those of human beings.
![Page 6: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/6.jpg)
How can we make How can we make computers “see” and computers “see” and “feel” ?“feel” ?n Blue eyes uses sensing technology to
identify a user’s actions and to extract key information.
n Information is then analyzed to determine the user’s physical ,emotional ,or informational state.
![Page 7: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/7.jpg)
What is What is Blue EyesBlue Eyes ? ?
n The complex solution for human-operator monitoring:– Visual attention monitoring– Physiological condition– Operator’s position detection– Wireless data acquisition using bluetooth
![Page 8: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/8.jpg)
n Doesn’t predict nor interfere with operator’s thoughts
n Cannot force directly the operator to work
What is BlueEyes not?What is BlueEyes not?
![Page 9: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/9.jpg)
IBM researchIBM research
n BLUE EYE – EMOTIONAL MOUSEsensors in the mouse ,sense the
physiological attributes which are correlated to emotions using correlation model
-by simply touching the mouse ,the computer will be able to determine a person’s emotional state.
n BLUE EYE enabled TELEVISION – could become active when the user makes an eye contact.
![Page 10: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/10.jpg)
Physiological Physiological foundationsfoundations
Preconscious brain
• manages the knowledge • logical reasoning• makes decisions
• alogical• „automatic”
Conscious brain
Human brain
![Page 11: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/11.jpg)
Conscious brain - operation Conscious brain - operation modesmodes
Exploratory mode Planning mode
![Page 12: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/12.jpg)
The eye movement The eye movement sensorsensorn Off-shelf eye movement sensor – Jazz
multisensorn Supplies raw digital data regarding- Eye position- Level of blood oxygenation- Acceleration along horizontal and
vertical axes- Ambient light intensity
![Page 13: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/13.jpg)
Eye movementsEye movements
Saccade - a rapid eye jump to a new location assigned by the conscious attention process.
Saccades presence Exploratory mode
• Central vision andthe conscious brain
• Retina - central vision (macula)
![Page 14: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/14.jpg)
Jazz multisensorJazz multisensor
Jazz MultisensorJazz Multisensor
![Page 15: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/15.jpg)
JazzJazz
![Page 16: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/16.jpg)
• A reliable real-time buffering, processing and recording
• A clear visualization interface• Mechanisms for creating custom data
analyzers• To ensure a possibility of distributing the
processing among a number of computers
Design objectivesDesign objectivesn A mobile data acquisition device
![Page 17: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/17.jpg)
Implementation and Implementation and engineering engineering considerationsconsiderations-THREE groups of usersn OPERATOR -a person whose physiological parameters are
supervised - authorization in the system - receiving alertsn SUPERVISOR – a person responsible for analyzing operator’s condition and performance.n SYSTEM ADMINISTRATORS- maintains the system deliver tools for adding - new operator’s to the database - defining alarm conditions - configuring logging tools - creating new analyzer modules
![Page 18: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/18.jpg)
DesigningDesigning
n A personal area network for linking all the operators and the supervising system
n Two major units - DAU (data acquisition unit )- CSU (central system unit )
![Page 19: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/19.jpg)
System overviewSystem overview
Central System UnitData Acquisition Unit
Atmel 89C52microcontroll
er
JazzMultisensor
ConnectionManager
Data Analysis
Visualization
Module
Bluetooth
Bluetooth
Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system.
DataLogger
![Page 20: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/20.jpg)
DAUDAU
The DAU consists of the following componentsn ATMEL 8952 microcontrollern BLUE TOOTH MODULE – supports synchronous voice
data transmissionn PCM CODEC –used to transmit operator’s voice and
central system sound feedbackn UART –communication between bluetooth module and
microcontroller (115200 bps)n MAX232 –level shiftern ALPHAUNUMERIC LCD displayn LED indicatorsn ID CARD interface
![Page 21: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/21.jpg)
DAU - featuresDAU - features
n Lightweightn Runs on batteries -
low power consumption
n Easy to use - does not disturb the operator working
n ID cards for operator authorization
Data Acquisition Unit
Atmel 89C52microcontroll
er
JazzMultisensor
Bluetooth
is
![Page 22: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/22.jpg)
BlueEyesData Acquisition Unit
ID card
DAU - componentsDAU - components
MC 145483PCM codec
ID cardinterface
Simplekeyboar
d
Jazz Multisensor
Beeper
LCD display
LED indicators
Atmel 89C52microcontroller
UARTMAX2
32
UART
Bluetooth
modul
microphone
earphone
![Page 23: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/23.jpg)
Microcontroller & blue Microcontroller & blue tooth moduletooth module
![Page 24: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/24.jpg)
EEPROMS & the PCBEEPROMS & the PCB
![Page 25: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/25.jpg)
Implementation - DAUImplementation - DAU
n Bluetooth module insiden Jazz sensor
interfacen Microphone and
earphone socketsn 5-key keyboardn Two lines, 8
characters LCDn Powered from
6 AA batteries
n• Made from scratch by the team
![Page 26: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/26.jpg)
Implementation - DAUImplementation - DAU
Bluetooth module
PCM codec board
Batteries andvoltage level monitor
Microcontroller board
![Page 27: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/27.jpg)
CSU - featuresCSU - featuresCentral System Unit
ConnectionManager
Data Analysis
Visualization
Module
Bluetooth
n Connection Connection managementmanagement
n Data Data processingprocessing
n VisualizationVisualizationn Data recordingData recordingn AccessAccess
verification verification n System maintenanceSystem maintenance
ConnectionManager
DataLogger
![Page 28: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/28.jpg)
CSU – componentsCSU – components
n CONNECTION MANAGER – main task to perform low-level blue tooth communication
n DATA ANALYSIS MODULE – performs the analysis of the raw sensor data in order to obtain information about operator’s physiological condition
n DATA LOGGER MODULE – provides support for storing the monitored data.
n VISULAIZATION MODULE – provides user interface for the supervisors
![Page 29: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/29.jpg)
CSU - Visualization CSU - Visualization ModuleModule
n Raw and processed data visualization Raw and processed data visualization using:using:
n audio and videoaudio and videodata playbackdata playback
– VU-metersVU-meters– pie-chartspie-charts– time seriestime series
![Page 30: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/30.jpg)
CSU - saccade detectorCSU - saccade detectorn Saccade detector and conscious brain
involvement level calculationEye movement velocity
-250-200-150-100
-500
50100150200250
0 8000 16000 24000 32000Time [ms]
Ve
locit
y
Raw eyemovement
data
Differentiation
Conscious brain involvement level
Visual attention level
0
10
20
30
40
50
60
0 8000 16000 24000 32000
Time [ms]
Att
en
tio
n le
vel
(L
va
)
![Page 31: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/31.jpg)
• Initializes Bluetooth modules
• Inquires for new devices in range
• Establishes connections • Supports operator
roaming• Feeds Operator Manager
with physiological data
Bluetooth
Bluetooth
...
...
CSU - Connection CSU - Connection ManagerManager
BluetoothConnection
Manager
OperatorData
Manager
Alerts
Separateddata streams
...
...
OperatorManager
Alert buffer
Raw DataBuffer
Alerts
OperatorManager
Alert buffer
Raw DataBuffer
•Data demultiplexer•Data producer queues
![Page 32: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/32.jpg)
CSU - Data AnalysisCSU - Data Analysis• Custom data analyzers - decision tree based classifiers
• Automatic decision tree coefficients generation - C4.5 based algorithm
Data samplesAssignedcategory
“Yes”
No answer
“Yes”
“No”
No answer
... ... Decision tree coefficients
![Page 33: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/33.jpg)
CSU - Data AnalysisCSU - Data Analysisn Custom data analyzers - decision tree
based classifiers
Example: Operator gesture recognition(simplified)
Vertical Acceleration variance value high
Horizontal Acceleration variance high
Horizontal Acceleration variance high
Y N
Y N YN
No answer “NO”“YES”
![Page 34: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/34.jpg)
CSU - intermodule CSU - intermodule communicationcommunication
n Uses buffered thread-safe queues and producer-consumer scheme
Dataproducer
Dataqueue
Dataconsumer
Dataconsumer
creates
Dataconsumer
Processing
Processing
Dataconsumer &
producer
Dataconsumer &
producer
Dataq
ueue
Dataq
ueue
register at
Dataconsumer
![Page 35: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/35.jpg)
CSU - intermodule CSU - intermodule communicationcommunication
Data AnalysisVisualization
ModuleData
Logger
Operator manager (OM)
OM
OM
ConnectionManager
...
Separated physiological data streams
Processed data
Alerts
Recorded (off-line) data
Data AnalysisVisualization
Module
Operator manager (OM)
OM
OM
ConnectionManager
...
![Page 36: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/36.jpg)
Implementation - CSUImplementation - CSU
n Custom-builtBluetoothcommunicationroutines
n Intel USB cameran ID card programmern Developed using Visual C++ 6.0 n Deployment environment:
Windows 2000,MS SQL Server
• Standard MDI userinterface
![Page 37: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/37.jpg)
Future Future improvementsimprovements
DAUsmall CMOS camera to monitor the operator’s point of gazesingle PCB (SMD technology)low voltage ICs - LiIO batteries power
CSUdata mining algorithms advanced database encryption using e.g. AES algorithm
![Page 38: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/38.jpg)
Data securityData security
n Only registered mobile devicescan connect to the system
n Bluetooth connection authenticationn Bluetooth connection encryptionn Access rights restrictionsn Personal and physiological data
encryption
![Page 39: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/39.jpg)
App licat ions ofApp licat ions of Blue EyesBlue Eyes
n Generic control rooms(System can be applied in every working
environmentrequiring permanent operator’s attention)
– Power station– Captain bridge– Flight control centers– Operating theatres – anesthesiologists
![Page 40: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/40.jpg)
Blue E yes Blue E yes - benefits- benefits
n Prevention from dangerous incidentsPrevention from dangerous incidents
n Minimization ofMinimization of– ecological consequencesecological consequences– financial lossfinancial loss– a threat to a human lifea threat to a human life
n The reconstructionThe reconstructionof the course ofof the course ofoperator’s workoperator’s work
![Page 41: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/41.jpg)
ReferencesReferences
n www.seminar4u.comn www.howstuffswork.comn www.cs.put.poznan.comn www.whitepapers.comn www.ibmresearchcenter.comn www.almaden.ibm.com
![Page 42: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/42.jpg)
ConclusionConclusion
n In the near future ,ordinary household devices- such as television , refrigerators ,ovens may be able to do their jobs when we look at them and speak to them.
n Future applications of blue eye technology is limitless
![Page 43: Blue eye technology - 123seminarsonly.com · Bluetooth technology provides means for creating a Personal Area Network linking the operators and the central system. ... modul microphone](https://reader033.vdocuments.mx/reader033/viewer/2022041622/5e3f70aa2d8e560e3d49a18a/html5/thumbnails/43.jpg)
Thank you