Download - Driver Drowsiness Detection
Driver Drowsiness Detection
Laveen Prabhu Selvaraj (Im No. 303657)
Abstract
Sleep disorders and various common acute directly or indirectly affect the quality and
quantity of one‘s sleep or otherwise cause excessive daytime fatigue. About 29 600
Norwegian accident-involved drivers received a questionnaire about the last accident
reported to their insurance company. About 9200 drivers (31%) returned the questionnaire.
The questionnaire contained questions about sleep or fatigue as contributing factors to the
accident. In addition, the drivers reported whether or not they had fallen asleep some time
whilst driving, and what the consequences had been. Sleep or drowsiness was a
contributing factor in 3.9% of all accidents, as reported by drivers who were at fault for the
accident. This factor was strongly over-represented in night-time accidents (18.6%), in
running-off-the-road accidents (8.3%), accidents after driving more than 150 km on one trip
(8.1%), and personal injury accidents (7.3%).The most frequent consequence of falling
asleep—amounting to more than 40% of the reported incidents—was crossing of the right
edge-line before awaking, whereas crossing of the centre line was reported by 16%. Drivers‘
lack of awareness of important precursors of falling asleep—like highway hypnosis, driving
without awareness, and similar phenomena—as well as a reluctance to discontinue driving
despite feeling tired are pointed out as likely contributors to sleep-related accidents. The
envisioned vehicle-based driver drowsiness detection system would continuously and
unobtrusively monitor driver performance (and ―micro-performance‖ such as minute steering
movements) and driver psychophysiological status (in particular eye closure). The system
may be programmed to provide an immediate warning signal when drowsiness is detected
with high certainty, or, alternatively, to present a verbal secondary task via recorded voice as
a second-stage probe of driver status in situations of possible drowsiness. The key
requirements and R&D challenges for a successful countermeasure include low
countermeasure cost, true unobtrusiveness, an acceptably-low false alarm rate, non-
disruption of the primary driving task, compatibility and synergy with other IVHS crash
avoidance countermeasures, and a warning strategy that truly sustains driver wakefulness or
convinces him/her to stop for rest.
1. Introduction
Fatigued or drowsy drivers have long been
acknowledged to constitute a potential
traffic safety hazard, and several research
studies have addressed various aspects of
the problem. Brown (1994) has presented
a comprehensive review and discussion of
the research literature on the nature of
fatigue and its effects on driver behavior
and traffic accidents, on the basis of which
he concluded that ‗fatigue is insufficiently
recognized and reported as a cause of
road accidents‘. In discussing the effects
of fatigue, Brown further points out that the
main effect is ‗a progressive withdrawal of
attention from road and traffic demands‘. A
most extreme form of withdrawal of
attention is obviously the closing of eyes
due to sleepiness. To prevent accidents
related to drowsiness and sleeping behind
the wheel, it is important to acquire
precise knowledge about the extent of the
problem as well as its preconditions and
consequences. The following issues, all of
which are investigated empirically in the
present study, are considered relevant for
a better understanding of sleep-related
accidents.
1. What is the proportion of accidents
caused by fatigue or sleep behind the
wheel, and what are the most likely types
of accidents to occur under these
conditions?
2. How prevalent is the problem of actually
falling asleep while driving?
3. What are the most frequent
consequences of falling asleep while
driving?
4. To what extent is sleeping behind the
wheel related to characteristics of the
driver, the road and traffic conditions, and
the trip?
These drowsiness detection methods can
be categorized into three major
approaches:
• Active driving or on-board detection:
Preventing accidents caused by
drowsiness behind the steering wheel is
highly desirable but requires techniques
for continuously estimating driver‘s
abilities of perception, recognition and
vehicle control abilities. This paper
proposes methods for drowsiness
estimation that combine the
electroencephalogram (EEG) log subband
power spectrum, correlation analysis,
principal component analysis, and linear
regression models to indirectly estimate
driver‘s drowsiness level in a virtual-
reality-based driving simulator. Results
show that it is feasible to quantitatively
monitor driver‘s alertness with concurrent
changes in driving performance in a
realistic driving simulator.
• Imaging processing techniques: this
approach analyzes the images captured
by cameras to detect physical changes of
drivers, such as eyelid movement, eye
gaze, yawn, and head nodding. For
example, the PERCLOS system
developed by W. W. Wierwile et. al. used
camera and imaging processing
techniques to measure the percentage of
eyelid closure over the pupil over time.
The three-in-one vehicle operator sensor
developed by Northrop Grumman Co. also
used the similar techniques. Although this
vision based method is not intrusive and
will not cause annoyance to drivers, the
drowsiness detection is not so accurate,
which is severely affected by the
environmental backgrounds, driving
conditions, and driver activities (such as
turning around, talking, and picking up
beverage). In addition, this approach
requires the camera to focus on a relative
small area (around the driver‘s eyes). It
thus requires relative precise camera
focus adjustment for every driver.
• Physiological signal detection
techniques: this approach is to measure
the physiological changes of drivers from
biosignals, such as the
electroencephalogram (EEG),
electrooculograph (EOG), and
electrocardiogram (ECG or EKG). Since
the sleep rhythm is strongly correlated
with brain and heart activities, these
physiological biosignals can give accurate
drowsiness/sleepiness detection.
However, all the researches up to date in
this approach need electrode contacts on
drivers‘ head, face, or chest. Wiring is
another problem for this approach. The
electrode contacts and wires will annoy
the drivers, and are difficult to be
implemented in real applications.
2. Active Driving or On board Detection
Basic Concepts
As indicated earlier, the basic idea behind
vehicle-based detection is to monitor the
driver unobtrusively by means of an on-
board system that can detect when the
driver is materially impaired by
drowsiness. The concept involves sensing
various drivers related and driving related
variables. Computing measures from
these variables online and then using the
measures in a combined manner to detect
when drowsiness is occurring. Measures
are combined because no single
unobtrusive operational measure appears
adequate in reliably detecting drowsiness.
The most promising approach uses
mathematical optimization procedures to
develop algorithms with the highest
potential detection accuracy. Techniques
normally employed include multiple
regression and linear discriminated
analyses. More exotic techniques could
also be employed in the future, including
neural networks, pattern recognition and
fuzzy logic. Optimization of algorithms for
detection of drowsiness requires a
definitional measure of ―actual‖
drowsiness. Such a measure may be
based on physiological, performance, or
subjective attributes and need not be
obtainable operationally. However, the
measure must be available in experiments
so that operational detection algorithms
can be ―trained‖ to indicate the value of
the definitional measure. This concept is
depicted in
Figure 1.
Figure 1. concept of using operational
measures to predict definitional measures
of drowsiness
On the left are measures that can be
obtained in the driving environment. These
measures (with the exception of
secondary task measures) are obtainable
operationally from the vehicle without
disturbing the driver. They can be used in
various combinations for algorithm
development. On the right are various
candidate definitional measures. AVEOBS
is an observer rating measure, EYEMEAS
and PERCLOS are measures of slow eye-
closure, and NEWDEF is a measure
composed of slow eye-closure, various
EEG waveform amplitudes (Alpha, Beta,
and Theta), and mean heart rate. A given
algorithm would be directed at indicating
the level of only one definitional measure,
or possibly a linear combination of them.
In any case, operationally available
measures (on the left) are used to detect
the level of the definitional measure of
drowsiness (on the right), with thresholds
set to indicate when drowsiness has
exceeded a pre-specified level.
Figure 2: Application of on board
detection in Mercedes
On-Board Detection System
Figure 3: Block Diagram of on board
detection
The on-board drowsiness detection
system would gather signals from sensors
on the vehicle, process these signals into
measures, and then compute the
algorithm (or algorithms) to determine if
the drowsiness threshold has been
exceeded.
Figure 3 shows a block diagram of the
envisioned system. Aspects of the
envisioned system already determined
through research efforts include the
following:
Signals input to the microcomputer
will include:
Steering-related signals
o A lateral
accelerometer-
related signal
and
o A lane position
signal (assumes
availability of
machine vision
technology for
optical tracking
of existing
highway lane
markings).
Measures will be computed using
six-minute running averages
(which provide the best prediction
accuracy).
An adjustable drowsiness
threshold feature will allow
sensitivity to be set according to
conditions.
. A step-up/step-down routine will ensure
that, when all incoming signals are valid,
the best algorithm is used. When one or
more of the incoming signals is invalid (for
example, inability to establish a lane
track), then the best algorithm excluding
the invalid signal(s) would be used. This
procedure will ensure that at least one
detection algorithm is always capable of
being computed.
A ―baselining‖ procedure will be used to
tailor detection algorithms to the individual
driver. It will record each driver‘s
performance measures on-line initially and
then subtract such values from all
subsequent values. Accordingly,
measures obtained are actually deviations
from the driver‘s own baseline.
Domain of Application
On-board drowsiness detection systems
will be applicable primarily to driving on
rural and other ―open‖ highways, such as
limited-access highways, at speeds at or
above 50 mph. There are two reasons for
limiting the drowsiness detection system
to this domain. First, as discussed earlier,
most drowsiness-related crashes occur on
these roads at these speeds. Second, it
appears that this domain is the one in
which feasibility is maximized. The
influences of stop-and-go traffic, traffic
signals, turning maneuvers, etc., would
probably introduce sufficient ―noise‖ into
the detection process that unobtrusive
detection would be unfeasible. As we can
see, we have the fortuitous circumstance
of ―feasibility in the most needed domain,‖
or in other words ―the coin we are
searching for was lost under the
streetlamp, where the light gives us the
best chance of finding it.‖
Nature and Accuracy of Algorithms
To provide a better idea of what a typical
algorithm looks like and what its
anticipated level of accuracy would be, a
specific algorithm will be described. It is
one of perhaps 120 that were recently
derived in a major, moving-base driving
simulator experiment using sleep deprived
drivers (Wreggit, Kim, and Wierwille,
1993). The algorithm was derived using
multiple regression analysis with
PERCLOS (the proportion of total time
that the driver‘s eyelids are closed 80% or
more) as the definitional measure. Figure
4 shows the actual values of PERCLOS
(open circles) and the algorithm-predicted
values (closed triangles) for 12 driver
subjects. Each interval on the abscissa
corresponds to a six-minute average, with
25 intervals per driver-subject. Increasing
values of the ordinate represent increasing
drowsiness levels. The algorithm generally
does an excellent job of mimicking the
values of
PERCLOS, particularly in the intermediate
ranges of PERCLOS where the threshold
would most likely be set Figure 4 shows
the specific thresholds used on the
definitional measure (PERCLOS) for the
determination of prediction accuracy. The
data in Figure 4 correspond to ―circle‖
values in Figure 4.
. Figure 4 algorithm-predicted values
(closed triangles) for 12 driver subjects
As can be seen, two thresholds have been
specified, thus breaking the plot into three
regions: alert, questionable, and drowsy.
When these thresholds are applied to the
output of the detection algorithm, which
provides an assessment of accuracy. In
the table, the boldface diagonal values
show the number of correct classifications.
The off-diagonal elements represent
errors, and in particular, the upper-left and
lower-right cells represent large errors. As
can be seen, three intervals were
classified (predicted) as drowsy when the
driver was observed as alert (false
alarms), and another three intervals were
classified as alert when the driver was
observed as drowsy (failure to detect).
Since there were 300 intervals, two
percent were seriously misclassified,
resulting in an apparent accuracy rate of
0.98 (for large errors).Of course there
were smaller classification errors as well,
but these are not as serious -- for
example, the 16 intervals in which the
system diagnosed a drowsy driver when
the driver‘s actual status was
―questionable‖ (i.e., somewhat drowsy).
Overcoming the False Alarm Problem
As already indicated, a major research
objective will be to overcome the false
alarm problem inherent to the
identification/diagnosis of low-probability
events. Since drowsiness is infrequent in
relation to all time spent driving, false
alarm rates must be very low. If not, the
number of false alarms will greatly
outnumber correct detections (―hits‖), even
if drowsiness is correctly detected with
100% accuracy (Knipling, 1993). This
problem may be overcome through
refinements to the performance
measurement algorithms, addition of
qualitatively different measures (i.e., direct
psychophysiological measures and/or
secondary tasks), and the use of graded
alarm intensities for different degrees of
drowsiness or levels of certainty. In
particular, the false alarm problem
appears less daunting from the
perspective of multiple degrees of
alertness and intensities of
warnings/advisories. Figure 5, which is
similar in concept to the two-threshold
algorithm concept, shows a theoretical
relation between ―actual‖ drowsiness level
(and thus actual risk of loss-of-alertness)
and ―operational‖ drowsiness level as
measured/derived by a detection system.
Figure 5. Three-level detection matrix
Three levels of ―actual‖ and ―operational‖
drowsiness are shown in the figure, but
note that dashed lines are used for the
three ―actual‖ drowsiness levels since the
variable represents a continuum without
qualitative breakpoints. Since the system
is not perfect, its data points would form
an ellipse rather than a straight line. Within
this scheme, zones G, E, and C represent‘
perfect classification, zones D, B, H, and F
represent small is classifications (or ―half
right‖ classifications), and zones A and I
represent large misclassifications. Drowsy
driver detection algorithms must be refined
to a point where zones A and I are very
small or non-existent. The effects of small
misclassifications (Zones D, B, H, and F)
on crash prevention, driver performance,
and driver acceptance must be
determined through further research. For
example, the ―half-false alarm‖ zones D
and B may be a source of irritation to
drivers or, on the other hand, they may
have the positive effect of reassuring the
driver that the system is functioning
continuously. Another way to increase
detection and reduce false alarms might
be to consider not just the current
measurement time interval but also trends
evident from preceding intervals. Were
there early signs of developing drowsiness
based either on the overall operational
measure or among specific indicators?
Fuzzy logic may be employed to further
enhance the accuracy of diagnosis by
considering the driver‘s recent time-history
of drowsiness.
3. Image processing
Image processing is a method to convert
an image into digital form and perform
some operations on it, in order to get an
enhanced image or to extract some useful
information from it. It is a type of signal
dispensation in which input is image, like
video frame or photograph and output may
be image or characteristics associated
with that image. Usually Image
Processing system includes treating
images as two dimensional signals while
applying already set signal processing
methods to them.
It is among rapidly growing technologies
today, with its applications in various
aspects of a business. Image Processing
forms core research area within
engineering and computer science
disciplines too.
Image processing basically includes the
following three steps.
· Importing the image with optical
scanner or by digital photography.
· Analyzing and manipulating the
image which includes data
compression and image
enhancement and spotting patterns
that are not to human eyes like
satellite photographs.
· Output is the last stage in which
result can be altered image or
report that is based on image
analysis.
Purpose of Image processing
The purpose of image processing is
divided into 5 groups. They are:
1. Visualization - Observe the objects
that are not visible.
2. Image sharpening and restoration -
To create a better image.
3. Image retrieval - Seek for the image
of interest.
4. Measurement of pattern – Measures
various objects in an image.
5. Image Recognition – Distinguish the
objects in an image.
Each application that benefit from smart
video processing has different needs, thus
requires different treatment. However,
they have something in common moving
objects. Thus, detecting regions that
correspond to moving objects such as
people and vehicles in video is the first
basic step of almost every vision system
since it provides a focus of attention and
simplifies the processing on subsequent
analysis steps. Due to dynamic changes in
natural scenes such as sudden
illumination and weather changes,
repetitive motions that cause clutter (tree
leaves moving in blowing wind), motion
detection is a difficult problem to process
reliably. Frequently used techniques for
moving object detection are background
subtraction, statistical methods, temporal
differencing and optical flow.
Figure6. A generic framework for smart
video processing algorithms.
There are various techniques for moving
object detection and tracking like optical
flow, low change of illumination,
segmentation, background subtraction,
frame difference, etc. The problem is
formulated in a sequential manner. It has
different step with different set of operation
will take place at each step and the output
of that step will be used as the input to the
other step .Each step is in charge of
specific function which it will perform on
each frame of the video sequence and the
final result of that step will be used in the
another step and each step will follow the
same things. The last step will give the
final output in the form of a video in a well
structured way. The formulation of step
are defined as follows-
1. Take video from Vision System.
2. Read 1st image to avi read that is
reference image
3. Read other image
4. Take subtraction of them and set
Thresholding
5. Applied Gaussian filter for noise remove
6. Applied morphological operation like
dilation and erosion for small noise
removes
7. Fill holes in resulted image
8. Take label connected component with
its properties like bounding box, centroid
and area of all no of object move in this
scene.
9. For i=1:n % n is no of object move
A=(length of object) Find(L==1) % find
white pixel whose length is A
If (A>100 && A<8000)
Then draw rectangle plot centroid
of that rectangle end to end
10. Take distance of centroid to
reference point
11. Take velocity estimation by ratio of
distance to time per Frame.
12. Take acceleration estimation by ratio
of velocity to time per Frame
Head movement measures
Head movement was measured using an
accelerometer that has 3 degrees of
freedom. This three dimensional
accelerometer has three one dimensional
accelerometers mounted at right angles
measuring accelerations in the range of 5g
to +5g where g represents earth
gravitational force.
Facial Action Classifiers
The facial action coding system (FACS)
[12] is arguably the most widely used
method for coding facial expressions in
the behavioral sciences. The system
describes facial expressions in terms of 46
component movements, which roughly
correspond to the individual facial muscle
movements. An example is shown in
Figure 7. FACS provides an objective and
comprehensive way to analyze
expressions into elementary components,
analogous to decomposition of speech
into phonemes. Because it is
comprehensive, FACS has proven useful
for discovering facial movements that are
indicative of cognitive and affective states.
In this paper we investigate whether there
are Action units (AUs) such as chin raises
(AU17), nasolabial furrow
deepeners(AU11), outer(AU2) and inner
brow raises (AU1) that are predictive of
the levels of drowsiness observed prior to
the subjects falling sleep.
Figure 7. Example facial action
decomposition from the Facial Action
Coding System.
In previous work we presented a system,
named CERT, for fully automated
detection of facial actions from the facial
action coding system [10]. The workflow of
the system is based is summarized in
Figure 3. We previously reported detection
of 20 facial action units, with a mean of
93% correct detection under controlled
posed conditions, and 75% correct for less
controlled spontaneous expressions with
head movements and speech. For this
project we used an improved version of
CERT which was retrained on a larger
dataset of spontaneous as well as posed
examples. In addition, the system was
trained to detect an additional 11 facial
actions for a total of 31 (Table 1). The
facial action set includes blink (action unit
45), as well as facial actions involved in
yawning (action units 26 and 27). The
selection of this set of 31 out of 46 total
facial actions was based on the availability
of labeled training data. The facial action
detection system was designed as follows:
First faces and eyes are detected in real
time using a system that employs boosting
techniques in a generative framework [13].
The automatically detected faces are
aligned based on the detected eye
positions, cropped and scaled to a size of
96 × 96 pixels and then passed through a
bank of Gabor filters. The system employs
72 Gabor spanning 9 spatial scales and 8
orientations. The outputs of these filters
are normalized and then passed to a
standard classifier. For this paper we
employed support vector machines. One
SVM was trained for each of the 31 facial
actions, and it was trained to detect the
facial action regardless of whether it
occurred alone or in combination with
other facial actions. The system output
consists of a continuous value which is the
distance to the separating hyper plane for
each test frame of video. The system
operates at about 6 frames per second on
a Mac G5 dual processor with 2.5 ghz
processing speed.
Figure 8. Overview of fully automated
facial action coding system.
Drowsiness prediction
The facial action outputs were passed to a
classifier for predicting drowsiness based
on the automatically detected facial
behavior. Two learning-based classifiers,
Adaboost and multinomial ridge
regression are compared. Within-subject
prediction of drowsiness and across-
subject (subject independent) prediction of
drowsiness were both tested.
Table 1. Full set of action units used for
predicting drowsiness
Within subject drowsiness prediction.
For the within-subject prediction, 80% of
the alert and non-alert episodes were
used for training and the other 20% were
reserved for testing. This resulted in a
mean of 19 non-alert and 11 alert
episodes for training, and 5 non-alert and
3 alert episodes for testing per subject.
The weak learners for the Adaboost
classifier consisted of each of the 30
Facial Action detectors. The classifier was
trained to predict alert or non-alert from
each frame of video. There was a mean of
43,200 training samples, (24+11)×60×30,
and 1440 testing samples, (5 + 3) × 60 ×
30, for each subject. On each training
iteration, Adaboost selected the facial
action detector that minimized prediction
error given the previously selected
detectors. Adaboost obtained 92% correct
accuracy for predicting driver drowsiness
based on the facial behavior.
Classification with Adaboost was
compared to that using multinomial ridge
regression (MLR). Performance with MLR
was similar, obtaining 94% correct
prediction of drowsy states. The facial
actions that were most highly weighted by
MLR also tended to be the facial actions
selected by Adaboost. 85% of the top ten
facial actions as weighted by MLR were
among the first 10 facial actions to be
selected by Adaboost.
Table 2. The top 5 most discriminant
action units for discriminating alert from
nonalert states for each of the four
subjects. A‘ is area under the ROC curve.
Across subject drowsiness prediction.
The ability to predict drowsiness in novel
subjects was tested by using a leave one-
out cross validation procedure.
Table 3. Performance for drowsiness
prediction, within subjects. Means and
standarddeviations are shown across
subjects.
The data for each subject was first
normalized to zero-mean and unit
standard deviation before training the
classifier. MLR was trained to predict
drowsiness from the AU outputs several
ways. Performance was evaluated in
terms of area under the ROC. For all of
the novel subject analysis, the MLR output
for each feature was summed over a
temporal window of 12 seconds (360
frames) before computing A‘. MLR trained
on all features obtained an A‘ of .90 for
predicting drowsiness in novel subjects.
Table 4. MLR model for predicting
drowsiness across subjects. Predictive
performance of each facial action
individually is shown.
Brain Waves
Relationship between the EEG
Spectrum and Subject Alertness
To investigate the fluctuations in driving
performance to concurrent changes in the
EEG spectrum, correlations between
changes in the EEG power spectrum and
driving performance to form a correlation
spectrum is measured. The spatial
distributions of these positive correlation
spectra on the scalp at dominant
frequency bins are investigated 7, 12, 16
and 20Hz, separately, as shown in Fig. 9.
The correlations are particularly strong at
central and posterior channels, which are
similar to the results of previous studies in
the drowsy experiments. The relatively
high correlation coefficients of EEG log
power spectrum with driving performance
suggests that using EEG log power
spectrum may be suitable for drowsiness
(micro-sleep) estimation, where the
subject‘s cognitive state might fall into the
first stage of the non-rapid-eye-movement
(NREM) sleep. To be practical for routine
use during driving or in other occupations,
EEG-based cognitive assessment
systems should use as few EEG sensors
as possible to reduce the preparation time
for wiring drivers and the computational
load for estimating continuously the level
of alertness in near real time. According to
the correlations shown in Fig. 9, we
believe it is adequate to use 2-channel
EEG signals having the highest correlation
coefficients to assess the alertness level
of drivers.
Figure 9: Scalp topographies for the
correlations between EEG power and
driving performance at dominant
frequencies 7, 12, 16, and 20 Hz,
computed separately for 40 EEG
frequencies between 1 and 40 Hz.
Next, we compared correlation spectra for
individual sessions to examine the stability
of this relationship over time and subjects.
The time interval between the training and
testing sessions of the lane-keeping
experiments distributes over one day to
one week long for the selected five
subjects. Fig. 10 plots correlation spectra
at cites Fz, Cz, Pz and Oz, of two
separate driving sessions with respect to
subjects A. The relationship between EEG
power spectrum and driving performance
is stable within the subjects, especially the
spectrum below 20 Hz. These analyses
provide strong and converging evidence
that change in subject alertness level
indexed by driving performance during a
driving task are strongly correlated with
the changes in the EEG power spectrum
at several frequencies at central and
posterior cites. This relationship is
relatively variable between subjects, but
stable within subjects. It is consistent with
the findings from a simple auditory target
detection task. These findings suggest
that information available in the EEG can
be used for real time estimation of
changes in alertness of human operators.
However, to achieve maximal accuracy,
the estimation algorithm should be
capable of adapting to individual
differences in the mapping between EEG
and alertness.
Figure 10. Correlation spectra between
the EEG power spectrum and the driving
performance at Fz, Cz, Pz, and Oz
channels in two separate driving sessions
with respect to subject A. Note that the
relationship between EEG power
spectrum and driving performance is
stable within this subject.
EEG-based Driving Performance
Estimation/Prediction
In order to estimate/predict the subject‘s
driving performance based on the
information available in the EEG power
spectrum, a 50-order linear regression
models with a least-square-error cost
function is used. We used only two EEG
channels with the highest correlation
coefficients in place of using all 33
channels to avoid introducing more
unexpected noise. Fig. 11 plots the
estimated and actual driving performance
of a session with respect to subject A. The
linear regression model in this figure is
trained with and tested against the same
session, i.e. within-session testing. As can
been seen, the estimated driving
performance matched extremely well with
the actual driving performance (r = 0.91).
Figure 11. Driving performance estimates
for a session with respect to subject A,
based on a linear regression (red line) of
PCA-reduced EEG log spectra at two
scalp sites, over plotted against actual
driving performance time series for the
session (solid line). The correlation
coefficient between the two time series is r
= 0.91.
When the model was tested against a
separate test session with respect to the
same subject, the correlation between the
actual and estimated driving performance
though decreased but remained high (r =
0.87) as shown in Fig. 12. Across 10
sessions, the mean correlation coefficient
between actual driving performance time
series for within session estimation is 0.85
± 0.11, whereas the mean correlation
coefficient for cross-session estimation is
0.82 ± 0.07. These results suggest that
continuous EEG-based driving
performance estimation using a small
number of data channels is feasible, and
can give accurate information about
minute-to-minute changes in operator
alertness.
Figure. 12. Driving performance estimates
for a test session, based on a linear
regression (red line) of PCA-reduced EEG
log spectra trained from a separate
training session with respect to the same
subject, over plotted against actual driving
performance time series of the test
session (solid line). The correlation
coefficient between the two time series is r
= 0.87. Note that the training and testing
data in this study were completely
disjoined.
Conclusion
This paper presented a system for
automatic detection of driver drowsiness
from video. Previous approaches focused
on assumptions about behaviors that
might be predictive of drowsiness. Here, a
system for automatically measuring brain
waves, facial expressions and vehicle
monitoring was employed to datamine
spontaneous behavior during real
drowsiness episodes. This is the first work
to our knowledge to reveal significant
associations between facial expression
and fatigue beyond eye blinks. The project
also revealed a potential association
between head roll and driver drowsiness,
and the coupling of head roll with steering
motion during drowsiness. Of note is that
a behavior that is often assumed to be
predictive of drowsiness, yawn, was in fact
a negative predictor of the 60-second
window prior to a crash. It appears that in
the moments before falling asleep, drivers
yawn less, not more, often. This highlights
the importance of using examples of
fatigue and drowsiness conditions in which
subjects actually fall sleep. The real
advantages of these following techniques
are, these can be combined into one
system and integration without affection or
interrupting each other detection and
function. This helps to give a exact alert
level to avoid the accidents due to the
driver drowsiness.
ACKNOWLEDGMENT
I like to thank Prof. Dr.-Ing. Olfa
Kanoun, Head of the Chair for
Measurement and sensor technology, TU
Chemnitz for organizing this course and
giving an opportunity to develop our
presentation and report writing skills.
References
[1] Knipling, R., & Wang, J, Crashes and
fatalities related to driver
drowsiness/fatigue. Washington, DC:
National Highway Traffic Safety
Administration (1994).
[2] Fridulv Sagberg, Road accidents
caused by drivers falling asleep, Norway,
23 December 1998.
[3] S. F. Liang, C. T. Lin, R. C. Wu, Y. C.
Chen, T. Y. Huang and T. P. Jung,
Monitoring Driver‘s Alertness Based on
the Driving Performance Estimation and
the EEG Power Spectrum Analysis,
Engineering in Medicine and Biology 27th
Annual Conference Shanghai, China,
September 1-4, 2005.
[4] Yi˘githan Dedeo glu, Moving object
detection, tracking and classification for
smart video surveillance, August, 2004
[5] Esra Vural, Mujdat Cetin, Aytul Ercil,
Gwen Littlewort, Marian Bartlett and Javier
Movellan, Drowsy Driver Detection
Through Facial Movement Analysis,
Grenoble, France (2000).
[6] Lawrence Barr, Heidi Howarth,
Stephen Popkin, Robert J. Carrol, A
review and evaluation of emerging driver
fatigue detection measures and
technologies, Washington, DC.
[7] Jiyan Pan, Quanfu Fan, Sharath
Pankanti, Robust abandoned object
detection using region-level analysis, IBM
T.J. Watson Research Center, Hawthorne,
NY, U.S.A.
[8]Kalpesh R Jadav, Prof. M.A.
Lokhandwala,Prof.A.P.Gharge,Vision
based moving object detection and
tracking,Limda,vadodara,India 13-14 May
2011.
[9] Smith, P.,& Shah, M.,& da Vitoria Lobo,
N. ,―Determining driver visual attention
with one camera‖ Intelligent
Transportation Systems, IEEE
Transactions, Dec, 2003.
[10] Bergasa, L.M.,& Nuevo, J.,& Sotelo,
M.A.,& Barea, R.,& Lopez, M.E., ―Real-
time system for monitoring driver
vigilance‖ Intelligent Transportation
Systems, IEEE Transactions, March,
2006.
[11] Qiang Ji,& Zhiwei Zhu,& Lan, P.,
―Real-time nonintrusive monitoring and
prediction of driver fatigue‖ Vehicular
Technology, IEEE Transactions, July,
2004.