Abstract: Driver drowsiness is the most critical cause of traffic accidents, thus drowsiness detection play a vital role in preventing traffic accidents. By developing an automatic solution for alerting drivers of drowsing, before an accident occurs, this could reduce the number of traffic accidents. Therefore, this research proposes a real-time detection approach for driver drowsiness. The proposed approach has two phases: image processing and machine learning. The role of image processing phase is to recognize the face of the driver and then extracts the image of the eyes of the driver. This phase uses Haar face detection algorithm that takes captured frames of image as input and then the detected face as output. Next, Haar is also used to extract the eyes image from the detected face which will be used as an input for the machine learning phase. The main role of the machine learning is to classify either the eyes of the driver are closed or opened using Support Vector Machine (SVM). If the result of the classification indicates that the driver’s eyes is closed for a predefined period of time, the eyes of the driver will be considered closed and hence an alarm will be started to alert the driver. The proposed methodology has been tested on available benchmark data. The result demonstrates the accuracy and robustness of the hybridized of image processing technique with machine learning technique. Thus, it can be concluded that the proposed approach is an effective solution method for a real-time of driver drowsiness detection.
INTRODUCTION
Driver fatigue is one of the most significant factors in a large number of car accidents. Each year, there are about 100,000 crashes in the USA alone due to driver drowsiness or fatigue (Knipling and Wang, 1995). Those accidents are estimated by NHTSA (US National Highway Traffic Safety Administration). Developing various technologies for monitoring and preventing drowsiness while driving is a major trend and challenge in the domain of accidence avoidance systems. There are many technologies for drowsiness detection and can be divided into three main sections. The first one is the biological indicators. This type measures biological factors such as eyes movement, brain waves and heart rate. These techniques are generally characterized with the best detection accuracy but they require physical contact with the human (the driver). They are intrusive. Hence, they are not flexible to use in practical situation (Kokonozi et al., 2008; Khushaba et al., 2011; Yang et al., 2010). The second one is vehicle behavior. This type measures vehicle behaviors such as speed, exacerbation angle and position. These techniques may be considered as non-intrusively but they have multiple limitations such as the type of the car, the driving conditions, etc. Furthermore, it needs special equipment and preparation and can be expensive and not practical (Liu et al., 2009; Lee et al., 2011; Khan and Aadil, 2012). Third one is computer vision analysis. This type benefit from the dynamic behavior of the human face and eye since they have a high degree of variability, face detection is considered to be a difficult problem in computer vision research, whereas the eyes can be considered salient and relatively stable feature on the face in comparison with other facial features. Those features will be provided to a machine learning classifier to determine whether those features represent an opened or closed eye (Kumari, 2014; Fan et al., 2009; Zhang and Zhang, 2010; Yin et al., 2009).
Many researches have been developed based on analyzing the eye-image. Khushaba et al. (2011) proposed a method that detected the face using the facial features such as lip contour, jaw contour and the shape of the entire face and the eyes were located and tracked using the Eigen eye method and finally the eye status is detected using edge detection and correlation methods.
A method for driver fatigue detection using image processing techniques has been proposed by Eriksson and Papanikotopoulos (1997) and Singh and Papanikolopoulos (1999). The presented method by Eriksson and Papanikotopoulos (1997) detected the facial areas of an image using symmetric property of faces and then it located the vertical position of the eyes using the pixels differences. While the proposed method by Singh and Papanikolopoulos (1999) used the Gaussian distribution of skin color in order to locate the face and then it located the eye using the Sobel vertical edge operator. In addition, they used template matching for detecting and tracking the eye images.
A vision-based real time driver fatigue detection system based on the distance of the eyelid has built by Dong and Wu (2005). It located the face using the characteristics of skin colors and then it detected the eyes using projection and dynamic template matching.
Regarding to the issue of variable lighting conditions in real time feature recognition; a solution presented by Albu et al. (2008) which detected the eye-states using a customized template matching technique on a frame-by-frame basis. It provided an approach that tolerated some variability in the distance from the eye plane to the webcam.
A method for handling the eye images captured using IR camera and IR illumination has been proposed by Bhardwaj et al. (2013). They first got the bright pupil and then detected the eye image and the fatigue status using the SVM and the Kalman filter.
An approach for computer vision-based automatic driver drowsiness detection has been presented by Ji et al. (2004). It used Support Vector Machine in order to increase the accuracy of eye detection and then he used a method that’s based on eye closure (PERCLOS) to detect the driver drowsiness. PERCLOS was used as an important symptom for detection drowsiness.
A detecting system for vision-based automatic driver drowsiness detection has been proposed by Sacco and Farrugia (2012); it used the Viola-Jones object detection framework to detect the face, eyes and mouth in successive frames, along with correlation coefficient template matching to determine the state of features. Then they used Support Vector Machine classification on the combination of the three features to detect the overall fatigue level of the driver.
An approach for vision-based automatic driver drowsiness detection has been developed by Park et al. (2011) which is similar to the approach proposed by Sacco and Farrugia (2012). It proposed an algorithm that consists of two processes, face tracking process and drowsiness detection process. The first process finds face regions and monitors the motion data which is the driver’s facial feature change (eye and mouth state) and head gesture using the Active Shape Model (ASM). At the same time, the drowsiness detection process determines a driver’s drowsiness state using the extracted facial features from both eye and mouth regions based on Support Vector Machine (SVM).
An approach for vision-based automatic driver drowsiness detection using Haar-Cascade classifier and Support Vector Machine classifier has been proposed by Patil et al. (2013). The proposed approach starts first by detecting the face then the eye using Haar-Cascade classifier, then the method extracts the features of eye parameters like eyelid movement and eyebrow and finally the method classify whether the eye is opened or closed using SVM function.
The vision-based systems have been widely used because of its accuracy and effectiveness (Zhu and Ji, 2004). Visual cues such as eye states (i.e., whether they are open or closed) can generally give a strong indication about the driver’s level of fatigue (drowsiness). Therefore, an automatic and robust approach to extract the eye states from input images is very important and essential task to our works. The focus will be emphasized on designing a system that will accurately monitor the open or closed state of the driver’s eyes by continues monitoring and analysis of the detected eye images.
The aim of this study is to develop a drowsiness detection system. The present study basically used the same general approach of starting with basic facial feature detection like face and eyes but our research differs in that it simplifies the process further in order to run efficiently in real time situation, so it doesn’t detect additional features like mouth or head gesture, nor it uses additional technique such as template matching to keep tracking the detected eye and also the present study used the pre-processed eye image as a direct input to the SVM classifier instead of selecting certain features, this also has provided more simplicity and efficiency to the end drowsiness detection system.
MATERIALS AND METHODS
The selected methodology in this study is divided into two main parts:
| • | The first part is responsible for processing the video frames, detecting the face image and then detecting the eye image |
| • | The second part is responsible for building and using the classification model |
Flowchart in Fig. 1 describes the overall methodology that was used. The proposed system will start by capturing the video frames one by one. For each frame, the system will detect the face in the frame image and then it will detect the eye image in the face image.
The eye-image will be pre-processed (by converting it into gray scale) and then the eye-image will be classified using a machine learning classifier to detect whether it was opened or closed. If the eye is closed for a certain amount of time, then the system will start an alarm to notify the driver.
First part-face and eye detection: Regarding detecting the face and the eye image, the Haar algorithm will be used, Haar algorithm is a well-known robust feature-based algorithm that can detect the face image efficiently (Viola and Jones, 2001).
| Fig. 1: | Methodology flowchart |
The face detection algorithm looks for specific Haar features of the face. When one of the features is found, the algorithm will allow the face candidate to pass to the next stage of detection.
Haar algorithm uses a cascade of stages that is used to remove the candidates that are non-face. And each stage consists of many different Haar features. And each feature in turn is classified by a Haar feature classifier (Viola and Jones, 2001). Also, there is a variant of the Haar algorithm that can detect the eye image efficiently (Zhou et al., 2009).
Second part-classifying the eye image: To classify the pre-processed eye image, a machine learning classifier was first built using the SVM algorithm. SVMs can efficiently solve linear or non-linear classification problems, it maximizes the margin around the separating hyper-plane and hence it can find an optimal classifier (Fig. 2) (Berwick and Idiot, 2003).
| Fig. 2: | Support vector machines |
The training set was consist of a set of eye-images that are opened and a set of eye-images that are closed, when the training model is built, it will be ready to use to classify any new pre-processed eye-image.
RESULTS AND DISCUSSION
The selected and used hardware in the experiments of this thesis consists of the following:
| • | The standards webcam of the HP laptops |
| • | HP laptop (Pavilion DV6) |
| • | CPU-Core-I5, 2.4 GH |
| • | RAM-4.0 GB |
| • | Graphic card: GeForce GT 230M |
| • | 64-bit windows operating system |
| • | The video frames were acquired at 60 frames per second (also the system was tested on lower and higher frame rates) |
The inexpensive hardware was selected in order to demonstrate that the proposed approach is efficient and can work under low-quality images generated by the standard laptop webcam.
Table 1 describes the results of the proposed system in this study for six test instances (each experiment instance has been conducted by a different user); the following terms describe the used measures in the experiments:
| • | Total frame means the total number of frames in each produced experiment instance |
| • | Detection failure means the count of drowsiness detection failures |
| Table 1: | Experiment instances, measures 1 |
| Table 2: | Experiment instances, measures 2 |
| • | Correct rate of drowsiness detection is defined as in the below equation which is ratio of (Total Frame-Detection Failure) to total frames. Figure 3 represents the calculated values of the correct rate for each tested instance: |
As described in Table 1, the correct rate of drowsiness detection is higher than 99.2% and the average correct rate can achieve 99.45%.
Table 2 describes other measures regarding the conducted experiments (same instances of measure (1) to find the precision rate. Figure 4 represents the calculated ratio of precision for each instance:
The gained result shows the efficiency of the proposed learning system. The result varies with respect to the following factors:
| • | No. of captured frames |
| • | Size of the eye |
| • | Eye clearance (with or without eyeglass) |
Furthermore, the training data play the main role in indicating the performance of the system. The performance directly proportional with quantity (number of the eye images) and the quality (variety of eye images) of the training data.
| Fig. 3: | Correct rate for each experiment instance |
| Fig. 4: | Precision rate for each experiment instance |
The experiment results demonstrated the effectiveness of the proposed methodology. In this methodology the most promising and efficient techniques have been selected and used in the developed system (Haar Face detection algorithm, Haar cascade eye detection algorithm, Support Vector Machine for machine learning classification). As compared with other similar researches in the literature (Park et al., 2011) achieved a correct rate of 93.74%, whereas our approach achieved correct rate of 99.45%.
CONCLUSION
The aim of this study is to address a solution to one of the major causes of the road accident, the driver drowsiness; the proposed solution does track the driver’s eyes and then notify him when his eyes get closed in order to avoid losing the control of the car and causing traffic accidents.
The present proposed method based mainly on two main phases, the first phase is to detect and pre-process the eye images using the image processing technique and the second phase is to build a classification model that will be able to classify whether the eye is opened or closed and then start an alarm accordingly.
The most important value this research has added to the literature in this domain is to find the simplest and most efficient approach to solve the automatic drowsiness detection problem; using simplest approach in order to utilize this system in the real time situation, so the processing time will be minimized. This study has achieved this simplicity and efficiency through the following:
| • | The most promising and efficient approach to locate the eye image efficiently, using Haar Cascade techniques were used |
| • | This study bypassed the template matching step used in the literature and instead performed histogram equalization and then entered the pre-proceed eye image directly to the SVM classifier |
The results show that this method is flexible for developing practical and ready to use drowsiness detection application and comprehensive solution.
ACKNOWLEDGMENT
The authors would like to express their gratitude to the National University of Malaysia for supporting this research.