Development of Driver States Detection System Based on Electrocardiography using Artificial Neural Network

Abstract

Cognitive overload and drowsy driving are considered as significant contributing factors of traffic accidents and well-known as significant causes of traffic accidents. These risky driving conditions deteriorate driver’s performance, impair vigilance, and degrade reaction time to driving environments, in which may lead to vehicle crashes. Thus, early detection system of cognitive overload and/or drowsy driving with preeminent accuracy is needed to reduce the occurrence probability of accidents and improve the traffic safety on the road. The present study was intended to develop artificial neural network (ANN) models that enable to identify the level of a driver’s states whether under cognitive load or drowsiness based on electrocardiography (ECG) by taking into account the individual variability in heart responses. This study has four specific objectives: (1) developed an ANN model to classify driver’s cognitive load levels based on ECG, (2) developed ANN model to classify driver’s drowsiness levels, (3) developed ANN model to classify driver’s states, and (4) developed a real-time detection systems of the driver’s cognitive load levels. First, an ANN model was proposed to classify the level of a driver's cognitive load levels using ECG. The ECG was measured on 15 male participants while they performed a simulated driving task as a primary task with/without an N-back task as a secondary task. Six ECG measures in time and frequency domains were quantified. To deal with the individual differences in heart response, a three-step data processing procedure was established for each participant: (1) selection of two most sensitive ECG measures, (2) definition of three (low, medium, and high) cognitive load levels, and (3) normalization of the selected ECG measures. The ANN model was established using a feed-forward network and scaled conjugate gradient as a back-propagation learning rule. The accuracy of the ANN classification model was found satisfactory for both of learning data (95%) and testing data (82%). Second, an ANN model that could detect an early sign of drowsy driving (fighting-off drowsiness) was developed based on ECG. The ECG of 20 out of 43 participants who suffered drowsiness while performing a simulator-based monotonous driving for 20 minutes were used in the analysis. Three driver states (normal, fighting-off drowsiness, and drowsy) were determined by means of participant’s subjective report and video recording analysis. Six ECG measures in time and frequency domains were derived from the ECG and pre-processed to compensate individual variations in heart response in two steps: (1) normalization and (2) selection of two sensitive ECG measures for ANN inputs. The ANN model was trained using a feed-forward network with a scaled conjugate gradient, and its average accuracy was over 99% for the training and testing data. Third, we established several ANN models to find the accurate model that can cope with individual variability in detecting the driver’s state based on ECG. The ECG was measured for 65 participants while driving a vehicle simulator under cognitive load or drowsiness. We defined five driver’s states from low alertness to high alertness (drowsiness, fighting-off drowsiness, normal, low load, and high load). Analysis of variance (ANOVA) on the ECG revealed significant changes as the driver’s state changed from low to high alertness. The results showed that the classification accuracy of ANN models with capable of controlling individual variability were over 95% for both of training and testing data sets. Lastly, a real-time detection system was developed to classify the driver’s status into either normal or overload using multi-layer ANN model based on ECG. The ECG obtained from 22 participants under two different driving conditions (1: normal driving, 2: driving while doing a two-back task). The real-time detection system was developed using the ANN model, and its usefulness was evaluated by a simulator-based driving experiment with two participants who drove under two driving conditions (normal driving, driving while performing an arithmetic task). As a result, the real-time detection system successfully detected the change of participant status with a reasonable time delay (mean = 4.5 seconds). The proposed driver’s states classification models either under cognitive load or drowsiness presented in this study would be useful and could be adapted into the development of driver’s states detection system to provide early warning to the driver at the onset of potentially risky driving states.