Detecting Driver Drowsiness with Multi-Sensor Data Fusion Combined with Machine Learning

Abstract:

According to the National Highway Traffic Safety Administration, in 2017 drowsy driving resulted in 50,000 injuries across 91,000 police-reported accidents, as well as almost 800 deaths. Through the application of visual and radar sensors combined with machine learning, this research developed a drowsy driver detection system aimed to prevent potentially fatal accidents. The working prototype of Advanced Driver Assistance Systems can be installed in present-day vehicles to detect drowsy drivers with over 95% accuracy. It integrates two types of visual surveillance to examine the driver for signs of drowsiness. A camera is used to monitor the driver’s eyes, mouth and head movement in order to recognize when a discrepancy occurs in the driver's eye blinking pattern, yawning incidence, and/or head drop, thereby signaling that the driver may be experiencing fatigue or drowsiness. The micro-Doppler sensor in the system allows the driver's head movement to be captured at all times. Through data fusion and deep learning, the system quickly analyzes and classifies a driver's behavior under various conditions in real-time monitoring. This research could be implemented to reduce drowsy driving, thereby, making the roads safer for everyone and ultimately saving lives.

Publications:

Detecting Driver Drowsiness with Multi-Sensor Data Fusion Combined with Machine Learning (Full Report)

Research Brief

Authors:

HOVANNES KULHANDJIAN

Dr. Hovannes Kulhandjian is an Associate Professor in the Department of Electrical and Computer Engineering at California State University, Fresno (Fresno State). He joined Fresno State in Fall 2015 as a tenure-track faculty member. Prior to that, he was an Associate Research Engineer in the Department of Electrical and Computer Engineering at Northeastern University. He received his BS degree in Electronics Engineering with high honors from the American University in Cairo (AUC) in 2008, and his MS and PhD degrees in Electrical Engineering from the State University of New York at Buffalo in 2010 and 2014, respectively. His current research interests are in applied machine learning, wireless communications, and networking, with applications to underwater and visible light communications and networking geared towards Intelligent Transportation Systems (ITS).

Dr. Kulhandjian has received three research grants from Fresno State Transportation Institute (FSTI) and he also received the Claude C. Laval III Award for Commercialization of Research, Innovation and Creativity 2021 as well as the Claude C. Laval Award for Innovative Technology and Research 2020 at Fresno State. In April 2021, as a PI he received a grant from the Department of Defense (DOD) Research and Education Program for Historically Black Colleges and Universities and Minority-Serving Institutions (HBCU/MI) Equipment/Instrumentation, to establish a “Secure Communications and Embedded Systems Laboratory at California State University, Fresno".

Dr. Kulhandjian is an active member of the Association for Computing Machinery (ACM) and the Institute of Electrical and Electronics Engineers (IEEE). He is a Senior Member of IEEE. He is serving as a guest editor for Journal of Sensor and Actuator Networks (JSAN) - Special Issue on Advances in Intelligent Transportation Systems (ITS). He has also served as a Guest Editor for IEEE Access Special Section Journal, Session Co-Chair for IEEE UComms’20 Conference, Session Chair for ACM WUWNet’19 Conference, Publicity Co-Chair for IEEE BlackSeaCom’21 Conference. He also serves as a member of the Technical Program Committee (TCP) for ACM and IEEE Conferences such as GLOBECOM 2021, WD 2021, WTS 2021, WD 2018, ICC 2018, WUWNet 2018, and VTC Fall 2017. He is a recipient of the Outstanding Reviewer Award from ELSEVIER Ad Hoc Networks and ELSEVIER Computer Networks.

Published:

September 2021

Keywords:

Intelligent transportation systems (ITS)

Drowsy Driver Detection

Applied Machine Learning

Accident Prevention

Advanced Driver Assistance systems (ADAS)