Detecting Driver Drowsiness with Multi-Sensor Data Fusion Combined with Machine Learning

You are here

Detecting Driver Drowsiness with Multi-Sensor Data Fusion Combined with Machine Learning


In this research work, we develop a drowsy driver detection system through the application of visual and radar sensors combined with machine learning. The system concept was derived from the desire to achieve a high level of driver safety through the prevention of potentially fatal accidents involving drowsy drivers. According to the National Highway Traffic Safety Administration, drowsy driving resulted in 50,000 injuries across 91,000 police-reported accidents, and a death toll of nearly 800 in 2017. The objective of this research work is to provide a working prototype of Advanced Driver Assistance Systems that can be installed in present-day vehicles. By integrating two modes of visual surveillance to examine a biometric expression of drowsiness, a camera and a micro-Doppler radar sensor, our system offers high reliability over 95% in the accuracy of its drowsy driver detection capabilities. The camera is used to monitor the driver’s eyes, mouth and head movement and recognize when a discrepancy occurs in the driver's blinking pattern, yawning incidence, and/or head drop, thereby signaling that the driver may be experiencing fatigue or drowsiness. The micro-Doppler sensor allows the driver's head movement to be captured both during the day and at night. Through data fusion and deep learning, the ability to quickly analyze and classify a driver's behavior under various conditions such as lighting, pose-variation, and facial expression in a real-time monitoring system is achieved. 



Dr. Hovannes Kulhandjian is an Associate Professor in the Department of Electrical and Computer Engineering at California State University, Fresno (Fresno State). He joined Fresno State in Fall 2015 as a tenure-track faculty member. Prior to that, he was an Associate Research Engineer in the Department of Electrical and Computer Engineering at Northeastern University. He received his BS degree in Electronics Engineering with high honors from the American University in Cairo (AUC) in 2008, and his MS and PhD degrees in Electrical Engineering from the State University of New York at Buffalo in 2010 and 2014, respectively. His current research interests are in applied machine learning, wireless communications, and networking, with applications to underwater and visible light communications and networking geared towards Intelligent Transportation Systems (ITS).

Dr. Kulhandjian has received three research grants from Fresno State Transportation Institute (FSTI) and he also received the Claude C. Laval III Award for Commercialization of Research, Innovation and Creativity 2021 as well as the Claude C. Laval Award for Innovative Technology and Research 2020 at Fresno State. In April 2021, as a PI he received a grant from the Department of Defense (DOD) Research and Education Program for Historically Black Colleges and Universities and Minority-Serving Institutions (HBCU/MI) Equipment/Instrumentation, to establish a “Secure Communications and Embedded Systems Laboratory at California State University, Fresno".

Dr. Kulhandjian is an active member of the Association for Computing Machinery (ACM) and the Institute of Electrical and Electronics Engineers (IEEE). He is a Senior Member of IEEE. He is serving as a guest editor for Journal of Sensor and Actuator Networks (JSAN) - Special Issue on Advances in Intelligent Transportation Systems (ITS). He has also served as a Guest Editor for IEEE Access Special Section Journal, Session Co-Chair for IEEE UComms’20 Conference, Session Chair for ACM WUWNet’19 Conference, Publicity Co-Chair for IEEE BlackSeaCom’21 Conference. He also serves as a member of the Technical Program Committee (TCP) for ACM and IEEE Conferences such as GLOBECOM 2021, WD 2021, WTS 2021, WD 2018, ICC 2018, WUWNet 2018, and VTC Fall 2017. He is a recipient of the Outstanding Reviewer Award from ELSEVIER Ad Hoc Networks and ELSEVIER Computer Networks.

September 2021
Intelligent transportation systems (ITS)
Drowsy Driver Detection
Applied Machine Learning
Accident Prevention
Advanced Driver Assistance systems (ADAS)



Contact Us

SJSU Research Foundation   210 N. 4th Street, 4th Floor, San Jose, CA 95112    Phone: 408-924-7560   Email: