A Multimodal Approach for Monitoring Driving Behavior and Emotions

A Multimodal Approach for Monitoring Driving Behavior and Emotions

Abstract: 

Studies have indicated that emotions can significantly be influenced by environmental factors; these factors can also significantly influence drivers’ emotional state and, accordingly, their driving behavior. Furthermore, as the demand for autonomous vehicles is expected to significantly increase within the next decade, a proper understanding of drivers’/passengers’ emotions, behavior, and preferences will be needed in order to create an acceptable level of trust with humans. This paper proposes a novel semi-automated approach for understanding the effect of environmental factors on drivers’ emotions and behavioral changes through a naturalistic driving study. This setup includes a frontal road and facial camera, a smart watch for tracking physiological measurements, and a Controller Area Network (CAN) serial data logger. The results suggest that the driver’s affect is highly influenced by the type of road and the weather conditions, which have the potential to change driving behaviors. For instance, when the research defines emotional metrics as valence and engagement, results reveal there exist significant differences between human emotion in different weather conditions and road types. Participants’ engagement was higher in rainy and clear weather compared to cloudy weather. More-over, engagement was higher on city streets and highways compared to one-lane roads and two-lane highways.

Authors: 

VAHID BALALI

The Principal Investigator, Vahid Balali, PhD, is an Assistant Professor in the Department of Civil Engineering and Construction Engineering Management at California State University Long Beach. Dr. Balali’s research focuses on visual data sensing and analytics, virtual design and construction for civil infrastructure and interoperable system integration, and smart cities in transportation for sustainable decision-making. He also has experience as a visual data analyst and has developed a video-based construction resource tracking and action recognition for activity analysis of operators at Caterpillar. He has the knowledge, technical skills, and experience that are crucial to the successful completion of the proposed work.

Dr. Vahid Balali has been named a recipient of the 2020 Early Academic Career Excellence Award by California State University Long Beach. He was also selected as one of the Top 40 under 40 by the Consulting-Specifying-Engineer for the year 2017 and as the top young professional in California by the Engineering News Record (ENR) for the year 2016. He received the 2014 second-best poster award from the Construction Research Congress, as well as the 2013 CMAA national capital chapter scholarship award. He is currently an associate member of ASCE and CMAA, a committee member of the ASCE Data Sensing and Analysis and ASCE Visual Information Modeling and Simulation committees, and a friend member of relevant TRB committees. He is also serving as a reviewer of several top-tier journals. He is actively collaborating with industrial partners and is involved in professional and outreach activities.

Published: 

July 2020

Keywords: 

Naturalistic driving study
Emotions and behaviors
Human sensing
Computer vision
Human-in-the-loop systems