A Multimodal Approach for Monitoring Driving Behavior and Emotions

Abstract:

Studies have indicated that emotions can significantly be influenced by environmental factors; these factors can also significantly influence drivers’ emotional state and, accordingly, their driving behavior. Furthermore, as the demand for autonomous vehicles is expected to significantly increase within the next decade, a proper understanding of drivers’/passengers’ emotions, behavior, and preferences will be needed in order to create an acceptable level of trust with humans. This paper proposes a novel semi-automated approach for understanding the effect of environmental factors on drivers’ emotions and behavioral changes through a naturalistic driving study. This setup includes a frontal road and facial camera, a smart watch for tracking physiological measurements, and a Controller Area Network (CAN) serial data logger. The results suggest that the driver’s affect is highly influenced by the type of road and the weather conditions, which have the potential to change driving behaviors. For instance, when the research defines emotional metrics as valence and engagement, results reveal there exist significant differences between human emotion in different weather conditions and road types. Participants’ engagement was higher in rainy and clear weather compared to cloudy weather. More-over, engagement was higher on city streets and highways compared to one-lane roads and two-lane highways.

Publications:

A Multimodal Approach for Monitoring Driving Behavior and Emotions (Full Report)

Research Brief

Authors:

ARASH TAVAKOLI
Arash Tavakoli is a PhD candidate, Engineering Systems and Environment (ESE) at the Link Lab of the University of Virginia. He has earned his BSc. and MSc. in Civil Engineering from the Sharif University of Technology and Virginia Tech respectively. Arash’s research interest lies on the intersection of psychology, transportation engineering, and computer science. Specifically, Arash implements psychological theories and experimental design together with affective computing and computer vision techniques to enhance human-autonomous vehicle interaction.

VAHID BALALI
The Principal Investigator, Vahid Balali, PhD, is an Assistant Professor in the Department of Civil Engineering and Construction Engineering Management at California State University Long Beach. Dr. Balali’s research focuses on visual data sensing and analytics, virtual design and construction for civil infrastructure and interoperable system integration, and smart cities in transportation for sustainable decision-making. He also has experience as a visual data analyst and has developed a video-based construction resource tracking and action recognition for activity analysis of operators at Caterpillar. He has the knowledge, technical skills, and experience that are crucial to the successful completion of the proposed work.

Dr. Vahid Balali has been named a recipient of the 2020 Early Academic Career Excellence Award by California State University Long Beach. He was also selected as one of the Top 40 under 40 by the Consulting-Specifying-Engineer for the year 2017 and as the top young professional in California by the Engineering News Record (ENR) for the year 2016. He received the 2014 second-best poster award from the Construction Research Congress, as well as the 2013 CMAA national capital chapter scholarship award. He is currently an associate member of ASCE and CMAA, a committee member of the ASCE Data Sensing and Analysis and ASCE Visual Information Modeling and Simulation committees, and a friend member of relevant TRB committees. He is also serving as a reviewer of several top-tier journals. He is actively collaborating with industrial partners and is involved in professional and outreach activities.

ARSALAN HEYDARIAN
Arsalan Heydarian is an Assistant Professor in the Department of Engineering Systems and Environment at the University of Virginia and he's part of the the UVA Cyber-physical systems lab, the Link Lab. His research broadly focuses on user-centered design, construction, and operation of intelligent infrastructure with the objective of enhancing their sustainability, adaptability, and resilience. Dr. Heydarian received his PhD in Civil Engineering from the University of Southern California (USC), MSc in System Engineering from USC, and B.Sc. and M.Sc in Civil Engineering from Virginia Tech.

Published:

July 2020

Keywords:

Naturalistic driving study

Emotions and behaviors

Human sensing

Computer vision

Human-in-the-loop systems