AI-based Pedestrian Detection and Avoidance at Night using an IR Camera, Radar, and a Video Camera

Abstract:

In 2019, the United States experienced more than 6,500 pedestrian fatalities involving motor vehicles which resulted in a 67% rise in nighttime pedestrian fatalities and only a 10% rise in daytime pedestrian fatalities. In an effort to reduce fatalities, this research developed a pedestrian detection and alert system through the application of a visual camera, infrared camera, and radar sensors combined with machine learning. The research team designed the system concept to achieve a high level of accuracy in pedestrian detection and avoidance during both the day and at night to avoid potentially fatal accidents involving pedestrians crossing a street. The working prototype of pedestrian detection and collision avoidance can be installed in present-day vehicles, with the visible camera used to detect pedestrians during the day and the infrared camera to detect pedestrians primarily during the night as well as at high glare from the sun during the day. The radar sensor is also used to detect the presence of a pedestrian and calculate their range and direction of motion relative to the vehicle. Through data fusion and deep learning, the ability to quickly analyze and classify a pedestrian’s presence at all times in a real-time monitoring system is achieved. The system can also be extended to cyclist and animal detection and avoidance, and could be deployed in an autonomous vehicle to assist in automatic braking systems (ABS).

Publications:

AI-based Pedestrian Detection and Avoidance at Night using an IR Camera, Radar, and a Video Camera (Full Report)

Research Brief

Authors:

HOVANNES KULHANDJIAN, PHD

Dr. Kulhandjian is an Associate Professor in the Department of Electrical and Computer Engineering at California State University, Fresno (Fresno State), which he joined in Fall 2015 after having been an Associate Research Engineer in the Department of Electrical and Computer Engineering at Northeastern University. He received his BS degree in Electronics Engineering with high honors from the American University in Cairo (AUC) in 2008, and his MS and PhD degrees in Electrical Engineering from the State University of New York at Buffalo in 2010 and 2014, respectively. His current research interests are in digital signal processing, wireless communications, and networking, with applications to underwater and visible light communications, as well as networking geared towards Intelligent Transportation Systems (ITS).

Dr. Kulhandjian has received numerous awards and research grants while at Fresno State including: four research grants from Fresno State Transportation Institute (FSTI); the Claude C. Laval III Award for Commercialization of Research, Innovation and Creativity 2021; and the Claude C. Laval Award for Innovative Technology and Research 2020. In April 2021, he received a grant as a PI from the Department of Defense (DOD) Research and Education Program for Historically Black Colleges and Universities and Minority-Serving Institutions (HBCU/MI) Equipment/Instrumentation to establish a secure Communications and Embedded Systems Laboratory at California State University, Fresno.

Dr. Kulhandjian is an active member of the Association for Computing Machinery (ACM) and the Institute of Electrical and Electronics Engineers (IEEE) including the IEEE Vehicular Technology Society (VTS). He is a Senior Member of IEEE. He is serving as a Guest Editor for the Special Issue “Advances in Intelligent Transportation Systems (ITS)”. He has served as a Guest Editor for IEEE Access Special Section Journal, Session Co-Chair for IEEE UComms’20 Conference, Session Chair for ACM WUWNet’19 Conference, and Publicity Co-Chair for the IEEE BlackSeaCom Conference. He also serves as a member of the Technical Program Committee (TCP) for ACM and IEEE Conferences such as GLOBECOM 2022, WTS 2022, WD 2021, WD 2018, ICC 2018, WUWNet 2020, and WiMob2019. He is a recipient of the Outstanding Reviewer Award from ELSEVIER Ad Hoc Networks and ELSEVIER Computer Networks.

Published:

November 2022

Keywords:

Intelligent transportation systems (ITS)

Applied Machine Learning

Accident Prevention

Advanced Driver Assistance systems (ADAS)

Pedestrian detection