Leveraging Generative AI in Vehicles for Enhanced Driver Safety and Advanced Communication Systems

Abstract

This paper proposes an integrated artificial intelligence–based driver assistance system for electric vehicles (EVs) that combines computer vision–based drowsiness detection with a generative artificial intelligence (GenAI)–driven conversational interaction framework to enhance driver safety and human–vehicle interaction. The primary objective of this work is to reduce fatigue-related driving risks while enabling natural, hands-free, and context-aware communication between the driver and the vehicle. The core idea is to tightly couple real-time driver state monitoring with intelligent conversational feedback, allowing safety alerts and voice interactions to adapt dynamically to the driver’s condition. Driver drowsiness is detected using non-intrusive visual indicators, namely eye closure duration and blink rate, extracted from an in-vehicle camera. A drowsy state is identified when eye closure exceeds 10 s or when the blink rate exceeds 6 blinks within a 6 s interval. Upon detection, the system generates multi-modal alerts consisting of audio warnings and vibration feedback, while a GenAI-based natural language processing module provides real-time, hands-free voice interaction. Experimental evaluation was conducted on an ESP32-based embedded prototype across five predefined driving scenarios representing normal and fatigued conditions. The results show stable face and eye detection under normal driving and achieved 100% correct alert triggering in all drowsiness-related cases (3 out of 5 scenarios), with zero false positives observed during non-drowsy conditions (2 out of 5 scenarios). The system demonstrated consistent real-time response and reliable alert activation under fatigue conditions. The main contribution and novelty of this research lie in the real-time integration of generative AI–driven conversational intelligence with embedded computer vision–based drowsiness detection within a unified, resource-constrained platform, which is rarely addressed jointly in existing systems. Overall, the proposed framework provides a practical, scalable, and human-centered solution for intelligent driver assistance in semi-autonomous and future autonomous EV environments.