A Lightweight Multi-Modality Learning Model for Detecting Drivers’ Drowsiness on Vehicle CAN-bus

Abstract

This paper proposes a lightweight multimodality learning model for detecting driver's drowsiness or fatigue based on multiple sources of input data (modality), i.e. vision data or modality (provided by in-cabin camera) and sensor data or modality collected from the vehicle's CAN bus network (i.e. brake force, throttle force, steering angle). This proposed model helps to improve the performance for detecting drivers' drowsiness compared to the existing models which are usually based on single source of input data such as sensing data or vision data. Experimental results show that the proposed multi-modal detection model significantly reduces the false alarm rate (caused by eyes' momentary closure or head rotation) compared to single-modal models proposed in the literature, and achieves approximately 93% accuracy for detecting drivers' drowsiness.