Safety decision-making for autonomous vehicles integrating passenger physiological states by fNIRS

In recent years, several serious traffic accidents have exposed the shortcomings of current autonomous driving systems in making safe decisions. Traditional decision-making methods, due to functional deficiencies or machine performance limitations, struggle to address potential risky behaviors, leading to a continued need for human intervention in complex driving scenarios. To address this, researchers have begun exploring the use of human physiological states as an information source to improve the safety decision-making of autonomous vehicles. “Functional Near-Infrared Spectroscopy (fNIRS), as a non-invasive real-time brain activity monitoring method, can provide cognitive information related to human risk perception and emotional states, and is thus considered a tool that can enhance autonomous driving systems.” said the author Xiaofei Zhang, a professor at Tsinghua University, “Our study introduces an intelligent decision-making algorithm based on fNIRS by analyzing passengers' physiological states, aiming to improve the safety and decision-making efficiency of autonomous vehicles when facing risky scenarios.”

The research process of this paper can be divided into the following parts: Initially,  an intelligent safety decision-making algorithm that integrates passengers’ physiological states (detected via fNIRS) into the decision-making process of autonomous driving is presented. This algorithm is based on Twin-Delayed Deep Deterministic Policy Gradient (TD3) and incorporates passenger risk assessment information to help the system make safer decisions in risky scenarios. The algorithm utilizes human guided deep reinforcement learning mechanisms to switch to a more conservative intelligent driving model (IDM) when passengers are detected to be in a risky state, thereby accelerating the learning process and improving the safety and comfort of the system. The experimental results show that this method is superior to the traditional methods in convergence speed, safety and driving comfort, which shows the potential application in auto drive system.

This study demonstrates that integrating passenger physiological states detected by fNIRS into the decision-making algorithm for autonomous driving effectively enhances safety and comfort in risky scenarios. Compared to traditional methods, this algorithm shows superior performance in learning convergence speed, safety, and driving comfort. However, the experimental scenarios were relatively simplified, and the participants had a narrow age range and homogeneous background, which may limit the generalizability of the findings. Additionally, due to time constraints, the learning process of the algorithm did not fully explore the optimal strategy. “Future research will aim to validate the algorithm in more complex and realistic driving scenarios and further enhance the accuracy and robustness of driving risk assessment by integrating information from vehicle sensors..” said Xiaofei Zhang.

Authors of the paper include Xiaofei Zhang, Haoyi Zheng, Jun Li, Zongsheng Xie, Huamu Sun, and Hong Wang.

This work was supported by the National Science Foundation of China Project 52072215, 52221005 and 12361105, Beijing Natural Science Foundation L243025, National key R&D Program of China 2022YFB2503003 and State Key Laboratory of Intelligent Green Vehicle and Mobility.

The paper, “Safety Decision-Making for Autonomous Vehicles Integrating Passenger Physiological States by fNIRS” was published in the journal Cyborg and Bionic Systems on May 13, 2025, at  10.34133/cbsystems.0205.

END