Biblio

In an effort towards predicting mental workload while driving, previous research found interactions between working memory load and visuospatial demands, which complicates the accurate prediction of momentary mental workload. To investigate this interaction, the cognitive concepts working memory load and visuospatial attention were integrated into a cognitive driving model using the cognitive architecture ACT-R. The model was developed to safely drive on a multi-lane highway with ongoing traffic while performing a secondary n-back task using speed signs. To manipulate visuospatial demands, the model must drive through a construction site with reduced lane-width in certain blocks of the experiment. Furthermore, it is able to handle complex driving situations such as overtaking traffic while adjusting the speed according to the n-back task. The behavioral results show a negative effect on driving performance with increasing task difficulty of the secondary task. Additionally, the model indicates an interaction at a common, task-unspecific level.

Turning through oncoming traffic at unsignalized intersections can lead to safety-critical situations contributing to 7.4% of all non-severe vehicle crashes. One of the main reasons for these crashes are human errors in the form of incorrect estimation of the gap size with respect to the Principle Other Vehicle (POV). Vehicle-to-vehicle (V2V) technology promises to increase safety in various traffic situations. V2V infrastructure combined with further integration of sensor technology and human intention prediction could help reduce the frequency of these safety-critical situations by predicting dangerous turning manoeuvres in advance, thus, allowing the POV to prepare an appropriate reaction. We performed a driving simulator study to investigate turning decisions at unsignalized intersections. Over the course of the experiments, we recorded over 5000 turning decisions with respect to different gap sizes. Afterwards, the participants filled out a questionnaire featuring demographic and driving style related items. The behavioural and questionnaire data was then used to fit a Bayesian Network model to predict the turning intention of the subject vehicle. We evaluate the model and present the results of a feature importance analysis. The model is able to correctly predict the turning intention with an accuracy of 74%. Furthermore, the feature importance analysis indicates that user specific information is a valuable contribution to the model. We discuss how a working turning intension prediction could reduce the number of safety-critical situations.

In the future, mixed traffic Highly Automated Vehicles (HAV) will have to resolve interactions with human operated traffic. A particular problem for HAVs is the detection of human states influencing safety, critical decisions, and driving behavior of humans. We demonstrate the value proposition of neurophysiological sensors and driver models for optimizing performance of HAVs under safety constraints in mixed traffic applications.

We performed a driving simulator study to investigate merging decisions with respect to an interaction partner in time-critical situations. The experimental paradigm was a two-alternative forced choice, where the subjects could choose to merge before human vehicles or highly automated vehicles (HAV). Under time pressure, subjects showed a significantly higher gap acceptance during merging situations when interacting with HAV. This confirmed our original hypothesis that when interacting with HAV, drivers would exploit the HAV's technological advantages and defensive programming in time-critical situations.
 

Driving is a complex task concurrently drawing on multiple cognitive resources. Yet, there is a lack of studies investigating interactions at the brain-level among different driving subtasks in dual-tasking. This study investigates how visuospatial attentional demands related to increased driving difficulty interacts with different working memory load (WML) levels at the brain level. Using multichannel whole-head high density functional near-infrared spectroscopy (fNIRS) brain activation measurements, we aimed to predict driving difficulty level, both separate for each WML level and with a combined model. Participants drove for approximately 60 min on a highway with concurrent traffic in a virtual reality driving simulator. In half of the time, the course led through a construction site with reduced lane width, increasing visuospatial attentional demands. Concurrently, participants performed a modified version of the n-back task with five different WML levels (from 0-back up to 4-back), forcing them to continuously update, memorize, and recall the sequence of the previous 'n' speed signs and adjust their speed accordingly. Using multivariate logistic ridge regression, we were able to correctly predict driving difficulty in 75.0% of the signal samples (1.955 Hz sampling rate) across 15 participants in an out-of-sample cross-validation of classifiers trained on fNIRS data separately for each WML level. There was a significant effect of the WML level on the driving difficulty prediction accuracies [range 62.2-87.1%; χ2(4) = 19.9, p < 0.001, Kruskal-Wallis H test] with highest prediction rates at intermediate WML levels. On the contrary, training one classifier on fNIRS data across all WML levels severely degraded prediction performance (mean accuracy of 46.8%). Activation changes in the bilateral dorsal frontal (putative BA46), bilateral inferior parietal (putative BA39), and left superior parietal (putative BA7) areas were most predictive to increased driving difficulty. These discriminative patterns diminished at higher WML levels indicating that visuospatial attentional demands and WML involve interacting underlying brain processes. The changing pattern of driving difficulty related brain areas across WML levels could indicate potential changes in the multitasking strategy with level of WML demand, in line with the multiple resource theory.

Mental workload is a popular concept in ergonomics as it provides an intuitive explanation why exceedingly cognitive task demands result in a decrease in task performance and increase the risk of fatal incidents while driving. At the same time, affective states such as frustration, also play a role in traffic safety as they increase the tendency for speedy and aggressive driving and may even degrade cognitive processing capacities. To reduce accidents due to dangerous effects of degraded cognitive processing capacities and affective biases causing human errors, it is necessary to continuously assess multiple user states simultaneously to better understand potential interactions. In two previous studies, we measured brain activity with functional near-infrared spectroscopy (fNIRS) for separate brain based prediction of working memory load (WML) (Unni et al., 2017) and frustration levels (Ihme et al. submitted) while driving. Here, we report results from a study designed to predict simultaneously manipulated WML and frustration using data driven machine learning approaches from whole-head fNIRS brain activation measurements. 

Experiencing frustration while driving can harm cognitive processing, result in aggressive behavior and hence negatively influence driving performance and traffic safety. Being able to automatically detect frustration would allow adaptive driver assistance and automation systems to adequately react to a driver’s frustration and mitigate potential negative consequences. To identify reliable and valid indicators of driver’s frustration, we conducted two driving simulator experiments. In the first experiment, we aimed to reveal facial expressions that indicate frustration in continuous video recordings of the driver’s face taken while driving highly realistic simulator scenarios in which frustrated or non-frustrated emotional states were experienced. An automated analysis of facial expressions combined with multivariate logistic regression classification revealed that frustrated time intervals can be discriminated from non-frustrated ones with accuracy of 62.0% (mean over 30 participants). A further analysis of the facial expressions revealed that frustrated drivers tend to activate muscles in the mouth region (chin raiser, lip pucker, lip pressor). In the second experiment, we measured cortical activation with almost whole-head functional near-infrared spectroscopy (fNIRS) while participants experienced frustrating and non-frustrating driving simulator scenarios. Multivariate logistic regression applied to the fNIRS measurements allowed us to discriminate between frustrated and non-frustrated driving intervals with higher accuracy of 78.1% (mean over 12 participants). Frustrated driving intervals were indicated by increased activation in the inferior frontal, putative premotor and occipito-temporal cortices. Our results show that facial and cortical markers of frustration can be informative for time resolved driver state identification in complex realistic driving situations. The markers derived here can potentially be used as an input for future adaptive driver assistance and automation systems that detect driver frustration and adaptively react to mitigate it.