Machine Learning Systems for HMI

Insights from our World Café session at Auto.AI Europe 2022

Johan von Forstner
September 29, 2022
3 min read

PARADOX CAT recently participated in Auto.AI Europe 2022 and hosted a World Café session, discussing machine learning applications in autonomous vehicles, focusing on in-cabin AI systems for human-machine interaction (HMI). Key takeaways included leveraging advanced sensors like interior cameras and radar for safety and convenience, avoiding false positives in gesture and voice recognition systems, and exploring federated and active learning for privacy-conscious AI development. PARADOX CAT’s new division, PARADOX AI, focuses on innovative in-cabin AI solutions, helping automotive clients develop prototypes and integrate AI into production HMI systems.

Overview

Leveraging from new and existing sensors
Avoiding false positives
Deploying active and federated learning systems
PARADOX CAT and in-cabin AI

Auto.AI takeaways

Recently, PARADOX CAT took part in Auto.AI Europe 2022 in Berlin. Auto.AI is one of the leading conferences on the applications of Deep Learning in the automotive industry, including but not limited to Level 4 and 5 of autonomous driving. As a company that specializes in in-cabin AI applications, we hosted a World Café session on the topic of architecting robust machine learning systems for HMI interaction within autonomous vehicles, moderated by Johan von Forstner (Principal Machine Learning Engineer) and Alexander Schaub (Director Development).

Our World Café session focused on several important challenges in the development of AI systems for natural interaction in the vehicle, and many attendees participated in the interesting discussions around these topics.

Leveraging from new and existing sensors

Safety-critical in-cabin sensing applications such as driver monitoring or child presence detection, but also convenience features, like gesture control, greatly benefit from dedicated sensors such as interior cameras with RGB and near-infrared capabilities or depth sensors like time-of-flight cameras. Modern millimeter-wave radars, which are traditionally used as exterior sensors for autonomous driving, are also quickly gaining popularity in the interior space due to their low cost and power consumption. They are even accurate enough to detect vital signals, including breathing and heart rate. On the other hand, participants stressed, that it should be possible to leverage the existing sensors in the cabin for new use cases with machine learning and sensor fusion, including microphones, steering wheel and seat occupancy sensors.

Avoiding false positives

In gesture or voice interaction systems, false positives can easily lead to frustration and increased driver distraction if they trigger unwanted actions in the HMI. There is always a tradeoff between making the system as natural and easy to use as possible while also keeping the gestures or voice commands distinct enough to avoid triggering them accidentally.

In other words: Should the system be trained to work well for every user, or should the user need to be trained to use the system correctly? This problem is hard to solve in general, so it requires tailored solutions depending on the application. Possible approaches include personalization of the system to the user through active learning, as well as combination with other modalities like gaze and pose detection to provide additional context.

Deploying active and federated learning systems

Just like autonomous driving systems, in-cabin sensing applications usually benefit from large and diverse datasets collected from the fleet. However, sending data back from customer vehicles may raise privacy concerns, especially in the case of interior cameras.

Federated learning, where only gradients of the deep learning model are transmitted, provides a privacy-friendly approach to this, however it is not directly applicable in cases where we cannot benefit from self-supervised techniques to label the data. In these cases, it could be combined with active learning, where the user can directly help to improve the AI system by providing labels for a couple of data samples. In the car, special care needs to be taken to choose a suitable UX design for such active learning sessions.

PARADOX CAT and in-cabin AI

PARADOX CAT has created a new division (PARADOX AI) with focus on artificial intelligence and machine learning, which has a startup character within our already established software development business. We strongly believe that this technology will have the greatest impact on everyone’s lives in the future. Focusing on new in-cabin applications of AI, we enable our automotive customers to develop prototypes in this field based on cutting-edge research. In addition, with our experience in series development of automotive HMI systems, we can support customers with the integration of AI applications into these platforms to bring them into production. Do you want to learn more about our PARADOX AI team and services? We are looking forward to get in touch.