Our Visit at International Conference on Computer Vision (ICCV) 2023

Paradox Cat

TECH BLOG

Our Visit at International Conference on Computer Vision (ICCV) 2023

PARADOX CAT at #ICCV2023 🥐

Dr. Johan von Forstner

AI R&D Architect @ PARADOX CAT GmbH

International Conference on Computer Vision Oral Session
One of the oral sessions, Photo: ICCV

In October 2023, I attended the ICCV in Paris—one of the top global conferences in Computer Vision. Representing Paradox Cat’s AI division, which focuses on interior sensing and HMI systems for vehicles, I explored the latest research trends and innovations in the field. A particularly insightful tutorial covered the In-Camera rendering pipeline, highlighting how raw sensor data is transformed into images using modern AI techniques. The conference showcased a wide range of interdisciplinary work, underlining how rapidly the field is evolving and blending into other domains.

Overview

Attending ICCV 2023 in Paris

Johan standing infront of a ICCV23 banner
Finally arrived at the entrance to ICCV

In October 2023, I had the great opportunity to join the International Conference on Computer Vision (ICCV), one of the top scientific conferences in the Computer Vision field, which was held in Paris this year. Our AI division at Paradox Cat is working on new technologies for interior sensing and HMI interaction in vehicles, so joining ICCV was a perfect way to learn about the latest research in the field using my Paradox Cat training budget.

Kicking Off with Cameras and Algorithms

After getting off the train on which I had been speeding through the French countryside at 320 km/h, I headed directly to the conference venue at Paris Expo Porte de Versailles, where the workshops and tutorial sessions were already ongoing. I joined the tutorial on Understanding the In-Camera Rendering Pipeline and the Role of AI/Deep Learning, which gave an extensive background on how modern cameras perceive the world and which algorithms are needed to turn the raw sensor output into beautiful images.

Main Conference Highlights

The following three days constituted the main conference, with two tracks of oral sessions, as well as poster sessions in between. Certainly by now, it became clear that the field of Computer Vision and AI is really exploding — with many poster sessions rivaling the Paris Métro in terms of crowding, it was not always easy to push one’s way through towards the poster one wanted to see. Especially presentations from the well-known Big Tech companies and the award-winning papers like Tracking Everything Everywhere All at Once, Segment Anything and ControlNet received their well-deserved attention.

many people looking at the poster session at ICCV23
Packed poster session at ICCV 2023

Interdisciplinary Trends and Insights

It was also interesting to see that Computer Vision is becoming very interdisciplinary, in the sense that many methods are not just bound to specific problems, but are often applied all across the field and combined in new creative ways. This also meant that it probably became pretty hard for the conference organizers to group the papers into distinct sessions, and there really was something interesting in almost every session.

Inspiring Keynotes and Takeaways

Besides the oral and poster sessions, the final two days also incorporated two invited keynote sessions. Especially the talk from Pushmeet Kohli (Google DeepMind) was very inspiring, giving an insight to how DeepMind is using AI to help solve hard problems in science, such as their recent success predicting protein structures with AlphaFold.

HuggingFace 🤗 Open Source AI community event at STATION F
HuggingFace 🤗 Open Source AI community event at STATION F
johan and a llama infant of iccv
Trying to talk to the LLaMAs at the entrance

Merci, Paris!

Au revoir, et merci beaucoup, Paris 🥐🇫🇷!
Thanks a lot to the ICCV 2023 organizers for this great event, and thank you to Paradox Cat for letting me participate! I hope that we can also present some of our own work at a future conference.

Related articles

Machine Learning systems for HMI interaction

As a company that specializes in in-cabin AI applications, we hosted a World Café session on the topic of architecting robust machine learning systems for HMI interaction within autonomous vehicles, moderated by Johan von Forstner (Principal Machine Learning Engineer) and Alexander Schaub (Director Development).

Follow us on our platforms!

If you want to be up-to-date feel free to follow us on our social media channels like Instagram, LinkedIn and Medium!

Latest Blog Post

Machine Learning Systems for HMI

Paradox Cat

TECH BLOG

Machine Learning Systems for HMI

Insights from our World Café session at Auto.AI Europe 2022

Dr. Johan von Forstner

AI R&D Architect @ PARADOX CAT GmbH

Machine Learning systems for HMI interaction

PARADOX CAT recently participated in Auto.AI Europe 2022 and hosted a World Café session, discussing machine learning applications in autonomous vehicles, focusing on in-cabin AI systems for human-machine interaction (HMI). Key takeaways included leveraging advanced sensors like interior cameras and radar for safety and convenience, avoiding false positives in gesture and voice recognition systems, and exploring federated and active learning for privacy-conscious AI development. PARADOX CAT’s new division, PARADOX AI, focuses on innovative in-cabin AI solutions, helping automotive clients develop prototypes and integrate AI into production HMI systems.

Overview

Auto.AI takeaways

Recently, PARADOX CAT took part in Auto.AI Europe 2022 in Berlin. Auto.AI is one of the leading conferences on the applications of Deep Learning in the automotive industry, including but not limited to Level 4 and 5 of autonomous driving. As a company that specializes in in-cabin AI applications, we hosted a World Café session on the topic of architecting robust machine learning systems for HMI interaction within autonomous vehicles, moderated by Johan von Forstner (Principal Machine Learning Engineer) and Alexander Schaub (Director Development).

Machine Learning systems for HMI interaction
Johan and Alexander moderate our World Café session at Auto.AI

Our World Café session focused on several important challenges in the development of AI systems for natural interaction in the vehicle, and many attendees participated in the interesting discussions around these topics.

The participants’ ideas were collected on the board and further discussed in following World Café rounds

Leveraging from new and existing sensors

Safety-critical in-cabin sensing applications such as driver monitoring or child presence detection, but also convenience features, like gesture control, greatly benefit from dedicated sensors such as interior cameras with RGB and near-infrared capabilities or depth sensors like time-of-flight cameras. Modern millimeter-wave radars, which are traditionally used as exterior sensors for autonomous driving, are also quickly gaining popularity in the interior space due to their low cost and power consumption. They are even accurate enough to detect vital signals, including breathing and heart rate. On the other hand, participants stressed, that it should be possible to leverage the existing sensors in the cabin for new use cases with machine learning and sensor fusion, including microphones, steering wheel and seat occupancy sensors.

Avoiding false positives

In gesture or voice interaction systems, false positives can easily lead to frustration and increased driver distraction if they trigger unwanted actions in the HMI. There is always a tradeoff between making the system as natural and easy to use as possible while also keeping the gestures or voice commands distinct enough to avoid triggering them accidentally.

In other words: Should the system be trained to work well for every user, or should the user need to be trained to use the system correctly? This problem is hard to solve in general, so it requires tailored solutions depending on the application. Possible approaches include personalization of the system to the user through active learning, as well as combination with other modalities like gaze and pose detection to provide additional context.

Deploying active and federated learning systems

Just like autonomous driving systems, in-cabin sensing applications usually benefit from large and diverse datasets collected from the fleet. However, sending data back from customer vehicles may raise privacy concerns, especially in the case of interior cameras.

Federated learning, where only gradients of the deep learning model are transmitted, provides a privacy-friendly approach to this, however it is not directly applicable in cases where we cannot benefit from self-supervised techniques to label the data. In these cases, it could be combined with active learning, where the user can directly help to improve the AI system by providing labels for a couple of data samples. In the car, special care needs to be taken to choose a suitable UX design for such active learning sessions.

PARADOX CAT and in-cabin AI

PARADOX CAT has created a new division (PARADOX AI) with focus on artificial intelligence and machine learning, which has a startup character within our already established software development business. We strongly believe that this technology will have the greatest impact on everyone’s lives in the future. Focusing on new in-cabin applications of AI, we enable our automotive customers to develop prototypes in this field based on cutting-edge research. In addition, with our experience in series development of automotive HMI systems, we can support customers with the integration of AI applications into these platforms to bring them into production. Do you want to learn more about our PARADOX AI team and services? We are looking forward to get in touch.

Related articles

International Conference on Computer Vision Oral Session

Paradox Cat TECH BLOG Our Visit at International Conference on Computer Vision (ICCV) 2023 PARADOX CAT at #ICCV2023 🥐 Johan von Forstner …

Follow us on our platforms!

If you want to be up-to-date feel free to follow us on our social media channels like Instagram, LinkedIn and Medium!

Latest Blog Post

International Conference on Computer Vision Oral Session

Paradox Cat TECH BLOG Our Visit at International Conference on Computer Vision (ICCV) 2023 PARADOX CAT at #ICCV2023 🥐 Johan von Forstner …