Building an Intelligent Cockpit with NVIDIA DRIVE IX

The transition to intelligent vehicles is more than just autonomous driving — it extends to the in-cabin experience as well.

With centralized, high-performance compute and open, scalable software platforms, developers can build a wide variety of innovative applications for a safer, more convenient ride for all occupants.

NVIDIA DRIVE IX is an open software platform that delivers interior sensing for innovative AI cockpit solutions. It provides perception applications to access features and deep neural networks for advanced driver and occupant monitoring, AR/VR visualization, and natural language interactions between the vehicle and occupants. DRIVE IX also offers features such as child or child seat detection, valuables detection, gesture recognition and seat belt detection.

In a webinar led by NVIDIA experts, we’ll cover the architecture of the NVIDIA DRIVE IX open platform and how to build intelligent occupant-centric applications, as well as future capabilities coming in the near future.

By building on this platform, developers can create truly software-defined vehicles, improving the occupant experience alongside automated and autonomous driving features.

Convenience Inside and Out

A key feature of NVIDIA DRIVE IX is occupant understanding. Using interior-facing cameras, the system can monitor the state of the driver, personalize the cockpit experience and detect emergency situations.

AI-powered driver monitoring tracks driver activity, head position and facial movements to analyze whether the driver is paying attention, drowsy or distracted. This capability is critical as automated driving functions become more commonplace across vehicle fleets, ensuring the human at the wheel is alert and paying attention.

Using the same driver-facing cameras, DRIVE IX can also personalize the vehicle’s interior depending on who is at the wheel. Once a driver is identified, the system can adjust seating, temperature controls, infotainment and more to their personal preferences.

This personalization extends outside the vehicle as well. The system can recognize the vehicle owner as they approach, allowing them to unlock and access the vehicle solely using their identity. 

Using the occupant-monitoring cameras, DRIVE IX can detect if the driver and passengers are wearing their seatbelts correctly, sitting in the correct posture and detect if someone has left behind a laptop, backpack or any other valuables. It can also detect the presence of car seats and children in the vehicle to ensure no one is forgotten in the backseat. Each seat can also be personalized using the DRIVE IX SDK. 

Intelligent Interaction

Through visualization, speech recognition and full car fusion, DRIVE IX enables seamless interaction between the vehicle and its passengers, building trust for a safer and more effective driving experience.

With a crystal-clear confidence view, drivers can get a sense of what the sensors on the car see, quickly and easily. Augmented reality heads-up displays and virtual reality views of the vehicle’s surroundings deliver the most important data (such as parking assistance, directions, speed and oncoming obstacles) without disrupting the driver’s line of sight.

Using embedded Natural Language Processing, DRIVE IX enables easy access to search queries, like finding the best coffee shops or sushi restaurants along a given route. The same system that monitors driver attention can also interpret gesture controls, providing another way for drivers to communicate with the cockpit without having to divert their gaze.

Developers can integrate applications that create a real life human interaction, e.g. trigger speech based on the user’s gaze or gesture. The system is capable of maintaining multiple contexts, permitting each occupant to have different conversations simultaneously. 

When run on the high-performance, energy-efficient DRIVE AGX platform, DRIVE IX can fuse sensors both inside and outside the vehicle to ensure passengers don’t exit the vehicle when another road user is passing their door, enhancing safety for everyone on the road.

Developers can learn how to implement these features and more on NVIDIA DRIVE IX for a safer, more convenient driving experience in our webinar on Aug. 19.