NVIDIA DRIVE IX
NVIDIA DRIVE IX empowers AV developers with means to interacting with the vehicle. DRIVE IX is the integrated vision, voice, and graphics user experience that is a companion stack to DRIVE AV.
DRIVE Mapping is suited for the following objectives:
- Driver Monitoring.
- Human/Machine Interface (HMI) i.e. user interface using voice/speech/natural language and gesture for driver and vehicle occupants.
- Visualization - both for production as well as for development.
- Production visualization may include confidence view in the instrument cluster.
- Development visualization may include camera/sensor inputs augmented with useful statistics.
Details
AI Assistant
AI Assistant enables voice recognition, lip reading, face identification and emotions detection for an AI powered human/machine interface (HMI). Some of the main capabilities of AI Assistant are: automatic speech recognition (ASR), natural language understanding (NLU), face detection and emotion detection.
AI CoPilot
AI CoPilot provides driver monitoring system using a driver-facing camera, IR LED, and sophisticated deep learning software running on the NVIDIA DRIVE AGX system in the car. This module enables development of applications to ensure drivers stay alert, or take action if a driver is distracted or drowsy. For example, by tracking head and eyes to understand where driver is paying attention, and monitoring blink frequency to assess fatigue and drowsiness. Some of the key capabilities of AI Co-Pilot include: 3 dimensional gaze detection, drowsiness detection, distraction detection and head pose detection.
Visualization - DRIVE AR
DRIVE AR takes input from Drive AV and AI Cockpit applications to provide real-time information on the instrument cluster for the driver as well as visual information of what the autonomous vehicle or driver monitoring systems are viewing. AutoPilot Monitor or Confidence View is the virtualized view for the instrument cluster; of what the autonomous driving application perceives, and acts upon. AV Viz and IX Viz applications take camera and sensor feeds from DRIVE AV and DRIVE IX applications augmented with important useful information (such as bounding boxes, stats etc) for a software developer.
Confidence View
AV Viz
IX Viz
The intelligent user experience cockpit, powered by NVIDIA DRIVE IX, leverages AI to ensure the driver’s attention is always on the road and deliver seamless convenience features. With advanced driver monitoring, the system can determine if a driver is drowsy or distracted, and recognize the driver for easy vehicle access.
Developing with DRIVE IX
How to set up
You will need:
- Host Development PC
- DRIVE AGX Developer Kit
- Driver facing camera + Camera calibration kit (Consult the DRIVE IX Calibration Guide for details)
- Optional: DRIVE Hyperion Kit
Steps:
- Install DRIVE Perception via the SDK Manager.
- Consult the DRIVE IX and DRIVE IX API sections of DRIVE Software Documentation
- Consult the DRIVE AR and DRIVE AR API sections of DRIVE Software Documentation
How to develop
| Development Tasks | Getting Started |
|---|---|
| Build applications using the APIs for Driver Monitoring |
DriveIX client library provides a C-styled interface to interact with NVIDIA's AI driven DriveIX service. User can obtain data inferences provided by DriveIX via this interface. Available features are:
Consult the DRIVE IX API section of the DRIVE Software Documentation for details of NVIDIA DRIVE IX Client Lib APIs. |
| Use applications to visualize DRIVE AV and DRIVE IX | Requires DRIVE Hyperion Kit Consult the DRIVE AR section included in the DRIVE Software Documentation |
Additional Development Resources:
Documentation
- All NVIDIA DRIVE documentation can be found here: https://developer.nvidia.com/drive/documentation