NVIDIA Container Toolkit
An open-source container toolkit that gives Docker workloads access to NVIDIA GPUs on Linux. Requires the NVIDIA driver, but not the host CUDA Toolkit.
NVIDIA Container Toolkit lets Docker containers use NVIDIA GPUs on a Linux machine or server. It's for developers and server operators who need GPU acceleration for containerized workloads, including self-hosted AI software. The project is open source under the Apache 2.0 license.
The toolkit includes a container runtime library and utilities that automatically configure containers to use the machine's NVIDIA GPUs. Its role is to provide GPU access for the software inside those containers. That makes it relevant when you're choosing the infrastructure for an AI service packaged as a Docker container; the service itself supplies the models and user interface.
The Linux host needs an NVIDIA GPU and the NVIDIA driver. It doesn't need the CUDA Toolkit installed on the host. GPU acceleration comes from the hardware attached to your own machine or server.
