dstack

A self-hosted AI compute orchestrator runs training and inference across GPU clouds, Kubernetes and on-prem servers, with NVIDIA and AMD support.

dstack is a self-hosted AI compute orchestration tool for teams running development, training and inference across cloud providers and their own servers. It gives those workloads a shared interface across Kubernetes, VMs and bare-metal clusters.

Compute fleets: Provision and manage clusters across GPU clouds, Kubernetes and on-prem infrastructure.
Training and batch jobs: Run workloads on a single node or across a cluster, with job queues and recovery handling for failed runs or unavailable compute.
Model serving: Deploy inference through secure endpoints with autoscaling and networking management.
Development environments: Give IDEs and AI agents access to compute; skills let Claude, Codex and Cursor manage fleets and submit workloads.
Persistent storage: Use instance and network volumes to retain data for workloads.

The server runs on Linux, macOS and Windows through WSL 2. Supported accelerators include NVIDIA and AMD GPUs, Google TPU and Tenstorrent hardware. Workloads can use cloud compute or your own on-prem servers, so self-hosting the control server doesn't mean every workload runs locally. The code is open source under the Mozilla Public License 2.0 (MPL-2.0).