dstack

Overview

dstack is an open-source orchestration layer for AI workloads, aimed at teams that need to provision and schedule jobs across cloud and on-premises compute. Its YAML-based workload model and broad accelerator support make it a strong fit for heterogeneous AI infrastructure, though its default plaintext storage and limited TPU configuration warrant attention.

The server can run on a laptop or another environment with access to the cloud and on-premises clusters in use. Workloads include fleets, development environments, tasks, services, presets and volumes. Users manage resources through the CLI or HTTP API, and Kubernetes is supported. dstack also supports GPU utilization metrics, but not quota controls, which may matter to teams that need enforced resource limits.

Listed platforms are API, Linux, macOS, self-hosted, web and Windows. This is infrastructure software, not a general-purpose desktop or mobile app.

Key features

Provisioning and workload scheduling

dstack combines infrastructure provisioning with job scheduling across GPU clouds, Kubernetes, virtual machines and bare-metal clusters. Auto-scaling, port forwarding and ingress help cover both scheduled work and services. YAML configuration gives teams a consistent way to define varied workloads, but teams that depend on quota enforcement will need another way to manage resource limits.

Accelerators and frameworks

NVIDIA, AMD, TPU and Tenstorrent accelerators are supported out of the box. The task guidance names Accelerate, torchrun, Ray and Spark for distributed workloads, and the maker describes compatibility with hardware, open-source tools and frameworks generally. TPU support has a concrete boundary: only single-host TPUs are supported, up to eight cores per instance.

Services, gateways and interfaces

Services can publish inference endpoints through gateways with HTTPS, custom domains, auto-scaling and rate limits. Users can manage resources through the CLI; the HTTP API also covers functionality beyond the CLI and integrations that need to call the server directly. Documented backends include AWS, Azure, GCP, Kubernetes, multiple GPU cloud providers and remote SSH hosts, with Slurm marked experimental.

Security and support

Server data, including project-scoped secrets managed by project admins, is stored in plaintext by default. Administrators can configure AES-256-GCM encryption for stored data; teams handling sensitive credentials should make that configuration part of deployment rather than assume encryption is on by default. Support channels are GitHub issues and the dstack Discord server.

Pricing

dstack OSS

free. The open-source orchestration stack is self-hosted. It suits teams prepared to operate the server and their infrastructure themselves; there is no paid tier to unlock the core open-source stack in this plan.

dstack Sky GPU Marketplace

Custom pricing. GPU usage is pay-as-you-go, charged per GPU-hour at rates that vary by provider and GPU. Prices are shown in the console before provisioning, and usage is billed against prepaid credits. The plan offers on-demand and spot GPU compute, so costs depend on the resources chosen rather than a fixed subscription price.

dstack Sky does not currently charge for BYOC mode. dstack Factory is the commercial offering, extending the open-source product with advanced multi-tenancy, usage metering, billing automation and optimized inference presets for frontier open models.

Platforms

dstack is listed for API, Linux, macOS, self-hosted, web and Windows. Its hybrid deployment model lets the server run in an environment that can reach the cloud and on-premises clusters being used.

Who it's for

dstack is best suited to AI teams that need one orchestration layer for jobs and services spanning different accelerators and compute environments. Its YAML workloads, API and documented cloud, Kubernetes and SSH backends suit teams integrating varied infrastructure. It is less suitable for organizations that require built-in quota controls, or for TPU users whose workloads require multi-host configurations or more than eight cores per instance.

Pros and cons

• Pros: One workload model covers fleets, development environments, tasks, services, presets and volumes across varied compute environments.
• Pros: Out-of-the-box support for four accelerator families and named distributed frameworks gives AI teams room to match workloads to available hardware.
• Pros: Service gateways include HTTPS, custom domains, auto-scaling and rate limits for publishing inference endpoints.
• Pros: The open-source stack is free, and Sky BYOC mode has no current charge.
• Cons: Plaintext is the default for server data and secrets, so administrators must configure AES-256-GCM encryption when stored-data protection is required.
• Cons: There are no quota controls, limiting its fit for teams that need the orchestrator to enforce resource allocations.
• Cons: TPU support stops at single-host instances with a maximum of eight cores.
• Cons: GPU Marketplace prices vary by provider and GPU, and usage requires prepaid credits, so spend depends on the selected compute.

Alternatives

For a broader comparison, see GPU Cluster Management Software.

• HTCondor is another free option for readers comparing cluster-management software across the same listed platforms.
• ClearML is worth considering if a freemium option and a self-hosted version described as 100% open source on GitHub better match your requirements.
• GPUStack offers a free, open-source GPU cluster manager for readers focused on that description.
• Backend.AI may suit home users who want an open-source option for their own hardware, or readers seeking a free try-out plan.
• Koordinator is a free alternative for readers looking for API, Linux and self-hosted platform support.
• NVIDIA ShadowPlay is a free Windows option, rather than a cross-environment orchestration layer.
• Kueue is a free open-source Kubernetes project for readers whose scope is Kubernetes.
• OpenPBS offers a free AGPL 3.0 open-source edition, with community forum support that carries no guarantees.

Verdict

Choose dstack if you need open-source orchestration for AI workloads across cloud and on-premises compute, with support for multiple accelerator families and both jobs and services. Its strongest case is the breadth of infrastructure and workload types in one YAML-driven stack. Look elsewhere if built-in quota controls, encrypted-by-default storage or multi-host TPU support is essential.