KAI Scheduler
KAI Scheduler is a free, open-source Kubernetes scheduler for allocating GPUs to AI workloads such as data processing, training, and inference.
What KAI Scheduler is
KAI Scheduler is a free, open-source Kubernetes scheduler for allocating GPUs to AI workloads such as data processing, training, and inference. It supports interactive notebooks through multi-node distributed training in one cluster and is designed for large GPU clusters and high-throughput workloads. Hierarchical queue trees provide quotas, limits, priorities, and over-quota borrowing between teams. Gang scheduling places distributed-workload pods together, while elastic workloads adjust replica counts within configured minimum and maximum values as capacity changes. GPU sharing supports time slicing, MPS, and MIG; the default NonMemoryEnforced mode does not enforce runtime GPU-memory isolation. Scheduling also includes topology-aware placement, time-based fairshare, and priority-based preemption. KAI documents native Ray integration on Kubernetes and Grove integration. It runs on cloud or on-premises infrastructure, including Karpenter environments. Deployment uses a production Helm chart, source build, or ArgoCD, with Kubernetes, Helm, and NVIDIA GPU Operator prerequisites.
Good
- Open-source Apache-2.0 project
- Hierarchical quotas and resource borrowing
- Gang and elastic scheduling
- Time slicing, MPS, and MIG sharing
- Cloud and on-premises support
Watch out
- Requires a running Kubernetes cluster and Helm
- NVIDIA GPU Operator required for GPU requests
- Default sharing mode lacks runtime memory isolation
Who it suits
KAI Scheduler suits teams running GPU workloads on Kubernetes, from notebooks to distributed training. It fits cloud and on-premises environments that need queue controls, sharing, and coordinated scheduling.
Verdict
KAI Scheduler combines GPU allocation, queue governance, coordinated placement, and sharing for Kubernetes AI workloads. Its prerequisites and default GPU-sharing limitation should be considered before deployment.
The facts
Each one read from KAI Scheduler's own pages, with the date it was checked.
What it does
- Cloud and on-premises
- The project describes compatibility with dynamic cloud infrastructure, including Karpenter autoscalers, and static on-premises deployments.checked 7 Oct 2026
- Deployment
- The project provides production Helm installation, source build, and ArgoCD GitOps installation options for Kubernetes.checked 7 Oct 2026
- Elastic workloads
- Elastic workloads can grow or shrink within configured minimum and maximum replica counts as capacity changes.checked 5 Oct 2026
- Fairness
- Time-based fairshare tracks historical GPU usage over a configurable window to distribute over-quota resources fairly over time.checked 7 Oct 2026
- Gang and elastic scheduling
- Gang scheduling places distributed workloads all at once, while elastic workloads can grow or shrink within minimum and maximum replica counts as capacity allows.checked 7 Oct 2026
- Gang scheduling
- Gang scheduling places all pods in a distributed workload together or waits until resources are available.checked 5 Oct 2026
- GPU sharing
- GPU sharing supports time slicing, MPS, and MIG so workloads can share GPU devices.checked 7 Oct 2026
- GPU sharing limitation
- The GPU-sharing documentation says the default NonMemoryEnforced mode schedules fractional GPU workloads without KAI enforcing runtime GPU memory isolation.checked 7 Oct 2026
- Hierarchical queues
- Multi-level queue trees support quotas, limits, and over-quota borrowing across teams.checked 7 Oct 2026
- Integrations
- The project states it is natively integrated with KubeRay for Ray workloads on Kubernetes and describes integrations with Grove and Dynamo for complex multi-component workloads.checked 7 Oct 2026
- Intended workloads
- It supports workloads ranging from interactive notebooks to multi-node distributed training in the same cluster.checked 7 Oct 2026
- License
- The public GitHub repository identifies the project as open source and lists an Apache-2.0 license.checked 7 Oct 2026
- License and project
- KAI Scheduler is an open-source Apache-2.0 project and a CNCF sandbox project.checked 5 Oct 2026
- Prerequisite
- The installation guide says NVIDIA GPU-Operator is required to schedule workloads requesting GPU resources.checked 5 Oct 2026
- Prerequisites
- The installation guide lists a running Kubernetes cluster and Helm CLI, and requires NVIDIA GPU Operator for workloads requesting GPU resources.checked 7 Oct 2026
- Priority and preemption
- Per-queue and per-workload priorities let critical jobs preempt while non-critical jobs back off.checked 7 Oct 2026
- Purpose
- KAI Scheduler is a Kubernetes-native scheduler for AI workloads that optimizes GPU allocation across data processing, training, and inference while keeping resource shares fair across teams.checked 7 Oct 2026
- Queues
- Hierarchical queues support quotas, limits, priorities, and borrowing resources across teams.checked 5 Oct 2026
- Scale
- The project says it is designed and continuously tested for large GPU clusters with thousands of nodes and high-throughput workloads.checked 5 Oct 2026
- Security
- Security vulnerabilities should be reported through GitHub Security Advisories rather than public GitHub issues.checked 5 Oct 2026
- Support
- The project directs users to GitHub issues for bugs, feature requests, or technical help, and also lists CNCF Slack, community calls, and a mailing list.checked 7 Oct 2026
- Support channel
- The project directs users to open GitHub issues for bugs, feature suggestions, or technical help.checked 5 Oct 2026
- Supported infrastructure
- The project states it supports cloud and on-premise deployments, including cloud auto-scalers such as Karpenter.checked 5 Oct 2026
- Topology-aware placement
- KAI supports topology-aware scheduling, including hierarchical topology-aware scheduling for hierarchical PodGroups.checked 7 Oct 2026
