KServe

What it does

Deployment API

KServe provides a Kubernetes custom resource definition that handles autoscaling, networking, health checks, and server configuration for model deployments.checked 5 Oct 2026

Frameworks

Documented serving runtimes include TensorFlow Serving, Triton, Hugging Face, LightGBM, XGBoost, AutoGluon, PMML, scikit-learn, and PaddlePaddle.checked 5 Oct 2026

Installation

KServe documents installation with Kustomize components or Helm charts and supports multiple deployment modes.checked 5 Oct 2026

Install options

KServe can be installed with Kustomize components or Helm charts, and can serve predictive models alone or alongside separate LLM and model-caching components.checked 5 Oct 2026

Integrations

Documented integrations include Istio, Knative Serving, Envoy Gateway, Envoy AI Gateway, and the vLLM runtime.checked 5 Oct 2026

Known limitation

The optional LocalModel component currently supports InferenceService workloads; support for LLMInferenceService is planned for future releases.checked 5 Oct 2026

LLM API

The Hugging Face runtime uses vLLM and provides OpenAI-compatible API endpoints for generative tasks including text generation, embeddings, and reranking.checked 5 Oct 2026

LLM features

Its generative AI features include vLLM and llm-d backends, OpenAI protocol compatibility, GPU acceleration, model caching, KV cache offloading, and request-based autoscaling.checked 5 Oct 2026

Predictive AI features

It supports TensorFlow, PyTorch, scikit-learn, XGBoost, and ONNX, with routing, InferenceGraph deployments, explainability, monitoring, and scale-to-zero.checked 5 Oct 2026

Predictive model features

Predictive serving supports canary rollouts, inference pipelines and ensembles, scale-to-zero, explainability, and monitoring features such as payload logging and drift detection.checked 5 Oct 2026

Project affiliation

The site says KServe has joined CNCF and identifies it as a Series of LF Projects, LLC project.checked 5 Oct 2026

Protocols

KServe documents gRPC support for its V1 and V2 inference protocols, including binary tensor data and streaming responses.checked 5 Oct 2026

Purpose

KServe is a Kubernetes platform for serving generative and predictive AI models across multiple frameworks.checked 5 Oct 2026

Requirements and limits

KServe runs on Kubernetes; its optional LocalModel component currently supports InferenceService workloads, while support for LLMInferenceService is described as planned for a future release.checked 5 Oct 2026

Rollouts

KServe supports canary rollouts and A/B testing.checked 5 Oct 2026

Security

The Istio integration documents mutual TLS, automatic certificate management, and fine-grained access-control and authorization policies.checked 5 Oct 2026

Standard API

It provides a Kubernetes Custom Resource Definition that abstracts autoscaling, networking, health checks, and server configuration.checked 5 Oct 2026

Support

Users can get help through the documentation, the #kserve Slack channel, GitHub Discussions, and community meetings.checked 5 Oct 2026

What it does

KServe is an open-source platform for serving generative and predictive AI models on Kubernetes.checked 5 Oct 2026

Who it is for

The administrator guide covers production management of both predictive and generative inference workloads on Kubernetes.checked 5 Oct 2026