Kiln

Kiln is an AI workbench for teams developing, evaluating, and optimizing AI systems. It fits engineers, data scientists, product managers, QA specialists, and subject-matter experts who need visibility into agent behavior and reproducible evaluation work. The free Individual plan is a practical entry point, while model limits, collaboration support, and enterprise controls determine whether a paid tier is necessary.

Overview

Kiln combines evaluation, synthetic-data generation, agent development, fine-tuning, and optimization in one workspace. Describe an evaluation goal and it can generate prompts and synthetic datasets, then benchmark automated judges against human ratings. That emphasis suits teams building a repeatable assessment process more than users seeking a broad spreadsheet or dashboard tool.

Prompts, outputs, ratings, and fine-tuning jobs are kept as JSON files on the user's filesystem. Calls to providers go directly from the user's machine to the selected service, and Kiln says the application can run fully offline with Ollama, vLLM, Llama.cpp, or another OpenAI-compatible local server. This local-first design is valuable for controlled environments, although desktop UI analytics through PostHog is a consideration for organizations with strict telemetry policies.

Key features

Evaluation and agent hierarchies

Goal-based generation gives teams a starting set of evaluation prompts and synthetic examples without hand-authoring every case. Comparing judge results with human ratings helps reveal whether automated scoring reflects real review. Kiln also lets teams compose agents and subagents, inspect tool calls, and evaluate each level of the hierarchy. That granularity is particularly useful when a final answer looks acceptable but an intermediate tool call or subagent is unreliable.

Providers, MCP, and optimization

MCP server support and connections for AWS Bedrock, Azure OpenAI, Vertex AI, OpenAI, Anthropic, Fireworks, Gemini, Ollama, and OpenAI-compatible endpoints reduce the need to redesign workflows around one vendor. Automatic agent optimization is reserved for Team, so Individual users can build and evaluate but do not receive that paid optimization capability.

Git collaboration and data control

Datasets, traces, and evals sync automatically to Git, making experiment history easier to review and share. Enterprise services name GitHub Enterprise, GitLab, Bitbucket, and Azure DevOps. CSV import, natural-language queries, and team collaboration broaden the workflow, while filesystem JSON storage keeps core artifacts inspectable and portable.

Pricing

Individual — free. This tier includes the open-source MIT-licensed Python library, source-available desktop app, local datasets with Git sync, Kiln Pro standard models with rate limits, and community support. It suits solo practitioners and teams proving a workflow, but rate-limited standard models may constrain frequent evaluation or larger experiments.

Team — custom pricing, billed “Request access.” It adds enhanced models with higher limits, automatic agent optimization, priority access to new features, and email support, while retaining everything in Individual. Teams with active iteration and a need for faster model access are the natural audience; the lack of a published price means budgeting requires a sales conversation.

Enterprise — custom pricing, billed “Custom.” Everything in Team is joined by SSO/SAML, annual contracts and procurement support, SLA and priority support, a dedicated solutions engineer, and custom onboarding and training. SSO, VPC deployment, and telemetry controls can also be included in enterprise services, making this the appropriate tier for governed organizational rollouts.

Platforms

Kiln supports API, Linux, macOS, Windows, and self-hosted deployments. Offline operation with local OpenAI-compatible servers gives self-hosting a substantive role rather than merely a packaging option.

Who it's for

Choose Kiln when an AI project needs structured evals, inspectable agent hierarchies, Git-based collaboration, or local data handling. It is less suitable as a general data-analysis front end, and organizations unwilling to use desktop UI analytics should account for the PostHog collection policy; the Python library itself sends no analytics.

Pros and cons

Pros

• Local-first storage: prompts, outputs, ratings, and fine-tuning jobs remain JSON files on the user's filesystem.
• Deep agent inspection: tool calls and each hierarchy level can be evaluated, not just the final response.
• Broad provider support: major hosted services, Ollama, MCP servers, and compatible endpoints can share one workflow.
• Free starting point: the Individual plan includes the Python library, desktop app, and Git-synced datasets without a subscription fee.

Cons

• Free model limits: Individual uses rate-limited standard models, which can slow intensive evaluation work.
• Paid collaboration upgrades: enhanced limits, automatic optimization, and email support begin on Team at custom pricing.
• Desktop telemetry: the desktop app collects UI analytics through PostHog, requiring enterprise controls if that is unacceptable.

Alternatives

For spreadsheet-oriented and general AI data analysis, consider the AI Data Analysis Tools directory. Quadratic is a better fit for users who want a freemium spreadsheet-style workspace across web, desktop, mobile, and self-hosted platforms, with limited AI usage on its Personal plan. Julius AI suits people seeking web and mobile data analysis with free daily credits and a Google Drive connector. Byaan is worth choosing for a free macOS app or a self-hosted, MIT/AGPL-oriented team deployment. Chartt fits lightweight web analysis with a 100-credit free tier and file and export caps. ExcelGPT is aimed at AI-assisted formulas, charts, and data analysis through web or API access. MySheetAI is a simpler web option with a free plan, five monthly questions, and small uploads. Loam is designed for grounded questions over uploaded files and connected AI tools. PerSight targets desktop users querying databases with capped question and connection allowances.

Verdict

Kiln is a strong choice for teams that need one, Git-synchronized workspace for evaluation, agent hierarchies, and optimization while keeping core data on local machines. Its free Individual plan lowers the barrier to adoption, and its provider breadth and offline options support varied deployments. Look elsewhere if rate-limited standard models are insufficient, if you need transparent published Team pricing, or if desktop UI analytics conflict with your governance requirements.