PromptWizard

B tier on AI Prompt GeneratorsScore 7.3 · #2 of 29

Android app: Not listed
Free plan: Yes
Runs on: api, Linux, Mac, self-hosted, Windows

microsoft.github.io

Summary

PromptWizard is a free, MIT-licensed open-source framework for optimizing prompts and examples through an iterative feedback process. It generates prompt variations, scores them, critiques results, and refines instructions; it can also optimize in-context examples and synthesize task-relevant ones. The project describes workflows with no examples, synthetic examples, or training data. Supported datasets listed include GSM8k, SVAMP, AQUARAT, and Instruction Induction (BBII). Custom datasets use JSONL samples with question and answer fields, and require task-specific answer extraction, evaluation functions, configuration, and data files. PromptWizard can generate chain-of-thought reasoning for examples, with an option to disable it to reduce prompt size. Installation is through its GitHub repository as a Python package in development mode, with instructions for Windows, macOS, and Linux. Model access setup supports OpenAI API keys and Azure OpenAI endpoints, so credentials are needed to access those services. The README reports average optimization times of around 20–30 minutes in its listed-dataset experiments, with timing depending on the dataset, and notes that human supervision can help tune generated prompts.

Who it is for

It suits developers who want to optimize prompts and examples in a Python-based workflow, including with custom datasets. Users should be prepared to configure evaluation for custom data and provide model API credentials.

What is good

Optimizes prompt instructions and in-context examples.
Supports synthetic examples and training-data workflows.
Can disable reasoning generation to reduce prompt size.
MIT-licensed and free.

What to know first

Model access requires OpenAI or Azure OpenAI credentials.
Custom datasets need task-specific evaluation functions.
Generated prompts may benefit from human supervision.

Verdict

PromptWizard offers an iterative, code-based approach to prompt and example optimization, with support for several dataset workflows. Custom data requires additional evaluation setup, and model access needs API credentials.

PromptWizard plans and pricing

All plans

MIT-licensed open-source software Free No paid plans listed; API access requires OpenAI or Azure OpenAI credentials github.com · 3 Oct 2026

Compared on AI prompt generators

Free plan: Yesmicrosoft.github.io
Model support: multiplemicrosoft.github.io
Optimization mode: automatedmicrosoft.github.io
Prompt testing: Yesmicrosoft.github.io
API access: Yesmicrosoft.github.io

Facts

Product: PromptWizard is an open source framework for automated prompt and example optimization using a feedback-driven critique and synthesis process.microsoft.github.io · 2 Oct 2026
Prompt optimization: It iteratively generates, scores, critiques, and refines prompt instructions.github.com · 2 Oct 2026
Example optimization: It optimizes in-context examples alongside prompt instructions and can synthesize diverse, task-relevant examples.github.com · 2 Oct 2026
Reasoning: It can generate chain-of-thought reasoning for in-context examples, and this option can be disabled to reduce prompt length or token count.github.com · 2 Oct 2026
Use cases: The repository describes use with no examples, synthetic examples, or training data, including custom datasets.github.com · 2 Oct 2026
Model API integrations: The setup instructions support OpenAI API keys and Azure OpenAI endpoints for LLM access.github.com · 2 Oct 2026
Installation: The project is installed from its GitHub repository as a Python package in development mode, with setup instructions for Windows, macOS, and Linux.github.com · 2 Oct 2026
Dataset format: Custom datasets are expected as JSONL files with question and answer fields in each sample.github.com · 2 Oct 2026
Supported datasets: The README lists GSM8k, SVAMP, AQUARAT, and Instruction Induction (BBII) as supported datasets.github.com · 2 Oct 2026
Optimization time: The README says optimization took around 20–30 minutes on average in its experiments on the listed datasets, with time depending on the dataset.github.com · 2 Oct 2026
Customization: Custom datasets require dataset-specific answer extraction and evaluation functions, along with configuration and data files.github.com · 2 Oct 2026
Human review: The README says generated prompts are usually detailed and that user supervision can help tune them for the task.github.com · 2 Oct 2026
License: The repository identifies the project as MIT licensed.github.com · 2 Oct 2026
Security: The repository links a security policy, but the pages opened do not state specific security controls or compliance certifications.github.com · 2 Oct 2026
Maker: The project page names Microsoft Research and lists Eshaan Agarwal, Joykirat Singh, Vivek Dani, Raghav Magazine, Tanuja Ganu, and Akshay Nambi as authors.microsoft.github.io · 2 Oct 2026
Purpose: PromptWizard is an open-source framework for automated, task-aware prompt and example optimization.microsoft.github.io · 3 Oct 2026
Prompt refinement: It generates prompt variations, scores them, critiques their successes and failures, and refines prompts over iterations.github.com · 3 Oct 2026
Reasoning chains: It can generate chain-of-thought reasoning for in-context examples, and its configuration can turn reasoning generation off to reduce prompt size.github.com · 3 Oct 2026
Usage scenarios: The README describes optimizing prompts without examples, generating synthetic examples, and optimizing prompts with training data.github.com · 3 Oct 2026
Dataset support: The README lists GSM8k, SVAMP, AQUARAT, and Instruction Induction (BBII) as supported training datasets.github.com · 3 Oct 2026
Custom data requirements: Custom datasets are expected in JSONL format with question and answer fields for each sample.github.com · 3 Oct 2026
Installation platforms: Installation instructions cover virtual environments on Windows, macOS, and Linux and package installation in development mode.github.com · 3 Oct 2026
Security reporting: The repository security policy asks people to report vulnerabilities to the Microsoft Security Response Center rather than through public GitHub issues.github.com · 3 Oct 2026
Human supervision: The README says generated prompts are usually detailed and that user supervision can help tune them further for a task.github.com · 3 Oct 2026