Promptfoo
Promptfoo is an open-source CLI tool and library for testing and evaluating LLM prompt quality.
Free + paidWebCLIAPIOpen sourceKoreanLLM-basedMultimodal
Visit websitepromptfoo.dev
Compare with book-to-skillExplore Promptfoo alternativesOverview
Promptfoo is an open-source CLI tool and library for testing and evaluating LLM prompt quality. It enables systematic benchmarking of prompts against multiple models and test cases, featuring side-by-side comparisons, red-teaming, and CI/CD integration for automated regression testing.
Key features
- Prompt matrix testing
- Side-by-side comparison view
- AI-based automated grading
- Red-teaming security tests
- CI/CD pipeline integration
- Local data processing
- Custom evaluation scripts
Pricing
Use cases
- Regression testing for prompt changes
- Model selection and comparison
- LLM security vulnerability scanning
- Large-scale dataset benchmarking
Who it is for
Prompt EngineersAI Product ManagersQA Engineers
Integrations
GitHub ActionsCircleCISlackHugging Face
How we verified this
Company, pricing, and feature details come from the primary sources below and our latest verification pass. When sources disagree, the official source and the most recent check win.
Last verified 08/30/2026Verified sources: 1
Alternatives
Tools you can use instead
book-to-skill
Converts technical books and document collections into structured, on-demand skills for AI coding agents.
★ 28.8KOpen source
Developer Tools
GPT-6 Astra
OpenAI frontier model for difficult end-to-end reasoning and professional work.
API
Developer Tools
