Promptfoo
Category: AI Developer Tools
An open-source platform for AI security, enabling developers to test, evaluate, and secure large language model (LLM) applications through red-teaming and performance benchmarking. Promptfoo was founded in 2024. The company is led by Ian Webster. Based in San Francisco, California. Team size: 1-10. Total funding raised: $18.4M. Latest round: Series A. Key investors include ["Insight Partners","Andreessen Horowitz"].
- Founded
- 2024
- Headquarters
- San Francisco, California
- Team size
- 1-10
- Total funding
- $18.4M
Value proposition
To eliminate risks in AI applications by providing tools for red-teaming, evaluation, and security, thereby helping to find and fix vulnerabilities, maximize output quality, and prevent regressions.
Products and solutions
["Open-source CLI tool for LLM evaluation","AI red-teaming and vulnerability scanning","Performance and quality testing for prompts, agents, and RAGs","Comparison of different LLM providers (e.g., GPT, Claude, Gemini, Llama)"]
Unique value
Promptfoo's uniqueness lies in its developer-first, open-source approach to LLM security and evaluation. It combines automated red-teaming with a test-driven development (TDD) framework, allowing for systematic and continuous testing of LLM applications. This approach is designed to be fast and developer-friendly, with features like live reloads and caching.
Target customer
Developers, AI engineers, and teams building applications using Large Language Models (LLMs).
Industries served
["Software Development","Artificial Intelligence","Cybersecurity"]
Technology advantage
The primary advantage is its strong open-source community, with over 100,000 developers using the tool. This provides a significant distribution channel and a continuous feedback loop for product improvement. The founding team's experience in scaling AI products at major tech companies like Discord and GitHub provides a strong technical and business advantage.
How they differentiate
Promptfoo differentiates itself by being an open-source, developer-first platform that combines LLM evaluation with AI red-teaming and security vulnerability scanning. It is designed for easy integration into CI/CD pipelines and focuses on providing a fast, declarative, and locally runnable testing framework.
Main competitors
["Langfuse","Arize AI","Weights & Biases"]
Key partnerships
["Portkey"]
Notable customers
["While specific enterprise customer names are not widely publicized, its large open-source user base suggests adoption across various companies."]
Major milestones
["Secured $18.4M in Series A funding led by Insight Partners and a16z.","Established itself as a popular open-source tool with over 100,000 developers.","Launched a comprehensive platform for AI security, including evaluation, red-teaming, and vulnerability scanning."]
Growth metrics
The open-source tool is used by over 100,000 developers.
Market positioning
Positioned as the open-source standard for LLM evaluation and security, targeting developers and teams who need to test and secure their AI applications systematically. It bridges the gap between development and security in AI.
Geographic focus
Primarily global, with a strong focus on the North American market, particularly the US, given its location in San Mateo, California, and its key investors.
Patents and IP
No public patents were found.
About Ian Webster
Led LLM engineering and developer platform teams at Discord, scaling AI products to 200M users. Co-founder of Kickback (YC-backed).
Official website: https://www.promptfoo.dev/