Skip to main content

Promptfoo

Category: AI Developer Tools

An open-source platform for AI security, enabling developers to test, evaluate, and secure large language model (LLM) applications through red-teaming and performance benchmarking. Promptfoo was founded in 2024. The company is led by Ian Webster. Based in San Francisco, California. Team size: 11-50. Total funding raised: $23M. Latest round: Acquired. Key investors include Insight Partners, Andreessen Horowitz.

Founded
2024
Headquarters
San Francisco, California
Team size
11-50
Total funding
$23M

Value proposition

To eliminate risks in AI applications by providing tools for red-teaming, evaluation, and security, thereby helping to find and fix vulnerabilities, maximize output quality, and prevent regressions.

Products and solutions

Open-source CLI tool for LLM evaluation, AI red-teaming and vulnerability scanning, Performance and quality testing for prompts, agents, and RAGs, Comparison of different LLM providers (e.g., GPT, Claude, Gemini, Llama)

Unique value

Promptfoo's uniqueness lies in its developer-first, open-source approach to LLM security and evaluation. It combines automated red-teaming with a test-driven development (TDD) framework, allowing for systematic and continuous testing of LLM applications. This approach is designed to be fast and developer-friendly, with features like live reloads and caching.

Target customer

Developers, AI engineers, and teams building applications using Large Language Models (LLMs).

Industries served

Software Development, Artificial Intelligence, Cybersecurity

Technology advantage

The primary advantage is its strong open-source community, with over 100,000 developers using the tool. This provides a significant distribution channel and a continuous feedback loop for product improvement. The founding team's experience in scaling AI products at major tech companies like Discord and GitHub provides a strong technical and business advantage.

How they differentiate

Promptfoo differentiates itself by being an open-source, developer-first platform that combines LLM evaluation with AI red-teaming and security vulnerability scanning. It is designed for easy integration into CI/CD pipelines and focuses on providing a fast, declarative, and locally runnable testing framework.

Main competitors

Langfuse, Arize AI, Weights & Biases

Key partnerships

Portkey

Notable customers

156 of the Fortune 500 companies (25%+ of Fortune 500), Top retailers, telecoms, financial services, healthcare, and insurance enterprises

Major milestones

Secured $23M total funding ($5M seed led by a16z + $18.4M Series A led by Insight Partners)., Grew to 350,000+ developers and 130,000 monthly active users., Trusted by 156+ Fortune 500 companies (25%+ of Fortune 500)., Acquired by OpenAI on March 9, 2026 to integrate into OpenAI Frontier agent platform.

Growth metrics

More than 350,000 developers have used Promptfoo, with 130,000 active monthly users. Trusted by 156 of the Fortune 500 (25%+).

Market positioning

Positioned as the open-source standard for LLM evaluation and security, targeting developers and teams who need to test and secure their AI applications systematically. It bridges the gap between development and security in AI.

Geographic focus

Primarily global, with a strong focus on the North American market, particularly the US, given its location in San Mateo, California, and its key investors.

Patents and IP

No public patents were found.

About Ian Webster

Led LLM engineering and developer platform teams at Discord, scaling AI products to 200M users. Co-founder of Kickback (YC-backed).

Latest news about Promptfoo

Also named in

More AI Developer Tools companies

Official website:

Promptfoo - AI Startup Profile | AI Market Watch