About Promptfoo
Promptfoo builds open-source tools for secure, reliable AI applications. Promptfoo is based in San Francisco, California, and is now part of OpenAI. Investors and operators backed Promptfoo early, and Promptfoo builds open-source, application-focused AI security.
Promptfoo's red teaming simulates real users to uncover application-specific vulnerabilities. Promptfoo creates thousands of context-aware attacks tailored to the customer's application. Promptfoo says its tests understand business logic, RAG, agents and integrations, and cover 50+ vulnerability types from injection to jailbreaks.[4]
Promptfoo connects to AI apps, agents, and workflows. Promptfoo provides remediation guidance directly in pull requests and developer workflows. Promptfoo lets teams track remediation progress and get suggested steps for each issue.[4][5]
The Community version includes all core features for local testing, evaluation, and vulnerability scanning. The open-source Community version includes up to 10k probes per month at no charge. Enterprise customers can purchase additional probes for larger scale testing. Enterprise pricing is customized based on a team's size and needs.[5]
Promptfoo lets customers manage LLM applications with teams-based access control and SSO, granular permission profiles, and customizable API access. Promptfoo provides real-time alerts and automated evaluations to show LLM security status across all projects. Promptfoo supports customizable plugin collections, scan configurations, and target settings that can be shared among colleagues.[5]
Promptfoo's LLM evaluation can be integrated into a CircleCI pipeline. Promptfoo can be integrated with test frameworks like Jest and Vitest to evaluate prompts as part of existing testing and CI workflows. Promptfoo's application-level jailbreak creation can be integrated with Burp Suite's Intruder feature for security testing of LLM-powered applications. Promptfoo can import eval test cases directly from Microsoft SharePoint CSV files using certificate-based authentication with Azure AD. Promptfoo configurations can reference prompts managed in Langfuse.[6][7][8][9][10]
Promptfoo offers real estate plugins that test every AI touchpoint, from property search to lending to advertising, across all 7 federally protected classes. Promptfoo tests AVM and appraisal AI for demographic correlation in valuations. Promptfoo's real estate testing covers home search assistants, neighborhood recommendation engines, property matching tools, and school district comparison features. Promptfoo's real estate testing can run entirely within a customer's infrastructure, with no applicant or transaction data leaving the environment. Promptfoo integrates with CI/CD pipelines to catch fair housing regressions before deployment. Promptfoo generates structured reports that map directly to Fair Housing Act and ECOA requirements.[11]
Promptfoo says its contributors are from companies like OpenAI, Google, Microsoft, and Amazon.[2][4][5][6][11]
Promptfoo on the record