CISO Tribune

Vendor comparison · 2 companies

Fiddler AI vs Promptfoo

Fiddler AI sells the AI Control Plane, a platform for first-party and third-party agents from the creation layer to production. Promptfoo builds open-source tools for secure, reliable AI applications. CISO Tribune files both under AI model and agent security and AI firewalls and guardrails. This page sets the two records side by side: what each sells, where it is based, who owns and funds it, the customers it names and its recent news, each fact numbered to its source.

Change the companies

Up to four companies side by side.

  • Fiddler AI
  • Promptfoo
Fiddler AI vs Promptfoo: Products, Customers and Funding Compared
Fiddler AIFiddler AI sells the AI Control Plane, a platform for first-party and third-party agents from the creation layer to production.fiddler.aiPromptfooPromptfoo builds open-source tools for secure, reliable AI applications.promptfoo.dev
What it sells
  • Agentic Observability[2][4]Monitors every layer of the agentic hierarchy, from application health to agent interactions to individual spans.
  • ML Observability[2][5]Detects data and concept drift in predictive models and provides root cause analysis and fairness metrics.
  • Fiddler Centor Models[2][3][6]Power guardrails and evaluations in-environment, with no external LLM API calls.
  • Agent Registry[7]Lists every agent application in Fiddler, including first-party, third-party and coding agents.
  • Red Teaming[2][4][5]Simulates real users to uncover application-specific vulnerabilities.
  • Evaluations[2][5]LLM evaluation of prompts and models that can run inside CI workflows.
Categories
Headquarters
Not stated on its pagesSan Francisco, United States[1][3]
Founded
2018[1]2024[1][12]
Ownership and funding
Privately held[12]Owned by OpenAI[12]
Named customers
Not stated on its pagesNot stated on its pages
Recent news
  • January 27, 2026Fiddler AI announced a $30 million Series C funding round led by RPS Ventures.[1]
  • March 9, 2026OpenAI agreed to buy Promptfoo, a startup helping businesses find and fix security issues in AI models during development.[13]
  • March 9, 2026OpenAI announced it has acquired Promptfoo, an AI security startup founded in 2024 to protect LLMs from adversarial attacks.[12]
G2 and Gartner Peer Insights
  • G2: listed, seen November 10, 2026[12]
  • Gartner Peer Insights: Not stated on its pages
  • G2: Not stated on its pages
  • Gartner Peer Insights: Not stated on its pages
Who it fits
From its own pages, Fiddler AI sells mainly to organizations deploying AI agents and predictive models, across all industries and in regulated industries such as financial services, healthcare, insurance and government. The typical users are software and AI engineers, data scientists, platform and DevOps teams, and security and compliance teams. Its pages address insurers, healthcare organizations and financial institutions deploying AI across underwriting, fraud and trading. Deployment options on the Enterprise plan are SaaS, VPC or on-premises.Not written yet
Last verified
November 6, 2026November 4, 2026

Where they overlap

  • AI model and agent security is the primary category of both Fiddler AI and Promptfoo; both are also filed under AI firewalls and guardrails.

Where they differ, field by field

  • Fiddler AI is also filed under AI governance. Promptfoo is also filed under Static application security testing and Breach simulation and automated pen-testing.
  • Fiddler AI lists Agentic Observability, ML Observability, Fiddler Centor Models and Agent Registry. Promptfoo lists Red Teaming and Evaluations.
  • Promptfoo is headquartered in San Francisco, United States.
  • Fiddler AI was founded in 2018. Promptfoo was founded in 2024.
  • Fiddler AI is privately held. Promptfoo is owned by OpenAI.

How they differ

CISO Tribune files both Fiddler AI and Promptfoo under AI model and agent security as the primary category, and both also appear under AI firewalls and guardrails. Fiddler AI sells Agentic Observability, ML Observability, Fiddler Centor Models and Agent Registry, and describes an AI Control Plane for first-party and third-party agents. Promptfoo sells Red Teaming, which simulates real users to uncover application-specific vulnerabilities, and Evaluations, which can run inside CI workflows. Fiddler AI also files under AI governance, while CISO Tribune files Promptfoo additionally under Static application security testing and Breach simulation and automated pen-testing.

Fiddler AI was founded in 2018 and is privately held. On January 27, 2026, it announced a $30 million Series C funding round led by RPS Ventures. Promptfoo was founded in 2024, is headquartered in San Francisco, United States, and is owned by OpenAI. OpenAI announced on March 9, 2026 that it had acquired Promptfoo.

Fiddler AI says it sells to organizations deploying agents and predictive models, including those in financial services, healthcare, insurance and government, and it states that Fortune 100 organizations use it. Fiddler AI can be deployed as SaaS, VPC or AWS GovCloud, or in any secure cloud environment or air-gapped. Promptfoo offers an open-source Community version with up to 10k probes per month at no charge, and it offers real estate testing plugins tied to Fair Housing Act and ECOA requirements.

What it sells: Fiddler AI

What Agentic Observability does[2][4]
Fiddler Agentic Observability monitors every layer of the agentic hierarchy from application health to agent interactions to individual spans. Fiddler gives aggregated and granular visibility from application to session to agent to trace to span.
What ML Observability does[2][5]
Fiddler ML Observability provides root cause analysis and fairness metrics such as disparate impact, demographic parity and equal opportunity. Fiddler tracks performance, drift, data integrity and model bias to observe and improve model health.
What Fiddler Centor Models does[2][3][6]
Fiddler Centor Models power guardrails and evaluations in-environment, with no external LLM API calls. Customers can choose Fiddler Centor Models that run in their environment, custom evaluators for domain-specific logic, or external judges for advanced reasoning.
What Agent Registry does[7]
Agent Registry lists every agent application in Fiddler, including first-party, third-party and coding agents. Fiddler records evaluation and policy outcomes with full traceability and generates audit evidence aligned with frameworks such as GDPR, HIPAA, NAIC, SR 11-7, NIST AI RMF, ISO/IEC 42001 and the EU AI Act.

What it sells: Promptfoo

What Red Teaming does[2][4][5]
Promptfoo creates thousands of context-aware attacks tailored to the customer's application. The open-source Community version includes up to 10k probes per month at no charge.
What Evaluations does[2][5]
Promptfoo's LLM evaluation can be integrated into a CircleCI pipeline. Promptfoo can be integrated with test frameworks like Jest and Vitest to evaluate prompts as part of existing testing and CI workflows. Promptfoo provides real-time alerts and automated evaluations to show LLM security status across all projects.

Questions about Fiddler AI and Promptfoo

What do Fiddler AI and Promptfoo sell?
Fiddler AI sells Agentic Observability, ML Observability, Fiddler Centor Models and Agent Registry. Promptfoo sells Red Teaming and Evaluations.
Are Fiddler AI and Promptfoo in the same category?
Yes. AI model and agent security is the primary category of both on CISO Tribune.
Where are Fiddler AI and Promptfoo headquartered?
Promptfoo is headquartered in San Francisco, United States.
When were Fiddler AI and Promptfoo founded?
Fiddler AI was founded in 2018. Promptfoo was founded in 2024.
Who owns Fiddler AI and Promptfoo?
Fiddler AI is privately held. Promptfoo is owned by OpenAI.
Who are Fiddler AI and Promptfoo sold to?
From its own pages, Fiddler AI sells mainly to organizations deploying AI agents and predictive models, across all industries and in regulated industries such as financial services, healthcare, insurance and government.
What have Fiddler AI and Promptfoo announced recently?
January 27, 2026: Fiddler AI announced a $30 million Series C funding round led by RPS Ventures. March 9, 2026: OpenAI agreed to buy Promptfoo, a startup helping businesses find and fix security issues in AI models during development.

About Fiddler AI

Fiddler AI sells the AI Control Plane, a platform for first-party and third-party agents from the creation layer to production. Fiddler combines continuous evaluation, monitoring, enforceable policy and auditable governance for enterprises. Fiddler Centor Models power guardrails and evaluations in-environment, with no external LLM API calls.[2][3]

Fiddler Agentic Observability monitors every layer of the agentic hierarchy from application health to agent interactions to individual spans. Fiddler ML Observability provides root cause analysis and fairness metrics such as disparate impact, demographic parity and equal opportunity. Fiddler Guardrails enforce policy inline, at the agent's request and response path, in under 80ms. Fiddler detects and redacts PII, PHI and secrets on the same request, before a prompt reaches the model and before a response reaches the developer. The Free plan includes real-time guardrails powered by Fiddler Centor Models at under 80ms latency, protecting against hallucinations, toxicity, PII/PHI, prompt injection and jailbreak attempts. Fiddler tracks 100+ out-of-the-box and custom metrics including hallucination, toxicity, PII/PHI leakage and jailbreak attempts.[2][3][4][5][6]

Fiddler AI provides centralized governance and oversight for agents and predictive models across the enterprise, with a registry of your agents, enforced guardrails and the audit evidence needed to meet evolving regulatory requirements. Agent Registry lists every agent application in Fiddler, including first-party, third-party and coding agents. Fiddler records evaluation and policy outcomes with full traceability and generates audit evidence aligned with frameworks and regulations such as GDPR, HIPAA, NAIC, SR 11-7, NIST AI RMF, ISO/IEC 42001 and the EU AI Act. Guardrails run in your own environment, so prompts and responses never leave your infrastructure. Teams can monitor predictive models and agents for drift, bias and performance degradation, run deep diagnostics to find root causes, and create custom reports for MRM and compliance reviews. Fiddler extends governance to the creation layer, monitoring the IDE, CLI and MCP boundary to validate tool invocations against allowlists, scan for secrets, and prevent coding agents from committing insecure code or accessing unauthorized repositories.[5][7]

Fiddler is offered in SaaS, VPC and AWS GovCloud. Fiddler can be deployed in any secure cloud environment or air-gapped. Fiddler states it is SOC 2 Type II compliant, with role-based access control and SSO. Fiddler states it is HIPAA ready. Fiddler AI Observability can run within Amazon SageMaker Studio. Fiddler is model- and cloud-agnostic, working across Azure OpenAI, Amazon Bedrock, LangGraph and 100+ other providers. The Developer plan is usage-based at $0.002 per trace and adds unified observability for agentic and predictive systems, custom evaluators and SSO. The Enterprise plan adds flexible deployment of SaaS, VPC or on-premises, along with white-glove support with a named Customer Success Manager and customized onboarding.[3][5][8]

Fiddler AI is used by organizations deploying agents across all industries, as well as highly regulated industries like financial services, healthcare, insurance and government. The typical users are software and AI engineers, data scientists, platform and DevOps teams, and security and compliance teams. Fiddler positions its AI Observability and Security platform as the Unified AI Command Center for insurers. Fiddler provides a unified AI observability and security platform for healthcare organizations. Fiddler describes itself as the control plane for AI agents in financial services, covering monitoring, policy and governance. Financial institutions deploying AI across underwriting, fraud and trading rely on Fiddler for unified observability, policy enforcement and regulatory-ready audit trails. Fiddler monitors Natural Language Processing and Computer Vision models to track and get real-time alerts on anomalies and data drift. Fiddler explains predictions in ML models powering autonomous vehicles, such as aerial drones and unmanned underwater vehicles. Fortune 100 organizations use Fiddler for agentic and predictive applications. For customer-facing agents, Fiddler governs agents built in-house or integrated from third-party providers in a unified dashboard with audit evidence for GDPR, CCPA, HIPAA and NAIC compliance.[2][3][4][5][6][8][9][10]

Fiddler's founders began with explainability and then expanded to observability to track how models change over time. Fiddler describes itself today as the Control Plane for AI Agents. IQT first invested in Fiddler in 2019.[2][4][6][8][10][11]

Fiddler AI on the record

Sources

Fiddler AI full record

  1. [1]Fiddler Raises $30M Series C to Deliver the First Control Plane for AIFiddler AI · published January 27, 2026 · Tier A
  2. [2]fiddler.aiFiddler AI · fiddler.ai · undated page, accessed November 6, 2026 · Tier A
  3. [3]Fiddler AI: The Control Plane for Enterprise AI AgentsFiddler AI · undated page, accessed November 10, 2026 · Tier A
  4. [4]Fiddler AI Observability for Healthcare LLM ApplicationsFiddler AI · undated page, accessed November 10, 2026 · Tier A
  5. [5]Fiddler AI Control Plane for Financial Services | Fiddler AIFiddler AI · undated page, accessed November 10, 2026 · Tier A
  6. [6]Customer Experience AI Observability for Agents | Fiddler AIFiddler AI · undated page, accessed November 10, 2026 · Tier A
  7. [7]AI Governance, Risk Management, and Compliance (GRC) | Fiddler AIFiddler AI · undated page, accessed November 10, 2026 · Tier A
  8. [8]Fiddler AI Observability for Insurance LLM Applications | Fiddler AIFiddler AI · undated page, accessed November 10, 2026 · Tier A
  9. [9]AI Observability for Lending and Trading | Fiddler AIFiddler AI · undated page, accessed November 10, 2026 · Tier A
  10. [10]Government | Fiddler AIFiddler AI · undated page, accessed November 10, 2026 · Tier A
  11. [11]About Us | Fiddler AIFiddler AI · undated page, accessed November 10, 2026 · Tier A
  12. [12]Fiddler AI Reviews 2026: Details, Pricing, & FeaturesG2 · undated page, accessed November 10, 2026 · Tier C

Promptfoo full record

  1. [1]PressPromptfoo · promptfoo.dev · undated page, accessed November 4, 2026 · Tier A
  2. [2]promptfoo.devPromptfoo · promptfoo.dev · undated page, accessed November 4, 2026 · Tier A
  3. [3]About Promptfoo | AI Security Experts | PromptfooPromptfoo · undated page, accessed November 10, 2026 · Tier A
  4. [4]Build Secure AI Applications | PromptfooPromptfoo · undated page, accessed November 10, 2026 · Tier A
  5. [5]Pricing | PromptfooPromptfoo · undated page, accessed November 10, 2026 · Tier A
  6. [6]Setting up Promptfoo with CircleCI | PromptfooPromptfoo · undated page, accessed November 10, 2026 · Tier A
  7. [7]Testing prompts with Jest and Vitest | PromptfooPromptfoo · undated page, accessed November 10, 2026 · Tier A
  8. [8]Finding LLM Jailbreaks with Burp Suite | PromptfooPromptfoo · undated page, accessed November 10, 2026 · Tier A
  9. [9]SharePoint Integration | PromptfooPromptfoo · undated page, accessed November 10, 2026 · Tier A
  10. [10]Langfuse integration | PromptfooPromptfoo · undated page, accessed November 10, 2026 · Tier A
  11. [11]AI Security for Real Estate | PromptfooPromptfoo · undated page, accessed November 10, 2026 · Tier A
  12. [12]OpenAI acquires Promptfoo to secure its AI agents | TechCrunchTechCrunch · published March 9, 2026 · Tier B
  13. [13]OpenAI Buying AI Security Startup Promptfoo to Safeguard AI Agents - BloombergBloomberg · published March 9, 2026 · Tier B

Spot a factual error? Request a correction.