Promptfoo Alternatives (2026): What to Use After the OpenAI Acquisition

Since OpenAI acquired Promptfoo in March 2026, “what should we use instead” has become one of the most common questions in GenAI QA. The short answer: the best Promptfoo alternatives in 2026 are Garak, Giskard, PyRIT, and DeepTeam for red-teaming, DeepEval for CI metric gates, RAGAS for RAG evaluation, and Opik, Langfuse, or Arize Phoenix when you want evals plus observability in one platform. No single tool replaces everything Promptfoo does, and plenty of teams should simply stay put. This guide covers both paths.

What happened: the Promptfoo OpenAI acquisition

On March 9, 2026, OpenAI announced it was acquiring Promptfoo, the open-source LLM evaluation and red-teaming company founded in 2024 by Ian Webster and Michael D’Angelo. The numbers behind the Promptfoo OpenAI acquisition explain the interest: Promptfoo had raised roughly $23 million, was valued at $86 million after its mid-2025 round, and its tooling was used by more than 25% of Fortune 500 companies.

OpenAI’s stated plan is to integrate Promptfoo’s technology into OpenAI Frontier, its enterprise agent platform - automated red-teaming, security evaluation of agentic workflows, and risk and compliance monitoring for agents. OpenAI also said it expects to continue building out the open-source offering, and as of late 2026 the MIT-licensed project is still maintained.

So why is everyone searching for alternatives? Because an evaluation and red-team stack owned by a foundation-model vendor raises two legitimate questions:

1. Neutrality. Promptfoo’s killer feature was impartial multi-model comparison - the same suite run across OpenAI, Anthropic, Google, Mistral, and open-source models. That impartiality is harder to take on faith when the tool’s roadmap is set inside one of the vendors being compared.
2. Roadmap gravity. Features that serve OpenAI Frontier customers will naturally lead. Teams on Bedrock, Vertex, or self-hosted models worry about becoming second-class users of the open-source project.

Neither concern means the tool stopped working. It means the decision is now strategic, not just technical.

Promptfoo alternatives at a glance

The pattern to notice: Promptfoo bundled three jobs - red-teaming, assertions in CI, and multi-model comparison - into one CLI. The open source LLM testing tools landscape splits those jobs across categories, which is why most migrations end with two tools, not one.

Best alternatives by job to be done

Red-teaming and adversarial testing

This was Promptfoo’s crown jewel, and it is the hardest to replace.

• Garak is the closest like-for-like scanner. Originally created by Leon Derczynski and now maintained with NVIDIA support, it runs probes across a defined taxonomy of LLM failure modes: prompt injection, jailbreaks, hallucination, data leakage, toxicity. It is a scanner rather than a test framework, so it slots into CI the same way Promptfoo’s red-team mode did.
• Giskard covers red-teaming plus hallucination and RAG testing from an independent European vendor with no frontier-model parent - a selling point for teams whose objection to the acquisition is precisely vendor ownership. See our Giskard vs DeepEval comparison for how it fits alongside a metric framework.
• PyRIT is Microsoft’s open-source red-team framework, strongest for attack orchestration and multi-modal probing, weakest on out-of-the-box polish.
• DeepTeam brings adversarial testing into the DeepEval ecosystem, which is convenient if you consolidate on Python.

Honest assessment: none of these matches Promptfoo’s combination of 40+ plugins, YAML config, and a clean web UI. Expect to trade polish for neutrality.

CI/CD metric gates

DeepEval is the strongest replacement for Promptfoo’s assertion workflow. It is pytest-native, Apache 2.0, and its metric library has grown well past 50 metrics covering hallucination, faithfulness, answer relevancy, bias, toxicity, and agent tool correctness. Deterministic Promptfoo assertions map to plain Python asserts; llm-rubric assertions map to G-Eval custom metrics. Our Promptfoo vs DeepEval comparison still holds as a feature-by-feature map - just read it now as a migration guide.

RAG evaluation

If you used Promptfoo’s context assertions to sanity-check a RAG pipeline, RAGAS is the purpose-built upgrade: faithfulness, context precision, context recall, and answer relevancy with published methodology. The DeepEval vs RAGAS head-to-head covers when you need one or both, and Promptfoo vs DeepEval vs RAGAS shows how the three-way stack looked before the acquisition reshuffled it.

Multi-model comparison and monitoring

For continuous evaluation rather than point-in-time suites, the open-source observability platforms now cover much of what Promptfoo’s eval matrix did:

• Opik (Comet) is fully Apache 2.0 with 30+ judge metrics, online evaluation rules, and guardrails.
• Langfuse, acquired by ClickHouse in January 2026, remains MIT-licensed and self-hostable, with tracing, datasets, and evals in one platform.
• Arize Phoenix is OpenTelemetry-native with strong eval primitives.

We compare all three in Opik vs Langfuse vs Phoenix.

Managed platforms

If the acquisition pushed you toward “just buy a platform,” Braintrust offers the best eval-first experimentation workflow and LangSmith the deepest LangChain integration - see Braintrust vs LangSmith. Note the irony: both are also proprietary, so this path trades one vendor dependency for another, just not a model vendor.

Who should stay on Promptfoo

Staying is the right call more often than the search volume suggests. Stay if:

• Your suites work today and your usage is CI red-teaming plus assertions on a stable application. Nothing broke on acquisition day.
• You are an OpenAI-primary shop. If your production models are OpenAI models, roadmap gravity works in your favor, and Frontier integration may become a feature rather than a threat.
• You have no procurement constraint around vendor-owned tooling, and switching costs exceed the neutrality benefit.
• You depend on the red-team plugin library. Replicating that coverage requires combining two alternatives; if adversarial breadth is your core need, the grass is genuinely not greener yet.

Who should migrate

Migrate, or at least start a parallel track, if:

• Vendor neutrality is a hard requirement - you are a security consultancy, an auditor, or a regulated enterprise whose assurance reports cannot rest on model-vendor-owned tooling.
• Impartial multi-model comparison is the point. If your eval program exists to decide between OpenAI, Anthropic, Google, and open-source models, run it on neutral rails.
• You are Python-consolidated anyway. Teams who always resented the Node.js dependency now have a clean excuse to move to DeepEval plus DeepTeam or Garak.
• Your compliance framework flags it. Some EU AI Act and internal AI governance reviews now ask who controls the testing toolchain, not just the model.

Migration effort: what to budget

• Deterministic assertions (contains, equals, regex): near-mechanical translation to pytest or any framework. Hours.
• llm-rubric assertions: map to G-Eval style custom metrics in DeepEval or judge metrics in Opik and Langfuse, but scores will not match one-for-one - re-calibrate thresholds against a golden set before enforcing gates. Days.
• Red-team plugins: the long pole. Map each plugin category you actually rely on (prompt injection, PII, jailbreaks, excessive agency) to Garak probes or Giskard scans, and accept that some categories need custom attack cases. Days to weeks.
• CI wiring and dashboards: a day or two per pipeline.

For a mid-sized suite, budget one to two engineer-weeks total, run old and new stacks in parallel for a sprint, and only decommission Promptfoo once the new gates have caught a real regression.

Related reading

• Opik vs Langfuse vs Phoenix - the three open-source observability platforms that absorb Promptfoo’s monitoring role
• Promptfoo vs DeepEval - the head-to-head that now doubles as a migration map
• Promptfoo vs DeepEval vs RAGAS - the pre-acquisition three-way landscape
• Giskard vs DeepEval - the vendor-neutral scanner vs the pytest framework
• OWASP LLM Top 10 testing checklist - the attack categories any replacement red-team stack must cover

Getting help

We run vendor-neutral evaluation and red-team engagements and have migrated Promptfoo suites to DeepEval, Garak, and Giskard stacks without losing coverage. A genai.qa Readiness Assessment maps your current suite to the right alternatives and delivers a working replacement pipeline in 2-3 weeks. Engagements from AED 15k.

Book a free scope call.