---
title: "AI Feature Launch Checklist for PMs"
description: "An AI feature launch checklist for product managers: eight go/no-go gates on evals, safety, cost, latency, observability, fallback, rollout and disclosure."
canonical_url: "https://builderscamp.com/guides/other/ai-feature-launch-checklist"
date_published: "2026-10-19"
date_modified: "2026-10-19"
author: "Joy Chatterjee"
publisher: "Builders Camp"
guide_class: "other"
---

# The AI feature launch checklist product managers run before go/no-go

**TL;DR:** Do not launch an AI feature until eight gates pass: evals, safety and guardrails, cost, latency, observability, fallback and human escalation, a staged rollout with a tested kill switch, and user disclosure. Each gate needs a named owner and a written no-go condition, decided before the launch meeting, so the go/no-go is a check of evidence rather than a debate.

An AI feature goes out on evidence, and most of that evidence has to exist before anyone books the launch meeting. The quality gate is the one teams skip, so start there: Hamel Husain and Shreya Shankar's [AI evals FAQ](https://hamel.dev/blog/posts/evals-faq/) puts the minimum viable setup in one line, "Spend 30 minutes manually reviewing 20-50 LLM outputs whenever you make significant changes." Thirty minutes of reading is the floor, not proof of quality, but a team that has not done even that has no basis for a go decision.

The other seven gates cover what an eval set cannot see: abuse, spend, speed, visibility in production, what happens when the model fails, how you turn it off, and what the user is told. A code launch checklist like the one in [building a launch checklist with Claude Code](https://builderscamp.com/guides/tools/claude-code-launch-checklist) still applies to the release itself. The list below sits on top of it and only covers what is specific to a model in the loop.

## What are the eight go/no-go gates for an AI feature?

The eight gates are quality, safety, cost, latency, observability, fallback, rollout and disclosure, and each one has a condition you can check before launch day.

| Gate | Go when | No-go when | Usual owner |
|---|---|---|---|
| Quality and evals | The eval set passes on the launch build, with failures sized and accepted | Nobody has read real outputs from the launch build | PM |
| Safety and guardrails | Prompt injection and misuse tests were run and the guardrails held | Untrusted input reaches the model with tool access and no test | Security or engineering |
| Cost | Cost per successful outcome is under a written ceiling, with per-user quotas | No ceiling exists, or a single user can trigger unlimited calls | PM and engineering |
| Latency | p95 latency is under the ceiling at expected load, with a timeout set | Latency was only measured on a developer laptop | Engineering |
| Observability | Every request is traced with input, output, model version and cost | A user complaint cannot be traced to the request behind it | Engineering |
| Fallback and escalation | Every failure state leads to a manual path or a human | A refusal or timeout leaves the user stuck | PM and support |
| Rollout and kill switch | Staged rollout planned, kill switch flipped once in production | Turning the feature off requires a deploy | Engineering |
| User disclosure | Users can see the output is AI generated and what it cannot do | The feature presents model output as fact with no signal | PM and legal |

Treat every no-go column as binding. If one row has no owner or no evidence, the answer is not yet, and the meeting is a review of what still has to be done.

## How do you prove the quality gate, not just claim it?

You prove the quality gate by showing a frozen eval set, its pass rate on the exact build that will ship, and a written list of the failures you decided to accept. The [guide to writing evals for AI products](https://builderscamp.com/guides/tools/how-to-write-evals-for-ai-products) covers how to build that set; the launch question is narrower. Ask three things in the review: was the set re-run after the last prompt or model change, are the hard cases weighted in rather than a random sample of easy traffic, and does every known failure mode have a size attached.

The [FAQ by Husain and Shankar](https://hamel.dev/blog/posts/evals-faq/) also says "We recommend reviewing at least 100 traces" when you are looking for failure modes, which is the better bar for a first launch than the 20 to 50 output minimum. Neither number tells you the feature is good. They tell you somebody looked, which is the part a launch review can actually check.

## Which safety and guardrail tests must pass before launch?

Prompt injection tests must pass for any feature where untrusted text, such as a user message, an uploaded file or a web page, reaches the model. The OWASP GenAI Security Project ranks [prompt injection](https://genai.owasp.org/llmrisk/llm01-prompt-injection/) first in its LLM Top 10 for 2025, defining it as a vulnerability that "occurs when user prompts alter the LLM's behavior or output in unintended ways." The same entry says it is "unclear if there are fool-proof methods of prevention for prompt injection."

No fool-proof prevention means the launch gate is about blast radius, not perfection. Before go, list every tool, data source and action the model can reach, then try to make it misuse each one with hostile input. Anything that can send, delete, pay or expose data should sit behind a confirmation step or a [human-in-the-loop](https://builderscamp.com/guides/glossary/human-in-the-loop) approval. Output filters, topic limits and refusal rules belong here too; the [AI guardrails](https://builderscamp.com/guides/glossary/ai-guardrails) glossary entry covers the main types.

## What cost limits does an AI feature need on day one?

An AI feature needs a cost ceiling per successful outcome and a per-user quota before day one, because usage growth on a model feature can grow the bill faster than the value. OWASP's [Unbounded Consumption entry](https://genai.owasp.org/llmrisk/llm102025-unbounded-consumption/) describes attackers who, "By initiating a high volume of operations," exploit "the cost-per-use model of cloud-based AI services," and its first defence is to "Apply rate limiting and user quotas to restrict the number of requests a single source entity can make" in a given time period.

Set the ceiling in the unit the business cares about. Divide expected model and infrastructure spend by the number of accepted answers, resolved tickets or completed tasks, the same ratio described in [how to measure AI feature success](https://builderscamp.com/guides/other/how-to-measure-ai-feature-success), and write down the number at which you would pause the rollout. A ceiling written after the first invoice is a postmortem, not a gate.

## How fast does the feature have to be, and what happens when it is slow?

The feature has to meet a p95 latency ceiling you set from the user's task, and it needs a timeout that sends the user to the fallback when the ceiling is missed. An inline suggestion that arrives after the user has finished typing is a failure even if the text is perfect; a background report can take far longer. Measure latency at the load you expect in week one, on production infrastructure, with the longest realistic inputs rather than the short ones from the demo.

Model latency also moves when a provider changes capacity or you switch model versions, so put the ceiling on an alert rather than in a slide. The no-go case is a feature that has only ever been timed by the engineer who built it.

## What does the team need to see in production on launch day?

The team needs a trace for every request: the input, the retrieved context, the output, the model and prompt version, latency, cost, and any user signal such as an edit, a retry or a thumbs down. Without traces, a complaint that the feature gave a wrong answer yesterday cannot be reproduced, and a cost spike cannot be pinned to a prompt change.

Agree on alert thresholds before launch for error rate, latency, cost per hour and the quality proxy you chose, and name who receives each alert. Store traces in a way that respects your privacy policy, since they contain user input. The traces also feed the eval set: every failure found in production is a new test case for the next release.

## What should happen when the model fails or refuses?

Every failure state should leave the user with a next step, either a manual path or a person. Google's People + AI Guidebook chapter on [errors and graceful failure](https://pair.withgoogle.com/chapter/errors-failing/) starts from a blunt premise: "AI is probabilistic by nature, and like all systems, will fail at some point." The chapter then says your primary goal in a failstate "should be to prevent undue harm to the user and help them move forward with their task."

Walk the failure states one by one in the launch review: timeout, refusal, low confidence, guardrail block, provider outage and a wrong answer the user notices. For each, name what the user sees and where they go. The [same PAIR chapter](https://pair.withgoogle.com/chapter/errors-failing/) notes that "When an AI system fails, often the easiest path forward is to let the user take over." It also warns that "In some cases, reverting to manual control could be risky or dangerous." For those cases, the escalation path is a human with the context already attached, and support needs to know the feature exists before it launches.

## How do you roll out an AI feature you can switch off?

Roll it out to a small cohort first behind a runtime flag, and confirm before launch that flipping the flag routes users to the fallback without a deploy. Pete Hodgson's [Feature Toggles article](https://martinfowler.com/articles/feature-toggles.html) on martinfowler.com describes Ops Toggles in plain terms: "These flags are used to control operational aspects of our system's behavior." Its canary example reads "A cohort of canary users is created via a random sampling of 1% of the user base" and uses the same flag system to expose the feature to that group only.

Write the rollback triggers next to the rollout plan: the cost, latency, error and quality thresholds at which the next stage is paused or the switch is flipped. Then flip the switch once in production with the team watching. The common failure is a kill switch that exists in the code, has never been tested, and turns out to disable the whole screen instead of only the AI path.

## What do users need to be told about an AI feature?

Users need to know they are dealing with AI output, what the feature is for, and what it cannot do. Microsoft Research's [Guidelines for Human-AI Interaction](https://www.microsoft.com/en-us/research/project/guidelines-for-human-ai-interaction/) were validated through a 4-step process described in a CHI 2019 paper, and recommend how AI systems should behave upon initial interaction, during regular interaction, when they are inevitably wrong, and over time.

Google's [Explainability and Trust chapter](https://pair.withgoogle.com/chapter/explainability-trust/) states the goal plainly: "Ideally, users have the appropriate level of trust given what the system can and cannot do."

Some disclosure is also a legal question. [Article 50 of the EU AI Act](https://artificialintelligenceact.eu/article/50/) opens with "Providers shall ensure that AI systems intended to interact directly with natural persons are designed and developed" so that people are informed they are interacting with an AI system, unless that is obvious from context, and the artificialintelligenceact.eu page lists the article as coming into force on 2 August 2026. Nothing on this page is legal advice: put disclosure on the checklist as a gate, and have legal or compliance confirm what applies to your product and market.

## What is the strongest argument against a launch checklist, and is it right?

The strongest argument against a checklist is that it slows teams down and turns judgment into box-ticking, so a team passes eight gates and still ships something users do not want. The objection is partly right. A checklist cannot tell you whether the feature should exist; that question belongs in discovery and in the [PRD for an AI feature](https://builderscamp.com/guides/templates/how-to-write-a-prd-for-ai-features), before any of these gates.

What the checklist does well is narrower: it stops the specific, avoidable failures of a model in production, such as an unbounded bill, an injected prompt or a timeout with no way out. Keep it short enough to run in one meeting. If a gate keeps passing without ever changing a decision, cut it; if an incident happens that no gate would have caught, add one.

## Where do you practise running AI launch decisions?

Builders Camp's [AI Product Management bootcamp](https://builderscamp.com/bootcamps/ai-product-management) runs for 2 weeks with 4 live sessions and 7 self-paced microlessons, and its stated goal is to help you design AI-native experiences, evaluate model behaviour, manage risk and deliver AI features users trust. The bootcamp is part of the [AI Product Expert Track](https://builderscamp.com/tracks/ai-product-expert), and its practical challenge is rated advanced.

## Frequently asked questions

### What should be on an AI feature launch checklist?

Eight gates: a passing eval set, safety and guardrail tests including prompt injection, a cost ceiling per successful outcome, a latency ceiling, tracing and alerts, a fallback with a human escalation path, a staged rollout with a kill switch, and a clear disclosure that the user is working with AI. Each gate needs an owner and a written go or no-go condition.

### How is an AI feature launch different from a normal feature launch?

A normal feature behaves the same way on every request, so once QA passes the main launch risk is adoption. An AI feature returns a different output per input, costs money on every call and can be manipulated through its own input, so quality, spend and safety have to be checked as launch gates, not left to post-launch monitoring.

### How many outputs should a PM review before launching an AI feature?

Hamel Husain and Shreya Shankar's AI evals FAQ describes a minimum viable setup of 30 minutes manually reviewing 20-50 LLM outputs whenever you make significant changes, and recommends reviewing at least 100 traces when you are hunting for failure modes. Review real outputs from the launch build, not a demo set.

### What is a kill switch for an AI feature?

A kill switch is a runtime flag that turns the AI path off and routes users to the fallback without a deploy. Pete Hodgson's Feature Toggles article on martinfowler.com calls these Ops Toggles, flags that control operational aspects of a system's behaviour. Test it in production before launch: a switch nobody has flipped is a hope, not a control.

### Do you have to tell users a feature uses AI?

For a feature that talks to people directly, plan on yes. Article 50 of the EU AI Act requires providers to make sure people are told they are interacting with an AI system unless that is obvious from context. Beyond the legal question, users who know what the system is calibrate their trust better. This is not legal advice; check your obligations with counsel.

### Who signs off on an AI feature go/no-go?

The PM runs the review and makes the call, but each gate has its own owner: engineering for latency, cost and the kill switch, whoever owns security for prompt injection tests, support for the escalation path, and legal or compliance for disclosure. A gate with no named owner counts as a no-go.

### Can you launch an AI feature with a known failure mode?

Yes, if the failure is documented, rare enough in your eval set to accept, caught by a guardrail or a human review step, and covered by a fallback the user can act on. What you cannot launch with is a failure you found and did not size, or one that harms the user before anyone can intervene.

## Sources

- [Hamel Husain and Shreya Shankar: AI Evals FAQ](https://hamel.dev/blog/posts/evals-faq/)
- [OWASP GenAI Security Project: LLM01 Prompt Injection](https://genai.owasp.org/llmrisk/llm01-prompt-injection/)
- [OWASP GenAI Security Project: LLM10 Unbounded Consumption](https://genai.owasp.org/llmrisk/llm102025-unbounded-consumption/)
- [Google People + AI Guidebook: Errors and Graceful Failure](https://pair.withgoogle.com/chapter/errors-failing/)
- [Google People + AI Guidebook: Explainability and Trust](https://pair.withgoogle.com/chapter/explainability-trust/)
- [Pete Hodgson on martinfowler.com: Feature Toggles](https://martinfowler.com/articles/feature-toggles.html)
- [Microsoft Research: Guidelines for Human-AI Interaction](https://www.microsoft.com/en-us/research/project/guidelines-for-human-ai-interaction/)
- [EU AI Act, Article 50: Transparency obligations](https://artificialintelligenceact.eu/article/50/)

## How this guide was made

Researched from Builders Camp's bootcamp, track and masterclass material and the sources listed on this page, drafted with AI, and fact-checked against every source cited.