--- title: "How to Use AI for Product Analytics" description: "Three product analytics questions AI answers without an analyst, the ways it returns a confidently wrong number, and the cross-foot check that catches them." canonical_url: "https://builderscamp.com/guides/tools/ai-for-product-analytics" date_published: "2026-09-18" date_modified: "2026-09-18" author: "Andre Albuquerque, Mário Araújo" publisher: "Builders Camp" guide_class: "tools" --- # How to use AI for product analytics without waiting on an analyst **TL;DR:** Three question types are safe to hand a model with no analyst in the room: recomputation, definition archaeology, and shape reading. Everything past those, starting with any claim about cause, needs the person who knows how the event got into the table. The check that catches most confidently wrong answers is cross-footing, recomputing the total as the sum of its segments and chasing the gap. ## Which product analytics questions can AI answer without an analyst? Three, and the boundary between them matters more than which model you use. The first is recomputation: the metric already has an agreed definition and you want it again over a different window, segment or denominator. The second is definition archaeology: which event fires when a user does the thing, which column holds the account id, what separates `signup_completed` from `account_created` in a schema nobody documented. The third is shape reading: whether last month's dip is a step change on one date or a gradual slide, because that single distinction decides whether you go looking for a release or for a market. Everything past those three depends on knowing how the data got into the table, and a model reading an export cannot know that. It can read what is in front of it with more patience than you have. It cannot tell you that the mobile SDK stopped firing on the 14th. ## What does the model get confidently wrong? The failures are boringly consistent, and none of them look like errors. A partial final period is the most common: the export ends mid-week, the model computes a weekly series, and the last bar drops off a cliff that is really just Tuesday. The second is the denominator. "Activation rate" computed over signups in the period and "activation rate" computed over signups that have had 30 days to activate are different numbers, and a model asked for the former when you meant the latter will return a clean, wrong figure with no signal that a choice was made. The third is the shape of the export itself. Google Analytics condenses low-frequency dimension values into a row labelled (other) once a table exceeds its row limit, and its own documentation treats any dimension carrying more than 500 values as high cardinality for exactly this reason. The worked example in Google's help centre is blunt: a property with 150,000 unique pages against a 100,000-row limit gets its least common 50,000 pages folded into one row. Separately, Google withholds data from a report altogether when a threshold applies, to stop anyone identifying an individual from demographic signals. A model reading that export sees rows. It has no way to see the rows that are not there. That is the part worth sitting with. Every one of those failures produces a number that is internally consistent, formatted correctly, and wrong. ## The verification step: recompute the number a second way Cross-footing is the cheapest check in analytics and almost nobody runs it on an AI answer. Ask for the headline figure, then ask for the same figure split by a dimension whose parts must sum to the whole: plan, country, platform, acquisition channel. Then add the parts up. If the segments sum to the total, you have ruled out most of the failure list in one step. If they fall short, the gap is the finding. A 4 percent shortfall usually means nulls in the segmenting column. A shortfall that lands on one suspiciously round bucket means you have found the condensed row. A shortfall that changes when you change the date window means a filter is being applied at one step and not the other. The second check is order of magnitude, stated before you see the answer. Write down what you expect, then read the result. A model will not warn you that 8 percent is implausible for your product; you will, if you committed to a range first. ## What does a good first prompt actually contain? Four things, and a prompt missing any of them will still produce an answer. Name the metric's definition in words, not just its name. Name the window, including the timezone the timestamps are in. Name the denominator explicitly. Name the exclusions: internal accounts, test users, the free tier, refunded orders. A request that meets that bar reads like a spec rather than a question: compute weekly activation rate for accounts created between 1 June and 31 August in UTC, where activation means the account completed its first inspection submission, denominator is accounts created in that week, excluding accounts whose email domain matches the company's own, and flag any week where fewer than 50 accounts entered the denominator. The last clause is the one people skip. A weekly rate computed over 12 accounts is not a rate; it is a rumour with a decimal point. ## Where does a person who knows the data still have to be in the room? Instrumentation semantics, causal claims, and any number that allocates money. Instrumentation first, because the gap between what an event is named and what it actually fires on is where most bad analysis starts, and it is invisible in the data. The Product Analytics bootcamp builds its practical challenge around exactly this failure: a company with 400 signed-up accounts reporting 120 as active, where "active" means any login in the last 30 days, so an account that opened the app once and never returned counts as a success. The board reads 30 percent activation and concludes the product is broken. The real finding is that nobody defined activation, and no tool and no model can recover a definition that was never made. Causal claims are the second. A drop that coincides with a release is a coincidence until something isolates it. ## When is the model faster than the dashboard, and when is it slower? Faster when the question is one you would otherwise have queued, and when the answer needs a shape the dashboard does not have: an odd date window, a segment nobody set up, a comparison between two definitions you are still arguing about. Slower when a saved report already answers it, because reproducing a configured funnel from a raw export costs more than opening the report. The honest limit is that a model can only be as good as the export, and the export is the thing you did not check. If your instrumentation is thin, AI makes bad analysis faster rather than making good analysis possible. That is not an argument against using it. It is an argument for spending the first week on the event plan and the second on the questions. ## What does this change about how a product team works? The analyst stops being a queue. When a product manager can recompute a known metric in four minutes, the analyst's time moves to the things that actually need judgement: defining activation at the account level, deciding what a product qualified lead is, choosing which two metrics per funnel stage earn a place on a weekly scorecard. That reallocation is worth more than any individual query the model runs. It also raises the bar on definitions, because a model will answer whatever you ask precisely. Vagueness that a human analyst would have quietly corrected now ships as a number. ## Start with the metric your team already argues about Take the one figure that gets disputed in every review, write its definition down in a sentence, and have the model recompute it against the raw export using that definition and nothing else. Then cross-foot it. Half the time the argument was never about the number. [See the Product Analytics bootcamp](https://builderscamp.com/bootcamps/product-analytics?utm_source=guide&utm_medium=organic&utm_campaign=ai-for-product-analytics) Product Analytics runs one week, two live sessions and nine microlessons, aimed at B2B product-led growth: instrumentation, activation, account-level rollups and cohort retention. If you want the broader base first, Data for Product Managers covers metrics, funnels, cohorts and experiments across two weeks, and both sit inside the [Data & Analytics Specialist Track](https://builderscamp.com/tracks/data-analytics-specialist). For the mechanics of a single analysis session, see [analysing product metrics with Claude Code](https://builderscamp.com/guides/tools/claude-code-metric-analysis-for-pms) and [pulling product data without SQL](https://builderscamp.com/guides/tools/claude-code-data-pull-without-sql). The definitions underneath this page are in [cohort analysis](https://builderscamp.com/guides/glossary/cohort-analysis), [activation rate](https://builderscamp.com/guides/glossary/activation-rate) and [metrics tree](https://builderscamp.com/guides/glossary/metrics-tree). ## Frequently asked questions ### Can AI replace a product analyst? Not for anything causal, and not for anything where the answer depends on how an event got into the table. It replaces the queue, not the analyst: recomputing a known metric over a new window or segment no longer needs a ticket, which frees the analyst for the questions where the definition itself is in dispute. ### What is the one check that catches most wrong numbers? Cross-footing. Ask for the headline number and the same number broken into segments that should sum to it, then check the sum. A gap means a null segment, a high-cardinality bucket, a filter applied at one step and not the other, or a join that dropped rows. All four are common and all four are invisible in a single number. ### Why does the model's number disagree with my dashboard? Usually the timezone, the attribution window or the denominator, in that order. Dashboards are configured once and then forgotten, so the dashboard's definition is often not the one anybody would write down today. Make the model state its definition in words before you compare, or you will spend an hour reconciling two numbers that were never measuring the same thing. ### Do I need a warehouse connection, or is a CSV export enough? A CSV export is enough for recomputation and shape reading, which covers most of what a product manager needs week to week. A live connection matters when the question changes shape every few minutes and re-exporting becomes the bottleneck. Start with the export, because a file you can re-read is also a file a reviewer can check. ### What is the (other) row, and why does it break an AI answer? When a table has more dimension values than its row limit, Google Analytics keeps the most common values and condenses the rest into a row labelled (other). Any dimension with more than 500 values counts as high cardinality and raises that risk. A model reading the export sees a legitimate-looking row and will happily treat it as a real segment. ### What should never be decided from an AI-produced number alone? Anything that allocates money or headcount, and anything that asserts cause. A number that survives a cross-foot check is a number you can quote; it is still not evidence that the release caused the change. Causal claims need a control group, a holdout period, or an experiment. ### Which Builders Camp bootcamp covers this? Product Analytics runs one week, 2 live sessions and 9 self-paced microlessons, and is built around B2B product-led growth: instrumentation, activation definitions, account-level rollups and cohort retention. Data for Product Managers is the broader 2-week version covering metrics, funnels, cohorts and experiments. ## Sources - [Google Analytics Help: About the (other) row](https://support.google.com/analytics/answer/13331684) - [Google Analytics Help: About data thresholds](https://support.google.com/analytics/answer/9383630) - [Builders Camp: Product Analytics bootcamp](https://builderscamp.com/bootcamps/product-analytics) - [Builders Camp: Data for Product Managers bootcamp](https://builderscamp.com/bootcamps/data-for-pm) ## How this guide was made Researched from Builders Camp's bootcamp, track and masterclass material and the sources listed on this page, drafted with AI, and fact-checked against every source cited.