How to collect feedback on an MVP from first users, and what to change in a vibe-coded app

Who should your first ten users be?

Pick them by hand, one at a time, from people who have the problem your MVP solves right now. Paul Graham's essay Do Things that Don't Scale opens on exactly this: "The most common unscalable thing founders have to do at the start is to recruit users manually." For a vibe-coded MVP, manual recruiting has a second benefit. Every person you invite is someone you can ask for 15 minutes on a call, and those calls are where the useful feedback comes from.

Use three filters when you choose the ten:

• They have the problem this month, not in principle. A person who split a utility bill with flatmates last week beats one who might move into a shared flat next year.
• They use something else today: a spreadsheet, a group chat, a competitor's app. You learn most from watching someone compare your app with a habit they already have.
• They match the one user type in your spec. If your mini-PRD names one user, do not invite the second type yet. Their requests will pull the next build in two directions.

Friends are fine only if they pass the first filter. A friend without the problem will be generous about the interface and tell you nothing about whether the app is worth using.

Why split ten users into two rounds of five?

Because most of what five people find, the next five would find again. Jakob Nielsen's analysis at Nielsen Norman Group found that a single test user reveals almost a third of a design's usability problems, using a typical rate of 31 percent per user, and that five users find about 85 percent. His conclusion: "After the fifth user, you are wasting your time by observing the same findings repeatedly but not learning much new." He recommends spending the budget on several small studies, because "The ultimate user experience is improved much more by 3 studies with 5 users each than by a single monster study with 15 users."

That research is about usability problems among comparable users, and Nielsen notes the formula "only holds for comparable users who will be using the site in fairly similar ways." It does not tell you whether people want the product, or how many would pay. What it still gives a solo builder is a schedule: five sessions, fix what they showed you, five new people on the fixed version. The second round tells you whether your fixes worked, which a single round of ten never can.

How do you ask someone to try your MVP?

Ask for a short call with a single task, not for opinions. A link with "let me know what you think" gets replies from the polite few, written after they have forgotten where they got stuck.

Hi [name], I built a small app for [problem in one line], because [why you think they have it]. Could you give me 15 minutes on a call this week, with your screen shared, to try it for the first time? I will not explain anything in advance; seeing where it confuses you is the point. [Two time options]

Send it individually, from your own account, and expect to send more messages than the sessions you need. The people who say yes and have the problem are your first ten.

What does a five-minute watch-them-use-it session look like?

Give one realistic task, then stay quiet while they do it. The method is thinking aloud, which Nielsen describes in Thinking Aloud: The #1 Usability Tool: "you ask test participants to use the system while continuously thinking out loud." His instruction to the person running the session is blunt: "Shut up and let the users do the talking." Every hint you give changes what you learn.

The script, 15 minutes in total:

1. One minute of setup. "I'm testing the app, not you. Please say what you're thinking as you go. I'll stay quiet; if you get stuck, tell me what you'd do if I weren't here."
2. The task, in their world. Not "add an expense" but "You paid the €84 electricity bill yesterday. Use this to sort it out with your flatmates."
3. Five minutes of watching. Write down where they pause, what they say, what they tap that does nothing. Do not rescue them. If they are stuck for a full minute, note it and move on.
4. Three questions afterwards. "What did you expect to happen when you [moment they paused]?" "What would you have used instead of this app?" "Is there anything you tried to do that wasn't there?"
5. One request. "If this comes up for real this week, try it then. No need to tell me." Then check the data a week later.

Step 5 is the one that tells you whether the app is useful rather than merely usable. A completed task in a watched session is the first; an unprompted second use is the second.

What should you write down in the feedback log?

One page per round, filled in during or straight after each session. The verbatim columns matter most, because next week you will not remember whether someone asked "can I split it unevenly?" or "why is it split evenly?", and those lead to different fixes.

Feedback log, one page per round

• App and version: [name, checkpoint or commit name]
• Round: [1 or 2] · Dates: [from, to]
• Core task: [the one task every session uses]

#	User (first name, how you found them)	Finished the core task? (yes / with a hint / no)	Where they stopped (screen and action)	What they asked (verbatim)	What they tried that does not exist (verbatim)	Bug seen	Came back within 7 days?
1
2
3
4
5

Decisions for the next build

• Fix: [items that blocked 2+ of 5, and any bug touching money or data]
• Add: [items 3+ of 5 tried, on the path to repeating the core task]
• Cut: [features nobody touched]
• Deferred, with count: [everything else]

What does a filled feedback log look like?

Here is round one for a fictional MVP: an expense splitter for shared flats, built with an AI coding platform in a weekend. The core user is the flatmate who pays the bills; the core task is adding a bill and seeing who owes what. The entries are illustrative.

#	User	Finished?	Where they stopped	What they asked	What they tried that does not exist	Bug seen	Came back?
1	Inês, flatmate group chat	No	Add flatmates screen: invite needs an email, she only had phone numbers	"Do they all need an account?"	Tapped the balance to mark it paid	None	No
2	Tom, local subreddit	Yes	None	"Can I split it unevenly? Sam has the big room"	Tapped the balance to mark it paid	Typed 84,50 and it saved €8,450	Yes, added rent
3	Priya, former colleague	With a hint	Add flatmates screen, same as Inês	None	None	None	No
4	Joel, student housing forum	Yes	None	"Where do I see last month?"	Tapped the balance to mark it paid	Typed 12,90 and it saved €1,290	No
5	Ana, flatmate group chat	No	Add flatmates screen: gave up after a minute	"Why do I need their email?"	Tapped the balance to mark it paid	None	No

Patterns jump out of five rows faster than out of fifty survey answers. Three of five stopped at the same screen. Four of five reached for something that does not exist. Two of five hit the same data bug, and both typed amounts the European way, with a decimal comma. Only one came back, and he added rent, which is the core task repeating on its own.

How do you decide what to fix, cut or add?

Four rules, applied to every line of the log:

• Fix anything that stopped two or more of five people finishing the core task, and any bug that touches money or data, however rare.
• Add a feature only if three or more of five tried to use it and it sits on the path to repeating the core task.
• Cut any feature nobody touched in five sessions that is not part of the core task.
• Defer everything else, with the count beside it, so round two can confirm or bury it.

Applied to the expense splitter:

Finding	Count	Decision	Why
Decimal comma saves the wrong amount	2 of 5	Fix first	Money is wrong, so it is fixed whatever the count
Invite requires an email	3 of 5 blocked	Fix	Blocks the core task; let the payer add flatmates by name, invites later
Mark a balance as paid	4 of 5 tried	Add	Settling up is how the next bill gets added, so it is on the path to repeat use
Uneven split	1 of 5 asked	Defer (1)	Real, but one person; watch for it in round two
See last month	1 of 5 asked	Defer (1)	Same
Expense categories dropdown	0 of 5 used	Cut	Nobody touched it, and it adds a field to every entry

The uneven split is the decision that feels wrong. Tom's reason was good, and it would be a nice feature. One request out of five is still one person, and adding it now means building the second user situation (rooms of different sizes) before the first one works for three people who could not even add their flatmates.

How do you turn the log into the next prompts?

One decision becomes one prompt, sent and tested before the next, with a named checkpoint between each. That is the feature-by-feature rule from vibe coding mistakes to avoid, and it matters more now that real users have data in the app.

For the expense splitter, round one produced four prompts in this order:

1. "Amounts typed with a comma as the decimal separator, such as 84,50, are saved as 8450. Accept both 84.50 and 84,50 as eighty-four euros fifty. Do not change anything else. Then show me how existing wrong entries can be found."
2. "On the add flatmates screen, let the payer add a flatmate by name only. Remove the email requirement. Keep the existing email invite as an optional second step."
3. "Add a 'Mark as paid' button next to each balance. Pressing it records a payment from that flatmate to the payer and updates the balance. Show who marked it and when."
4. "Remove the expense category dropdown from the add expense form and stop requiring a category. Keep existing data untouched."

Each prompt names the change, the boundary ("do not change anything else") and something you can check in thirty seconds. If a fix breaks something, roll back to the checkpoint instead of stacking a second fix on top; how to debug AI-generated code covers that loop.

After the four changes, recruit five new people and run the same task. Round two succeeds if the add flatmates screen no longer appears in the "where they stopped" column and more than one person in five comes back on their own. Then tell round one what changed, especially Inês and Ana, who gave up; how to close the loop on customer feedback has message templates you can shorten to two lines.

What can ten users not tell you?

The obvious objection is that ten people is a tiny sample, and it is. Ten sessions say nothing about how big the market is, what people would pay or whether a stranger finds the app without you. Watched sessions also overstate commitment: people try harder when someone is on the call.

What ten users can tell you is where the app breaks for people who have the problem, which is what has to be fixed before any larger number is worth measuring. Graham makes the case for the small number directly: "The feedback you get from engaging directly with your earliest users will be the best you ever get." The column that guards against politeness is the last one. Words in a session are cheap; a second use, a week later, with nobody watching, is not.

What comes before and after this feedback loop?

The feedback loop assumes the app is live at a link you can share and that data survives a refresh. If it is not there yet, how to deploy an AI-built web app for the first time covers the hosting choice and the five checks to pass before the first session, and how to build an MVP with Replit covers the build before that. For interviews before you build, rather than sessions after, see how to run customer interviews.

Zero to Shipped with Vibe Coding is a 1 week Builders Camp bootcamp in the Vibe Coding Expert Track, with 2 live sessions and 11 self-paced microlessons, for going from idea to a working MVP with AI-assisted building tools. Using AI to Build Your First Product runs over 2 weeks and lists collecting early feedback without breaking trust, and using feedback and metrics to plan v2, in its syllabus.

See the Zero to Shipped with Vibe Coding bootcamp

Copy the log template before your first session, not after it. The column people skip when filling it in from memory is "what they tried that does not exist", and in the example above it was the one that decided the next feature.