What Is a System Prompt?

What does a system prompt mean?

A system prompt is the instruction that sits above every conversation with a language model and defines its role, tone, constraints and output format for one specific use. Product Talk's glossary says that unlike user prompts, "the system prompt remains constant and establishes the context within which all user interactions occur" (Product Talk). Anthropic's own guidance makes the practical case in one line: "Setting a role in the system prompt focuses Claude's behavior and tone for your use case" (Anthropic prompting best practices).

For a product manager, the system prompt is the product decision. It is where the team writes down who the feature serves and what a good answer looks like, once, instead of hoping every user phrases their request well.

How do the system, user and assistant roles fit in one thread?

Every chat with a model is one model reading one list of messages, and each message carries a role. The system message holds the standing instructions. User messages hold what the person asked. Assistant messages hold what the model answered. OpenAI's API now names the first role differently, and its text generation guide says "developer messages are instructions provided by the application developer, prioritized ahead of user messages." Anthropic's API takes the same instructions as a separate system field. The idea is identical in both.

Seen as data, a two-turn thread for an internal meeting-notes tool looks like this:

system:    You summarise B2B sales call notes for the product team...
user:      [notes from Tuesday's call with a logistics customer]
assistant: [summary of Tuesday's call]
user:      Now compare that with last week's call notes: [notes]

In ChatGPT, Claude or Gemini as a consumer, you never see the first line. The app writes its own system prompt and your typing starts at the first user message. When you build a feature on the API, that first line is yours to write.

Why does the whole thread get resent on every turn?

A model does not remember the previous message. Anthropic's documentation is direct about it: "The Messages API is stateless, which means that you always send the full conversational history to the API" (Anthropic: Working with the Messages API). Every new question travels with the system prompt, every earlier question and every earlier answer. Anthropic's context windows page describes the same thing as progressive accumulation, where "each user message and assistant response accumulates within the context window."

How much room that leaves depends on the model. Anthropic lists a 1M-token context window for its current models, including Claude Sonnet 5, and a 200k-token window for older ones such as Claude Sonnet 4.5 (Anthropic: Context windows). A large window tells you how much text fits, not how well the model attends to an instruction written 40 turns ago. STRV's prompting guide gives the working rule: "keep your chat threads short to avoid context rot" (STRV).

Two product consequences follow. Cost and latency grow with every turn, because the model reads the whole history each time. And a rule you set at the start weakens as the thread fills up, which is why a long chat drifts off format even though the system prompt never changed.

What belongs in the system prompt, and what belongs in the user message?

The dividing line is change. If a sentence would be the same for the next thousand requests, it belongs in the system prompt. If it describes this request, it belongs in the user message.

Instruction	System prompt	User message
Who the model acts as and who it writes for	Yes	No
Output format, length and headings	Yes	Only to override once
What to refuse or flag as out of scope	Yes	No
Definitions of your categories or labels	Yes	No
Two or three example answers	Yes, or as example turns	No
The notes, ticket or document to process	No	Yes
Today's specific question	No	Yes

The most common mistake is the reverse of this table: a user pasting the same five lines of formatting rules into every chat, or a team hardcoding one customer's data into the system prompt. Both cost tokens and both drift.

What does a PM-ready system prompt look like?

Here is a system prompt for a fictional internal tool that turns sales call notes into a summary the product team can scan. Every line answers a question the model would otherwise guess.

ROLE
You summarise B2B sales call notes for a product team at an invoicing
software company. Your readers are product managers, not salespeople.

OBJECTIVE
Pull out product evidence: problems the customer described, features
they asked for, and competitors they named.

OUTPUT FORMAT
Three headed lists: Problems, Requests, Competitors.
One line per item. Quote the customer's words where the notes have them.

RULES
- Only use what is in the notes. If something is not stated, write
  "not stated". Never infer company size or budget.
- A request is only a request if the customer asked for it. Sales
  suggestions do not count.
- If the notes are not from a sales call, reply only: "Not a sales call."

Notice what it leaves out: the notes themselves. They arrive in the user message, fresh on every call. For the full sectioned structure and more PM tasks, see the prompt template for product managers.

How do example turns make output consistent?

Rules describe a format. Examples show it, and models copy what they are shown more reliably than what they are told. The API version of this trick is to write the first exchange yourself. Anthropic's documentation says plainly: "Earlier conversational turns don't necessarily need to actually originate from Claude. You can use synthetic assistant messages" (Anthropic: Working with the Messages API).

So after the system prompt above, you add one invented user turn with short sample notes and one invented assistant turn with the exact summary you want back. The real notes go in the third message. The model now has a concrete answer to imitate, including how short "one line per item" really is. This is few-shot prompting delivered as conversation history instead of pasted into one message.

Can a user override a system prompt?

Not reliably, and the word reliably is doing the work. The API ranks system or developer instructions above user input, so a casual "ignore your instructions" usually fails. A patient user with cleverly framed input can still pull a model off its rules, and a document the model is asked to read can carry instructions of its own.

The fix is not a longer system prompt. Anything that must never happen, such as exposing another customer's data, needs a check in code that runs after the model answers. Treat the system prompt as the default behaviour of the feature, and treat your code as the guarantee.

Where does the AI Prompting for Product bootcamp fit?

AI Prompting for Product is a 1 week Builders Camp bootcamp with 2 live sessions and 19 self-paced microlessons, directed by Andre Albuquerque. Its public syllabus lists prompt structure (roles, context, constraints and success criteria) and evaluation and iteration loops among its topics, and its practical challenge asks you to write customer service reply prompts in three tones and score each output on clarity, warmth, resolution and brand fit.

See the AI Prompting for Product bootcamp

For the model underneath, read large language model, and for how instructions travel when you switch between models, multi-model prompting. The broader discipline is prompt engineering.

Before you edit a live system prompt, save ten real user inputs and the current outputs for each. Run the same ten through the new version and compare them side by side. A system prompt change is a product release that ships to every user at once, and ten saved inputs are the cheapest regression test you will ever run.