Two prompts can ask the same model to do the same job and still get very different results. The difference usually comes down to structure: what the prompt makes clear, what it leaves open, and whether the model can tell instructions apart from data.

An effective prompt is more than a question. In an application, it works like a small contract for the model call. It says what the model should do, what information it should use, what limits it should respect, and what shape the output should have.

This chapter shows the building blocks of a strong prompt and a practical way to improve prompts by testing real failures instead of guessing.

Most useful prompts combine some of these five parts. Not every prompt needs all five. The important skill is knowing which part solves which kind of problem.

Let's look at each part with concrete before-and-after examples.

1. Role / Persona

The role tells the model what perspective to use. It does not turn the model into a real expert. It simply nudges the answer toward the language, priorities, and level of detail common in that domain.

Without role:

This produces a generic answer that might cover mobile push notifications, email, or even physical alarm systems. The model has no signal about what kind of notification system you mean.

With role:

Now the model is more likely to discuss queues, delivery guarantees, fan-out, idempotency, retries, and observability. The role affects more than tone. It changes the assumptions behind the answer.

2. Task / Instruction

The task is the main instruction. It tells the model what to do. A common mistake is naming a topic instead of giving an action.

Compare these two versions:

Vague: "Tell me about database indexing."

Clear: "Explain how B-tree indexes work in PostgreSQL. Cover how the index is structured, how lookups traverse it, and what types of queries benefit most from B-tree indexes."

The vague version could produce a short summary, a long tutorial, or something in between. The clear version tells the model what ground to cover.

3. Context

Context is the background information the model needs to do the job well. Without it, the model fills in missing details with generic assumptions. Those assumptions often do not match your product, your users, or your data.

Without context:

With context:

The first version might produce something like "Upload failed. Error code 413." The second version produces something a nurse can understand and act on.

4. Format Specification

Format specification tells the model how to structure its output. In production, this is not cosmetic. Downstream code, UI components, validators, and analytics often depend on the response shape.

Without the format specification, the model might return one long paragraph. With it, you get a response that is easier to show in a UI or split into sections.

5. Constraints / Guardrails

Constraints define the boundaries of acceptable behavior. They can limit length, keep the answer on topic, and express product or safety rules the model would not otherwise know.

Constraints matter most when the model talks directly to users or works with tools. They are not a security boundary by themselves, but they make expected behavior explicit and give validators something concrete to check.

Putting All Five Together

A prompt that uses all five components together:

Not every prompt needs all five parts. A quick internal query may only need a task. A customer-facing workflow usually needs task, context, format, constraints, and sometimes examples. Add the part that fixes the failure you see. Do not add extra instructions just to make the prompt look complete.

Most chat-style LLM APIs represent requests as messages with roles such as system, user, and assistant. The role matters because it helps the API distinguish application instructions, user requests, and prior assistant responses. The practical question is what belongs in each role.

What Goes Where

The system prompt sets behavior that should stay stable for the call or conversation: role, boundaries, style defaults, and non-negotiable rules. Treat it as application policy, not user-visible content.

The user prompt contains the specific request and request-specific data. It changes with every interaction.

A practical way to think about it:

System Prompt Persistence

In stateless chat-completion style APIs, the system prompt is sent with every API call. Some platforms provide server-side conversation abstractions, but the model still receives those instructions as part of the context it uses. Either way, long system prompts consume context budget and can increase cost.

For long conversations, a lengthy system prompt becomes expensive. If your system prompt is 500 tokens and you make 20 stateless turns, that is 10,000 input tokens just for repeated policy text. Keep system prompts clear, stable, and as short as the job allows.

There is a real tension in prompt writing. If the prompt is too vague, the model guesses. If it is too rigid, the model may focus on arbitrary rules instead of the actual task. Strong prompts specify what matters and leave harmless choices flexible.

The same task at five levels of specificity shows how this plays out.

Level 1: Too vague

"Write something about APIs."

This could produce almost anything: an essay, a poem, a tutorial, or a rant. The model has no signal about what you want.

Level 2: Slightly better

"Write a blog post about REST APIs."

Now we have a format (blog post) and a topic (REST APIs), but we still do not know the audience, length, depth, or purpose.

Level 3: Good for most use cases

"Write a 500-word blog post explaining REST APIs to junior developers. Cover what REST stands for, the main HTTP methods, and a simple example using a bookstore API."

This gives the model a clear target. Audience, length, scope, and an example domain are all specified.

Level 4: Highly specific (production quality)

"Write a 500-word blog post explaining REST APIs to junior developers who know basic HTTP but have never built an API. Cover: (1) what REST stands for and the core principles, (2) GET, POST, PUT, DELETE with one-sentence descriptions, (3) a bookstore API example showing a GET request and response. Use a conversational tone. Do not use jargon without defining it first."

For many production applications, this level is enough. The model has a clear target without being trapped by arbitrary rules.

Level 5: Over-specified (counterproductive)

"Write exactly 500 words. First paragraph must be 3 sentences. Use the word 'API' exactly 12 times. Every section header must be a question. The third paragraph must contain a code block of exactly 4 lines. End with a question starting with 'Have you...'."

At this level, the model spends effort satisfying arbitrary constraints instead of producing useful content. The result is often brittle and unnatural.

Finding the Right Level

The right level of specificity depends on your use case:

The general principle: specify the required outcome and constraints clearly, but avoid constraining details that do not affect quality, safety, or parseability.

Vague instructions produce vague outputs. A few simple habits make prompts much harder to misread.

Use Imperative Verbs

Start instructions with action verbs. Instead of "It would be nice if you could explain...", write "Explain...". Instead of "Can you maybe list some...", write "List the top 5...".

Direct instructions are easier for the model to follow. You are not being rude; you are reducing ambiguity.

Decompose Complex Tasks

When a task has multiple parts, break it into numbered steps. Numbering helps the model organize its thinking and makes it less likely to skip steps.

One broad instruction (easy to miss parts):

Decomposed instruction (easier to follow):

The decomposed version is longer, but it gives the model a checklist. The broad version often skips one of the requested items because the task boundaries are unclear.

Specify the Output Format Explicitly

If you need structured output, show the model exactly what you want. A template is usually clearer than a paragraph describing the format.

When the format is this explicit, the model is more likely to follow it. For production systems, prefer provider-supported structured output or schema validation when available. Prompt formatting is useful, but parsers and validators should still enforce the contract.

Use Delimiters for Input Data

When your prompt includes data the model should process, use clear delimiters. Delimiters do not stop prompt injection, but they reduce ambiguity and make the instruction/data boundary explicit.

The delimiters ---BEGIN REVIEW--- and ---END REVIEW--- signal that the text between them is data. A malicious review can still try to hijack the model, so pair delimiters with instruction hierarchy, output validation, and least-privilege tool design.

Sometimes the clearest way to specify behavior is to show examples. Few-shot prompting is useful when the task depends on a local convention that is hard to describe completely in prose.

Zero-Shot vs One-Shot vs Few-Shot

The terminology is simple:

The difference is clearest with a realistic task: classifying customer support tickets.

Zero-shot (instructions only):

For straightforward cases, zero-shot prompting may be enough. Ambiguous tickets are harder. A model may classify a login issue as TECHNICAL when your support organization wants login and identity problems routed to ACCOUNT.

Few-shot (examples set the standard):

The fifth example is the key one. It shows a login issue classified as ACCOUNT, not TECHNICAL. This teaches the local routing policy: identity and access issues go to the account team, even when the user describes them as a technical problem.

When Examples Help

Examples help most when:

When Examples Hurt

More examples are not always better. The tradeoffs are real:

A practical rule: start zero-shot, test, then add examples for the failures you actually see. More examples are justified when they cover distinct edge cases, not when they repeat the obvious case.

The diagram shows the basic decision flow. Start simple, then add examples only when evaluation shows a specific need.

The important habit is to treat prompts as versioned system behavior. You do not write a prompt once and assume it is done. You draft, test, inspect failures, refine, and regression-test, just as you would with application code.

The Iteration Workflow

The loop is simple:

Suppose you are building a feature that generates product descriptions from raw product specs.

Step 1: Draft

Step 2: Test and Identify Problems

Running this against several products reveals three common failures:

Step 3: Refine

Step 4: Test Again

Run v2 against the same products plus a few edge cases:

Step 5: Keep Refining

If v2 still has issues, make targeted changes. If the model rewrites pricing, add: "Display the price exactly as provided; do not convert or reformat it." If it mishandles technical specs, add: "For technical terms, add a brief parenthetical explanation only when the target audience may not know the term."

This process continues after launch. As new edge cases appear, update the prompt and the evaluation set together. Prompt changes without new tests are hard to trust.

References