Ask a large language model a hard math problem outright and it will often give you a confident, wrong answer. Ask it to reason through the problem one step at a time first, and the same model will frequently arrive at the correct result. That gap — between answering directly and reasoning explicitly — is the entire premise of chain-of-thought prompting, one of the most impactful prompt engineering techniques ever formalized.

This guide explains what chain-of-thought (CoT) prompting is, why it works, the different forms it takes, when to use it, when to avoid it, and how to write CoT prompts that consistently produce better, more reliable reasoning from models like GPT, Claude, and Gemini. Whether you are building an AI-powered tutoring app, writing prompts for business analysis, or just trying to stop a chatbot from hallucinating on multi-step problems, chain-of-thought prompting belongs in your toolkit.

What Is Chain-of-Thought Prompting?

Chain-of-thought prompting is a technique where you explicitly instruct a model to break a problem into intermediate reasoning steps before producing a final answer. Instead of jumping from question to conclusion, the model generates a visible chain of reasoning — each step building on the last — and only then commits to an answer.

The technique is usually traced back to a 2022 paper by Google researchers ("Chain-of-Thought Prompting Elicits Reasoning in Large Language Models," Wei et al.) which showed that prompting models to reason step-by-step produced dramatic improvements on arithmetic, commonsense, and symbolic reasoning benchmarks. The key insight was simple but powerful: large language models are better at generating the next reasonable step than at producing a final answer in a single leap. By forcing the model to generate the intermediate steps, you let it use its strongest capability — next-token prediction — repeatedly, and each intermediate result becomes context for the next step.

In practice, chain-of-thought prompting usually takes one of two forms:

Zero-shot chain-of-thought

You instruct the model to reason step-by-step using a general directive. The classic example is appending the phrase "Let's think step by step" to the end of your question. Despite its simplicity, this single addition reliably improves performance on reasoning-heavy tasks.

Few-shot chain-of-thought

You give the model several worked examples, each one showing the full reasoning chain from question to answer. The model then follows the same pattern on a new question. This is more reliable on difficult or specialized tasks because the examples establish the exact reasoning style you want.

Both forms share the same core idea: make the model's reasoning explicit so its final answer is grounded in a traceable, checkable chain of steps.

Why Chain-of-Thought Prompting Works

To use a technique well, it helps to understand why it works. There are three overlapping reasons chain-of-thought prompting improves results.

1. Reasoning needs more compute per step

When a model answers in a single shot, it gets one forward pass worth of computation to transform the question into the answer. When it reasons step-by-step, every generated reasoning token becomes input context for the next token, giving the model many more forward passes of "thinking time" before it commits to a final answer. Hard problems simply need more computation, and CoT effectively buys that computation by spending more output tokens on reasoning.

2. Intermediate results reduce the working-memory load

Multi-step problems require keeping track of partial results. A model that writes down each intermediate result can refer back to it in context instead of holding everything in an internal, implicit state. This is analogous to why humans write out long division rather than doing it entirely in their heads: externalizing intermediate steps lowers the chance of dropping or mangling a value along the way.

3. A longer reasoning trace constrains the answer space

A short, direct answer can plausibly be any token the model knows. A long, correct reasoning chain narrows the set of answers that are consistent with it, making the correct final answer more likely. In other words, the chain constrains the conclusion; the model is less likely to contradict itself when it has just laid out the logic on the page.

Understanding these mechanisms matters because it tells you when chain-of-thought helps (hard, multi-step problems) and when it is wasted effort (simple recall or single-step tasks where extra reasoning tokens just add latency and cost).

Classic Chain-of-Thought Examples

Let's make this concrete. Here is a zero-shot CoT prompt template you can adapt for almost any reasoning task.

Zero-shot CoT template

You are a careful problem solver. Read the question below, then reason
through it step by step out loud before giving your final answer.

Question: {your question here}

Let's think step by step.

And here is a worked example using a classic math word problem.

Worked example: the sheep and dogs problem

Prompt:

Roger has 3 sheep and 2 dogs. Each sheep has 4 legs. Each dog has 4 legs.
How many legs do all of Roger's animals have in total?

Let's think step by step.

Expected reasoning chain:

Step 1: Count the legs from the sheep.
Roger has 3 sheep, and each sheep has 4 legs.
3 × 4 = 12 legs from the sheep.

Step 2: Count the legs from the dogs.
Roger has 2 dogs, and each dog has 4 legs.
2 × 4 = 8 legs from the dogs.

Step 3: Add them together.
12 + 8 = 20.

Final answer: Roger's animals have 20 legs in total.

Critically, a model asked the same question without "Let's think step by step" might say "5 animals × 4 legs = 20 legs" — which happens to be correct here, but only because every animal has four legs. Change one number (say, a dog with 3 legs) and the shortcut answer is wrong, while the explicit chain of reasoning still works.

Few-shot CoT template

Here are some examples of how to solve word problems by reasoning step by step.

Example 1:
Q: A store has 15 apples. A customer buys 4, then the store receives a
delivery of 6 more. How many apples does the store have now?
A: The store starts with 15 apples. A customer buys 4, leaving
15 - 4 = 11 apples. A delivery adds 6, giving 11 + 6 = 17 apples.
The store now has 17 apples.

Example 2:
Q: A train travels 60 miles in 1 hour, then another 90 miles in 2 hours.
What is the train's average speed across the whole trip?
A: The train travels 60 + 90 = 150 miles total. The trip takes
1 + 2 = 3 hours. Average speed = total distance / total time =
150 / 3 = 50 miles per hour. The average speed is 50 mph.

Now solve this question the same way:

Q: {your question here}
A:

The few-shot form is especially useful when the reasoning style matters — when you want the model to show its work in a particular format, check assumptions explicitly, or use a specific notation.

When to Use Chain-of-Thought Prompting

Chain-of-thought is not a universal upgrade. It shines on certain task types and is unnecessary or even counterproductive on others.

Tasks where CoT helps a lot

Multi-step math and arithmetic — word problems, unit conversions, percentage chains, compounding calculations.
Logical and deductive reasoning — syllogisms, constraint-satisfaction puzzles, "who sits where" logic puzzles.
Planning and decomposition — breaking a goal into ordered sub-tasks, drafting a project plan, outlining a multi-part document.
Causal and commonsense reasoning — explaining why something happened, predicting consequences of an action, answering "if X then Y" questions.
Code reasoning — tracing through an algorithm, debugging existing code, explaining what a function does step by step.
Quantitative comparisons — "which option is cheaper over two years," "is A a better investment than B," and similar multi-factor calculations.
Exam preparation and tutoring — solving practice problems with a visible method students can follow and learn from.

Tasks where CoT is usually unnecessary

Simple factual recall — "What is the capital of France?" does not benefit from a reasoning chain.
Single-step classification — sentiment labels, spam detection, simple intent classification.
Creative generation — writing a poem or a marketing tagline; reasoning chains can make the output stiff and over-explained.
Pure lookup or extraction — pulling a field out of a document, summarizing an article.
Any task where correctness is obvious — if the model already answers reliably in one shot, adding a chain only burns tokens and adds latency.

A simple rule of thumb: if the answer requires combining more than one piece of information in an ordered way, CoT probably helps. If the answer is a single fact or a single creative leap, skip it.

How to Write Better Chain-of-Thought Prompts

A bare "let's think step by step" gets you most of the way, but a few refinements make CoT prompting notably more reliable, especially on harder problems.

Tell the model to reason before answering, not to narrate after

A common mistake is asking the model to "explain your reasoning" after giving the answer. By then the answer is already committed and the "explanation" is post-hoc rationalization. Instead, instruct the model to reason first and only produce the final answer at the end.

Weak:

Q: ... A: [final answer]. Explain your reasoning.

Strong:

Reason through this step by step, then give your final answer on a new line
starting with "Answer:".

Constrain the output format

For each step:
1. State what you are computing and why.
2. Show the calculation or logic.
3. State the intermediate result.
Only after all steps, write "Answer:" followed by the final answer.

Constraining the format keeps the chain legible, makes it easier to verify, and stops the model from wandering.

Ask the model to check its own work

For high-stakes problems, add a self-verification step to the prompt:

After you reach an answer, re-examine each step for arithmetic or logic
errors. If you find one, fix it and update the answer. Only give the
final answer once you have checked the chain.

Models are surprisingly good at catching their own mistakes when explicitly prompted to review, and this self-check pass catches the most common source of CoT errors — a single dropped or mis-transcribed number.

Use few-shot examples for specialized reasoning

Example reasoning:
Q: ... A: Step 1: ... Step 2: ... Answer: ...

When the reasoning involves domain-specific steps (legal analysis, medical triage reasoning, financial reconciliation), provide 2-3 worked examples so the model follows your exact method rather than inventing its own.

Break very hard problems into sub-prompts

Step 1 (sub-prompt): Extract the relevant numbers and constraints.
Step 2 (sub-prompt): Set up the equation using those numbers.
Step 3 (sub-prompt): Solve and verify.

For the hardest problems, you can split the chain across multiple prompt calls — this is sometimes called prompt chaining — keeping each step focused and verifiable.

Common Chain-of-Thought Mistakes

Even with the technique in hand, several pitfalls trip up practitioners.

Using CoT on simple tasks. Extra reasoning on single-step problems wastes tokens, adds latency, and occasionally confuses the model into second-guessing a correct answer. Match the technique to the task.
Asking for the answer and the explanation at once. If the model outputs the answer first, the explanation is no longer driving the answer — it is just describing it. Always have the model reason before answering.
Not constraining the reasoning length. Unconstrained CoT can produce meandering, paragraphs-long traces that never converge. Set a step budget ("reason in at most 5 steps") or a format that forces brevity.
Trusting the chain uncritically. A plausible-looking chain can still be wrong at a single step, and the final answer inherits that error. On important problems, verify the key intermediate results yourself.
Copying a famous CoT prompt verbatim. The "let's think step by step" line is a great default, but for specialized domains a custom few-shot example set outperforms it. Tailor the chain to your task.
Forgetting that CoT can increase hallucination on creative tasks. Forcing a model to "reason" about a subjective or open-ended question can produce confident-sounding but arbitrary justifications. Reserve CoT for problems that actually have a correct answer.

Chain-of-Thought Prompting in Practice: A Worked Business Example

To make this concrete, here is a full chain-of-thought prompt you could use for a realistic pricing problem.

You are a pricing analyst. Reason through the following problem step
by step, showing each calculation, and only give the final answer at
the end.

Problem:
A SaaS company charges $30 per user per month on its Pro plan.
Customers who pay annually get a 20% discount on the total. A team
currently has 12 users and wants to know how much they would pay for
one year if they switch from monthly billing to annual billing.

Reason step by step. Show:
1. The total annual cost under monthly billing.
2. The total annual cost under annual billing (with discount).
3. The difference, and which option is cheaper.

End with "Answer:" and the cheaper total on a new line.

A model given this prompt will methodically compute 12 × 30 × 12 = $4,320 for monthly billing, then $4,320 × 0.8 = $3,456 for annual billing, then state the $864 difference and that annual billing is cheaper. Each step is visible, each arithmetic operation is checkable, and the final answer is anchored to the chain. A model asked the same question in a single shot might still get it right, but it might also fumble the discount direction or the month count — a CoT prompt removes most of that risk.

This is the real value of chain-of-thought prompting in professional settings: it turns a black-box answer into a white-box reasoning trace you can audit, debug, and trust.

Variations and Related Techniques

Chain-of-thought has spawned a family of related techniques worth knowing.

Self-consistency

Run the same CoT prompt several times (with non-zero temperature), collect the different reasoning chains, and take the majority answer. Self-consistency reliably beats a single CoT run because correct answers tend to cluster while wrong answers scatter. It costs more tokens but is one of the cheapest reliability boosts available.

Tree-of-thought

Instead of a single linear chain, the model explores multiple reasoning branches, evaluates them, and keeps the most promising. This is more expensive and harder to implement in a plain prompt, but it shines on search-heavy problems like puzzle solving or multi-step planning.

Least-to-most prompting

The model first breaks the problem into a list of sub-questions, then answers each sub-question in order, feeding each answer into the next. This is a structured form of CoT that is especially good on compositional problems where later steps depend on earlier results.

ReAct (reason + act)

The model alternates between reasoning steps and external actions (tool calls, searches, lookups), using each action's result as input to the next reasoning step. ReAct is the backbone of most modern AI agent loops and is essentially chain-of-thought extended to interact with tools and external information.

All of these build on the same insight that made chain-of-thought powerful: explicit, externalized reasoning outperforms implicit, single-shot answers on hard problems.

Putting It All Together

Chain-of-thought prompting is one of the highest-leverage techniques in prompt engineering. It is simple to apply — sometimes a single appended phrase is enough — yet it consistently improves performance on the exact category of tasks where AI models are weakest: multi-step reasoning. The principles fit on one page:

Force the model to reason before it answers, not after.
Use zero-shot CoT ("let's think step by step") as a default, and few-shot CoT for harder or domain-specific tasks.
Constrain the reasoning format so the chain stays legible and checkable.
Add a self-verification step on high-stakes problems.
Reserve CoT for problems that actually have intermediate steps; skip it on simple recall or creative tasks.
Consider self-consistency, tree-of-thought, least-to-most, or ReAct when plain CoT is not enough.

Master chain-of-thought prompting and you will immediately and measurably improve your model's accuracy on math, logic, planning, analysis, and code reasoning — the tasks that actually matter in real work.

If you want to build, test, and deploy prompts like these without hand-editing markdown files, PromptWright gives you a managed environment for prompt versioning, evaluation, and rollout. Sign up for free at promptwright.net/signup and turn chain-of-thought prompting from a copy-pasted snippet into a tested, production-ready asset.

Chain-of-Thought Prompting: A Complete Guide to Step-by-Step AI Reasoning