Zero-Shot, Few-Shot, and Chain-of-Thought Prompts

Prompt engineering is the practice of designing input text to guide a large language model (LLM) toward a desired output. The three techniques—zero-shot, few-shot, and chain-of-thought (CoT)—vary in how much context or demonstration you provide. They are not mutually exclusive; you can combine them (e.g., few-shot with chain-of-thought). The AI-900 exam expects you to understand their definitions, use cases, and trade-offs.

Zero-shot prompting means you give the model a task without any examples. The model relies entirely on its pre-training knowledge to infer the desired output. For example, you might input:

Translate the following English sentence to French: 'Hello, how are you?'

The model outputs 'Bonjour, comment allez-vous?' without ever seeing an example of English-to-French translation in the prompt. Zero-shot works well for common tasks like translation, summarization, and simple classification. However, accuracy suffers for niche or ambiguous tasks. The exam tests that zero-shot is the default mode for many Azure OpenAI deployments, but you often need few-shot for better results.

Few-shot prompting provides a small number of examples (typically 2-5) in the prompt to demonstrate the task. The model learns from these examples and generalizes to new inputs. For instance:

Classify the sentiment of the following movie reviews as positive or negative.
Review: 'This movie was fantastic! I loved every minute.'
Sentiment: Positive
Review: 'A waste of time. Boring and predictable.'
Sentiment: Negative
Review: 'The acting was good, but the plot was confusing.'
Sentiment:

Here, two examples guide the model to output 'Positive' or 'Negative' for the third review. Few-shot is especially useful when the task is complex or domain-specific. The exam emphasizes that few-shot requires more tokens (cost and latency increase) but yields higher accuracy. A common wrong answer is to think more examples always improve performance—in reality, 3-5 examples often suffice; too many can confuse the model.

Chain-of-thought (CoT) prompting instructs the model to reason step by step before giving a final answer. This technique dramatically improves performance on arithmetic, logic, and multi-step reasoning tasks. For example:

Roger has 5 tennis balls. He buys 2 more cans of tennis balls. Each can has 3 tennis balls. How many tennis balls does he have now?
Let's think step by step.
Roger started with 5 balls.
2 cans of 3 balls each is 6 balls.
5 + 6 = 11.
The answer is 11.

The phrase 'Let's think step by step' triggers the model to produce intermediate reasoning. CoT can be combined with few-shot by providing examples that include reasoning steps. The exam tests that CoT reduces errors in mathematical and logical tasks but increases token usage. A common trap: candidates think CoT is only for math problems, but it applies to any multi-step reasoning, such as legal analysis or planning.

In Azure OpenAI Service, you configure prompts in the 'System Message' and 'User Message' fields. Zero-shot is the simplest: you just write the user message. Few-shot requires adding examples in the conversation history or within the user message. CoT is typically implemented by including a reasoning instruction in the system message or user message. The exam may ask you to identify which technique is being used given a sample prompt.

When using these techniques via Azure OpenAI's API, you control the model's behavior with parameters:

For few-shot and CoT, you must manage token limits because examples and reasoning steps consume tokens. The exam may ask about the trade-off between accuracy and token usage.

You can test these techniques in Azure OpenAI Studio's Chat Playground. There, you can set system messages, user messages, and adjust parameters. The exam expects you to know that the Chat Playground is the primary tool for prompt engineering experimentation.

These techniques are foundational for retrieval-augmented generation (RAG), where you combine prompts with external data. For example, you might use few-shot prompting to format retrieved documents into a coherent answer. CoT can help the model reason over multiple retrieved pieces of information. The AI-900 exam covers RAG separately but expects you to understand how prompts integrate with it.

At a high level, LLMs process prompts by tokenizing the input, running it through transformer layers, and generating output tokens one by one. Zero-shot relies solely on the model's learned representations. Few-shot adds examples that the model attends to via self-attention, effectively conditioning the output on the pattern. CoT forces the model to allocate attention to intermediate reasoning steps, which improves accuracy on tasks requiring multiple inferential hops. The exam does not require deep knowledge of transformers but expects you to understand these behavioral differences.

A large e-commerce company uses Azure OpenAI to automatically classify customer support tickets into categories like 'Billing', 'Technical Issue', 'Returns', etc. Initially, they used zero-shot prompting: 'Classify the following ticket: ...' Accuracy was only 70% because ticket language varied widely. They switched to few-shot prompting with 3 examples per category. Accuracy rose to 92%. They also added a chain-of-thought instruction for ambiguous tickets: 'Explain your reasoning before giving the category.' This helped the model handle edge cases like 'I want to return my order because the payment failed.' The system processes 10,000 tickets daily. Token costs increased by 15% due to examples and reasoning, but the reduction in manual reclassification saved $50,000/month. Misconfiguration: when they tried 10 examples per category, the context window was exceeded, causing truncation and errors. They learned to keep examples concise and limit to 5.

A law firm uses GPT-4 to summarize contracts. They need summaries that highlight key clauses and potential risks. Zero-shot prompts like 'Summarize this contract' produced generic summaries missing important details. They adopted few-shot prompting with 2 example summaries that included risk analysis. They also used chain-of-thought: 'Read the contract step by step. Identify each clause, then summarize.' This forced the model to list clauses before the final summary. The result was a structured output with bullet points. The firm processes 500 contracts per week. Token usage per contract increased from 2,000 to 4,000 tokens, but accuracy improved from 60% to 95%. A common pitfall was forgetting to clear the conversation history, causing the model to 'remember' previous contracts and produce inconsistent outputs. They now reset the session for each contract.

A software company uses Azure OpenAI to generate code snippets for internal tools. Zero-shot prompts like 'Write a Python function to sort a list' worked well for simple tasks. But for complex tasks like 'Write a function to parse a CSV file and generate a report', zero-shot often produced buggy code. They used few-shot with 2 examples of similar functions, including docstrings and error handling. They also added chain-of-thought: 'First, outline the steps: read CSV, validate data, compute statistics, generate report. Then write the code.' This reduced bugs by 80%. They deploy the generated code after review. The team learned that zero-shot is fine for boilerplate, but few-shot+CoT is essential for complex logic. One mistake was using too high temperature (0.9), which produced creative but incorrect code. They now use temperature 0.2 for code generation.