Prompting and Context Engineering practice questions

At the very beginning of the system prompt only.

Why it fails: System prompts are excellent for setting the persona and high-level rules, but for specific, data-heavy tasks, the model benefits from having task-specific instructions closer to the actual input. Relying solely on the system prompt can lead to the model losing focus when the user input becomes very long.

Embedded randomly within the middle of the input data.

Why it fails: Placing instructions in the middle of data blocks makes them harder for the model to distinguish from the content it is supposed to analyze. This leads to increased confusion and a higher likelihood that the model will treat instructions as part of the data rather than as a command.

In a separate API call before the main request.

Why it fails: Claude does not maintain state between independent API calls unless explicitly managed via a conversation history. Sending instructions in a separate call would result in the model having no knowledge of those instructions when it receives the main data in the subsequent request, causing the task to fail.

Translate the manual into a simpler language like Basic English.

Why it fails: Translating a technical manual into simplified language can lead to the loss of critical technical nuances and terminology. Claude is highly capable of understanding complex, professional-grade language, so simplifying the source text is unnecessary and potentially harmful to the accuracy of the output.

Set the max_tokens parameter to the lowest possible value.

Why it fails: Setting 'max_tokens' too low will cause the model's response to be cut off prematurely. For a complex analysis of a technical manual, the model likely needs a significant number of tokens to provide a complete and accurate answer, making a low limit counterproductive.

Removing all punctuation and special characters from the user's input.

Why it fails: Stripping punctuation is an ineffective security measure that also destroys the semantic meaning of the user's request. Prompt injection often relies on the logic and meaning of words rather than special characters, so this approach fails to address the underlying vulnerability while degrading usability.

Setting the model's temperature to 0 to prevent creative bypasses.

Why it fails: While setting the temperature to 0 makes the model's output more deterministic, it does not prevent the model from following a successful prompt injection. If the injection successfully re-routes the model's logic, a low temperature will simply ensure the model follows that malicious path consistently.

Only allowing users to choose from a pre-defined list of five questions.

Why it fails: While restricting user input to a fixed list is a form of security, it is not a 'prompting strategy' for a generative model; it is a UI-level restriction that eliminates the primary benefit of using an LLM. It avoids the problem rather than mitigating it through engineering.

It allows the model to access external databases during the thinking phase.

Why it fails: The 'thinking' phase is entirely internal to the model's generation process and does not grant it any additional capabilities like external database access. It is a prompting technique to improve logical flow, not a mechanism for expanding the model's available tools or data sources.

It reduces the total number of tokens generated, saving API costs.

Why it fails: Actually, including a thinking phase increases the total number of tokens generated, which will increase the cost of the API call. However, this cost is often justified by the significant improvement in the quality and correctness of the final answer for complex tasks.

It forces the model to use a specific version of its training data.

Why it fails: Prompting techniques cannot influence which version of the training data the model uses. The model's knowledge is fixed at the time of training. Thinking tags only help the model better navigate and apply the knowledge it already possesses in a more logical and structured manner.

Minimize the use of system prompts to ensure the model focuses purely on the user input.

Why it fails: System prompts are the optimal place for setting the persistent constraints, persona, and overall task strategy. Relying solely on user input removes the model's anchor for behavior, making it harder to maintain a consistent persona or strict constraints during the multi-step reasoning process, which usually leads to lower quality.

Provide all possible edge cases in a single large paragraph for the model to handle.

Why it fails: Paragraph-style instructions are difficult for models to parse effectively. Lists, tables, or itemized rules are far superior for defining edge cases. The model is better at attending to structured rules than interpreting dense, unstructured prose, which often leads to the model ignoring some conditions in a long paragraph.

Increase the max_tokens value to allow the model more space to generate a longer summary.

Why it fails: Increasing max_tokens only prevents truncation if the response is cut off, but it does not address the issue of the model intentionally skipping sections. If the model is not generating the content to begin with, increasing the limit will not change the quality or the completeness of the output.

Lower the temperature to zero to make the model more likely to follow instructions.

Why it fails: While lowering temperature makes output more deterministic, it does not fundamentally change how the model attends to the input document. If the underlying prompt does not explicitly instruct the model to pay attention to every section, a zero temperature will just result in the same incomplete, deterministic summary.

Add a few-shot example that shows the model summarizing a completely different type of document.

Why it fails: Few-shot examples are most effective when they are relevant to the specific domain and task. Providing examples of different document types can introduce noise and confuse the model's internal expectations about the structure of legal contracts, potentially leading to more errors rather than fixing the original skipping problem.

Use thousands of examples to ensure the model learns every possible user permutation.

Why it fails: Large sets of few-shot examples consume excessive tokens and lead to prompt bloat, which can degrade performance. LLMs are excellent at pattern matching and usually require only a handful (3-5) of high-quality, diverse examples to understand the pattern, task, and desired output format effectively.

Use examples that are completely distinct from the target task to reduce bias.

Why it fails: Examples must be relevant to the target task. If the examples are distinct from the task, the model will be confused about what it is supposed to be doing. Relevant examples provide the necessary context to guide the model's reasoning and ensure it understands the specific domain constraints.

Always put the examples at the very end of the prompt to maximize recency.

Why it fails: While recency is important, the best practice is to place examples logically, often after the initial system instructions and before the actual task input. Placing them at the very end can sometimes cause the model to confuse the example data with the actual current task input if not properly delimited.

Set the temperature to zero to force the model to be more direct.

Why it fails: Lowering the temperature reduces randomness but does not force the model to be concise. A deterministic model can still be very verbose if the prompt does not explicitly forbid it. You must provide clear instructions about length to effectively control the verbosity of the generated response.

Use the API parameter 'max_tokens' to cut off the response mid-sentence.

Why it fails: Cutting off a response with max_tokens results in incomplete and poor-quality data. It does not teach the model to be concise; it just causes the generation to stop abruptly. This will lead to broken JSON or incomplete sentences, which will fail any downstream parsing processes.

Include a long list of synonyms for 'concise' in the prompt to emphasize your point.

Why it fails: Over-emphasizing a concept with many synonyms can actually confuse the model and increase the likelihood of it hallucinating or ignoring the primary instruction. A single, clear, and direct directive is much more effective than providing an verbose list of adjectives that do not add functional clarity.

In the system prompt only, to set a global behavior for all subsequent turns.

Why it fails: While setting behavior in the system prompt is useful, for specific complex tasks, it is often more effective to include the 'think step-by-step' instruction within the user message context. This ensures the model applies the reasoning specifically to the current data rather than relying on a general behavioral instruction.

Inside an XML tag at the beginning of the prompt to establish the task early.

Why it fails: Placing reasoning instructions at the beginning can sometimes be less effective because the model may start generating the answer before fully 'digesting' the large amount of data that follows. In long-context scenarios, Claude performs best when the specific call-to-action is located near the end of the input.

Hidden within the few-shot examples without explicitly stating it in the instructions.

Why it fails: While few-shot examples can demonstrate reasoning, being explicit is always better in prompt engineering. Depending on the model to infer a step-by-step reasoning requirement solely from examples is less reliable than providing a direct instruction, especially for complex tasks that vary significantly from the provided examples.

Using only standard HTML5 tags to ensure the model's parser recognizes them.

Why it fails: Claude does not require standard HTML5 tags; in fact, custom tags like <document_content> or <legal_clause> are often more effective because they describe the semantic nature of the content. The model is trained to handle flexible XML-style structures, so developers should prioritize descriptive tag names over web standards.

Encoding the XML tags in Base64 to prevent the model from seeing them as text.

Why it fails: Encoding tags in Base64 would make them invisible and unreadable to Claude, defeating the entire purpose of using them for structure. The model needs to see the tags as plain text to use them as delimiters. Base64 encoding is used for binary data like images, not for structural prompting.

Set 'max_tokens' to a very low number to force the model to be concise.

Why it fails: Low 'max_tokens' will likely truncate the response before the JSON list is finished, especially if the model is also providing reasoning. This doesn't help with ambiguity; it just results in incomplete and likely invalid JSON that cannot be parsed by the developer's application.

Use a system prompt to tell the model 'You are a perfect data entry clerk.'

Why it fails: While role prompting can set a general tone, calling the model 'perfect' does not provide it with the technical tools or logical framework needed to resolve actual ambiguities in the data. Concrete strategies like CoT and schema definition are far more effective than simple persona-based adjectives.

Remove all XML tags and use only plain text to avoid distracting the model.

Why it fails: Removing XML tags actually makes the model's job harder. Without delimiters, the model may struggle to separate the instructions from the messy data. XML tags are a recommended best practice for Claude because they provide the structural clarity that helps the model focus on the relevant information.

Always set the temperature to 1.0 to increase recall.

Why it fails: Setting the temperature to 1.0 does not improve context recall and often introduces noise. For tasks requiring synthesis of long documents, lower temperatures provide more stable and reliable results, whereas high temperature can lead to hallucinations or inconsistent focus when dealing with complex, multi-part factual information.

Summarize the entire document into a single line before prompting.

Why it fails: Summarizing the document into a single line destroys the granular information that the context window is intended to preserve. Claude excels at processing large contexts natively, so manual summarization is unnecessary and often counterproductive as it removes the details required for specific queries or deep analysis tasks.

Repeat the instructions at the beginning and end of the document.

Why it fails: While repetition is sometimes used in other models, it is generally unnecessary for Claude and can lead to increased token usage without significant performance gains. It is more effective to use clear, well-structured system prompts and place the primary task instruction at the end of the document.

Set the temperature to 0 to maximize creativity in the reasoning.

Why it fails: A temperature of 0 minimizes randomness, which is desirable for reasoning tasks, but the term 'maximize creativity' is incorrect. Reasoning tasks require logical consistency, not creativity. The goal is to obtain the most probable, accurate path, and high temperature would only introduce risks of logical drift and errors.

Keep all instructions in a single, unformatted paragraph.

Why it fails: Formatting instructions into a single, dense paragraph makes them difficult for the model to parse and prioritize. Using sections, bullet points, and clear delimiters makes the instructions more accessible to the model, reducing the chance that key instructions are ignored or misunderstood during the generation of the response.

Store the system prompt in a variable that is accessible to the user.

Why it fails: Exposing the system prompt to the user is a major security risk. The system prompt should remain hidden from the user to prevent them from understanding the internal logic and identifying potential vulnerabilities. Making it accessible invites users to attempt to override or exploit the defined model behavior.

Place all system instructions in the user message to ensure visibility.

Why it fails: Mixing system instructions with user messages is highly insecure and creates a major vulnerability. System instructions must reside in the designated system prompt field, which is logically separated from the user input. Placing them in the user message allows users to easily overwrite or ignore the system's core instructions.

Use an informal tone in the prompt to make the model feel more relaxed and natural.

Why it fails: An informal tone can introduce ambiguity and affect the model's adherence to strict output requirements. When structural consistency is required, a direct, professional, and explicit tone is far more effective at ensuring the model respects the defined output schema without unnecessary conversational filler or stylistic deviations.

Instruct the model to 'try its best' to format the output as JSON consistently.

Why it fails: Vague instructions like 'try your best' provide no actual constraints to the model. Claude performs best with clear, directive instructions. For reliable JSON output, the prompt must explicitly define the expected fields and types, rather than relying on the model to guess the intended format.

Include the entire document as a raw string without any structural delimiters or tags.

Why it fails: Providing raw strings without delimiters makes it difficult for the model to parse the structure of the document. Using tags like <document> or <context> significantly improves the model's ability to maintain focus and prevents it from misinterpreting parts of the document as instructions for the task.

Include a brief summary of the document within the system prompt to guide the model.

Why it fails: Providing a summary in the prompt can introduce bias or 'priming' that causes the model to ignore nuances in the original source text. It is generally more effective to rely on the model's own analysis of the provided text rather than steering it with a pre-written summary.

Place the instructions for the summary task after the document text to keep the prompt clean.

Why it fails: Placing instructions after a long document can lead to 'lost in the middle' issues where the model overlooks the task requirements. Best practice dictates placing instructions at the beginning of the prompt to ensure they remain the highest priority context for the model to follow during generation.

Use a system prompt that instructs Claude to always output valid JSON.

Why it fails: While a system prompt can set a general directive, it does not specify the exact schema or how to handle missing fields. Claude might output valid JSON but with different key names or structures across reports. Without a concrete example, consistency is not guaranteed, making this less effective than providing a sample output.

Ask Claude to think step-by-step before producing the JSON output.

Why it fails: Chain-of-thought prompting can improve reasoning for complex tasks, but it does not guarantee a specific output format. Without an explicit schema example, Claude might still produce inconsistent JSON structures. The step-by-step reasoning may even introduce extraneous text that breaks JSON parsing, so it is not the best choice for format consistency.

Provide a detailed natural language description of the desired JSON schema in the prompt.

Why it fails: A natural language description alone is ambiguous and may lead to inconsistent field names or formatting. Claude might interpret variations differently across reports, especially when figures are missing. This approach lacks the concrete example needed to enforce exact key names and null handling, making it less reliable than providing an explicit template or example output.

Move the policy document to the end of the prompt, after the email and instruction.

Why it fails: Moving the policy document to the end places the instruction before the long context, which reduces the model's ability to attend to the task directive. Long inputs are best ordered with the document first and the query last, so this change would likely worsen the summaries rather than improve them.

Repeat the summarization instruction at both the beginning and the end of the prompt.

Why it fails: Duplicating the instruction may help slightly, but it does not address the root cause: long context placed before the instruction can dilute the model's focus. Without restructuring the input order, repetition alone rarely resolves missed details in policy-heavy tasks.

Place the email and instruction first, then the policy document, and ask Claude to reference the policy after reading it.

Why it fails: Putting the email and instruction before the policy document inverts the recommended long-context pattern. The policy becomes trailing context that the model may under-weight, so summaries would still miss key policy details even if Claude is asked to reference them.

Add 'Do not be generic' to the system prompt.

Why it fails: Negative constraints are often less effective than positive examples or clear instructions on what the model should actually do. Simply telling a model what to avoid does not provide a clear path for the desired behavior, often leading to unpredictable results or the model ignoring the constraint entirely during generation.

Set the temperature to 1.0 to encourage creative writing.

Why it fails: Increasing the temperature to a high value like 1.0 makes the model's output more random and less predictable. While this might increase creativity, it actively works against the goal of maintaining a strict and consistent brand voice, as the model is more likely to choose less probable tokens.

Use a single-sentence instruction at the very end of the prompt.

Why it fails: Placing instructions at the end can help with recency bias, but a single sentence lacks the necessary depth to convey a complex brand identity. Without detailed context or examples, the model has no baseline to understand what the specific voice entails, resulting in generic and inconsistent outputs.

Setting top_k to a value of 1 to minimize output variety.

Why it fails: While setting top_k to 1 makes the output deterministic, it does not improve the model's internal ability to extract or understand context. In fact, overly restrictive sampling can sometimes prevent the model from articulating complex reasoning steps necessary to correctly synthesize information from a large dataset.

Converting all text to uppercase to improve character recognition.

Why it fails: Converting text to uppercase actually removes important semantic information and casing cues that help the model understand language. It does nothing to improve performance and may actually degrade the model's ability to process the text naturally, as it deviates from the standard training data format.

Reducing the temperature to exactly 0.5 for balanced reasoning.

Why it fails: A temperature of 0.5 is a general-purpose setting and does not specifically optimize for context extraction. For high-precision extraction tasks, a lower temperature closer to 0 is generally preferred, but temperature adjustment alone is far less impactful than structural changes like XML tagging or CoT.