Prompt and Context Engineering practice questions

Move the system prompt inside the messages array as the first element.

Why it fails: The Messages API strictly separates the system prompt from the messages array. Including the system prompt inside the messages list as a user or assistant role would confuse the model's roles and violate the standard API structure, leading to unpredictable behavior and potentially poor performance.

Repeat the core instructions within the user's turn after the change request.

Why it fails: While this might work, it is inefficient and redundant. The system prompt is specifically designed to persist across the entire conversation. If it is being ignored, the issue lies in the clarity or specificity of the system prompt itself, rather than needing to repeat it manually every turn.

Reduce the number of turns in the messages array before sending the next prompt.

Why it fails: Truncating the conversation history does not solve the persona drift issue. If the model is not adhering to the system prompt, the problem is with the instruction clarity. Reducing context just limits the information available to the model without actually addressing the underlying failure to follow instructions.

Change the system prompt to: 'You are a system administrator who is very angry.'

Why it fails: Changing the persona to an angry administrator does not help the model analyze logs more accurately. In fact, it might lead to non-professional output that includes irrelevant venting, which makes the analysis harder to read and potentially less accurate due to the model's focus on tone over substance.

Put the logs before the instructions.

Why it fails: Putting instructions after the logs is a common cause of failure because the model might lose track of its task after reading thousands of lines of log data. Instructions should always come before the data or be repeated in a 'sandwich' pattern to ensure they remain the primary directive.

Reduce the number of logs to zero.

Why it fails: Reducing the input to zero would mean there is no data to analyze, which is completely counterproductive. The task requires analyzing logs; removing them makes the task impossible. The focus must be on improving the prompt's ability to extract information from the logs, not on removing the data.

Embed the constraints naturally into a single long paragraph.

Why it fails: Paragraph-based instructions are harder for the model to parse. When constraints are buried in text, they are more easily overlooked or forgotten. Itemized lists are superior because they provide visual and structural cues that help the model pay attention to each individual requirement, reducing the risk of missing a constraint.

Hope the model understands the spirit of the constraints.

Why it fails: Relying on the 'spirit' of instructions is a recipe for failure in production. Models perform best with literal, explicit, and structured guidance. Vague instructions force the model to guess your intent, which leads to inconsistent results and frequent violations of even simple constraints.

Ask the user to remind the model of the constraints every turn.

Why it fails: This is inefficient and places the burden on the user. If the system prompt is well-designed, the model should be able to adhere to the constraints throughout the conversation. Requiring user reminders is an admission that the prompt engineering is insufficient, and it degrades the user experience significantly.

Just add 'Be strict' to the system prompt.

Why it fails: Vague instructions like 'Be strict' are interpreted inconsistently by models. They lack the concrete, actionable guidance required to change output behavior. 'Strictness' is subjective, and the model needs a specific, unambiguous rule—like using XML tags or a strict schema—to guarantee that the output format remains consistent.

Ask the user to correct the model if it gets it wrong.

Why it fails: This is a reactive approach that doesn't solve the problem for the next turn. The goal of prompt engineering is to prevent errors from occurring in the first place, not to rely on manual user intervention, which is both expensive and a poor user experience for automated data extraction.

Use a very small max_tokens setting so the model runs out of tokens before it can write extra text.

Why it fails: This is a dangerous approach. If the model runs out of tokens, it will simply truncate the output, potentially cutting off the very data you need. This could result in partial, corrupted, or completely missing data, making it a highly unreliable method for enforcing strict output formatting.

Using the 'top_k' parameter to limit the model to only the most likely next tokens.

Why it fails: Top-k sampling is a technique for controlling output diversity, but it does not directly solve the problem of context recall or instruction following in long prompts. While it can make the output more deterministic, it doesn't help the model better understand or navigate the 100,000 tokens of input provided.

Compressing the context by removing all vowels and punctuation to save tokens.

Why it fails: Compressing text in a way that destroys natural language structure significantly degrades the model's ability to understand the content. Claude's performance is tied to its training on natural language; mangling the input to save tokens will lead to poor reasoning and a high rate of errors or hallucinations.

Switching to a smaller model version like Claude 3 Haiku for better long-context reasoning.

Why it fails: While Haiku is faster and cheaper, it generally has less reasoning capability than larger models like Sonnet or Opus. For very long and complex contexts, the larger models are typically more robust at maintaining instruction adherence and accurate recall, even if they are slower and more expensive to run.

Use high-resolution images even if they exceed the 5MB limit per image.

Why it fails: There are strict technical limits on image size and resolution for the API. Exceeding these limits will result in an error. Developers should resize or compress images to fit within the supported specifications while maintaining enough clarity for the model to identify the relevant features for the task.

Always convert images to grayscale to reduce token usage and improve speed.

Why it fails: Converting images to grayscale does not significantly reduce token usage, as the model's vision encoder processes fixed-size patches regardless of color. Furthermore, removing color can destroy vital information that Claude might need to answer questions correctly, such as identifying specific items, status lights, or UI elements.

Providing the final answer in the prompt and asking the model to justify it.

Why it fails: This is known as 'reverse-engineering' and is not a CoT technique. If the answer is already provided, the model's reasoning may be biased toward justifying that specific answer, even if it is wrong. CoT should involve the model discovering the answer through its own step-by-step logical process.

Setting the temperature to 0 to ensure the model doesn't make any creative errors.

Why it fails: While setting temperature to 0 helps with consistency and determinism, it is a sampling parameter, not a prompting technique. CoT refers to how the prompt is structured to guide the model's internal reasoning process, whereas temperature affects how the model selects tokens from its probability distribution.

Using a very short prompt to avoid distracting the model with extra tokens.

Why it fails: Short prompts are often the opposite of CoT. Reasoning usually requires more tokens, not fewer. By being too brief, you may not give the model enough 'room' or instruction to perform the necessary mental steps, which can lead to the model jumping to an incorrect conclusion too quickly.

The model has reached a higher 'intelligence' level and decided the instruction was irrelevant.

Why it fails: Models do not have the agency to decide which instructions are relevant or irrelevant based on their 'intelligence.' If an instruction is ignored, it is almost always due to a failure in prompt clarity, structural ambiguity, or the instruction being overwhelmed by other competing information in the context window.

The developer forgot to increase the temperature to account for the model's larger size.

Why it fails: Temperature does not need to be increased for larger models; in fact, instruction following usually benefits from lower temperatures. Increasing the temperature would likely make the model even less likely to follow specific instructions as it would introduce more randomness into the output generation process.

The new model requires all instructions to be written in all-caps for emphasis.

Why it fails: Writing in all-caps is not a requirement for any Claude model and does not inherently improve instruction following. While it might occasionally change the weight of certain tokens, it is a poor substitute for proper structural engineering using XML tags and a well-defined system prompt.

Add a long list of negative constraints at the end of the prompt.

Why it fails: Negative constraints are often ignored or counterproductive when overused. They can distract the model from the primary task instruction. It is much more effective to focus on positive instructions and clear schema definitions rather than trying to list every possible way the model might fail to generate JSON.

Set the maximum tokens for the response to exactly 10.

Why it fails: Limiting tokens to 10 will almost certainly truncate the output, making it impossible to produce a complete JSON object. This is a configuration error that prevents the model from fulfilling the request, rather than an engineering strategy to improve the quality or validity of the generated structured content.

Ask the model to generate the data as an XML file instead, as Claude is better at XML.

Why it fails: While Claude handles XML well, switching formats does not solve the underlying issue of output reliability. If the schema is complex, the model needs clear structural guidance regardless of the serialization format. XML is not inherently more 'reliable' than JSON when the prompt instructions lack clear schema definition.

Move the handbook to the end of the prompt and place the user question and instructions after it.

Why it fails: Placing the long document last pushes the actual question and instructions into a trailing position where attention is weaker, and the document still competes for attention across its length. This does not fix the positional degradation for late-document content and can harm instruction adherence. The scenario needs better use of the whole context, not a reordering that buries the query.

Place the long handbook in the middle of the prompt and surround it with the instructions and question at both ends.

Why it fails: Content in the middle of a long context tends to receive the weakest attention, so relocating the handbook there would likely worsen recall for the sections that already fail. Duplicating instructions at both ends adds tokens without restoring attention to mid-document policy text. This approach misunderstands where degradation concentrates and does not improve retrieval of late content.

Split the handbook into two equal prompts sent sequentially, asking Claude to remember the first half when answering the second.

Why it fails: The Messages API is stateless, so the model cannot 'remember' a prior call unless the full history is resent, which reintroduces the same long-context problem. Splitting also risks losing cross-references between sections. This does not address positional attention degradation and adds coordination complexity, so it is not the best fix for the stated symptom.

Increase the temperature setting to 1.0 to encourage more creative exploration.

Why it fails: Increasing temperature adds randomness to the token selection process, which significantly increases the likelihood of hallucinated content. In factual extraction tasks, lower temperatures are preferred to ensure the model makes the most statistically probable choices based on the provided text, minimizing the deviation from source data.

Ask the model to act as a financial expert and provide its own professional analysis.

Why it fails: Assigning an expert persona can improve the tone of the response but does not inherently prevent hallucinations. The model may use its training data to fill in gaps if the source text is incomplete, leading to external information being mixed with the provided input data.

Include a few-shot example that shows the model ignoring missing information.

Why it fails: While few-shot prompting is helpful, examples that show the model ignoring missing info contradict the goal of grounding. The examples must demonstrate strict adherence to the source text, showing the model identifying gaps, rather than teaching it to overlook missing data which could lead to further errors.

Include long, conversational filler text to make the model feel more comfortable.

Why it fails: Conversational filler text increases the token count without adding useful information. This 'prompt bloat' can distract the model and increase latency. Concise, direct instructions are always preferred as they minimize the chance of the model being misled by irrelevant information embedded within the conversational preamble.

Avoid using examples, as they encourage the model to copy rather than think.

Why it fails: Few-shot prompting is one of the most powerful techniques for improving performance. Providing examples helps the model understand the expected format, tone, and logic. Rather than preventing thought, examples provide a concrete baseline that helps the model achieve the desired output quality more reliably and efficiently.

Always set the maximum token count to 4096, regardless of the task.

Why it fails: Setting the maximum token count should be based on the requirements of the specific task. Setting a blanket high limit does not improve performance and can lead to unnecessary costs or unexpected truncations if the model continues outputting irrelevant content beyond what is actually needed for the response.

Ask the user to define the tone in the first prompt.

Why it fails: Relying on the user to define the tone creates inconsistency. Users may forget or choose inappropriate tones. The system prompt should enforce the desired experience, ensuring that the model maintains its intended persona from the very first interaction without needing additional configuration by the end-user.

Randomly change the prompt at every turn to keep the model alert.

Why it fails: Changing the system instructions dynamically during a session will confuse the model and lead to erratic behavior. The system prompt should remain static to maintain a consistent logical framework for the conversation, ensuring that the model's decision-making process is stable and predictable throughout the entire session.

Use the assistant role to remind itself of the tone every turn.

Why it fails: Self-reminders within the conversation history waste tokens and can lead to repetitive, robotic output. The system prompt is the appropriate place for persona instructions; repeating them in the message history is inefficient and detracts from the model's ability to focus on the user's actual request.

Place system instructions at the very end of the user prompt.

Why it fails: Placing system instructions at the end makes them highly vulnerable. Attackers can easily append a prompt injection that overwrites the instructions if they come after the user content. The system instructions must always be in the dedicated system block or at the very beginning of the prompt.

Disable the history feature so the model forgets previous inputs.

Why it fails: Disabling history destroys the model's ability to maintain context, which renders it nearly useless for a chat application. This is not a solution to prompt leakage; it is a fundamental degradation of the application's core functionality and does not prevent the model from leaking instructions in a single turn.

To increase the token limit for the evaluation process.

Why it fails: A hold-out set has no impact on token limits. The evaluation process is about checking the quality of the model's output, not about the technical limits of the API. Using a test set is purely for assessment purposes and does not change the model's operational capacity or parameters.

To ensure the model receives different system instructions every time.

Why it fails: System instructions should be consistent for evaluation. If you change them, you introduce another variable that makes it impossible to determine if the performance improvement is due to the prompt change or the new test input. Consistent prompts are necessary for scientifically valid performance testing.

To reduce the cost of API calls during the testing phase.

Why it fails: Using a hold-out set does not reduce costs. In fact, it requires you to run more evaluations, which increases the total cost. The benefit is in accuracy and reliability, not cost savings. It is a necessary investment for production-grade applications that prioritize stability and long-term performance.

Asking the model to provide only the final answer to save tokens.

Why it fails: Asking for only the final answer forces the model to skip the reasoning phase, which is exactly where errors occur. While it saves tokens, it significantly increases the likelihood of incorrect answers. For complex tasks, the performance benefits of CoT far outweigh the minor cost of the extra tokens.

Providing the model with a massive list of facts without any reasoning steps.

Why it fails: Providing facts without a logical framework does not help the model reason through the task. The model needs a method to process those facts. Without a chain of thought, the model may misinterpret the relationships between facts, leading to flawed conclusions even if the underlying data is accurate.

Using a very high temperature to ensure the model finds a unique reasoning path.

Why it fails: High temperature introduces randomness that disrupts logical chains. Reasoning tasks require high consistency, which is best achieved with a lower temperature. Randomness is the opposite of the structured, systematic approach required for robust reasoning, and will often lead to inconsistent and incorrect answers in complex tasks.

Always put everything in the user field to keep the API call simple.

Why it fails: While simple, putting everything in the user field is bad practice. The model treats system prompts with higher priority and as immutable, whereas user messages are treated as conversational data. This can make the model more susceptible to prompt injections and lead to inconsistent persona and instruction adherence.

Use the system field only for the first message, then switch to user.

Why it fails: The system prompt is not a message in the sequence; it is a parameter of the API call that persists. You cannot 'switch' to user; they coexist. The system prompt should contain all high-level directives that apply globally, and should not be replaced or abandoned as the conversation continues.

Use the system field to store user-specific information.

Why it fails: The system prompt should contain general instructions, not user-specific data. Storing user data in the system prompt would mean you need to change the system prompt for every user, which is inefficient and does not leverage the prompt's purpose of defining global assistant behavior and safety constraints.

Reduce the system prompt to a single word.

Why it fails: A single-word system prompt is insufficient for complex tasks. While it saves tokens, it provides no real guidance, leading the model to default to its base behavior. This sacrifices all control over the model's persona and logic, which is an unacceptable trade-off for such a small token saving.

Use as many synonyms as possible to explain the task.

Why it fails: Using synonyms increases the word count and introduces ambiguity. The goal of prompt engineering is precision. Using multiple words to mean the same thing just confuses the model and wastes tokens. A single, clear, direct word is always better than a list of synonyms for the same concept.

Placing the document content at the very end of the prompt after all instructions.

Why it fails: While placing data at the end can help with recency bias, it does not provide the structural clarity needed for complex extraction. Without clear delimiters, Claude may struggle to identify exactly where the instructions stop and the data begins, potentially leading to the inclusion of instructional text in the extracted results.

Increasing the temperature setting to 1.0 to ensure the model captures nuanced legal language.

Why it fails: High temperature increases randomness and the likelihood of the model deviating from the provided text. For extraction tasks where grounding is required, a lower temperature near 0.0 is preferred to ensure deterministic and faithful output. Raising temperature would likely introduce hallucinations or external legal knowledge not present in the source.

Using a few-shot approach with examples of general legal documents not related to the current task.

Why it fails: Few-shot prompting is powerful, but using unrelated examples can confuse the model regarding the specific domain or format required. Examples should always be representative of the actual task at hand. Providing irrelevant legal context may actually encourage the model to hallucinate or apply incorrect patterns from those unrelated examples.

Setting the top_p parameter to 1.0 to ensure the widest possible range of reasoning paths.

Why it fails: A top_p of 1.0 allows for maximum diversity, which can be detrimental to logic-heavy tasks where precision is required. While it explores many paths, it doesn't inherently improve the quality of reasoning and can lead to less consistent results compared to more constrained sampling methods in technical contexts.

Placing the most important instructions in the middle of a very long prompt to avoid bias.

Why it fails: Claude, like most large language models, can suffer from 'lost in the middle' phenomena where performance degrades for information placed in the center of long contexts. It is far more effective to place critical instructions at the beginning or end of the prompt to ensure they receive the highest attention.

Combining multiple unrelated tasks into a single prompt to maximize token efficiency.

Why it fails: Prompting for multiple unrelated tasks simultaneously often leads to degraded performance on all of them. Claude performs best when focused on a single objective or a set of highly related sub-tasks. Overloading a prompt increases the cognitive load on the model and usually results in missed instructions or errors.