Prompt Engineering practice questions

Temperature reduction to zero

Why it fails: Reducing the temperature parameter ensures the model selects the most probable next token, which increases consistency across multiple runs. However, this does not fundamentally change the reasoning architecture of the response. For complex log analysis, deterministic output might still lack the depth of reasoning required to identify root causes without explicit logical instructions to the model.

Chain-of-Thought (CoT) prompting

Why it fails: Requesting a step-by-step breakdown forces the model to process internal logic before arriving at a conclusion. This technique is particularly effective for multi-step reasoning tasks like log analysis where errors are hidden within sequential events. It helps the model maintain context and ensures that every part of the final answer is justified by the preceding logical steps.

Increasing the frequency penalty

Why it fails: Frequency penalties are used to discourage the model from repeating the same words or phrases too often in a single response. While this can make the output feel more varied and natural, it does not improve the model's ability to perform logical deduction. In a technical log context, it might actually be detrimental by discouraging the repetition of necessary technical terms.

Few-shot prompting with summaries

Why it fails: Providing examples of successful summaries helps the model understand the desired output format and style. While few-shot learning is powerful for pattern recognition, it does not inherently force the model to perform the detailed reasoning needed for root cause analysis. Without a chain-of-thought component, the model might still skip critical diagnostic steps despite seeing similar examples.

Using distinct delimiters for user input

Why it fails: Using markers like triple backticks or XML tags clearly separates the developer's instructions from the user's provided data. This structural boundary makes it significantly harder for a user to inject commands that the model might interpret as new instructions. It acts as a clear signal to the model about which parts of the prompt are static and which are variable.

Implementing recursive character splitting

Why it fails: Recursive character splitting is a technique primarily used in Retrieval-Augmented Generation to break down large documents into manageable chunks for embedding. It does not provide any security benefits against prompt injection or instruction hijacking. Using this method in a prompt engineering context would be irrelevant to the goal of maintaining model control and preventing malicious user overrides.

Placing user input before the system instructions

Why it fails: Positioning untrusted user input at the beginning of a prompt often makes the model more susceptible to injection attacks. Models typically place higher weight on the most recent instructions they have processed. By placing user content first, the developer risks having the subsequent system instructions ignored or overridden by malicious commands embedded early in the prompt structure by an attacker.

Utilizing a robust system-level prompt

Why it fails: The system prompt is the primary mechanism for defining the model's identity, tone, and safety constraints. By explicitly stating that the model must ignore any instructions contained within user-provided text that contradict its core mission, the developer creates a strong behavioral baseline. This serves as a critical defense layer that persists throughout the interaction, ensuring the model's primary directives remain active.

Applying a high Top-p value

Why it fails: Top-p sampling, or nucleus sampling, controls the diversity of the model's output by selecting from a cumulative probability threshold. While it affects the creativity and variety of the generated text, it has no impact on the security or robustness of the prompt against injection. High Top-p values might even lead to less predictable behavior, which is generally undesirable in high-stakes security contexts.

Presence penalty

Why it fails: Presence penalty is used to encourage the model to talk about new topics by penalizing tokens that have already appeared in the text. While this can prevent repetition within a single long output, it does not necessarily increase the overall 'creativity' or randomness of the initial token selection. It is more about expanding the breadth of content rather than the stylistic variety of individual phrases.

Temperature

Why it fails: Adjusting the temperature scales the logits before the softmax layer, directly influencing the model's tendency to choose lower-probability tokens. A higher value increases entropy, leading to more diverse and unexpected slogans. This is the standard mechanism for introducing stochasticity into the model's generation process, making it ideal for creative tasks where a single 'correct' or highly probable answer is not desired.

Max tokens

Why it fails: The max tokens parameter sets a hard limit on the length of the generated response but does not influence the content or style of the text itself. Increasing this value would simply allow the model to generate longer slogans or more of them, but the slogans themselves would remain repetitive if the underlying randomness controlled by temperature or top-p is not also increased accordingly.

Stop sequences

Why it fails: Stop sequences are specific strings that signal the model to cease generation immediately upon their appearance. They are useful for controlling the structure of the output, such as stopping after a single sentence or a list item. However, they do not affect the diversity of the tokens generated before the stop sequence is reached, making them ineffective for increasing the creativity of the slogans.

Label distribution balance

Why it fails: If a prompt contains mostly examples for one specific category, the model is likely to become biased and over-predict that category for new inputs. Maintaining an even distribution of labels across the few-shot examples ensures the model treats all classification options with appropriate weight. This balance is crucial for achieving high accuracy and fairness in multi-class classification tasks using in-context learning.

Example diversity and coverage

Why it fails: Examples should cover a wide range of linguistic styles and potential scenarios the model might encounter. If all examples are short and formal, the model may struggle with long or informal user inputs. Including diverse examples helps the model understand the boundaries between different categories more effectively, leading to more robust classification performance across various types of real-world customer feedback.

Semantic similarity to the target input

Why it fails: Selecting examples that are semantically similar to the current user query can significantly improve the model's performance. This technique, often used in dynamic few-shot selection, helps the model find the most relevant context for the specific problem it is solving. When the examples mirror the complexity and vocabulary of the input, the model's ability to map the input to the correct label is enhanced.

Alphabetical ordering of examples

Why it fails: The order of examples can influence the model's output due to recency bias, but alphabetical ordering is generally irrelevant to the model's understanding of the task logic. While the sequence matters, organizing examples by the first letter of their text does not provide any semantic or structural benefit that would improve classification accuracy. It is a random heuristic that does not contribute to the model's learning process.

Total character count of all examples

Why it fails: While total length is a consideration for context window limits, the specific character count itself is not a primary driver of example effectiveness. A prompt could be very long but contain low-quality or irrelevant examples that actually degrade performance. The focus should always be on the quality and representativeness of the content rather than hitting a specific character target, provided the total remains within the model's limits.

Least-to-most prompting

Why it fails: Least-to-most prompting involves breaking a complex problem into a series of simpler sub-problems and solving them sequentially. While this is effective for decomposition, it does not involve generating multiple parallel reasoning paths or using a majority vote to determine the final answer. It is a linear strategy focused on reducing task complexity rather than using consensus to increase the reliability of the output.

Self-consistency prompting

Why it fails: Self-consistency leverages the intuition that a complex problem often has multiple ways to reach the correct answer. By sampling diverse reasoning paths and taking a majority vote, the technique significantly improves performance on arithmetic and symbolic reasoning. It effectively filters out 'fluke' errors that might occur in a single greedy decode or a single Chain-of-Thought path, leading to much more robust results.

Tree-of-Thought prompting

Why it fails: Tree-of-Thought (ToT) is a framework where the model explores a tree of potential solutions, evaluating each step and backtracking if necessary. While it involves multiple paths, it is focused on a search and heuristic evaluation process rather than a simple majority vote on the final answer. ToT is generally more complex to implement and involves explicit state evaluation at each node of the reasoning tree.

ReAct prompting

Why it fails: ReAct combines reasoning and acting by allowing the model to generate reasoning traces and then execute external actions, such as searching a database. This is a dynamic, tool-use-oriented strategy. It does not rely on generating multiple reasoning paths for the same static problem to find a consensus answer, but rather on the interaction between the model's thoughts and the information it retrieves from its environment.

Consistent tone and stylistic alignment

Why it fails: Defining a persona, such as a 'friendly and helpful guide', ensures that the model maintains a specific emotional resonance and vocabulary throughout the interaction. This prevents the model from shifting between overly formal and overly casual language, which can happen in long conversations. It creates a more professional and coherent user experience that aligns with the brand's intended voice and communication standards.

Reduction in total inference latency

Why it fails: Persona prompting does not reduce the computational resources required for inference or the time it takes for the model to generate tokens. In fact, adding a persona description slightly increases the prompt length, which could technically lead to a marginal increase in processing time. Latency is primarily a function of model size, hardware performance, and the total number of input and output tokens.

Implicit context for domain-specific jargon

Why it fails: By telling the model it is a 'Senior Network Engineer', it will naturally prioritize technical terminology and explanations appropriate for that role. This reduces the need for the developer to explicitly define every term in the prompt. The model draws on its pre-trained knowledge associated with that persona to provide more accurate and contextually relevant answers that match the expertise expected of that role.

Elimination of the need for stop sequences

Why it fails: Persona prompting and stop sequences serve different purposes. A persona defines the 'how' and 'what' of the content, while stop sequences define 'where' the generation should end. Even with a well-defined persona, a model might continue to generate text beyond the desired length or structure. Therefore, stop sequences remain a necessary tool for controlling the physical boundaries of the model's response in many applications.

Automatic verification of factual accuracy

Why it fails: While a persona might make a model 'sound' more authoritative, it does not provide any mechanism for verifying the truthfulness of the statements it makes. A model role-playing as an expert can still hallucinate incorrect information with great confidence. Factual verification requires separate techniques such as RAG or external validation steps, as persona-based prompting is purely a stylistic and contextual steering mechanism for the generation.

The Instruction

Why it fails: The instruction part of a prompt tells the model what action to perform, such as 'summarize' or 'translate'. While it directs the model's task, it does not typically contain the raw data or external information required to complete that task. The instruction acts as the verb of the prompt, while the context acts as the noun or the source material that the verb operates upon.

The Context

Why it fails: Context serves as the reference material for the model. In a RAG pipeline, this is the section where the results of a vector search are placed. By providing this information, the model can answer questions about specific documents or data points it hasn't seen before. It is the foundation for grounded generation and is critical for maintaining accuracy in domain-specific applications.

The Output Indicator

Why it fails: An output indicator is a small hint at the end of a prompt, like 'Answer:', that helps the model start its response in the right direction. It is a structural tool to ensure the model doesn't include conversational filler before the actual answer. It does not provide the model with any factual information or external data needed to generate the content of the response itself.

The Few-shot Examples

Why it fails: Few-shot examples provide patterns of input and output to show the model how to format its response or handle specific logic. While they provide 'context' in a general sense, they are not the primary place where dynamic, query-specific information is placed. Few-shot examples are usually static across many queries, whereas the context block changes based on the specific question being asked by the user.

It generates the final answer for the user

Why it fails: The final answer is usually produced in a 'Final Answer' or 'Thought' step after the model has gathered enough information. The observation step is an intermediate phase where the model consumes data from an external tool or environment. It is not the conclusion of the process but rather a mid-point that informs the subsequent reasoning cycles until a final conclusion can be reached.

It provides the model with the result of an action

Why it fails: In the ReAct loop, the model first thinks, then acts (e.g., by calling a search API). The 'Observation' is the raw output from that API call. This information is then appended to the prompt, allowing the model to 'see' what happened. This external feedback is what enables the model to solve complex problems that require up-to-date information or multi-step interactions with external systems.

It defines the persona for the reasoning trace

Why it fails: Persona definition is typically handled in the system prompt at the beginning of the interaction and remains constant. The observation step is dynamic and changes based on the actions the model takes. While the persona might influence how the model 'thinks' about an observation, the observation itself is the data returned from a tool, not a stylistic instruction for the model's behavior.

It is used to penalize repetitive reasoning paths

Why it fails: Penalizing repetition is handled by hyperparameters like frequency and presence penalties, or by specific logic in the reasoning step that detects loops. The observation step is purely informative, providing the model with new data. If the model repeats the same action and gets the same observation, it is the model's 'Thought' process that must decide how to break the loop based on that observation.

Instructing the model to say 'I don't know' if unsure

Why it fails: Giving the model an 'out' is one of the most effective ways to prevent it from making things up. When a model is told it is acceptable to admit it doesn't have the answer based on the provided text, it is less likely to fill in gaps with plausible-sounding but false information. This constraint significantly improves the reliability of the model in information-retrieval tasks.

Adding a requirement for direct quotes and citations

Why it fails: Forcing the model to provide evidence for its statements grounds the output in the source material. If the model must cite a specific section or quote from the legal document, it is much harder for it to hallucinate entirely new facts. This requirement also makes it easier for human reviewers to verify the model's work, as they can directly check the provided references against the original text.

Increasing the Top-k parameter to 100

Why it fails: Top-k sampling limits the model's choices to the most likely next tokens. Increasing this value to 100 actually increases the pool of potential tokens the model can choose from, which can lead to more diverse but also potentially more erratic or hallucinated output. For accuracy-focused tasks like legal summarization, a lower Top-k or Top-p is usually preferred to keep the model's focus on the most probable and grounded tokens.

Removing all few-shot examples from the prompt

Why it fails: Few-shot examples generally help reduce hallucinations by demonstrating the correct way to handle information and format responses. Removing them takes away valuable guidance, making the model more likely to revert to its default generative behavior, which is more prone to fabrications. Examples provide a 'pattern' of groundedness that the model can follow, so removing them is usually counterproductive when trying to increase accuracy.

Using a very high temperature setting

Why it fails: High temperature increases the randomness of the model's output, which is the opposite of what is needed to reduce hallucinations. High randomness encourages the model to select less likely tokens, which often leads to creative but factually incorrect statements. For factual summarization, the temperature should be kept low (near 0) to ensure the model remains as deterministic and faithful to the source text as possible.

Chain-of-Thought prompting

Why it fails: While Chain-of-Thought also involves step-by-step reasoning, it typically does so in a single pass where the model explains its logic while answering. Least-to-most prompting is more explicit about the decomposition phase, often involving a two-stage process: first generating the sub-problems, and then solving them. It is specifically targeted at tasks where the model might fail to see the necessary sequence of operations upfront.

Least-to-most prompting

Why it fails: This technique is highly effective for complex reasoning, such as math word problems or long-form coding tasks. It works by prompting the model to list all the components needed to solve the main query. Once the sub-problems are identified, the model is prompted to solve them one by one, building the final answer from these smaller, verified blocks of logic, which significantly increases the overall success rate.

Zero-shot prompting

Why it fails: Zero-shot prompting involves giving the model a task without any examples or intermediate reasoning steps. It relies entirely on the model's pre-trained knowledge and its ability to understand the instruction in one go. This is the opposite of least-to-most prompting, as it does not encourage any form of decomposition or structured problem-solving, making it less effective for highly complex or multi-stage technical challenges.

Negative prompting

Why it fails: Negative prompting is a technique where the user specifies what the model should NOT include in its output. While useful for filtering out unwanted styles or content, it does nothing to help the model solve complex problems or break them down into smaller pieces. It is a constraint-based method rather than a reasoning-based method, and it is primarily used in image generation or stylistic text control.

Increasing the max token limit

Why it fails: The max token limit only controls how long the model's response can be. It does not provide any additional information to the model about the database structure. If the model doesn't know the schema, a longer response will just contain more incorrect or hallucinated SQL code. Max tokens is a resource management parameter, not a technique for improving the factual grounding of a specialized technical task.

Adding a 'think step-by-step' instruction

Why it fails: While 'think step-by-step' can help the model reason through the logic of a JOIN or a WHERE clause, it cannot help the model if it doesn't know the names of the tables and columns. Logical reasoning is distinct from factual knowledge. Even with perfect logic, the model will fail to produce executable SQL if it is guessing the schema, making this technique secondary to providing the actual schema context.

Including the DDL (Data Definition Language) in the context

Why it fails: Including the 'CREATE TABLE' statements or a similar schema description provides the model with a precise map of the database. This allows the model to correctly identify column names, data types, and foreign key relationships. By grounding the prompt with the actual DDL, the developer eliminates the model's need to guess, which is the primary cause of errors in natural-language-to-SQL tasks.

Using a more creative persona for the model

Why it fails: For technical tasks like SQL generation, creativity is usually a disadvantage. A 'creative' model might try to invent more elegant but non-existent table names or suggest clever but invalid syntax. For this use case, the model should be steered toward being a 'precise and literal database administrator' rather than anything creative. However, even a precise persona still needs the actual schema data to be effective.

To help the model distinguish between instructions and data

Why it fails: When a prompt contains a large amount of text, the model can lose track of where the rules end and the information begins. Delimiters act as clear boundaries, signaling to the model that the text following a specific marker is the data to be processed, rather than a continuation of the command. This clarity is essential for complex tasks like summarization or entity extraction from long files.

To reduce the total number of tokens used in the prompt

Why it fails: Delimiters actually add a small number of tokens to the prompt. While the impact is minimal, their purpose is organizational rather than efficiency-oriented. To reduce token count, one would use techniques like prompt compression, summarization of the context, or removing redundant instructions. Delimiters are used to improve the quality and reliability of the model's understanding, not to minimize the footprint of the input.

To prevent the model from following commands inside the data

Why it fails: Source documents might contain phrases that look like instructions, such as 'Summarize this section'. If these are not clearly delimited, the model might get confused and try to execute those internal commands instead of the primary task. Delimiters help the model understand that everything within the specified block is 'content' and should be treated as passive data rather than active instructions for the model's behavior.

To bypass the model's safety and content filters

Why it fails: Delimiters have no effect on the underlying safety filters or guardrails of a model. Those systems operate on the entire prompt and the generated output regardless of how the text is partitioned. Using delimiters to try and hide malicious intent is generally ineffective, as modern safety layers are designed to analyze the semantic meaning of the text rather than being fooled by simple structural markers like triple hashes.

To automatically translate the prompt into multiple languages

Why it fails: Delimiters are static characters and do not have any inherent translation capabilities. Translation is a task performed by the model itself based on instructions, not a function of how the prompt is partitioned. While you could use delimiters to separate different language versions of a text, the markers themselves do not facilitate the translation process or change how the model's multi-lingual capabilities are activated during inference.

Changing the temperature to 1.0

Why it fails: Increasing the temperature makes the model more random and potentially more wordy as it explores less probable token paths. This is the opposite of what is needed for conciseness. A high temperature might lead to more creative phrasing, but it usually results in less control over the length and structure of the output, making it an inappropriate choice for a task that requires strict adherence to a specific style.

Using a negative prompt like 'Do not use many words'

Why it fails: Negative prompts are often less effective than positive, specific instructions. Telling a model what NOT to do doesn't always provide a clear path for what it SHOULD do. A model might avoid 'many words' but still fail to reach the desired level of brevity because 'many' is subjective. Specific, positive constraints are generally more reliable for steering the model's output toward a particular structural goal.

Providing a few-shot example of a concise rewrite

Why it fails: Few-shot examples are very effective for showing the model the exact style and length you expect. By providing a 'Before' and 'After' pair where the 'After' is both professional and significantly shorter, the model can learn the specific transformation you are looking for. This grounds the subjective instruction in a concrete pattern that the model can then apply to the new paragraph with higher accuracy.

Repeating the word 'concise' five times in the prompt

Why it fails: Keyword stuffing or repetition in a prompt rarely improves performance and can sometimes confuse the model or cause it to focus too much on the repeated word at the expense of other instructions. It doesn't provide the model with any new information about how to achieve conciseness. Clearer instructions or examples are far more effective than simply increasing the frequency of a single descriptor within the prompt text.

Instruct the model to ignore context if it already knows the answer.

Why it fails: Encouraging a model to use its internal parametric knowledge instead of provided RAG context increases the risk of hallucination. The goal of RAG is to anchor the model to factual, external data; allowing it to prioritize pre-trained data defeats the purpose of the retrieval infrastructure.

Set the sampling temperature to 1.5 to maximize creativity.

Why it fails: High temperature values increase randomness and creative flair, which is counterproductive for factual accuracy. In RAG applications requiring grounded responses, temperature should typically be kept low, near 0.0 or 0.1, to ensure the model adheres strictly to the provided context rather than generating novel, potentially incorrect information.

Increase the number of context documents to over 100 for every query.

Why it fails: Adding too much context leads to 'lost in the middle' phenomena, where the model ignores critical information. Providing excessive, irrelevant data increases noise, which actually degrades the model's performance and potentially increases hallucination rates by forcing the model to sort through irrelevant, confusing, or contradictory information.

Use highly complex, ambiguous language to test robustness.

Why it fails: Ambiguous language introduces uncertainty into the model's interpretation, which is the opposite of what is needed for safe guardrails. Clarity and precision in prompt engineering are essential to ensure the model adheres to guidelines; complexity creates failure points and unpredictable outcomes that threaten the stability of the system.

Encourage the model to ignore user input if it sounds frustrated.

Why it fails: Ignoring user sentiment or frustration can lead to poor user experiences and does not address the actual safety or goal-alignment requirement. Guardrails should handle tone appropriately rather than simply discarding user input, as this degrades the quality of the service and fails to address the user's underlying needs.

A way to increase the model's token limit.

Why it fails: Delimiters are structural tools used to organize text; they do not impact the hardware-defined token limits or the architecture of the model. They are purely a formatting technique to enhance clarity for the model's attention mechanism, making the input structure explicit so the model can parse information accurately.

A method to compress text data before sending to the GPU.

Why it fails: Delimiters do not perform data compression. They are simple characters that actually consume tokens. Using them makes the prompt longer, not shorter. Compression is a function of algorithms or protocols, while delimiters are meant for semantic clarity and structural separation to guide the model's parsing logic effectively.

A method for adjusting the model's sampling temperature.

Why it fails: Sampling temperature is a parameter controlled via the API call or configuration file, not by the content of the prompt text. Delimiters have zero impact on the stochastic nature of the output generation process; they only help the model understand the boundaries of the provided data segments.

It forces the model to use more GPU memory per token.

Why it fails: Intermediate reasoning steps do increase token count, but they do not inherently change the GPU memory allocation per token. Memory constraints are defined by the model architecture and context window size rather than the specific prompt strategy employed during the inference phase of the NeMo deployment.

It increases the likelihood of the model selecting a random seed.

Why it fails: The random seed is a fixed parameter set in the inference configuration, not a variable manipulated by the model's output content. CoT prompting focuses on structured reasoning outputs, which remains entirely orthogonal to the underlying sampling parameters like temperature or top-p sampling used during execution.

It eliminates the need for system-level instructions entirely.

Why it fails: System instructions are foundational to defining the behavior and constraints of the model. CoT prompting is a technique for eliciting better reasoning from a model, not a replacement for fundamental safety guidelines or task-specific directives provided in the system prompt of the NVIDIA model instance.

Append 'Be as creative as possible' to the system prompt.

Why it fails: Increasing creativity in a technical support context is dangerous, as it promotes hallucination. Technical manuals require high precision and factual adherence; prompting for creativity encourages the model to drift away from the source material, which is exactly the behavior you want to avoid in professional documentation support.

Change the stop sequences to be empty.

Why it fails: Stop sequences are essential for preventing the model from continuing to generate text after the answer is finished, such as simulating a user response. Removing these markers increases the likelihood of runaway generation and model drift, which reduces the overall professionalism and utility of the generated response.

Increase the temperature to 0.9 to ensure varied answers.

Why it fails: Raising the temperature to 0.9 would introduce significant randomness into the output, which is detrimental for technical tasks. In professional support scenarios, you want deterministic and consistent answers based on documentation. High temperatures make the model unreliable, leading to inconsistent guidance that frustrates users seeking accurate technical information.

To reduce the latency of the underlying GPU cluster.

Why it fails: Few-shot prompting actually increases latency because it consumes more tokens in the prompt context. The prompt length directly impacts the processing time per request. It does not provide any optimizations for the underlying hardware cluster, which operates independently of the token length provided in the user's prompt.

To compress the model weights for deployment.

Why it fails: Prompting strategies operate at the inference layer and have no impact on the static weights of the model. Weight compression is achieved through techniques like quantization or pruning, which are model architecture processes, not prompt engineering techniques used to influence the conversational behavior or logic of the model.

To permanently store data in the model's internal memory.

Why it fails: LLMs do not update their weights based on prompt content during standard inference. Few-shot examples are ephemeral and exist only within the current context window. They provide temporary guidance rather than permanent learning, meaning the model will not retain the information after the context window is cleared.

Zero-shot prompting with broad general coding instructions.

Why it fails: Zero-shot prompting relies solely on the model's pre-existing knowledge. Since the syntax is internal and likely absent from the training set, the model will likely hallucinate standard library calls that do not exist, leading to non-functional code that requires manual correction and significant debugging effort.

Increasing the model temperature to encourage exploration.

Why it fails: Increasing the temperature encourages the model to generate diverse, unpredictable code. In a syntax-sensitive environment like coding, this is highly counterproductive. It will likely introduce syntax errors and invalid language constructs, making the resulting code useless for the internal compiler or runtime environment it is intended for.

Reducing the context window to force brevity.

Why it fails: Reducing the context window limits the model's ability to process the provided examples. If you are using few-shot prompting, you need sufficient space to include the examples. Constraining the context window will force the truncation of your examples, directly undermining the efficacy of the few-shot learning process.