Techniques to Improve Generative AI Model Output practice questions

Decrease the temperature to 0.1 to reduce randomness and force the model to stick to the input.

Why it fails: Low temperature reduces randomness but does not add factual grounding; the model may still omit or invent details.

Use prompt engineering to instruct the model to only include information explicitly mentioned in the conversation.

Why it fails: Prompt engineering helps but may not overcome the model's training biases; it can still hallucinate.

Add a human-in-the-loop step to review and correct every generated note before use.

Why it fails: Human review is slow and doesn't reduce the model's tendency to produce inaccuracies; it's a workaround not a fix.

Set top_p to 0.5 and increase repetition penalty

Why it fails: Top_p and repetition penalty do not directly control length or structure.

Use prompt caching and increase batch size

Why it fails: Caching and batch size affect performance, not output quality.

Increase temperature and use a larger model

Why it fails: Higher temperature increases randomness; larger model does not guarantee conciseness.

Modify the system instruction to request polite responses only

Why it fails: A system instruction only shapes the prompt context; it cannot suppress offensive tokens the fine-tuned weights have learned to emit. Prompt-level politeness requests are for steering tone in general assistants, not for enforcing safety boundaries against harmful content baked into the model.

Retrain the model on a larger dataset of chat logs

Why it fails: More chat logs amplify the same unfiltered distribution that produced the offensive outputs, so harmful patterns persist or worsen. Larger fine-tuning corpora suit improving coverage and domain accuracy, not removing unsafe content, which requires safety alignment or output filtering.

Reduce the temperature to 0.0

Why it fails: Temperature 0.0 makes decoding deterministic, selecting the highest-probability token; if that token is offensive, the same harmful output recurs every time. Low temperature suits reproducible, factual tasks such as classification or extraction, not safety remediation.

Incorrect prompt format

Why it fails: Affects input handling, not training quality.

Low temperature setting

Why it fails: Affects randomness, not functional correctness.

Insufficient training epochs

Why it fails: Would likely cause underfitting, not syntax-correct but wrong logic.

Use exact keyword matching instead of embedding similarity

Why it fails: Keyword matching is less contextually aware.

Increase chunk size of documents

Why it fails: Larger chunks may contain irrelevant content.

The evaluation metrics are missing, causing deployment to hang

Why it fails: Metrics are not required for deployment.

The training pipeline failed silently

Why it fails: Would appear in Errors field.

The model has no errors, so it is fine

Why it fails: If stuck for days, it's an issue despite no errors.

The developer needs the aiplatform.admin role

Why it fails: Admin role would work but not the only option.

The aiplatform.user role overrides the modelUser role

Why it fails: Roles are additive, not overriding.

The policy is missing the aiplatform.specialist role

Why it fails: The binding grants deployment permissions but omits the tuning-specific permission, so the missing element is a role or permission covering tuning jobs, not a specialist role name. It is tempting because role names sound authoritative, yet IAM evaluates granted permissions rather than titles.

Decrease topP to 0.5

Why it fails: Reduces token pool but may not fix determinism.

Increase topK to 100

Why it fails: Increases diversity but temperature is the root cause.

Decrease maxOutputTokens

Why it fails: Shorter output but still repetitive.

Deploy more model replicas to handle load.

Why it fails: More replicas improve throughput but do not reduce latency for a single request.

Enable prompt caching for repeated queries.

Why it fails: Caching helps only for identical prompts, not typical real-time use.

Batch multiple user requests together.

Why it fails: Batching increases latency for individual requests.

The maxOutputTokens is too low.

Why it fails: Lower token limit cuts responses shorter, but doesn't cause incoherence.

The safety threshold blocks too much content.

Why it fails: Safety filters block harmful content, not cause incoherence.

The topK value is too low.

Why it fails: Low topK reduces randomness; 40 is actually high, increasing diversity.

Disable the safety filter entirely to allow all topics.

Why it fails: Disabling safety filters risks harmful content and violates policy.

Adjust the safety category thresholds to allow VIOLENCE and SEXUAL content since it's medical.

Why it fails: Medical discussions do not require violence/sexual content; this may still be blocked.

Increase the input token limit to 2000.

Why it fails: Token limit does not affect safety filter blocking.

Fine-tune the model on a dataset with an equal number of positive and negative examples.

Why it fails: Fine-tuning is effective but time-consuming; prompt engineering is a lighter-weight approach often tried first.

Reduce the max output tokens to limit the model's tendency to generate positive language.

Why it fails: Max tokens only controls length, not sentiment bias.

Increase the temperature parameter to reduce model confidence.

Why it fails: Increasing temperature adds randomness and does not specifically address the bias toward positive sentiment.

Fine-tune the model on summaries that are exactly 200 words.

Why it fails: Overkill and time-consuming when a simple parameter setting works.

Add a system prompt that says 'Summarize in exactly 200 words.'

Why it fails: Prompt instructions may be ignored or inaccurately followed; a hard token limit is more reliable.

Lower the temperature to reduce variability in word choices.

Why it fails: Temperature does not control output length.

Using a prompt that explicitly instructs the model to avoid offensive language

Why it fails: While a prompt can instruct the model to avoid offensive language, it is not foolproof. The model may still generate inappropriate content, especially if the prompt is not perfectly crafted. A content filter or fine-tuning provides more reliable mitigation. Prompt engineering alone is less robust than these methods.

Reducing the max output tokens to shorten emails

Why it fails: Reducing max output tokens limits the length of the generated text but does not address the content's appropriateness. Offensive language can appear in short texts as well. This technique does not influence the model's tone or vocabulary, so it fails to solve the problem.

Increasing the temperature to make the model more creative

Why it fails: Increasing temperature makes the model more random and creative, which could increase the chance of generating inappropriate content. It does not help mitigate offensive language; instead, it may exacerbate the issue by exploring more diverse and potentially risky outputs.

Fine-tune the model on a dataset of past agent responses about returns.

Why it fails: Fine-tuning adapts style and task behavior, but it does not reliably teach a model to recite a specific, frequently changing policy clause, and it can bake in stale details. Because the policy PDF is the authoritative source and may change, tuning would need constant rework. Grounding in the live document is the appropriate mechanism for factual, verifiable answers.

Increase the temperature setting so the model explores more of the policy text.

Why it fails: Raising temperature increases randomness in token selection, which makes output more varied and less deterministic. It does not help the model locate or prioritize a specific fact inside a long document, and it can actually increase the chance of fabricating a plausible but incorrect return window. This option addresses creativity, not grounding, so it does not solve the missed-detail problem.

Reduce the maximum output tokens so responses are shorter and focused.

Why it fails: Limiting output tokens only truncates the response length; it does not change which source content the model attends to. A shorter answer could still omit the 30-day detail entirely, or worse, get cut off mid-sentence. The root problem is grounding in the source document, not verbosity, so controlling token count leaves the underlying retrieval and citation behavior unchanged.

Set the temperature to 0 to make the output deterministic.

Why it fails: Temperature 0 makes the model always pick the highest-probability token, which can reduce variability but does not guarantee syntactic validity. The model may still produce invalid SQL if its training or prompt leads it to. Determinism does not equal correctness, and syntax errors can persist even with greedy decoding.

Increase the max output tokens to give the model more room to correct itself.

Why it fails: More tokens do not inherently improve syntax; the model may simply generate longer but still invalid SQL. Max output tokens controls length, not correctness. Without a validation step, the model has no feedback to fix errors, so this does not ensure valid SQL.

Use controlled generation with a response schema that defines the SQL query structure.

Why it fails: A response schema can enforce a JSON structure but cannot validate SQL syntax itself. It might ensure the output is a JSON object with a 'query' field, but the SQL inside could still be invalid. Schema-based control is not designed to parse or validate SQL grammar, so it does not guarantee syntactically correct SQL.

Fine-tune the model on a dataset of discharge instructions that contain interaction warnings.

Why it fails: Fine-tuning could raise the base rate of warnings but cannot guarantee inclusion for every case where the source notes mention an interaction, especially rare drug pairs. It also demands curated clinical data, review, and retraining cycles. An explicit prompt constraint achieves the required reliability faster and without the regulatory burden of tuning a clinical model.

Raise the top-k sampling value so more candidate tokens are considered.

Why it fails: Top-k controls how many token candidates are sampled from, affecting diversity rather than content coverage. It does not direct the model to include a specific mandatory element. Increasing top-k could add variability and even introduce unsupported clinical statements, so it does not reliably force the drug-interaction warning into every relevant discharge instruction.

Apply a chain-of-thought prompt asking the model to reason step by step before writing.

Why it fails: Chain-of-thought improves multi-step reasoning but does not guarantee inclusion of a specific required element; the model can reason correctly and still omit the warning in the final text. It also adds latency and token cost. For a mandatory-content requirement, an explicit structural instruction or post-generation check is more reliable than reasoning elicitation.

Increase the amount of training data

Why it fails: Increasing training data volume does not alter the politeness-helpfulness trade-off, since the model already mirrors the tone distribution in its existing corpus; more of the same data reinforces that pattern. It is tempting because scaling data genuinely improves generalisation and factual coverage, and would be the right choice when the model lacks domain knowledge rather than exhibiting a tone imbalance.

Lower the top_k sampling value

Why it fails: Lowering top_k narrows token selection to fewer candidates, which reduces response diversity but does not address politeness or helpfulness — those stem from training data and prompt design, not sampling breadth. It is tempting because top_k tuning genuinely controls randomness and repetition, making it the right lever when outputs are incoherent or wildly off-topic.

Increase the temperature parameter

Why it fails: Raising temperature increases sampling randomness, which worsens the unhelpful, evasive phrasing rather than correcting it. Temperature suits creative brainstorming where varied outputs are wanted. The scenario needs prompt engineering or fine-tuning to steer tone and content toward helpful answers, not broader token sampling.

Run a syntax checker after generation and regenerate

Why it fails: Post-generation checking and regeneration adds latency and cost per attempt and still relies on the model eventually producing valid output, rather than constraining decoding. It is tempting because it needs no retraining and works as a fallback, and would be correct where validation is cheap and occasional retries are acceptable.

Add a system prompt that instructs the model to produce valid code

Why it fails: A system prompt only biases generation probabilistically; it cannot guarantee syntactic validity because the model may still emit malformed tokens. It is tempting because prompting is cheap and needs no retraining, and would be the right choice for steering style, tone or format rather than enforcing hard syntactic correctness.

Increase beam search width

Why it fails: Beam search widens the search for high-probability sequences but optimises likelihood, not grammar, so it can still return syntactically invalid code. It is tempting because wider beams often improve output quality, and would be the right choice when the goal is better overall fluency rather than guaranteed syntax.

Increase the output resolution

Why it fails: Higher resolution adds pixel detail but does not correct the model's learned facial structure, so distortion persists. It is tempting because resolution is a visible output setting, yet the defect stems from training data and architecture; fine-tuning on clear face images addresses the actual cause.

Increase the number of inference steps

Why it fails: More inference steps refine an existing latent trajectory but cannot correct a model that never learned accurate facial structure; they mainly sharpen detail and can even amplify distortion. It is tempting because step count genuinely governs sampling quality and convergence, and raising it is the right fix when outputs look noisy or under-resolved rather than anatomically wrong.

Reduce the classifier-free guidance scale

Why it fails: Lowering classifier-free guidance weakens prompt adherence, so faces drift further from the text description rather than gaining anatomical accuracy. It is tempting because guidance scale genuinely controls the creativity-fidelity trade-off, and reducing it is correct when outputs look over-saturated, contrasty or unnaturally rigid for a given prompt.