Generative AI Leader Techniques to Improve Generative AI Model Output — 100 Questions

Question 1

A healthcare company is using a fine-tuned version of PaLM 2 on Vertex AI to generate clinical notes from doctor-patient conversations. The model was fine-tuned on a dataset of 10,000 de-identified transcripts and corresponding notes. During testing, the generated notes are grammatically correct and well-structured, but they often contain subtle inaccuracies: for example, they might mention a medication that was not discussed, or omit a key symptom. The team has already tried increasing the training epochs and adjusting learning rates, with minimal improvement. They need a solution that can be implemented quickly to improve factual accuracy without retraining the entire model. The team has access to a large archive of verified clinical notes and a small set of recent conversation-to-note pairs that have been manually reviewed and corrected. The inference pipeline currently uses a single call to the model with the conversation transcript as input. What should the team do?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Implement retrieval-augmented generation (RAG) by retrieving similar verified notes from the archive and providing them as context in the prompt.. Retrieval-augmented generation (RAG) directly addresses the core issue of factual inaccuracy without retraining. By retrieving verified clinical notes similar to the current conversation from the archive and injecting them as context in the prompt, the model gains access to ground-truth examples that anchor its output to factual details. This approach leverages the team's existing archive and small set of corrected pairs to provide relevant, accurate context, improving precision without modifying the model's weights.

Answer

Decrease the temperature to 0.1 to reduce randomness and force the model to stick to the input.

Answer

Use prompt engineering to instruct the model to only include information explicitly mentioned in the conversation.

Answer

Add a human-in-the-loop step to review and correct every generated note before use.

Question 2

For a document summarization task, a team wants to produce concise summaries without losing key information. Which combination of techniques is most effective?

Accepted Answer

Use few-shot examples and reduce max output tokens. Using few-shot examples provides the model with explicit patterns of desired summarization behavior, guiding it to produce concise outputs, while reducing max output tokens enforces a hard length constraint that prevents verbosity. This combination directly addresses the goal of conciseness without losing key information by conditioning the model on high-quality examples and capping the response length.

Answer

Set top_p to 0.5 and increase repetition penalty

Answer

Use prompt caching and increase batch size

Answer

Increase temperature and use a larger model

Question 3

A travel company fine-tuned a language model on customer chat logs to provide travel recommendations. After deployment, they receive complaints that the model sometimes generates inappropriate or offensive content. What is the most effective approach to improve output safety while preserving overall performance?

Accepted Answer

Add a post-processing safety classifier that filters or rewrites unsafe outputs. A post-processing safety classifier acts as a guardrail that can detect and filter or rewrite unsafe outputs without altering the underlying model's weights or training data. This approach preserves the model's overall performance on safe, relevant recommendations while adding a dedicated safety layer that can be independently tuned and updated as new safety requirements emerge. Unlike prompt engineering or hyperparameter adjustments, a classifier provides a robust, policy-enforced mechanism to catch edge cases that the model might otherwise generate.

Answer

Modify the system instruction to request polite responses only

Answer

Retrain the model on a larger dataset of chat logs

Answer

Reduce the temperature to 0.0

Question 4

A team deployed a fine-tuned model for code generation. After training, the model produces syntactically correct but functionally wrong code. What is the most likely cause?

Accepted Answer

Overfitting to training data. Overfitting to training data causes the model to memorize specific code patterns and syntax from the training set without learning the underlying logic or functional requirements. This results in syntactically correct outputs that fail to generalize to new, unseen coding tasks, producing functionally wrong code despite proper syntax.

Answer

Incorrect prompt format

Answer

Low temperature setting

Answer

Insufficient training epochs

Question 5

A team notices the RAG pipeline sometimes retrieves irrelevant documents. Which THREE improvements should they consider? (Choose three.)

Accepted Answer

Add a reranking step. Option A (Add a reranking step) is correct because a cross-encoder or LLM-based reranker re-scores the initially retrieved candidates against the query, promoting truly relevant passages and demoting semantically similar but off-topic ones, which directly reduces irrelevant retrievals. Option D (Reduce the number of retrieved documents) is correct because lowering top-k limits the context to the highest-scoring matches, cutting the chance that marginal, irrelevant chunks are passed to the generator. Option E (Use a higher quality embedding model) is correct because better embeddings produce more discriminative vector representations, improving recall and precision of the similarity search so irrelevant documents are less likely to be retrieved. Option B is not appropriate because replacing embedding similarity with exact keyword matching sacrifices semantic recall and would worsen results for paraphrased or conceptual queries. Option C is not appropriate because increasing chunk size dilutes the embedding with multiple topics, typically making retrieval less precise rather than more relevant.

Answer

Use exact keyword matching instead of embedding similarity

Answer

Increase chunk size of documents

Question 6

Refer to the exhibit. A team runs 'gcloud ai models list --filter=displayName:qa-chat-v1' and sees the output. The model was tuned using supervised fine-tuning (SFT) but shows 'state: DEPLOYING' for days. What is the most likely issue?

Accepted Answer

The model is stuck in deployment due to insufficient quota. The model shows 'state: DEPLOYING' for days, which indicates the deployment process is stuck rather than failing with an error. In Vertex AI, a model stuck in DEPLOYING state for an extended period is typically caused by insufficient quota for the selected machine type (e.g., GPU or TPU accelerators). The deployment process cannot complete because the requested resources are not available, causing it to hang indefinitely.

Answer

The evaluation metrics are missing, causing deployment to hang

Answer

The training pipeline failed silently

Answer

The model has no errors, so it is fine

Question 7

Refer to the exhibit. A team's IAM policy for Vertex AI includes the following binding. They can deploy models but cannot create tuning jobs. Which statement is true?

Accepted Answer

The aiplatform.user role lacks permission to create tuning jobs. The roles/aiplatform.user role does not include permission to create tuning jobs (aiplatform.tuningJobs.create). The modelUser role does not override the user role, admin role is not needed, and specialist role doesn't exist.

Answer

The developer needs the aiplatform.admin role

Answer

The aiplatform.user role overrides the modelUser role

Answer

The policy is missing the aiplatform.specialist role

Question 8

Refer to the exhibit. A data scientist sends a prediction request to a text generation model with the following parameters and receives repetitive output. Which parameter should be changed?

Accepted Answer

Increase temperature to 0.5. Temperature controls the randomness of the model's output; a low temperature (e.g., 0.1) makes the model highly deterministic, often causing it to repeat the same high-probability token sequences. Increasing temperature to 0.5 allows the model to sample from a broader distribution, reducing repetition and producing more varied text. The other parameters (topP, topK, maxOutputTokens) affect sampling breadth or length but do not directly address the core issue of overly deterministic token selection.

Answer

Decrease topP to 0.5

Answer

Increase topK to 100

Answer

Decrease maxOutputTokens

Question 9

Which TWO techniques can help reduce latency for a real-time generative AI application? (Choose two.)

Accepted Answer

Use streaming responses to send tokens as generated.. Option A is correct because streaming responses deliver tokens to the client as soon as they are generated rather than waiting for the entire completion, which directly reduces perceived latency (time-to-first-token) for real-time generative AI applications. Option B is correct because quantizing the model to lower precision (e.g., FP16 to INT8 or INT4) reduces memory bandwidth and compute requirements per token, allowing faster inference and thus lower latency. Option C is not correct here because adding replicas primarily improves throughput and scalability under load, not the per-request latency of a single inference. Option D is not correct because prompt caching mainly reduces cost and latency for repeated identical prefixes, which is a narrower optimization and not a general latency-reduction technique for real-time generation. Option E is not correct because batching multiple user requests increases throughput but typically increases latency for individual requests, which is the opposite of what a real-time application needs.

Answer

Deploy more model replicas to handle load.

Answer

Enable prompt caching for repeated queries.

Answer

Batch multiple user requests together.

Question 10

A team configures a Vertex AI prediction request as shown. Users report that the model sometimes produces incoherent or off-topic responses despite moderate settings. What is the most likely cause?

Accepted Answer

The temperature is too high for coherent responses.. A is correct because temperature controls the randomness of token selection; a high temperature (e.g., >0.8) increases the probability of sampling low-probability tokens, leading to incoherent or off-topic responses even with moderate settings. Vertex AI's default temperature is 0.0 for deterministic output, and raising it without careful tuning often causes semantic drift.

Answer

The maxOutputTokens is too low.

Answer

The safety threshold blocks too much content.

Answer

The topK value is too low.

Question 11

A developer is building a chatbot for a medical application that discusses sensitive health topics. The chatbot consistently gets its outputs blocked. What should the developer do?

Accepted Answer

Review and refine the system instructions to avoid triggering safety filters, and consider using a different model endpoint that allows medical contexts.. Safety filters in generative AI models are triggered by content that violates predefined policies, often due to ambiguous or overly broad system instructions. By refining the system instructions to explicitly frame the medical context (e.g., 'This is a clinical discussion for educational purposes'), the developer can reduce false positives. Additionally, using a model endpoint fine-tuned for medical domains (e.g., Med-PaLM 2) bypasses generic safety restrictions while maintaining compliance with ethical guidelines.

Answer

Disable the safety filter entirely to allow all topics.

Answer

Adjust the safety category thresholds to allow VIOLENCE and SEXUAL content since it's medical.

Answer

Increase the input token limit to 2000.

Question 12

A company deploys a sentiment analysis model to classify customer reviews. The model consistently returns overly positive sentiment for all reviews, even when reviews contain negative feedback. Which technique would best resolve this issue?

Accepted Answer

Add a system prompt instructing the model to analyze the review for both positive and negative sentiment and output the overall classification.. Using a system prompt with explicit instructions to detect all sentiments, including negative, guides the model to consider the full emotional range. Option B is wrong because fine-tuning on a balanced dataset is a good practice but is not the quickest fix; prompt engineering is more immediate. Option C is wrong because reducing max tokens only affects output length, not sentiment bias. Option D is wrong because increasing temperature adds randomness and does not enforce balance.

Answer

Fine-tune the model on a dataset with an equal number of positive and negative examples.

Answer

Reduce the max output tokens to limit the model's tendency to generate positive language.

Answer

Increase the temperature parameter to reduce model confidence.

Question 13

A team uses a generative model to summarize lengthy legal documents. The summaries are accurate but often exceed the target length of 200 words, varying widely. Which simple adjustment should be applied to ensure consistent output length?

Accepted Answer

Set the max output tokens parameter to 200.. Setting the max output tokens parameter to 200 directly caps the number of tokens the model can generate, ensuring the summary cannot exceed the target length. This is a deterministic, model-level constraint that works regardless of prompt phrasing or training data, making it the most reliable adjustment for consistent output length.

Answer

Fine-tune the model on summaries that are exactly 200 words.

Answer

Add a system prompt that says 'Summarize in exactly 200 words.'

Answer

Lower the temperature to reduce variability in word choices.

Question 14

A team is using a generative AI model to create personalized email campaigns. They find that the emails sometimes include inappropriate jokes or offensive language. Which TWO techniques can help mitigate this issue? (Choose two.)

Accepted Answer

Fine-tuning the model on a dataset of professional email communications. A content filter provides a hard block against offensive terms, ensuring that no inappropriate language reaches the recipient. Fine-tuning on professional emails aligns the model's style with desired communication standards, reducing the likelihood of offensive jokes. Together, they offer both preventive and corrective measures.

Answer

Using a prompt that explicitly instructs the model to avoid offensive language

Answer

Reducing the max output tokens to shorten emails

Answer

Increasing the temperature to make the model more creative

Question 15

A retail company uses Gemini on Vertex AI to answer customer questions about its return policy. Agents report that answers are accurate but often miss the 30-day window detail buried in a long policy PDF that is passed in each request. The team wants the model to reliably ground its answers in that document. Which approach should they take?

Accepted Answer

Use grounding with Vertex AI Search and return citations to the policy document in the response.. Grounding with Vertex AI Search retrieves the most relevant chunks of the policy and constrains generation to that evidence, so a detail like the 30-day window is surfaced rather than left for the model to find in a long context. Citations make each answer auditable, which matters when agents relay policy to customers. Sampling parameters and output limits change style or length, not factual grounding.

Answer

Fine-tune the model on a dataset of past agent responses about returns.

Answer

Increase the temperature setting so the model explores more of the policy text.

Answer

Reduce the maximum output tokens so responses are shorter and focused.

Question 16

A developer is using the Gemini API to build an application that generates SQL queries from natural language questions. The generated queries sometimes have syntax errors. They want to ensure the output is always valid SQL. Which approach is most effective?

Accepted Answer

Use a function call with a defined SQL query parameter and validate the output against a SQL parser.. Function calling lets the model return a structured object with a SQL query field, which the application can then parse and validate with a SQL parser. This creates a verification loop: if the query is invalid, the app can reject it or ask the model to retry. Combining structured output with deterministic validation is the most reliable way to guarantee syntactically correct SQL. Other options affect randomness or length but do not validate syntax.

Answer

Set the temperature to 0 to make the output deterministic.

Answer

Increase the max output tokens to give the model more room to correct itself.

Answer

Use controlled generation with a response schema that defines the SQL query structure.

Question 17

A hospital's patient-education team uses a generative AI model to produce discharge instructions. Clinicians report that the model sometimes omits the required warning about drug interactions even though the source notes contain it. They want to reliably increase the chance the warning appears in every relevant output. Which technique is most appropriate?

Accepted Answer

Add an explicit output requirement in the prompt that every discharge instruction must include a drug-interaction warning when the source notes mention one.. The failure is a missing mandatory element, so the fix must explicitly require that element in the output specification. A prompt-level output constraint makes inclusion conditional on the source notes and is immediately auditable. Sampling parameters, reasoning prompts, and fine-tuning influence randomness, reasoning depth, or base rates, but none reliably guarantees a specific required statement.

Answer

Fine-tune the model on a dataset of discharge instructions that contain interaction warnings.

Answer

Raise the top-k sampling value so more candidate tokens are considered.

Answer

Apply a chain-of-thought prompt asking the model to reason step by step before writing.

Question 18

A team is building a generative AI model for customer support. They notice the model often produces overly polite but unhelpful responses. Which technique would best improve response quality without sacrificing helpfulness?

Accepted Answer

Apply reinforcement learning from human feedback (RLHF). RLHF directly addresses the misalignment between the model's training objective (e.g., predicting the next token) and the desired outcome (helpful, not just polite). By using human feedback to train a reward model, the system learns to optimize for response quality and helpfulness, reducing sycophantic or overly polite but uninformative outputs.

Answer

Increase the amount of training data

Answer

Lower the top_k sampling value

Answer

Increase the temperature parameter

Question 19

A generative AI model for code generation sometimes produces syntactically incorrect code. The team wants to reduce syntax errors without retraining the entire model. Which approach is most effective?

Accepted Answer

Implement constrained decoding with grammar rules. Constrained decoding with grammar rules directly enforces the syntax of the target programming language during token generation, preventing the model from producing invalid constructs. This approach modifies the decoding process (e.g., using a context-free grammar or a formal syntax specification) to mask or forbid tokens that would lead to a syntax error, without altering the underlying model weights. It is the most effective method because it guarantees syntactically correct output at generation time, rather than relying on post-hoc fixes or probabilistic adjustments.

Answer

Run a syntax checker after generation and regenerate

Answer

Add a system prompt that instructs the model to produce valid code

Answer

Increase beam search width

Question 20

A company uses a text-to-image model to generate marketing visuals. The outputs often contain distorted human faces. Which technique is most likely to improve face generation?

Accepted Answer

Fine-tune the model on a curated dataset of human faces. Fine-tuning the model on a high-quality dataset of human faces directly addresses the distortion issue by specializing the model for face generation. Option B (increasing output resolution) may improve overall image sharpness but does not specifically correct face distortions. Option C (increasing inference steps) can enhance image coherence but is not targeted at face quality. Option D (reducing classifier-free guidance scale) decreases prompt adherence, which could actually worsen face generation rather than improve it.

Answer

Increase the output resolution

Answer

Increase the number of inference steps

Answer

Reduce the classifier-free guidance scale