Generative AI Leader Techniques to Improve Generative AI Model Output — Timed 10 Questions

Question 1

A healthcare company is using a fine-tuned version of PaLM 2 on Vertex AI to generate clinical notes from doctor-patient conversations. The model was fine-tuned on a dataset of 10,000 de-identified transcripts and corresponding notes. During testing, the generated notes are grammatically correct and well-structured, but they often contain subtle inaccuracies: for example, they might mention a medication that was not discussed, or omit a key symptom. The team has already tried increasing the training epochs and adjusting learning rates, with minimal improvement. They need a solution that can be implemented quickly to improve factual accuracy without retraining the entire model. The team has access to a large archive of verified clinical notes and a small set of recent conversation-to-note pairs that have been manually reviewed and corrected. The inference pipeline currently uses a single call to the model with the conversation transcript as input. What should the team do?

10 minute time limit — choose an answer to begin.

Accepted Answer

Implement retrieval-augmented generation (RAG) by retrieving similar verified notes from the archive and providing them as context in the prompt.. Retrieval-augmented generation (RAG) directly addresses the core issue of factual inaccuracy without retraining. By retrieving verified clinical notes similar to the current conversation from the archive and injecting them as context in the prompt, the model gains access to ground-truth examples that anchor its output to factual details. This approach leverages the team's existing archive and small set of corrected pairs to provide relevant, accurate context, improving precision without modifying the model's weights.

Answer

Decrease the temperature to 0.1 to reduce randomness and force the model to stick to the input.

Answer

Use prompt engineering to instruct the model to only include information explicitly mentioned in the conversation.

Answer

Add a human-in-the-loop step to review and correct every generated note before use.

Question 2

For a document summarization task, a team wants to produce concise summaries without losing key information. Which combination of techniques is most effective?

Accepted Answer

Use few-shot examples and reduce max output tokens. Using few-shot examples provides the model with explicit patterns of desired summarization behavior, guiding it to produce concise outputs, while reducing max output tokens enforces a hard length constraint that prevents verbosity. This combination directly addresses the goal of conciseness without losing key information by conditioning the model on high-quality examples and capping the response length.

Answer

Set top_p to 0.5 and increase repetition penalty

Answer

Use prompt caching and increase batch size

Answer

Increase temperature and use a larger model

Question 3

A travel company fine-tuned a language model on customer chat logs to provide travel recommendations. After deployment, they receive complaints that the model sometimes generates inappropriate or offensive content. What is the most effective approach to improve output safety while preserving overall performance?

Accepted Answer

Add a post-processing safety classifier that filters or rewrites unsafe outputs. A post-processing safety classifier acts as a guardrail that can detect and filter or rewrite unsafe outputs without altering the underlying model's weights or training data. This approach preserves the model's overall performance on safe, relevant recommendations while adding a dedicated safety layer that can be independently tuned and updated as new safety requirements emerge. Unlike prompt engineering or hyperparameter adjustments, a classifier provides a robust, policy-enforced mechanism to catch edge cases that the model might otherwise generate.

Answer

Modify the system instruction to request polite responses only

Answer

Retrain the model on a larger dataset of chat logs

Answer

Reduce the temperature to 0.0

Question 4

A team deployed a fine-tuned model for code generation. After training, the model produces syntactically correct but functionally wrong code. What is the most likely cause?

Accepted Answer

Overfitting to training data. Overfitting to training data causes the model to memorize specific code patterns and syntax from the training set without learning the underlying logic or functional requirements. This results in syntactically correct outputs that fail to generalize to new, unseen coding tasks, producing functionally wrong code despite proper syntax.

Answer

Incorrect prompt format

Answer

Low temperature setting

Answer

Insufficient training epochs

Question 5

A team notices the RAG pipeline sometimes retrieves irrelevant documents. Which THREE improvements should they consider? (Choose three.)

Accepted Answer

Add a reranking step. Option A (Add a reranking step) is correct because a cross-encoder or LLM-based reranker re-scores the initially retrieved candidates against the query, promoting truly relevant passages and demoting semantically similar but off-topic ones, which directly reduces irrelevant retrievals. Option D (Reduce the number of retrieved documents) is correct because lowering top-k limits the context to the highest-scoring matches, cutting the chance that marginal, irrelevant chunks are passed to the generator. Option E (Use a higher quality embedding model) is correct because better embeddings produce more discriminative vector representations, improving recall and precision of the similarity search so irrelevant documents are less likely to be retrieved. Option B is not appropriate because replacing embedding similarity with exact keyword matching sacrifices semantic recall and would worsen results for paraphrased or conceptual queries. Option C is not appropriate because increasing chunk size dilutes the embedding with multiple topics, typically making retrieval less precise rather than more relevant.

Answer

Use exact keyword matching instead of embedding similarity

Answer

Increase chunk size of documents

Question 6

Refer to the exhibit. A team runs 'gcloud ai models list --filter=displayName:qa-chat-v1' and sees the output. The model was tuned using supervised fine-tuning (SFT) but shows 'state: DEPLOYING' for days. What is the most likely issue?

Accepted Answer

The model is stuck in deployment due to insufficient quota. The model shows 'state: DEPLOYING' for days, which indicates the deployment process is stuck rather than failing with an error. In Vertex AI, a model stuck in DEPLOYING state for an extended period is typically caused by insufficient quota for the selected machine type (e.g., GPU or TPU accelerators). The deployment process cannot complete because the requested resources are not available, causing it to hang indefinitely.

Answer

The evaluation metrics are missing, causing deployment to hang

Answer

The training pipeline failed silently

Answer

The model has no errors, so it is fine

Question 7

Refer to the exhibit. A team's IAM policy for Vertex AI includes the following binding. They can deploy models but cannot create tuning jobs. Which statement is true?

Accepted Answer

The aiplatform.user role lacks permission to create tuning jobs. The roles/aiplatform.user role does not include permission to create tuning jobs (aiplatform.tuningJobs.create). The modelUser role does not override the user role, admin role is not needed, and specialist role doesn't exist.

Answer

The developer needs the aiplatform.admin role

Answer

The aiplatform.user role overrides the modelUser role

Answer

The policy is missing the aiplatform.specialist role

Question 8

Refer to the exhibit. A data scientist sends a prediction request to a text generation model with the following parameters and receives repetitive output. Which parameter should be changed?

Accepted Answer

Increase temperature to 0.5. Temperature controls the randomness of the model's output; a low temperature (e.g., 0.1) makes the model highly deterministic, often causing it to repeat the same high-probability token sequences. Increasing temperature to 0.5 allows the model to sample from a broader distribution, reducing repetition and producing more varied text. The other parameters (topP, topK, maxOutputTokens) affect sampling breadth or length but do not directly address the core issue of overly deterministic token selection.

Answer

Decrease topP to 0.5

Answer

Increase topK to 100

Answer

Decrease maxOutputTokens

Question 9

Which TWO techniques can help reduce latency for a real-time generative AI application? (Choose two.)

Accepted Answer

Use streaming responses to send tokens as generated.. Option A is correct because streaming responses deliver tokens to the client as soon as they are generated rather than waiting for the entire completion, which directly reduces perceived latency (time-to-first-token) for real-time generative AI applications. Option B is correct because quantizing the model to lower precision (e.g., FP16 to INT8 or INT4) reduces memory bandwidth and compute requirements per token, allowing faster inference and thus lower latency. Option C is not correct here because adding replicas primarily improves throughput and scalability under load, not the per-request latency of a single inference. Option D is not correct because prompt caching mainly reduces cost and latency for repeated identical prefixes, which is a narrower optimization and not a general latency-reduction technique for real-time generation. Option E is not correct because batching multiple user requests increases throughput but typically increases latency for individual requests, which is the opposite of what a real-time application needs.

Answer

Deploy more model replicas to handle load.

Answer

Enable prompt caching for repeated queries.

Answer

Batch multiple user requests together.

Question 10

A team configures a Vertex AI prediction request as shown. Users report that the model sometimes produces incoherent or off-topic responses despite moderate settings. What is the most likely cause?

Accepted Answer

The temperature is too high for coherent responses.. A is correct because temperature controls the randomness of token selection; a high temperature (e.g., >0.8) increases the probability of sampling low-probability tokens, leading to incoherent or off-topic responses even with moderate settings. Vertex AI's default temperature is 0.0 for deterministic output, and raising it without careful tuning often causes semantic drift.

Answer

The maxOutputTokens is too low.

Answer

The safety threshold blocks too much content.

Answer

The topK value is too low.