Generative AI Leader Practice Test 36 — 15 Questions

Question 1

An AI team is building a customer support chatbot for a telecom company using a fine-tuned LLM on Vertex AI. The model performs well on common issues but fails to answer correctly for rare or novel problems, often providing plausible-sounding but incorrect solutions. The team has a large corpus of internal troubleshooting documents. They want to minimize incorrect answers while keeping latency low. Which approach should they take?

Accepted Answer

Implement a retrieval-augmented generation (RAG) pipeline using Vertex AI Search to fetch relevant documents before generating answers.. Option B is correct because RAG uses the troubleshooting documents as a knowledge base, providing grounded answers for rare issues without retraining. Option A is wrong because more fine-tuning on common issues won't help with rare ones. Option C is wrong because a larger model may increase latency and cost without solving the grounding problem. Option D is wrong because few-shot examples cannot cover all rare scenarios.

Answer

Switch to a larger base model (e.g., Gemini Ultra) without any retrieval.

Answer

Collect more data on rare issues and continue fine-tuning the model weekly.

Answer

Use a few-shot prompt with 10 examples of rare problems and solutions.

Question 2

A software development team builds an internal code assistant using a generative model. The assistant writes Python functions that often contain security vulnerabilities such as SQL injection or command injection. The team wants to mitigate these vulnerabilities without adding a manual review step for every code snippet, as that would slow development. They have access to a static analysis security scanner API. Which approach best addresses the vulnerabilities while maintaining developer velocity?

Accepted Answer

After each generation, automatically run the code through the static analysis scanner, and if vulnerabilities are found, send the output back to the model for revision with the scanner's feedback.. Option D is correct because post-generation automatic scanning with the security scanner catches vulnerabilities and can request regeneration with suggestions, maintaining speed. Option A is wrong because fine-tuning may not eliminate all vulnerabilities. Option B is wrong because top-k only affects output diversity, not security. Option C is wrong because system prompts are not reliably followed for security.

Answer

Increase top-k sampling to generate a wider variety of code tokens.

Answer

Fine-tune the model on a corpus of secure code examples.

Answer

Add a system prompt: 'Do not generate code with security vulnerabilities.'

Question 3

A data scientist is using the Gemini API to generate product descriptions for an e-commerce site. The descriptions are often too verbose and include speculative claims that are not in the product specifications. The scientist wants to reduce hallucinations and control the length of the output without retraining the model. What should they do?

Accepted Answer

Refine the prompt to be concise and include instructions to stick to facts and limit output to 50 words.. Option A is correct because verbose prompts often lead to verbose output; simplifying the prompt reduces length. Adding explicit constraints like 'only use provided facts' reduces hallucination. Option B is wrong because increasing token count would make descriptions longer, not shorter. Option C is wrong because lower temperature reduces randomness but doesn't prevent speculation. Option D is wrong because few-shot examples may help but do not directly enforce length or factuality.

Answer

Increase the max output token count to 2048 and decrease temperature to 0.1.

Answer

Add three few-shot examples of short, factual descriptions.

Answer

Set temperature to 0.0 and top_k to 1.

Question 4

A healthcare startup is using Vertex AI Imagen to generate synthetic medical images for training a diagnostic model. The images must comply with HIPAA regulations and cannot contain any real patient data. The team fine-tuned Imagen on a dataset of de-identified medical scans. However, during testing, they notice that some generated images closely resemble specific patients from the original dataset, even though the dataset was de-identified. They suspect that the model memorized some training examples. The team needs to address this issue without losing image quality. They have access to the original training data and Vertex AI tools. What action should they take?

Accepted Answer

Re-tune the model using differential privacy (DP-SGD) to prevent memorization of individual examples.. Option D is correct. Differential privacy during fine-tuning adds noise to prevent memorization. Option A (more data) might not help if the model is overfitting. Option B (stronger safety filters) won't prevent uniqueness recall. Option C (post-processing) only alters output after generation, doesn't fix memorization.

Answer

Use a post-processing step to blur or distort generated images.

Answer

Increase the size of the training dataset by adding more synthetic images.

Answer

Apply stricter output safety filters to block images that look like any known patient.

Question 5

A fintech startup is building a generative AI application that generates personalized investment advice based on user profiles and market data. They are using Vertex AI Agent Builder to create an agent that retrieves information from a BigQuery table containing user data and from a real-time market data API. The agent needs to ensure that responses comply with financial regulations, meaning the model must not give specific stock recommendations unless the user explicitly requests them after disclaimers. The team has implemented grounding with both sources. During testing, the agent sometimes spontaneously suggests buying a particular stock without being asked, which could lead to regulatory issues. The team wants to enforce strict control over the agent's behavior. What should the team do?

Accepted Answer

Implement a custom system instruction that explicitly prohibits unsolicited stock recommendations and requires a disclaimer before any advice. Option D is correct because a custom system instruction can explicitly constrain the agent's behavior. Option A is wrong because adding more data may not control behavior. Option B is wrong because fine-tuning may not cover all edge cases. Option C is wrong because a safety filter may not catch subtle recommendations.

Answer

Increase the safety filter sensitivity to block any financial recommendations

Answer

Add more historical data to the BigQuery table to improve grounding accuracy

Answer

Fine-tune the model on a dataset of compliant conversations

Question 6

An e-commerce company is using Vertex AI PaLM 2 for Text (via Model Garden) to generate product descriptions. They have an existing pipeline that calls the model with a prompt including product attributes. Recently, they migrated to the Gemini API. The team notices that the Gemini model sometimes outputs descriptions that are factually inconsistent with the input (e.g., wrong color or size). This was less frequent with PaLM 2. They have not changed the prompts. What is the most likely cause and solution?

Accepted Answer

Adjust the prompt to be more explicit about adhering to the input data, and reduce the temperature.. Option C is correct. Gemini may follow instructions differently; adjusting the prompt or temperature can help. Option A (increase model temperature) may increase randomness, worsening consistency. Option B (add negative prompts) might not address factual alignment. Option D (switch back to PaLM 2) does not solve the cause.

Answer

Revert to PaLM 2 since it was more reliable for this task.

Answer

Add negative prompts to discourage incorrect facts.

Answer

Increase the model's temperature to make outputs more deterministic.

Question 7

A large enterprise is deploying a multi-modal generative AI application that processes customer support emails (text) and attached screenshots (images). They need to run inference on over 10,000 requests per minute with strict latency requirements (p99 < 500ms). They have already selected Gemini 1.5 Pro as the model and deployed it on Vertex AI using a GPU-based endpoint with autoscaling. During testing, they observe that the p99 latency spikes to over 2 seconds during peak traffic. The application is stateless and requests are independent. The team has access to Cloud Observability and can modify the deployment configuration. Which course of action should the team take to meet the latency requirements while minimizing cost?

Accepted Answer

Enable Vertex AI Model Caching and deploy the endpoint on a managed instance group with larger GPU nodes (e.g., A100 40GB). Option C is correct because enabling model caching reduces redundant computation for repeated prompts, and using dedicated VMs (MIGs) with higher GPU count per replica reduces per-request latency. Option A is wrong because adding more replicas may help throughput but not per-request latency. Option B is wrong because CPU-based serving would be much slower for Gemini. Option D is wrong because preemptible VMs are not reliable for production latency.

Answer

Increase the maximum number of replicas in the autoscaling configuration to handle spikes

Answer

Use preemptible VMs for the endpoint to get priority scheduling

Answer

Switch to a CPU-based ml.c5 instance to reduce GPU contention

Question 8

A marketing agency wants to use Vertex AI to automatically generate social media posts for clients. They plan to use the Gemini API with few-shot prompting. The agency's developers have limited experience with generative AI and want the fastest way to prototype and iterate on prompts. They are already using Google Cloud for other services. Which approach should they take to quickly develop and test prompts?

Accepted Answer

Use Vertex AI Studio (Gen AI Studio) to design and test prompts interactively.. Option A is correct. Vertex AI Studio provides a no-code interface for prompt design and testing. Option B (write code) is slower. Option C (use Cloud Shell) is possible but less user-friendly. Option D (third-party tool) adds complexity.

Answer

Use a third-party platform like OpenAI Playground and migrate later.

Answer

Use Google Cloud Shell to invoke the model via curl commands.

Answer

Write Python scripts using the Vertex AI SDK and run them in Airflow.

Question 9

A large e-commerce company deploys a generative AI chatbot on Vertex AI for customer service. The chatbot is powered by a fine-tuned model on the company's historical support tickets. Despite high accuracy on training topics, the chatbot frequently gives irrelevant or off-topic answers when customers ask about new products or promotions. The company maintains a comprehensive product catalog and a knowledge base of current promotions. The chatbot's prompts include a system instruction to 'Answer based on your knowledge' and no other retrieval mechanism. The response time requirement is under 3 seconds. Which course of action should the team take?

Accepted Answer

Implement a RAG pipeline that retrieves relevant product and promotion data from the knowledge base and injects it into the prompt.. Option C is correct because implementing RAG with the product catalog allows real-time retrieval of current information, addressing the irrelevance for new products without needing retraining. Option A is wrong because fine-tuning again on outdated data won't help with new products. Option B is wrong because increasing temperature makes outputs more random and less focused. Option D is wrong because adding more safety filters doesn't improve topical relevance.

Answer

Increase the temperature to encourage the model to generate more diverse answers.

Answer

Add additional safety filters to block irrelevant responses.

Answer

Fine-tune the model again on a larger dataset that includes recent support tickets.

Question 10

A retail company is deploying a generative AI chatbot on Vertex AI to provide product recommendations. The chatbot uses a base foundation model with no fine-tuning. Users report that the chatbot sometimes gives offensive or insensitive responses. The team must quickly implement safety controls without modifying the model. They also want to reduce irrelevant off-topic answers. Which combination of techniques should they apply?

Accepted Answer

Enable Vertex AI Safety Filters and craft system instructions defining appropriate behavior.. Option D is correct because safety filters (e.g., Vertex AI Safety Settings) block harmful content, and prompt engineering with system instructions keeps the model on topic and respectful. Option A is wrong because temperature adjustment alone does not prevent toxicity. Option B is wrong because few-shot examples may not cover all safety scenarios. Option C is wrong because fine-tuning is not allowed per the constraint (no model modification).

Answer

Fine-tune the model on a curated dataset of safe retail conversations.

Answer

Set temperature to 0.0 and top_p to 0.1.

Answer

Provide 50 few-shot examples of safe interactions.

Question 11

A financial services firm is using a foundation model on Vertex AI to generate investment summaries from quarterly reports. The summaries are accurate but often miss key financial metrics and trends. The team cannot afford to fine-tune the model frequently. Which technique should they use to improve the completeness and relevance of the summaries without modifying the model?

Accepted Answer

Provide three few-shot examples in the prompt that highlight the desired metrics.. Option C is correct because adding few-shot examples that specifically include the desired metrics (e.g., revenue growth, profit margins) trains the model to include those details. Option A is wrong because increasing temperature increases randomness, which could omit key facts. Option B is wrong because stopping at newlines doesn't guarantee completeness. Option D is wrong because adjusting top_p does not target completeness.

Answer

Increase temperature to 0.9 to encourage more creative outputs.

Answer

Set stop sequences to ['

'] to ensure the model finishes each paragraph.

Answer

Lower top_p to 0.5 to reduce the sampling pool.

Question 12

A retail company has deployed a customer support chatbot using Vertex AI Agent Builder. The chatbot is configured with a knowledge base stored in BigQuery (user manuals) and Cloud Storage (product images). The agent uses a Gemini 1.5 Pro model for response generation. Users report that the chatbot frequently gives incorrect answers and sometimes does not reference the knowledge base at all. Logs show high latency (average response time > 10 seconds) and many responses are generic or hallucinated. The agent's grounding configuration currently uses the default settings. The development team is considering the following actions: A) Switch to a smaller model like Gemini 1.5 Flash to reduce latency. B) Increase the context window of the model to allow more knowledge base content. C) Enable Vertex AI Search for grounding and configure a search aggregation strategy that retrieves relevant documents from the knowledge base. D) Fine-tune the Gemini model with the company's historical chat logs to improve domain-specific responses. Which action should the team take FIRST to address the issues?

Accepted Answer

Enable Vertex AI Search for grounding and configure a search aggregation strategy that retrieves relevant documents from the knowledge base.. Option C is correct because the symptoms indicate the agent is not effectively retrieving and leveraging the knowledge base. Enabling grounding with Vertex AI Search and configuring search aggregation directly addresses the incorrect answers and lack of knowledge base usage. Reducing model size (A) might help latency but not accuracy. Increasing context window (B) could hurt performance further. Fine-tuning (D) is costly and may not fix retrieval issues without proper grounding.

Answer

Switch to a smaller model like Gemini 1.5 Flash to reduce latency.

Answer

Increase the context window of the model to allow more knowledge base content.

Answer

Fine-tune the Gemini model with the company's historical chat logs to improve domain-specific responses.

Question 13

A financial technology company has deployed a custom-tuned PaLM 2 model on Vertex AI to generate personalized investment recommendations for retail clients. The model was fine-tuned on a corpus of historical market data and advisory transcripts. Recently, the compliance team flagged that several recommendations contradicted SEC guidelines, and the model sometimes repeated prohibited statements from outdated training materials. The team has already implemented safety filters (e.g., blocking toxic content) and adjusted the model's system instructions to be more conservative. However, the issues persist. The model's deployment parameters are: temperature=0.4, top_p=0.9, max_output_tokens=500, and no grounding. The company must maintain compliance without significantly increasing latency. What should they do next?

Accepted Answer

Configure Vertex AI grounding using a curated data store of real-time SEC regulations and market data. Option D is correct because grounding with real-time authoritative sources (e.g., up-to-date SEC regulations and market data) ensures outputs are based on current, compliant information, directly addressing the root cause of outdated or prohibited content. Option A (fine-tuning) may introduce bias and doesn't guarantee real-time accuracy. Option B (temperature increase) would worsen variability. Option C (chain-of-thought) can improve reasoning but does not anchor to current compliance data.

Answer

Increase temperature to 0.7 to allow more diverse responses, and add a second model to verify outputs

Answer

Perform an additional fine-tuning round exclusively on the most recent SEC regulatory filings and compliance-approved content

Answer

Implement a chain-of-thought prompting technique that requires the model to explain its reasoning step by step

Question 14

You are a data scientist at a financial institution. You are using Vertex AI to fine-tune a large language model (LLM) for generating financial reports. You have prepared a dataset of 10,000 examples. During fine-tuning, you notice that the training loss is decreasing steadily, but the validation loss is increasing after 5 epochs. The model's generated reports on the validation set contain many factual errors and nonsensical statements. You suspect overfitting. You have limited compute budget and need to improve generalization. What should you do?

Accepted Answer

Implement early stopping with a patience of 2 epochs. Early stopping with a patience of 2 epochs is the correct approach because it directly addresses overfitting by halting training when the validation loss fails to improve for a specified number of epochs. This preserves the model's generalization ability without requiring additional compute or data, which aligns with the limited budget constraint. In Vertex AI, early stopping is a built-in hyperparameter tuning strategy that monitors validation metrics and stops the job to prevent further degradation.

Answer

Increase the learning rate

Answer

Increase the number of training epochs to 20

Answer

Add more training examples from a public dataset

Question 15

You are a machine learning engineer at a healthcare startup. Your team has developed a generative AI model that summarizes patient medical records. The model is deployed on Vertex AI Endpoints using a custom container. You have configured the endpoint with a single n1-standard-4 machine (4 vCPUs, 15 GB memory) without accelerators. The model uses a small transformer architecture. During load testing with 50 concurrent requests, you observe that the average latency is 8 seconds, which exceeds the requirement of 2 seconds. Additionally, some requests time out after 10 seconds. You suspect the CPU is the bottleneck. You also notice that the model inference code uses TensorFlow but is not optimized for inference. Which action should you take to reduce latency?

Accepted Answer

Deploy the model on a machine with a GPU and use TensorRT for inference optimization.. Option D is correct because the CPU is identified as the bottleneck, and deploying on a GPU with TensorRT optimization directly addresses this by accelerating the TensorFlow inference. TensorRT optimizes the model graph and fuses layers, significantly reducing latency for transformer-based models, which is essential to meet the 2-second requirement.

Answer

Reduce the model size by pruning and quantization, then redeploy.

Answer

Increase the request timeout to 30 seconds to accommodate the latency.

Answer

Enable autoscaling to handle the load with multiple replicas.