Generative AI Leader Practice Test 15 — 15 Questions

Question 1

A data scientist fine-tunes a foundation model on customer support transcripts. After evaluation, the model's responses are too formal. Which adjustment during fine-tuning is most likely to make responses more conversational?

Accepted Answer

Include examples of informal customer interactions in the fine-tuning data.. The training data directly influences the tone and style of model outputs. Including examples of informal conversations in the fine-tuning dataset teaches the model the desired conversational tone. Other options affect training dynamics but not the style.

Answer

Increase the batch size to stabilize training.

Answer

Decrease the number of fine-tuning steps to prevent overfitting.

Answer

Use a higher learning rate for faster adaptation.

Question 2

During a RAG pipeline implementation, the retrieval system frequently returns irrelevant documents, causing the generator to produce incorrect answers. Which change is most likely to improve the relevance of retrieved documents?

Accepted Answer

Add a re-ranking step using a cross-encoder model to refine the top results.. Re-ranking with a cross-encoder model evaluates query-document pairs more deeply, improving precision at the cost of latency. Increasing the number of documents may introduce more noise. Changing embedding dimension or chunk size may help but are less targeted.

Answer

Increase the number of documents retrieved from the vector store.

Answer

Use a different embedding model with higher vector dimension.

Answer

Decrease the chunk size of documents to reduce noise.

Question 3

A data scientist fine-tunes a large language model on Vertex AI but gets poor results on validation data. What is the most likely cause?

Accepted Answer

Insufficient training data. Fine-tuning a large language model on Vertex AI with poor validation results is most likely due to insufficient training data. Large language models have billions of parameters and require a substantial amount of high-quality, task-specific data to effectively adapt to a new domain or task; without enough examples, the model cannot learn the desired patterns and will perform poorly on unseen data.

Answer

Incorrect learning rate

Answer

Using wrong model family

Answer

Overfitting due to too many epochs

Question 4

A company wants to build a chatbot that answers questions based on internal documents. Which approach is most appropriate?

Accepted Answer

Use a prompt with the documents in the context. Option D is correct because Retrieval-Augmented Generation (RAG) allows the chatbot to dynamically include relevant internal documents in the prompt context without modifying the underlying model. This approach leverages the pre-trained model's language understanding while grounding answers in specific, up-to-date internal data, avoiding the cost and latency of fine-tuning or retraining.

Answer

Use a pre-trained model without any customizations

Answer

Train a custom model from scratch

Answer

Fine-tune a model on the documents

Question 5

A financial institution deploys a chatbot using Gemini Pro in Vertex AI. Compliance requires logging all user inputs and model outputs for audit. Which approach meets this requirement?

Accepted Answer

Enable Vertex AI Endpoint request-response logging. Vertex AI Endpoint request-response logging captures both the user's input prompt and the model's generated output, which is precisely what compliance auditing requires. This feature logs the exact payloads sent to and received from the deployed model, ensuring a complete audit trail without additional configuration.

Answer

Capture logs via Cloud Monitoring

Answer

Use Cloud Logging sink with a filter for Vertex AI requests

Answer

Enable Vertex AI Model Registry logging

Question 6

Which TWO are benefits of using pre-trained foundation models instead of training from scratch?

Accepted Answer

Lower training cost. Pre-trained foundation models have already been trained on vast datasets, so you only need to fine-tune them for your specific task. This dramatically reduces the compute resources, time, and data required compared to training from scratch, directly lowering training cost.

Answer

Complete control over model architecture

Answer

Eliminates the need for prompt engineering

Answer

Guaranteed absence of bias

Question 7

A company fine-tunes a model using Vertex AI and notices the model's performance drops on the original training task (e.g., language understanding) after fine-tuning for a new task (e.g., summarization). What could be the cause?

Accepted Answer

Catastrophic forgetting. Catastrophic forgetting occurs when a neural network loses previously learned knowledge upon being fine-tuned on a new task. In this scenario, fine-tuning the model for summarization overwrites the weights responsible for language understanding, causing performance degradation on the original task. This is a well-known limitation of sequential fine-tuning in deep learning.

Answer

Data leakage

Answer

Model quantization

Answer

Underfitting

Question 8

A company wants to build a chatbot that answers questions using their internal knowledge base. Which approach is most suitable?

Accepted Answer

Use Retrieval-Augmented Generation (RAG). Retrieval-Augmented Generation (RAG) combines retrieval of relevant documents from a knowledge base with generative responses, making it ideal for this use case.

Answer

Fine-tune a model on the knowledge base

Answer

Train a new model from scratch

Answer

Use zero-shot prompting with no context

Question 9

Refer to the exhibit. A team has deployed a model to an endpoint with the configuration shown. They notice that during peak traffic, the endpoint frequently returns 429 (Too Many Requests) errors. Which action should they take to resolve this issue?

Accepted Answer

Increase MIN_REPLICA_COUNT to 5. Increasing MIN_REPLICA_COUNT ensures a minimum number of replicas are always available to handle traffic bursts, reducing 429 errors. Other options would not help or would worsen the problem.

Answer

Change MACHINE_TYPE to n1-highmem-4

Answer

Decrease MAX_REPLICA_COUNT to 1

Answer

Disable autoscaling by setting MIN_REPLICA_COUNT equals MAX_REPLICA_COUNT

Question 10

An organization wants to use a generative model to automatically generate legal contracts. The model must produce clauses that are not only grammatically correct but also legally enforceable and consistent with current jurisdiction laws. Which combination of techniques best ensures legal compliance?

Accepted Answer

Fine-tune a model on a diverse set of enforceable contracts and incorporate an external compliance verifier that uses rule-based checks.. Option D is correct because fine-tuning on curated legal documents teaches domain-specific language and enforceability, while a verifier (an external logic/rule system) checks compliance with laws. Option A is incorrect because prompt engineering is unreliable for precise legal reasoning. Option B is incorrect because small model likely lacks legal knowledge. Option C is incorrect because RAG retrieves but does not verify enforceability.

Answer

Fine-tune a small model exclusively on legal contracts from a single jurisdiction and use it for generation.

Answer

Implement retrieval-augmented generation (RAG) with a vector database of all relevant laws.

Answer

Use a large instruction-tuned model with carefully engineered prompts describing jurisdiction details.

Question 11

Refer to the exhibit. This IAM policy is applied to a Vertex AI project. A user 'test@example.com' reports they cannot create a ModelEvaluationPipelineJob. Which action should the administrator take?

Accepted Answer

Add the user to the roles/aiplatform.admin role at the project level.. Roles/aiplatform.user does not have the permissions to create pipeline jobs; it only allows viewing and using models and endpoints. Roles/aiplatform.admin has full control, so adding the user to this role is the simplest fix. There is no roles/aiplatform.specialist; removing the service account would not help; and granting at the model level is insufficient for creating pipeline jobs.

Answer

Grant the user roles/aiplatform.specialist at the project level.

Answer

Add the user roles/aiplatform.user at the model level to allow pipeline creation.

Answer

Remove the service account from roles/aiplatform.admin.

Question 12

A data scientist notices that a Gemini model generates inconsistent responses to similar prompts. What is the likely cause?

Accepted Answer

The top_p or temperature parameters are set too high causing randomness. Option D is correct because high temperature (e.g., >1.0) or high top_p (e.g., >0.9) increases the randomness of token sampling, causing the model to select less probable tokens. This directly leads to inconsistent responses for similar prompts, as the model's output distribution becomes more uniform and less deterministic.

Answer

Model is not fine-tuned enough

Answer

The prompt is too short

Answer

The temperature setting is too low

Question 13

A developer is using Vertex AI Studio to test a text generation model. Which two actions can be performed in Vertex AI Studio? (Choose TWO)

Accepted Answer

Deploy a model to an endpoint. Option D is correct because Vertex AI Studio provides a direct interface to deploy a text generation model to an endpoint for serving predictions. This action is a core capability of the platform, allowing developers to test and then operationalize their models without leaving the Studio environment.

Answer

Manage IAM roles

Answer

Monitor model cost

Answer

Create a dataset

Question 14

Which TWO of the following are best practices for prompt engineering?

Accepted Answer

Provide context and examples in the prompt. Clear and specific instructions help guide the model, and providing context and examples improves output quality. Options B, D, and E are not recommended.

Answer

Append random noise to prompts to improve creativity

Answer

Always use the maximum possible number of tokens

Answer

Use negative prompts to discourage undesired outputs

Question 15

A team is tuning a large language model for a question-answering task. They notice the model gives high confidence scores to answers that are factually incorrect. Which evaluation metric should they primarily use to detect this overconfidence problem?

Accepted Answer

Expected Calibration Error (ECE). Expected Calibration Error (ECE) directly measures the alignment between a model's predicted confidence and its actual accuracy. In this scenario, high confidence on incorrect answers indicates miscalibration, and ECE quantifies this mismatch by binning predictions by confidence and computing the average absolute difference between accuracy and confidence per bin.

Answer

Perplexity

Answer

BLEU score

Answer

ROUGE-L