Databricks-GenAI-Assoc Practice Test 7 — 10 Questions

Question 1

Refer to the exhibit. What is the cause of this error when logging a RAG chain to MLflow?

Accepted Answer

The model was logged without a signature defining input and output types.. This error occurs because the model signature is missing or incorrectly defined, which is required for MLflow to perform input validation and inference. When serving a RAG chain, MLflow needs to know exactly what the input schema looks like so it can properly route requests and manage data types. Providing a valid signature ensures the model is deployable and functional.

Answer

The vector index is offline.

Answer

The user has insufficient Unity Catalog permissions.

Answer

The LLM endpoint is overloaded.

Question 2

A GenAI engineer is designing a retrieval-augmented generation application whose source documents are long PDFs. Early testing shows that answers are vague because retrieved chunks contain several unrelated topics, and the language model frequently cites content that does not support its claims. The engineer wants to improve chunk quality before indexing. Which TWO changes should the engineer make to the ingestion pipeline? (Choose two.)

Accepted Answer

Split documents along semantic boundaries such as sections and headings instead of a fixed character count. Chunk quality is determined at ingestion: splitting on semantic boundaries keeps each chunk topically focused, and overlap preserves meaning across boundaries. Together they raise the precision of retrieved context so the model has grounded material to cite. Embedding dimension, temperature, and oversized single chunks do not correct incoherent chunking.

Answer

Store the full PDF text in a single chunk per document

Answer

Increase the embedding dimension of the model used for the index

Answer

Lower the temperature setting on the language model

Question 3

A generative AI team runs a RAG chatbot whose Mosaic AI Model Serving endpoint is monitored in Unity Catalog inference tables. Over two weeks, the percentage of user questions that receive a refusal answer ('I don't have enough information') climbs from 4% to 31%, while retrieval latency and token counts stay flat. The team wants the earliest actionable signal that the retrieval corpus has gone stale rather than the prompt or model. Which monitoring signal should they inspect first?

Accepted Answer

The distribution of retrieval scores for the top-k chunks returned by the Vector Search index, tracked over time in the inference table.. Refusals rising while latency and token counts hold steady isolates the problem to retrieval quality rather than the model or the serving tier. Retrieval score distributions logged to the inference table show whether top-k chunks are becoming weaker matches, which is the earliest signal that the indexed corpus no longer covers current user questions. Checking capacity, token cost, or generation time cannot distinguish stale content from a healthy pipeline.

Answer

The average generation time per completion recorded by the serving endpoint's request logs.

Answer

The endpoint's provisioned concurrency utilization and queue depth metrics from the serving endpoint logs.

Answer

The count of prompt tokens consumed per request, compared week over week in the billing usage table.

Question 4

An engineer is preparing to deploy a Mosaic AI Agent to a Model Serving endpoint. The agent depends on a custom Python library that is not available on PyPI. Which approach ensures the library is available at serving time?

Accepted Answer

Include the library as a wheel file in the MLflow model's requirements or artifacts so it is installed in the serving environment.. Model Serving builds its container from the MLflow model's declared environment, so custom libraries must be packaged as artifacts and referenced in the model's requirements. This ensures the dependency is installed in the serving environment and the agent can import it reliably.

Answer

Copy the library source into the notebook that defines the agent and rely on the notebook's sys.path.

Answer

Install the library manually on the driver node of the cluster used to log the model.

Answer

Add the library to the workspace's global init script so all clusters install it at startup.

Question 5

A team runs an offline evaluation of a RAG chatbot using MLflow LLM Evaluation with an LLM judge. The same evaluation dataset produces noticeably different scores when re-run on different days, even though the application code and retrieved documents are unchanged. Which action best addresses this score instability?

Accepted Answer

Pin the judge model version and set temperature to zero for evaluation runs. Identical inputs producing different scores across runs indicate the judge itself is nondeterministic, typically through model version drift or sampling temperature. Pinning the judge model version and forcing temperature to zero makes scoring reproducible, so observed differences can be attributed to the application. The other options either change the system under test, destroy evaluation fidelity, or hide variance rather than eliminate it.

Answer

Increase the number of retrieved chunks per query so the judge has more context

Answer

Switch all metrics from LLM-judged to exact-match string comparison

Answer

Reduce the evaluation dataset to a single representative example

Question 6

An engineer has finished building a RAG chatbot and wants to expose it as a Databricks App so business users can reach it through a browser. The app needs a Python web server and a command that starts it. Which artifact in the app's project layout defines the runtime command and dependencies used when the app is deployed?

Accepted Answer

The app.yaml file, which declares the command to run and the environment or dependencies for the app.. A Databricks App is launched according to its app.yaml configuration, which specifies the command that starts the web server along with source and dependency information. The bundle file orchestrates deployment, requirements files list packages, and model signatures describe schemas, but only the app configuration tells the platform how to run the application.

Answer

The MLflow model signature stored with the logged model, which encodes the serving command.

Answer

The requirements.txt file alone, which both lists dependencies and specifies the server start command.

Answer

The databricks.yml bundle file, which contains the Python entrypoint and pip requirements for the app.

Question 7

A GenAI engineer is deploying a RAG application that uses Databricks Vector Search and a Foundation Model API. The solution must comply with governance policies that require all data access and model invocations to be auditable and access-controlled at a fine-grained level. Which two Unity Catalog features should the engineer leverage to meet these requirements? (Choose two.)

Accepted Answer

Unity Catalog privileges on the Vector Search index and the source Delta table.. Unity Catalog privileges on the Vector Search index and source table provide fine-grained access control, ensuring only authorized users can retrieve data. Inference tables for the Foundation Model API automatically log requests and responses, including user identity, creating an audit trail. Together, these features meet the access-control and auditability requirements for the RAG application.

Answer

Unity Catalog volumes to store the model weights and configuration files.

Answer

Cluster policies to restrict the types of clusters that can access the data.

Answer

Databricks SQL dashboards to visualize access patterns and model usage.

Question 8

Which of the following is a primary benefit of using Unity Catalog to manage models for a RAG application?

Accepted Answer

It provides centralized lineage and access control.. Unity Catalog provides a unified governance framework that manages permissions, lineage, and discovery across all data and AI assets. In a RAG context, this ensures that the entire pipeline—from the source Delta tables to the vector indexes and the final models—is traceable and governed. This consistency is essential for enterprise compliance and simplifies the operational management of the AI lifecycle by providing a centralized point for auditing and security policy application.

Answer

It automatically generates the optimal prompt for the model.

Answer

It eliminates the need for vector embeddings.

Answer

It forces all data to be stored in the cloud root.

Question 9

When deploying an application using Databricks Asset Bundles (DABs), which file is the primary entry point to define the project structure and configuration?

Accepted Answer

databricks.yml. Databricks Asset Bundles use a YAML-based configuration file, typically named 'databricks.yml', to define the project's resources, targets, and settings. This file acts as the single source of truth for the project's lifecycle, enabling consistent deployments across development, staging, and production environments. By using a declarative configuration, teams can version-control their entire infrastructure, ensuring that deployment patterns are standardized and repeatable across the organization.

Answer

requirements.txt

Answer

setup.py

Answer

bundle.json

Question 10

What is the primary purpose of the 'bundle validate' command in the Databricks Asset Bundles CLI?

Accepted Answer

To check for schema compliance and configuration errors.. The 'bundle validate' command performs a schema check and structural analysis on your configuration files. It ensures that the YAML is syntactically correct and that all referenced resources exist or will be created correctly. This step is crucial for preventing common errors before you attempt a deployment, saving time and avoiding partially successful deployments that could leave the environment in an inconsistent state.

Answer

To execute the deployment and verify it in the workspace.

Answer

To format the YAML files according to Databricks standards.

Answer

To automatically update the bundle version to the latest release.