Databricks-GenAI-Assoc Practice Exam 4 — 20 Questions

Question 1

Refer to the exhibit. What is the cause of this error when logging a RAG chain to MLflow?

Accepted Answer

The model was logged without a signature defining input and output types.. This error occurs because the model signature is missing or incorrectly defined, which is required for MLflow to perform input validation and inference. When serving a RAG chain, MLflow needs to know exactly what the input schema looks like so it can properly route requests and manage data types. Providing a valid signature ensures the model is deployable and functional.

Answer

The vector index is offline.

Answer

The user has insufficient Unity Catalog permissions.

Answer

The LLM endpoint is overloaded.

Question 2

A GenAI engineer is designing a retrieval-augmented generation application whose source documents are long PDFs. Early testing shows that answers are vague because retrieved chunks contain several unrelated topics, and the language model frequently cites content that does not support its claims. The engineer wants to improve chunk quality before indexing. Which TWO changes should the engineer make to the ingestion pipeline? (Choose two.)

Accepted Answer

Split documents along semantic boundaries such as sections and headings instead of a fixed character count. Chunk quality is determined at ingestion: splitting on semantic boundaries keeps each chunk topically focused, and overlap preserves meaning across boundaries. Together they raise the precision of retrieved context so the model has grounded material to cite. Embedding dimension, temperature, and oversized single chunks do not correct incoherent chunking.

Answer

Store the full PDF text in a single chunk per document

Answer

Increase the embedding dimension of the model used for the index

Answer

Lower the temperature setting on the language model

Question 3

A generative AI team runs a RAG chatbot whose Mosaic AI Model Serving endpoint is monitored in Unity Catalog inference tables. Over two weeks, the percentage of user questions that receive a refusal answer ('I don't have enough information') climbs from 4% to 31%, while retrieval latency and token counts stay flat. The team wants the earliest actionable signal that the retrieval corpus has gone stale rather than the prompt or model. Which monitoring signal should they inspect first?

Accepted Answer

The distribution of retrieval scores for the top-k chunks returned by the Vector Search index, tracked over time in the inference table.. Refusals rising while latency and token counts hold steady isolates the problem to retrieval quality rather than the model or the serving tier. Retrieval score distributions logged to the inference table show whether top-k chunks are becoming weaker matches, which is the earliest signal that the indexed corpus no longer covers current user questions. Checking capacity, token cost, or generation time cannot distinguish stale content from a healthy pipeline.

Answer

The average generation time per completion recorded by the serving endpoint's request logs.

Answer

The endpoint's provisioned concurrency utilization and queue depth metrics from the serving endpoint logs.

Answer

The count of prompt tokens consumed per request, compared week over week in the billing usage table.

Question 4

An engineer is preparing to deploy a Mosaic AI Agent to a Model Serving endpoint. The agent depends on a custom Python library that is not available on PyPI. Which approach ensures the library is available at serving time?

Accepted Answer

Include the library as a wheel file in the MLflow model's requirements or artifacts so it is installed in the serving environment.. Model Serving builds its container from the MLflow model's declared environment, so custom libraries must be packaged as artifacts and referenced in the model's requirements. This ensures the dependency is installed in the serving environment and the agent can import it reliably.

Answer

Copy the library source into the notebook that defines the agent and rely on the notebook's sys.path.

Answer

Install the library manually on the driver node of the cluster used to log the model.

Answer

Add the library to the workspace's global init script so all clusters install it at startup.

Question 5

A team runs an offline evaluation of a RAG chatbot using MLflow LLM Evaluation with an LLM judge. The same evaluation dataset produces noticeably different scores when re-run on different days, even though the application code and retrieved documents are unchanged. Which action best addresses this score instability?

Accepted Answer

Pin the judge model version and set temperature to zero for evaluation runs. Identical inputs producing different scores across runs indicate the judge itself is nondeterministic, typically through model version drift or sampling temperature. Pinning the judge model version and forcing temperature to zero makes scoring reproducible, so observed differences can be attributed to the application. The other options either change the system under test, destroy evaluation fidelity, or hide variance rather than eliminate it.

Answer

Increase the number of retrieved chunks per query so the judge has more context

Answer

Switch all metrics from LLM-judged to exact-match string comparison

Answer

Reduce the evaluation dataset to a single representative example

Question 6

An engineer has finished building a RAG chatbot and wants to expose it as a Databricks App so business users can reach it through a browser. The app needs a Python web server and a command that starts it. Which artifact in the app's project layout defines the runtime command and dependencies used when the app is deployed?

Accepted Answer

The app.yaml file, which declares the command to run and the environment or dependencies for the app.. A Databricks App is launched according to its app.yaml configuration, which specifies the command that starts the web server along with source and dependency information. The bundle file orchestrates deployment, requirements files list packages, and model signatures describe schemas, but only the app configuration tells the platform how to run the application.

Answer

The MLflow model signature stored with the logged model, which encodes the serving command.

Answer

The requirements.txt file alone, which both lists dependencies and specifies the server start command.

Answer

The databricks.yml bundle file, which contains the Python entrypoint and pip requirements for the app.

Question 7

A GenAI engineer is deploying a RAG application that uses Databricks Vector Search and a Foundation Model API. The solution must comply with governance policies that require all data access and model invocations to be auditable and access-controlled at a fine-grained level. Which two Unity Catalog features should the engineer leverage to meet these requirements? (Choose two.)

Accepted Answer

Unity Catalog privileges on the Vector Search index and the source Delta table.. Unity Catalog privileges on the Vector Search index and source table provide fine-grained access control, ensuring only authorized users can retrieve data. Inference tables for the Foundation Model API automatically log requests and responses, including user identity, creating an audit trail. Together, these features meet the access-control and auditability requirements for the RAG application.

Answer

Unity Catalog volumes to store the model weights and configuration files.

Answer

Cluster policies to restrict the types of clusters that can access the data.

Answer

Databricks SQL dashboards to visualize access patterns and model usage.

Question 8

Which of the following is a primary benefit of using Unity Catalog to manage models for a RAG application?

Accepted Answer

It provides centralized lineage and access control.. Unity Catalog provides a unified governance framework that manages permissions, lineage, and discovery across all data and AI assets. In a RAG context, this ensures that the entire pipeline—from the source Delta tables to the vector indexes and the final models—is traceable and governed. This consistency is essential for enterprise compliance and simplifies the operational management of the AI lifecycle by providing a centralized point for auditing and security policy application.

Answer

It automatically generates the optimal prompt for the model.

Answer

It eliminates the need for vector embeddings.

Answer

It forces all data to be stored in the cloud root.

Question 9

When deploying an application using Databricks Asset Bundles (DABs), which file is the primary entry point to define the project structure and configuration?

Accepted Answer

databricks.yml. Databricks Asset Bundles use a YAML-based configuration file, typically named 'databricks.yml', to define the project's resources, targets, and settings. This file acts as the single source of truth for the project's lifecycle, enabling consistent deployments across development, staging, and production environments. By using a declarative configuration, teams can version-control their entire infrastructure, ensuring that deployment patterns are standardized and repeatable across the organization.

Answer

requirements.txt

Answer

setup.py

Answer

bundle.json

Question 10

What is the primary purpose of the 'bundle validate' command in the Databricks Asset Bundles CLI?

Accepted Answer

To check for schema compliance and configuration errors.. The 'bundle validate' command performs a schema check and structural analysis on your configuration files. It ensures that the YAML is syntactically correct and that all referenced resources exist or will be created correctly. This step is crucial for preventing common errors before you attempt a deployment, saving time and avoiding partially successful deployments that could leave the environment in an inconsistent state.

Answer

To execute the deployment and verify it in the workspace.

Answer

To format the YAML files according to Databricks standards.

Answer

To automatically update the bundle version to the latest release.

Question 11

What is the primary function of a Storage Credential in Unity Catalog?

Accepted Answer

To authorize Databricks to access specific cloud storage.. A Storage Credential serves as a secure container for the cloud-provider credentials required to access external storage locations. By separating the credential from the data objects, Unity Catalog allows administrators to manage access to cloud storage in a centralized, governed way without exposing secrets to individual users. This is a fundamental security practice that prevents credential leakage.

Answer

To store user passwords for Databricks.

Answer

To define the schema of a table.

Answer

To cache table data for faster query performance.

Question 12

When monitoring a production LLM, you detect a drift where the model's responses are becoming increasingly verbose and less helpful compared to the baseline. Which strategy is most effective for detecting this quality decay?

Accepted Answer

Implement automated LLM-as-a-judge evaluations on a sample of production inputs.. Model quality decay in LLMs is best detected through continuous automated evaluation using a 'judge' model. By running a set of evaluation prompts through the production model and grading them against an LLM-as-a-judge, you can quantify performance trends. This proactive monitoring allows teams to identify when model behavior deviates from expected standards, triggering alerts before the issues impact a large number of end-users or require significant architectural changes.

Answer

Monitor system-level CPU and memory usage of the inference cluster.

Answer

Set up alerts for high request volume spikes in the API logs.

Answer

Require manual reviews for every single response generated by the model.

Question 13

A GenAI engineer is building a retrieval-augmented generation application on Databricks. They want to store document embeddings and perform fast approximate nearest-neighbor search without managing a separate vector database. They have already created a source Delta table with columns: id (string), text (string), and embedding (array<float>). Which Databricks feature should they use to create a Vector Search index that automatically syncs with the Delta table?

Accepted Answer

Databricks Vector Search with a Delta Sync index. Databricks Vector Search is the managed service designed for storing embeddings and performing fast similarity search. A Delta Sync index automatically syncs with a source Delta table, so when new documents or embeddings are added, the index updates without manual intervention. This eliminates the need to manage a separate vector database and integrates natively with Databricks workflows.

Answer

Databricks Feature Store with a training set

Answer

MLflow Model Registry with a custom PyFunc model

Answer

Delta Live Tables with a materialized view

Question 14

When designing an application that requires fine-tuning a small model (like Llama-3-8B) on Databricks, which THREE factors must be considered to ensure a successful training job?

Accepted Answer

Ensuring the training data is sufficiently representative of the target task.. Training small models on Databricks requires careful consideration of compute, data quality, and hyperparameter tuning. By ensuring sufficient GPU memory, clean data, and properly tuned learning rates, engineers can successfully adapt the model to new domains. These factors are critical to avoid common pitfalls like catastrophic forgetting or overfitting, which can render the fine-tuned model less useful than its base version.

Answer

Disabling checkpointing to save storage costs.

Answer

Increasing the batch size to the maximum possible for all datasets.

Question 15

Which command is used within the Databricks CLI to deploy a project defined by a Databricks Asset Bundle?

Accepted Answer

databricks bundle deploy. The 'databricks bundle deploy' command is the primary interface for deploying resources defined in a Databricks Asset Bundle. It interprets the 'databricks.yml' configuration, validates the resources, and then synchronizes the code and configuration with the targeted Databricks workspace. This CLI-first approach is essential for automating CI/CD pipelines, as it allows developers to trigger deployments programmatically without relying on manual browser-based interactions or fragile API scripts.

Answer

databricks workspace sync

Answer

databricks job create

Answer

databricks deploy project

Question 16

Your organization requires an auditable record of all LLM evaluation results for compliance. Which Databricks feature provides the best centralized storage for these evaluation runs?

Accepted Answer

MLflow Experiments. MLflow Experiments act as the centralized repository for all model development and evaluation runs. By logging evaluation results to MLflow, teams create a permanent, version-controlled audit trail. This allows stakeholders to compare different iterations of models, review evaluation metrics, and verify that quality gates were met before deployment, ensuring compliance with internal AI governance policies and external regulatory frameworks for model transparency.

Answer

Databricks File System (DBFS) text logs

Answer

Unity Catalog volume for raw CSV storage

Answer

Git commit history of the inference code

Question 17

A developer needs to store prompt templates, model parameters, and evaluation results for a GenAI application so that each iteration can be compared and reproduced later. Which Databricks capability should they use?

Accepted Answer

MLflow Tracking with experiments and runs. MLflow Tracking provides experiments and runs that capture parameters, metrics, and artifacts, making it the right tool to version prompts, record model settings, and store evaluation results for comparison and reproduction. Data pipelines, volumes, and dashboards serve different purposes and do not offer run-based experiment tracking.

Answer

Delta Live Tables pipelines

Answer

Databricks SQL dashboards

Answer

Unity Catalog volumes

Question 18

A team is preparing data for a RAG system and needs to remove duplicates from a large collection of PDF text extracts. What is the most efficient way to perform de-duplication in Databricks?

Accepted Answer

Execute a distributed 'dropDuplicates' operation in Spark.. Using Spark's 'dropDuplicates()' on the text content column or calculating a hash (e.g., MD5) of the text to identify duplicates is highly scalable. This is crucial because redundant information in the vector database can cause the retriever to prioritize duplicate entries, leading to biased results and inefficiency. Removing duplicates ensures that the search index remains focused and that the retrieved context is diverse and informative.

Answer

Use a Python 'set' in a local loop on the driver node.

Answer

Manually compare every document against every other document using nested loops.

Answer

Ignore duplicates, as vector databases automatically handle them during insertion.

Question 19

You are building an application that uses Model Serving to host a fine-tuned LLM. Which configuration is required to optimize for high-concurrency request throughput?

Accepted Answer

Enable 'Provisioned Throughput' on the serving endpoint configuration.. Enabling Provisioned Throughput for Model Serving allows for dedicated resources that can handle high-concurrency workloads efficiently. This is crucial for applications serving many users simultaneously. By optimizing resource allocation, you prevent bottlenecks during peak usage, ensuring consistent performance and minimizing request latency, which directly impacts the user experience and the scalability of the overall application architecture.

Answer

Set the serving endpoint to use CPU-only instances for all LLM inference tasks.

Answer

Disable auto-scaling and fix the number of replicas to one.

Answer

Configure the endpoint to use an external API key for every request.

Question 20

Which TWO factors should be prioritized when selecting an embedding model for a domain-specific RAG application on Databricks?

Accepted Answer

The semantic relevance of the model to the target domain's terminology.. Selecting the right embedding model requires balancing semantic accuracy within the specific domain and the operational cost of maintaining the model. By prioritizing domain-specific performance and resource efficiency, teams ensure that the RAG pipeline provides relevant search results without incurring excessive latency or compute costs, which are foundational for sustaining long-term generative AI production workloads.

Answer

The total number of parameters in the model regardless of the domain.

Answer

The availability of the model on the public Hugging Face repository.

Answer

The color scheme of the model's documentation page.