Databricks-GenAI-Assoc Practice Test 22 — 15 Questions

Question 1

Which design pattern is best for protecting the LLM from prompt injection attacks when building a customer-facing chatbot on Databricks?

Accepted Answer

Using a guardrail service to validate and sanitize user inputs.. Implementing an input-filtering layer using a separate, lightweight classification model or guardrail service effectively detects and blocks malicious prompts before they reach the LLM. This proactive defense is vital for securing AI applications, as prompt injection can lead to unauthorized data access or malicious behaviors. This layering approach creates a defense-in-depth architecture that keeps the generative model safe from adversarial inputs.

Answer

Embedding all user inputs into the vector database.

Answer

Relying on the LLM’s internal safety training to reject injections.

Answer

Sending all inputs through a standard SQL query first.

Question 2

Which component of the Databricks platform allows you to bundle together notebooks, model serving configurations, and pipeline definitions for repeatable deployment?

Accepted Answer

Databricks Asset Bundles.. Databricks Asset Bundles (DABs) are the standardized way to manage and deploy project artifacts. By bundling these components, teams can maintain version control over their entire application ecosystem. This approach is fundamental for implementing robust CI/CD pipelines, ensuring that the development, testing, and production environments remain synchronized and predictable throughout the release cycle.

Answer

Databricks SQL Warehouse.

Answer

Unity Catalog.

Answer

MLflow Tracking.

Question 3

Which THREE of the following are common challenges when monitoring LLM applications in production that differ significantly from traditional ML monitoring? (Choose three)

Accepted Answer

Non-deterministic output behavior.. LLMs introduce unique challenges because their outputs are non-deterministic, high-dimensional, and often lack a single 'correct' answer. Traditional monitoring focuses on numerical drift and binary classification metrics like precision/recall. LLM evaluation requires semantic understanding, nuance assessment, and the ability to handle unstructured text, necessitating specialized tools like LLM-as-a-judge or human-in-the-loop validation to manage the subjectivity inherent in generative AI outputs.

Answer

Lack of high-throughput API endpoints.

Answer

The model's inability to connect to internet data.

Question 4

Refer to the exhibit. The Databricks Vector Search index is currently in an 'ONLINE' status with incremental sync. If a developer performs a DELETE operation on the source Delta table, what happens to the vector index?

Accepted Answer

The record is automatically removed from the index during the next sync cycle.. When using incremental synchronization, the Databricks Vector Search service monitors the Delta table's transaction log for changes. A delete operation recorded in the Delta log will trigger a corresponding removal in the index. This ensures the index remains a true representation of the source data, which is critical for compliance and data quality, ensuring the RAG application does not retrieve outdated or removed sensitive information.

Answer

The record remains in the index until a full re-index is manually triggered.

Answer

The system throws an error because vector indexes do not support deletes.

Answer

The index enters a 'PENDING_DELETE' state and stops accepting queries.

Question 5

A team is using MLflow to track their model evaluation runs. They want to ensure that every evaluation run is reproducible. What is the best practice to achieve this in Databricks?

Accepted Answer

Log the model URI, dataset version, and code hash.. Reproducibility in evaluation requires logging not only the results but also the exact artifacts, environment, and code version used. MLflow's ability to log the model URI, the specific evaluation dataset version, and the code state (via git hash) ensures that an evaluation run can be perfectly recreated. This is mandatory for auditing and verifying improvements in LLM performance over time, preventing 'black box' results that cannot be validated.

Answer

Only log the final scalar metrics.

Answer

Hardcode the dataset paths in the notebook.

Answer

Use a global variable for model versioning.

Question 6

Which TWO of the following are benefits of using Delta Lake for data preparation over standard Parquet files on cloud storage?

Accepted Answer

ACID compliance for reliable write operations.. Delta Lake provides ACID transactions and time travel, which are essential for robust data engineering. ACID transactions ensure that data preparation pipelines never leave tables in a partially written state, while time travel allows for auditing and reverting changes. These features significantly simplify data lifecycle management, reduce the need for custom retry logic, and enhance the overall reliability of data preparation workflows in complex, multi-user production environments.

Answer

Automatic conversion of JSON to XML format.

Answer

Native support for cross-cloud streaming ingestion.

Answer

Automatic hardware upgrades for compute clusters.

Question 7

An engineer is using Mosaic AI Agent Evaluation to score a conversational agent that calls tools. The agent sometimes answers correctly but with fabricated citations. Which evaluation approach best surfaces this specific failure mode?

Accepted Answer

Enable the groundedness judge and supply the retrieved context so the judge can verify claims against sources.. The groundedness judge compares response claims against the retrieved context, which is precisely how fabricated citations are detected. A correctness judge evaluates answers against references and misses unsupported citations, operational metrics say nothing about grounding, and removing tools avoids the scenario rather than evaluating it.

Answer

Measure only token usage and latency to detect when the agent is hallucinating citations.

Answer

Run the agent without tools so it cannot produce citations at all.

Answer

Rely solely on the correctness judge, since a correct answer cannot contain fabricated citations.

Question 8

When building an application that retrieves context from Databricks Vector Search, what is the recommended data format for storing the document chunks?

Accepted Answer

Delta tables. Delta tables are the native, optimized storage format for Databricks. When using Vector Search, the index is automatically maintained by Delta tables. This provides a performant and reliable way to sync data between the source Delta table and the vector index. Storing data in Delta ensures that the vector search index can benefit from CDC (Change Data Capture) patterns, keeping the RAG knowledge base automatically updated and consistent.

Answer

JSON files in an S3 bucket

Answer

Parquet files on DBFS root

Answer

SQL Server database

Question 9

When deploying a model endpoint, which Databricks feature provides the capability to review and approve the model before it is promoted to production?

Accepted Answer

Unity Catalog Model Status Transitions. The Unity Catalog Model Registry provides a formal lifecycle management process, including status transitions such as 'Staging', 'Production', and 'Archived'. By using these status labels, organizations can implement a human-in-the-loop approval process where models must pass specific validation criteria—or manual reviews—before they are eligible for the production inference endpoint. This promotes safety, quality, and compliance in machine learning deployments by ensuring only vetted models reach end-users.

Answer

Delta Live Tables Quality Checks

Answer

Databricks Job Task Notifications

Answer

Git branch protection rules

Question 10

Which Databricks feature is specifically designed to monitor model quality and drift in production?

Accepted Answer

MLflow Model Monitoring.. MLflow Model Monitoring (now part of Mosaic AI) provides the necessary tooling to track model performance metrics and identify drift over time. For generative AI, this is critical because language models can exhibit performance degradation or shifts in behavior when input distributions change. Proactive monitoring allows engineers to detect these shifts early and trigger retraining or fine-tuning, ensuring consistent performance in the production environment.

Answer

Unity Catalog audit logs.

Answer

The Databricks SQL query history.

Answer

Cluster event logs.

Question 11

A developer needs to monitor the performance of an LLM application in production. They want to track the latency of their model endpoint. Where can they find this metric in the Databricks workspace?

Accepted Answer

In the 'Monitoring' tab of the Model Serving endpoint UI.. Databricks Model Serving provides built-in monitoring dashboards for every endpoint. These dashboards automatically track key performance indicators such as request latency, throughput, and error rates. Monitoring these metrics is vital for understanding the user experience and detecting potential performance bottlenecks. By providing this information natively, Databricks helps developers maintain high service quality without requiring external observability tools, streamlining the monitoring and optimization process for deployed models.

Answer

In the Unity Catalog table lineage view.

Answer

In the Databricks SQL Warehouse query history.

Answer

In the workspace audit logs.

Question 12

Which TWO evaluation approaches are most effective for measuring the quality of a RAG pipeline's retrieval stage?

Accepted Answer

Context Precision. Evaluating the retrieval stage requires assessing how well the system identifies relevant documents from the vector store before the LLM generates a response. Context Precision and Context Recall provide quantitative measures to ensure the retrieval engine is performing correctly. Without these metrics, it is impossible to determine if the LLM is failing due to poor generation or simply because it lacked the correct information to answer the query.

Answer

Model Answer Faithfulness

Answer

Answer Semantic Similarity

Answer

LLM Latency Monitoring

Question 13

Refer to the exhibit. An AI engineer is configuring a Databricks Asset Bundle (DAB) to deploy a generative AI application. When executing 'databricks bundle deploy --target prod', which workspace host will the bundle resources be deployed to, and why?

Accepted Answer

It will deploy to the production workspace at https://adb-987654321.azuredatabricks.net because the target flag explicitly selects the prod configuration block.. Databricks Asset Bundles allow developers to define multiple deployment targets within the databricks.yml configuration file. When the user explicitly passes the --target prod flag to the CLI command, the bundle execution context switches from the default target to the specified target configuration, overriding default settings and directing deployment to the production workspace host URL defined under the prod block.

Answer

The deployment will fail because the default target conflicts with the production target definition during workspace authentication initialization.

Answer

It will deploy to the development workspace because the dev target is explicitly marked with 'default: true'.

Answer

It will deploy to both workspaces simultaneously to maintain synchronization between development and production environments.

Question 14

A team is preparing a fine-tuning dataset from customer reviews stored in a Delta table. They need to filter out reviews shorter than 20 tokens and reviews flagged as spam by a classifier, then write the result to a Unity Catalog table for training. Which approach best fits Databricks best practices?

Accepted Answer

Use a Spark DataFrame with a token-count UDF and a filter on the spam flag, then write the result to a Unity Catalog table with .write.mode("overwrite").saveAsTable().. Batch preprocessing of an existing Delta table is best done with Spark DataFrame transformations, which distribute token counting and filtering across executors. Writing the cleaned result to a Unity Catalog table creates a governed, reproducible training artifact. Driver-side pandas and row-by-row writes do not scale, views recompute on every read, and streaming pipelines are overkill for a one-time preparation step.

Answer

Load the table into a pandas DataFrame on the driver, apply filtering, and write it back using the Databricks SQL connector row by row.

Answer

Create a view that filters short and spam reviews, then point the fine-tuning job at the view without materializing a new table.

Answer

Use Delta Live Tables with a streaming table that ingests all reviews and applies expectations to drop short and spam rows at write time.

Question 15

When preparing data for a fine-tuning task, you realize the dataset is severely imbalanced. Which Databricks technique should you use to create a more balanced dataset?

Accepted Answer

Use the sampleBy() method to perform stratified sampling.. Using Spark's 'sample()' or 'sampleBy()' transformation allows you to perform stratified oversampling or undersampling to balance classes. This ensures that the fine-tuned model doesn't become biased toward the most frequent categories in the dataset. Proper balancing is essential for ensuring robust model performance across all target classes, preventing the model from underperforming on rare but critical edge cases in real-world applications.

Answer

Run a standard 'SELECT * FROM table' query.

Answer

Increase the number of epochs during the training process.

Answer

Use a UDF to delete all rows that belong to majority classes.