Databricks-GenAI-Assoc Practice Test 17 — 15 Questions

Question 1

A generative AI engineer is designing an agent on Databricks that uses a LangChain agent with tool-calling capabilities. The agent must call a Databricks SQL warehouse to run queries and a Vector Search index for retrieval. Which design consideration is most important for controlling agent behavior in production?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Define clear tool descriptions and limit the agent's available tools to only those required for the task.. Agent behavior is driven by tool descriptions and the set of tools available. Clear descriptions help the model choose the right tool, and a minimal toolset reduces the chance of unintended calls. Iteration limits, logging, and model size affect performance or observability but do not directly control which tools the agent invokes.

Answer

Use the largest available foundation model for the agent to maximize reasoning quality.

Answer

Enable verbose logging of every intermediate step and store the logs in a Delta table for later review.

Answer

Increase the agent's maximum iterations to a very high number so it can always complete complex tasks.

Question 2

A team runs a RAG application on Mosaic AI Model Serving and logs all requests to an inference table. Reviewers report that answers are sometimes fluent but contradict the retrieved documents. The team wants a recurring, automated check that quantifies this contradiction on production traffic and alerts when it exceeds a threshold. Which approach best fits?

Accepted Answer

Schedule a Databricks job that runs mlflow.evaluate() with the groundedness judge over recent inference-table records and writes results to a Delta table for alerting.. Groundedness evaluation compares generated answers against retrieved context, which is precisely the failure mode described. Running it as a scheduled job over recent inference-table records produces a quantified, recurring metric, and persisting results to Delta supports alerting on threshold breaches. Operational metrics and manual review cannot detect semantically unsupported but fluent responses.

Answer

Enable inference table payload logging and inspect raw prompt and response text manually each week.

Answer

Increase the number of retrieved chunks per query so the model has more context to draw from.

Answer

Monitor the endpoint's p95 latency and error rate in the serving endpoint metrics and alert when either degrades.

Question 3

An engineer is building a GenAI application that must return structured JSON output conforming to a specific schema so downstream systems can parse it reliably. The team wants to enforce the schema at generation time rather than post-processing free-form text. Which approach should they use?

Accepted Answer

Use a structured output mode or response format that constrains generation to the provided JSON schema.. Structured output or response format settings constrain the model's decoding process to the supplied JSON schema, guaranteeing that generated content parses and conforms. This is more reliable than prompt instructions, fine-tuning, or regex extraction because it operates at generation time and eliminates malformed or incomplete output. Downstream systems can then parse results without defensive fallback logic.

Answer

Fine-tune the foundation model on examples of the desired JSON output.

Answer

Post-process the model output with a regular expression that extracts the first JSON object.

Answer

Set the LLM temperature to zero and instruct the model to return JSON in the system prompt.

Question 4

A generative AI engineer is designing a RAG application on Databricks that uses a foundation model served via Databricks Model Serving. The application must handle peak loads gracefully and provide consistent response times. The engineer is evaluating design patterns for scaling and reliability. Which TWO design choices should the engineer implement? (Choose two.)

Accepted Answer

Configure the Model Serving endpoint with autoscaling enabled and set a minimum and maximum number of concurrent requests per replica.. To handle peak loads and maintain consistent response times, the endpoint must scale elastically and the client must handle transient failures. Autoscaling with concurrency limits per replica ensures the serving infrastructure adapts to load, while client-side retries with backoff and timeouts improve resilience. These two choices together provide both server-side elasticity and client-side fault tolerance, which are essential for production reliability.

Answer

Deploy the model to a single large GPU replica with maximum memory to handle all traffic without autoscaling.

Answer

Precompute all possible user questions and their answers and store them in a lookup table for instant retrieval.

Answer

Cache the LLM responses for identical user queries in a Delta table and serve them without invoking the model.

Question 5

A GenAI engineer is designing an agentic application on Databricks that uses a foundation model to decide which external tools to call. The team wants the agent to be able to invoke a Databricks SQL warehouse query and a Python function registered as a Unity Catalog function, and they need the model to select tools based on natural language requests. Which design element is required for the model to select and invoke these tools correctly?

Accepted Answer

Provide tool definitions with names, descriptions, and parameter schemas to the model in the request.. Function-calling models rely on tool definitions passed in the request, including name, description, and parameter schema, to decide which tool to invoke and how to format arguments. Providing these definitions for the SQL warehouse query and Unity Catalog function gives the model the runtime contract it needs. Fine-tuning, volume storage, and inference tables do not supply that contract and therefore cannot enable correct tool selection.

Answer

Enable inference tables on the model serving endpoint to capture tool calls.

Answer

Store the tool list in a Unity Catalog volume and have the agent read it before each request.

Answer

Fine-tune the foundation model on the team's historical tool-call logs.

Question 6

A data science team wants to expose a RAG chain as a REST API so that an external web application can send questions and receive answers. The chain is developed with Databricks LangChain integrations and must be deployed with autoscaling and built-in monitoring. Which Databricks capability should they use?

Accepted Answer

Model Serving with a custom MLflow pyfunc model that wraps the chain.. Model Serving is the Databricks capability for hosting models and custom Python logic as REST endpoints with autoscaling and monitoring. Wrapping the LangChain chain in an MLflow pyfunc model makes it deployable through that service, so the web application can call a stable API. Scheduled jobs, SQL warehouses, and manual Flask apps do not provide the same managed serving characteristics.

Answer

A Databricks job that runs the chain on a schedule and writes answers to a Delta table.

Answer

An all-purpose cluster with a Flask app started manually on the driver node.

Answer

A Databricks SQL warehouse with a query that calls the chain through a UDF.

Question 7

A data scientist is developing a GenAI application that uses a foundation model served via Databricks Model Serving. The model endpoint is configured to log inference tables for monitoring. The data science team wants to ensure that the inference logs, which may contain sensitive user prompts, are protected according to Unity Catalog policies. Which Unity Catalog object should be used to store and govern the inference tables?

Accepted Answer

A managed table in a Unity Catalog schema with appropriate grants and column masks.. Inference tables logged by Databricks Model Serving are Delta tables that can be stored in Unity Catalog. Using a managed table in a Unity Catalog schema allows the team to apply Unity Catalog governance features such as grants, column masks, and row filters to protect sensitive data. This ensures that the inference logs are subject to the same security policies as other data assets, providing centralized governance and auditability.

Answer

A volume in Unity Catalog with file-level access controls.

Answer

An external table pointing to a cloud storage location with a storage credential.

Answer

A view that dynamically redacts sensitive columns from the inference logs.

Question 8

A Generative AI engineer is setting up MLflow LLM Evaluation for a RAG pipeline on Databricks that answers questions about product manuals. The team wants to evaluate both the retrieval stage and the generation stage in a single evaluation run. Which TWO metric groups should they include in the evaluation configuration? (Choose two.)

Accepted Answer

Retrieval metrics such as retrieval_relevance and retrieval_groundedness. A complete RAG evaluation run needs metrics for both stages. Retrieval metrics score the chunks returned by the retriever, while generation metrics such as groundedness and relevance score the final answer against the question and context. Together they let the team pinpoint whether a quality issue originates in retrieval or in generation.

Answer

System metrics such as CPU utilization and memory consumption of the serving endpoint

Answer

Training metrics such as loss and learning rate from the fine-tuning job

Answer

Data freshness metrics such as table update timestamps in Unity Catalog

Question 9

A team is using `databricks.agents.deploy()` to publish a Mosaic AI Agent to a serving endpoint. They must expose an environment-specific Vector Search index name and the endpoint name to the deployment without hardcoding values in the notebook, and the same notebook must run in dev and prod. Which approach should the engineer use?

Accepted Answer

Read the values from `dbutils.widgets` defined at the top of the notebook and pass them into `databricks.agents.deploy()`.. The cleanest way to run one notebook across dev and prod is to parameterize it with notebook widgets, which jobs and Databricks Asset Bundles can populate per target. The values then flow into `databricks.agents.deploy()` so the Vector Search index and endpoint names differ by environment without code changes. Duplicated notebooks, host-string branching, and Delta-table config stores all introduce drift or fragility that the scenario's requirements exclude.

Answer

Store the values in a Delta table and have the notebook query that table at deploy time.

Answer

Detect the environment by reading the current workspace URL inside the notebook and branch on it.

Answer

Hardcode the prod values and use a separate copied notebook for dev with the dev values.

Question 10

An engineer is building a multi-turn GenAI assistant on Databricks. The assistant must answer follow-up questions that reference earlier turns, such as 'what about its warranty?', while keeping each request within the model's context limit. Which design should the engineer implement?

Accepted Answer

Maintain the conversation in a Delta table, then on each turn build a prompt from a bounded recent window of turns plus a summary of older turns, along with retrieved context.. Bounding prompt size while preserving conversational antecedents is achieved by keeping a recent window of turns and summarizing older ones, with per-turn retrieval for grounding. Stateless endpoints, untrimmed history replay, and vector-encoded history all fail to give the model usable prior context within the context limit.

Answer

Send only the current user message to the model on every turn and rely on the model's built-in memory of prior requests.

Answer

Persist the conversation in a Delta table and, on each turn, resend the entire raw history along with retrieved context, trimming nothing.

Answer

Encode the entire conversation history into a single embedding and prepend that vector to the prompt so the model can reconstruct prior turns.

Question 11

A financial services company is designing a RAG assistant that must never return answers containing personally identifiable information from its knowledge base. The team plans to filter retrieved chunks before they are placed into the prompt. Which Databricks design element should they use to enforce this filtering consistently?

Accepted Answer

Store a sensitivity label or metadata field on each chunk in the Vector Search index and apply a metadata filter at query time to exclude restricted chunks.. Metadata filtering in Vector Search allows the application to exclude chunks based on attributes such as sensitivity level before they reach the prompt. This enforces the policy at the retrieval boundary, reducing both leakage risk and token usage. Prompt instructions and output regexes act later and cannot guarantee that sensitive content never enters the model context.

Answer

Add a system prompt instructing the model to ignore any personally identifiable information it sees in the context.

Answer

Run a post-processing regex on the model output to remove patterns that resemble personally identifiable information.

Answer

Grant the serving endpoint a service principal that lacks access to the sensitive tables so retrieval returns no restricted rows.

Question 12

An engineer is designing a Databricks GenAI application that summarizes customer meeting notes. The notes contain personally identifiable information, and company policy requires that the summarization model never sends note text to an external provider. The team already has a fine-tuned open model registered in Unity Catalog. Which deployment choice satisfies the policy?

Accepted Answer

Serve the registered open model on a Databricks Model Serving endpoint and call it from the application. The policy requires that note text never reach an external provider, so inference must run inside the company's Databricks environment. Serving the registered open model on a Databricks Model Serving endpoint keeps prompts and responses within the workspace while still offering a scalable API, and Unity Catalog continues to govern access to the model.

Answer

Use Databricks Foundation Model APIs with the pay-per-token option for a supported open model

Answer

Export the notes to a local workstation, run an open model there, and upload the summaries back

Answer

Call a third-party foundation model API directly from the notebook and pass the notes in the request body

Question 13

A GenAI engineer registers a vector search index in Unity Catalog that points to a Delta table containing customer support transcripts. The security team requires that when an end-user queries the index through a Databricks notebook, the underlying table's row filter and column mask policies are enforced. Which Unity Catalog feature should the engineer configure?

Accepted Answer

Enable row-level security and column masks directly on the Delta table, and query the vector search index with the user's identity propagated.. Row filters and column masks in Unity Catalog are applied dynamically based on the identity executing the query. When a vector search index is queried with end-user identity propagation, the source table's policies are evaluated, so sensitive transcript fields are masked and rows are filtered according to group membership. This satisfies the security team's requirement without duplicating policies on the index.

Answer

Apply tags to the Delta table columns and rely on tag-based access control to mask values at query time.

Answer

Create a separate vector search index for each user group and assign group-level permissions on those indexes.

Answer

Use a service principal with read access to the Delta table and query the index through that principal.

Question 14

A GenAI engineer is building a chatbot using Databricks Model Serving and needs to ensure that the endpoint can only be invoked by users who have been granted the 'genai_users' group. The endpoint is registered in Unity Catalog. Which configuration is required to enforce this access control?

Accepted Answer

Grant EXECUTE on the model serving endpoint to the 'genai_users' group.. Model serving endpoints registered in Unity Catalog are secured with the EXECUTE privilege. To allow only the 'genai_users' group to invoke the endpoint, an administrator must grant EXECUTE on the endpoint to that group. This ensures that only authorized users can call the endpoint, meeting the access-control requirement.

Answer

Grant USE CATALOG and USE SCHEMA on the catalog and schema containing the endpoint to the 'genai_users' group.

Answer

Attach the endpoint to a cluster that has been granted CAN ATTACH TO for the 'genai_users' group.

Answer

Set the endpoint's owner to the 'genai_users' group and rely on ownership for access.

Question 15

An engineer registers a GenAI agent to Unity Catalog and enables inference tables on its Mosaic AI Model Serving endpoint. They want to automatically detect when response quality degrades in production without waiting for human review. Which capability should they configure to achieve continuous automated quality monitoring?

Accepted Answer

A Databricks Lakehouse Monitoring monitor over the inference table, joined with a labeled evaluation dataset to compute quality metrics on a schedule.. Inference tables persist production request and response payloads as Delta, which makes them queryable by Lakehouse Monitoring. Building a monitor over that table, especially when joined with labeled data, yields scheduled quality and drift metrics that surface degradation automatically. This combination is the supported path for continuous, automated quality monitoring on Databricks rather than ad hoc manual inspection.

Answer

Configure autoscaling on the serving endpoint so additional model replicas absorb traffic spikes and stabilize response quality.

Answer

Set up a scheduled job that calls the endpoint with synthetic prompts and records latency percentiles to a Delta table for trending.

Answer

Enable model signature enforcement on the endpoint so requests that violate the expected schema are rejected before scoring.