Databricks-GenAI-Assoc Design Applications Practice Test 4 — 15 Questions

Question 1

A generative AI engineer is designing an agent on Databricks that uses a LangChain agent with tool-calling capabilities. The agent must call a Databricks SQL warehouse to run queries and a Vector Search index for retrieval. Which design consideration is most important for controlling agent behavior in production?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Define clear tool descriptions and limit the agent's available tools to only those required for the task.. Agent behavior is driven by tool descriptions and the set of tools available. Clear descriptions help the model choose the right tool, and a minimal toolset reduces the chance of unintended calls. Iteration limits, logging, and model size affect performance or observability but do not directly control which tools the agent invokes.

Answer

Use the largest available foundation model for the agent to maximize reasoning quality.

Answer

Enable verbose logging of every intermediate step and store the logs in a Delta table for later review.

Answer

Increase the agent's maximum iterations to a very high number so it can always complete complex tasks.

Question 2

An engineer is designing a Databricks RAG application where the same retrieved context must be reused across several prompt variants during evaluation. They want to reduce token cost and improve consistency between variants. Which design choice best supports this?

Accepted Answer

Compute the retrieval step once, store the retrieved documents in a Delta table keyed by query, and have each prompt variant read from that table.. Separating retrieval from generation and persisting the retrieved context lets several prompt variants share one retrieval result. This lowers cost because embeddings and Vector Search are invoked once, and it improves consistency because all variants see the same evidence. The alternatives change generation or chunking behavior but do not eliminate redundant retrieval.

Answer

Use a higher temperature during evaluation so the model explores more responses and the best one can be selected.

Answer

Configure the model serving endpoint to cache identical requests so repeated prompts with the same context are served from cache.

Answer

Increase the chunk size in the Vector Search index so that each retrieval returns more context, reducing the need to call the retriever multiple times.

Question 3

An engineer is designing a GenAI application where a Databricks workflow must call an external LLM provider that enforces a strict rate limit and occasionally returns transient errors. The engineer wants the application to degrade gracefully instead of failing the whole job. Which design choice best addresses this requirement?

Accepted Answer

Wrap the external call in retry logic with exponential backoff and a fallback response path. Resilience to external rate limits and transient failures comes from handling the call itself: bounded retries with exponential backoff smooth over temporary rejections, and a fallback response keeps the application functional when the provider stays unavailable. Infrastructure sizing, task-level retries, and response caching do not address the failure at its source.

Answer

Increase the driver node type on the job cluster

Answer

Cache all prompts and responses in a Delta table before calling the provider

Answer

Set the job to retry the entire task on failure

Question 4

A team is deploying a Databricks RAG chatbot that must serve interactive traffic with low latency while also allowing the data science team to test prompt variations safely. The engineer wants the production endpoint to keep serving stable traffic even while a new prompt template is being evaluated. Which approach best meets these requirements?

Accepted Answer

Create a separate Model Serving endpoint for the experimental prompt and route a small percentage of traffic to it for comparison. Separating the experimental prompt onto its own Model Serving endpoint preserves production stability while enabling controlled comparison. The team can direct a small share of traffic to the experimental endpoint, measure quality and latency, and promote the prompt only after it proves better, all without redeploying or interrupting the production endpoint.

Answer

Lower the production endpoint's concurrency limit and run the experiment on the same endpoint during off-peak hours

Answer

Edit the prompt template in the production endpoint's registered model version and restart the endpoint

Answer

Add the new prompt as a branch inside the production chain and select it at runtime using a random number generator

Question 5

A data science team wants to expose a RAG chain as a REST API so that an external web application can send questions and receive answers. The chain is developed with Databricks LangChain integrations and must be deployed with autoscaling and built-in monitoring. Which Databricks capability should they use?

Accepted Answer

Model Serving with a custom MLflow pyfunc model that wraps the chain.. Model Serving is the Databricks capability for hosting models and custom Python logic as REST endpoints with autoscaling and monitoring. Wrapping the LangChain chain in an MLflow pyfunc model makes it deployable through that service, so the web application can call a stable API. Scheduled jobs, SQL warehouses, and manual Flask apps do not provide the same managed serving characteristics.

Answer

A Databricks job that runs the chain on a schedule and writes answers to a Delta table.

Answer

An all-purpose cluster with a Flask app started manually on the driver node.

Answer

A Databricks SQL warehouse with a query that calls the chain through a UDF.

Question 6

A GenAI engineer is designing a retrieval-augmented generation application whose source documents are long PDFs. Early testing shows that answers are vague because retrieved chunks contain several unrelated topics, and the language model frequently cites content that does not support its claims. The engineer wants to improve chunk quality before indexing. Which TWO changes should the engineer make to the ingestion pipeline? (Choose two.)

Accepted Answer

Split documents along semantic boundaries such as sections and headings instead of a fixed character count. Chunk quality is determined at ingestion: splitting on semantic boundaries keeps each chunk topically focused, and overlap preserves meaning across boundaries. Together they raise the precision of retrieved context so the model has grounded material to cite. Embedding dimension, temperature, and oversized single chunks do not correct incoherent chunking.

Answer

Store the full PDF text in a single chunk per document

Answer

Increase the embedding dimension of the model used for the index

Answer

Lower the temperature setting on the language model

Question 7

A GenAI engineer is designing a Databricks application that must ground answers in a large corpus of internal policy documents. The corpus is updated by a nightly Delta job, and the application must cite the source document for every answer. The engineer is deciding how to structure the retrieval and generation stages. Which TWO design choices best satisfy the grounding and citation requirements? (Choose two.)

Accepted Answer

Instruct the model to answer only from the retrieved context and to include the provided document identifiers in its response. Grounding and citation depend on two things working together: retrieval that returns source identifiers with each chunk, and a generation prompt that restricts the model to that retrieved context while requiring it to surface the identifiers. Metadata columns supply verifiable provenance, and the constrained instruction keeps the answer tied to retrieved policy text instead of the model's own memory.

Answer

Rely on the model's pretrained knowledge of internal policies to fill gaps when retrieval returns no relevant chunks

Answer

Cache the full corpus in the prompt on every request so the model always has complete policy context

Answer

Increase the model's temperature setting so the generation stage explores a wider range of possible answers

Question 8

A financial services company is designing a RAG assistant that must never return answers containing personally identifiable information from its knowledge base. The team plans to filter retrieved chunks before they are placed into the prompt. Which Databricks design element should they use to enforce this filtering consistently?

Accepted Answer

Store a sensitivity label or metadata field on each chunk in the Vector Search index and apply a metadata filter at query time to exclude restricted chunks.. Metadata filtering in Vector Search allows the application to exclude chunks based on attributes such as sensitivity level before they reach the prompt. This enforces the policy at the retrieval boundary, reducing both leakage risk and token usage. Prompt instructions and output regexes act later and cannot guarantee that sensitive content never enters the model context.

Answer

Add a system prompt instructing the model to ignore any personally identifiable information it sees in the context.

Answer

Run a post-processing regex on the model output to remove patterns that resemble personally identifiable information.

Answer

Grant the serving endpoint a service principal that lacks access to the sensitive tables so retrieval returns no restricted rows.

Question 9

A generative AI team is building a customer-support assistant on Databricks. The assistant must answer questions using the company's private knowledge base, and the team wants to minimize latency while ensuring that the LLM only uses retrieved documents. They plan to use Databricks Vector Search with a Delta table as the source. Which design choice best balances low latency and grounded responses?

Accepted Answer

Use a Vector Search index with a relevance score threshold and retrieve the top 5 chunks, then include those chunks in the prompt with instructions to answer only from the provided context.. Grounding requires retrieving only relevant context and instructing the model to use it. A Vector Search index with a relevance threshold and a small top-k (e.g., 5) reduces prompt size and latency while filtering weak matches. Including those chunks with an explicit instruction to answer from context limits hallucination. This combination balances performance and accuracy better than unfiltered or keyword-only retrieval.

Answer

Use a keyword-based search over the Delta table and pass the top 10 matching rows to the LLM, bypassing Vector Search entirely.

Answer

Store the entire knowledge base in a single prompt and use a long-context LLM to answer questions without retrieval.

Answer

Configure a Vector Search index with a small embedding dimension and retrieve the top 20 chunks, then pass all chunks directly to the LLM without filtering.

Question 10

An engineer is designing a Databricks GenAI application that summarizes customer meeting notes. The notes contain personally identifiable information, and company policy requires that the summarization model never sends note text to an external provider. The team already has a fine-tuned open model registered in Unity Catalog. Which deployment choice satisfies the policy?

Accepted Answer

Serve the registered open model on a Databricks Model Serving endpoint and call it from the application. The policy requires that note text never reach an external provider, so inference must run inside the company's Databricks environment. Serving the registered open model on a Databricks Model Serving endpoint keeps prompts and responses within the workspace while still offering a scalable API, and Unity Catalog continues to govern access to the model.

Answer

Use Databricks Foundation Model APIs with the pay-per-token option for a supported open model

Answer

Export the notes to a local workstation, run an open model there, and upload the summaries back

Answer

Call a third-party foundation model API directly from the notebook and pass the notes in the request body

Question 11

An engineer is designing a Databricks RAG application that must support multi-turn conversations where follow-up questions refer to earlier turns. They want the retrieval step to remain accurate as the conversation progresses. Which TWO design elements should they include? (Choose two.)

Accepted Answer

Maintain a conversation history buffer and condense prior turns into a standalone query before invoking Vector Search.. Conversational RAG needs two coordinated elements: a rewritten, standalone query for retrieval, and conversation history for generation. Rewriting resolves references so Vector Search returns on-topic chunks, while including prior turns helps the model produce coherent follow-ups. Simply enlarging the retrieval window or removing filters does not address the ambiguity that multi-turn questions introduce.

Answer

Increase the number of retrieved chunks for every turn to compensate for conversational ambiguity.

Answer

Store the conversation history in the prompt and pass it unchanged to the retriever as the search query.

Answer

Disable metadata filtering during multi-turn conversations so more historical documents are eligible.

Question 12

An engineer is building a multi-turn GenAI assistant on Databricks. The assistant must answer follow-up questions that reference earlier turns, such as 'what about its warranty?', while keeping each request within the model's context limit. Which design should the engineer implement?

Accepted Answer

Maintain the conversation in a Delta table, then on each turn build a prompt from a bounded recent window of turns plus a summary of older turns, along with retrieved context.. Bounding prompt size while preserving conversational antecedents is achieved by keeping a recent window of turns and summarizing older ones, with per-turn retrieval for grounding. Stateless endpoints, untrimmed history replay, and vector-encoded history all fail to give the model usable prior context within the context limit.

Answer

Send only the current user message to the model on every turn and rely on the model's built-in memory of prior requests.

Answer

Persist the conversation in a Delta table and, on each turn, resend the entire raw history along with retrieved context, trimming nothing.

Answer

Encode the entire conversation history into a single embedding and prepend that vector to the prompt so the model can reconstruct prior turns.

Question 13

A generative AI engineer is designing a RAG application on Databricks that uses a foundation model served via Databricks Model Serving. The application must handle peak loads gracefully and provide consistent response times. The engineer is evaluating design patterns for scaling and reliability. Which TWO design choices should the engineer implement? (Choose two.)

Accepted Answer

Configure the Model Serving endpoint with autoscaling enabled and set a minimum and maximum number of concurrent requests per replica.. To handle peak loads and maintain consistent response times, the endpoint must scale elastically and the client must handle transient failures. Autoscaling with concurrency limits per replica ensures the serving infrastructure adapts to load, while client-side retries with backoff and timeouts improve resilience. These two choices together provide both server-side elasticity and client-side fault tolerance, which are essential for production reliability.

Answer

Deploy the model to a single large GPU replica with maximum memory to handle all traffic without autoscaling.

Answer

Precompute all possible user questions and their answers and store them in a lookup table for instant retrieval.

Answer

Cache the LLM responses for identical user queries in a Delta table and serve them without invoking the model.

Question 14

An engineer is designing a RAG application that uses Databricks Vector Search to retrieve documents and a foundation model endpoint to generate answers. The team wants to log all user queries, retrieved documents, and generated responses for auditing and continuous improvement. They also need to monitor for drift in retrieval quality over time. Which Databricks capability should they integrate into the application design?

Accepted Answer

MLflow Tracing with inference tables on the model serving endpoint.. MLflow Tracing provides detailed spans across the RAG pipeline, capturing queries, retrieved documents, and LLM responses. Inference tables on the serving endpoint automatically log request and response payloads for auditing. This combination enables both observability and drift monitoring. Other options focus on data pipelines, dashboards, or governance, which do not capture the runtime application behavior needed for continuous improvement.

Answer

Delta Live Tables with expectations to validate the retrieved documents.

Answer

Databricks SQL dashboards querying the source Delta table for document updates.

Answer

Unity Catalog audit logs and lineage for the Vector Search index.

Question 15

A GenAI application uses a Databricks Model Serving endpoint hosting a foundation model, and the team wants structured JSON output that conforms to a fixed schema for downstream parsing. Responses sometimes include prose or markdown fences that break the parser. Which design change is most likely to produce reliably parseable output?

Accepted Answer

Define the expected structure with a response format or structured-output parameter on the serving request so the endpoint constrains generation to the schema.. Declaring a response format or structured-output schema makes the endpoint constrain decoding so the generated text matches the required structure, removing prose and markdown fences by construction. Lowering temperature, raising the token limit, or stripping fences afterward only reduce or mask the symptom without guaranteeing conformance.

Answer

Add a regex-based post-processor that strips markdown fences and extracts the first balanced JSON object from the raw response.

Answer

Lower the temperature to zero and add the phrase 'return only JSON' to the end of the user message.

Answer

Increase max_tokens so the model has room to finish the JSON object without being truncated mid-structure.