Databricks-GenAI-Assoc Hard Questions — 25 Difficult Questions

Question 1

An organization is deploying a GenAI application using Databricks Model Serving. Which TWO steps are required to ensure the deployment environment handles model governance and observability effectively?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Enable Unity Catalog for the registered model and its versions.. Effective model governance and observability require integrating Unity Catalog for centralized access control and MLflow for tracking inference payloads. These components allow organizations to monitor model performance, lineage, and bias over time. Mastering these tools ensures that production deployments remain compliant, transparent, and auditable, which is essential for regulated industries using generative AI to make data-driven decisions while minimizing operational risks associated with model drift and unauthorized access.

Answer

Hardcode API credentials within the model inference script.

Answer

Disable automatic scaling to maintain consistent latency.

Answer

Manually deploy the model via the standard cluster user interface.

Question 2

Refer to the exhibit. What is the most likely cause for this error in a production RAG application?

Accepted Answer

Network connectivity between Model Serving and the vector store is misconfigured.. The error indicates a network timeout when connecting to the vector store. This is a common connectivity issue between the serving endpoint and the data source. Investigating network security groups, firewall rules, or DNS resolution issues within the Databricks environment is necessary to resolve it. Ensuring robust, low-latency connectivity to the retrieval source is fundamental for reliable RAG performance and minimizing application downtime in production.

Answer

The model is too large for the GPU memory.

Answer

The model weights are corrupted.

Answer

The input prompt is too long for the model.

Question 3

Refer to the exhibit. An engineer wants to perform a canary deployment by routing 10% of traffic to a new version (version 6). How should the JSON traffic configuration be modified?

Accepted Answer

Add a new route for version 6 and set its traffic_percentage to 10, then set version 5 to 90.. Canary deployments involve splitting traffic between an established model version and a new candidate. To achieve this, the 'routes' list must include both versions, with the 'traffic_percentage' values summing to 100%. This controlled rollout allows teams to monitor the performance and accuracy of the new model on a small segment of production traffic before committing the full load to the new version.

Answer

Set the traffic_percentage of version 6 to 10 and keep version 5 at 100.

Answer

Remove version 5 and set version 6 to 10.

Answer

Create a new endpoint entirely for version 6 and switch the DNS record.

Question 4

Refer to the exhibit. An engineer is configuring a canary deployment for a churn prediction model. Based on the provided traffic configuration, what is the expected behavior of the endpoint?

Accepted Answer

90% of requests are routed to churn_v1 and 10% to churn_v2.. The configuration implements a traffic-splitting strategy, directing 90% of requests to 'churn_v1' and 10% to 'churn_v2'. This is a standard pattern for A/B testing or canary releases, allowing teams to validate new models in production with minimal risk. By monitoring the performance of the 10% traffic slice, engineers can decide whether to promote the model, ensuring stability before a full rollout.

Answer

The endpoint will error because total traffic must equal 100% for each model version individually.

Answer

The endpoint will only route traffic to churn_v1 after churn_v2 reaches capacity.

Answer

The endpoint will round-robin requests between v1 and v2 regardless of the weights.

Question 5

Which CI/CD approach for model deployment best minimizes downtime during a model update?

Accepted Answer

Use a blue-green deployment pattern.. A blue-green deployment strategy is the gold standard for minimizing downtime. By spinning up a new version of the model (green) alongside the existing one (blue) and switching traffic only after verifying the health of the green endpoint, engineers ensure zero downtime and an immediate rollback path if issues occur. This approach is critical for high-availability AI services where any interruption would negatively impact users.

Answer

Delete the old model and immediately deploy the new version.

Answer

Deploy all updates directly to the production endpoint.

Answer

Schedule deployments during peak traffic hours.

Question 6

Which TWO of the following are mandatory requirements for developing an AI application using the Databricks Mosaic AI Model Serving environment?

Accepted Answer

The model must be registered as a model version within a Unity Catalog schema.. Mosaic AI Model Serving requires specific configurations to ensure secure and performant access. Firstly, the model must be registered in Unity Catalog to maintain governance and lineage. Secondly, the serving endpoint requires defined compute resources, typically GPU-accelerated for LLMs, to handle inference requests. These requirements are essential for productionizing models, as they ensure that models are discoverable, governed, and have the necessary hardware to meet low-latency performance targets in real-time scenarios.

Answer

The model must be stored in a legacy Databricks Workspace folder.

Answer

The model must be served using a shared-access mode interactive cluster.

Answer

The application code must perform manual model sharding across nodes.

Question 7

An engineer is designing a Databricks RAG application that must support multi-turn conversations where follow-up questions refer to earlier turns. They want the retrieval step to remain accurate as the conversation progresses. Which TWO design elements should they include? (Choose two.)

Accepted Answer

Maintain a conversation history buffer and condense prior turns into a standalone query before invoking Vector Search.. Conversational RAG needs two coordinated elements: a rewritten, standalone query for retrieval, and conversation history for generation. Rewriting resolves references so Vector Search returns on-topic chunks, while including prior turns helps the model produce coherent follow-ups. Simply enlarging the retrieval window or removing filters does not address the ambiguity that multi-turn questions introduce.

Answer

Increase the number of retrieved chunks for every turn to compensate for conversational ambiguity.

Answer

Store the conversation history in the prompt and pass it unchanged to the retriever as the search query.

Answer

Disable metadata filtering during multi-turn conversations so more historical documents are eligible.

Question 8

Which TWO factors are most important when selecting a chunking strategy for text data prior to vectorization?

Accepted Answer

The maximum input token length of the target embedding model.. Selecting a chunking strategy requires balancing semantic context and technical limitations like context window size. If chunks are too small, they lack meaning; if too large, they exceed the limits of the embedding model and introduce irrelevant noise. Proper chunking is vital for maximizing the accuracy of RAG systems, as it determines the granularity of the information available for the retriever to present to the LLM.

Answer

The file format of the source documents (e.g., PDF vs. Word).

Answer

The total number of documents in the corpus.

Answer

The storage cost of the resulting vector database.

Question 9

Refer to the exhibit. The user is a member of the 'finance_team'. Why might the user encounter an access error when executing this join query?

Accepted Answer

The user lacks the USAGE privilege on the 'sales' schema.. In Unity Catalog, users require the USAGE privilege on all parent objects—the catalog and the schema—to access a table. While the user has USAGE on the catalog and SELECT on the tables, they lack the USAGE privilege on the 'sales' schema. Without explicit USAGE permission on the schema, the user cannot traverse the hierarchy to reference the tables, causing a permission denied error.

Answer

The user lacks the SELECT privilege on the schema object.

Answer

The user requires the MODIFY privilege to perform joins.

Answer

The user needs to be an owner of the tables to perform joins.

Question 10

Refer to the exhibit. Why did the analyst group lose access after the table was recreated?

Accepted Answer

The new table is a different object, so the GRANT statement must be repeated.. In Unity Catalog, privileges are bound to the specific object ID. When a table is dropped and re-created, it becomes a new object with a new unique identifier. The previous GRANT statements, which were associated with the original object ID, do not automatically apply to the new table. This mechanism protects against security drift by ensuring that every new object must have its permissions explicitly managed.

Answer

The catalog owner needs to refresh the table metadata.

Answer

The analyst group needs to be re-added to the schema permissions.

Answer

The user who recreated the table is not the catalog owner.

Question 11

A developer deploys a new model version as shown in the exhibit. What is the purpose of this configuration?

Accepted Answer

To conduct A/B testing or a canary rollout.. This configuration implements a canary deployment strategy by splitting traffic between two model versions. By routing 90% of requests to the stable version and 10% to the new version, the team can monitor performance and accuracy in a real-world scenario with minimal risk. This is a standard practice for safely rolling out model improvements, allowing teams to collect data on the new version's behavior before a full-scale transition.

Answer

To increase compute capacity for the primary model.

Answer

To load balance between two different cloud regions.

Answer

To force all traffic to model-v1 when model-v2 errors.

Question 12

A team prepares customer support transcripts for fine-tuning. Each transcript is a JSON record with a nested messages array, and approximately 8 percent of records contain a null role field. The team wants records with malformed messages to be quarantined rather than silently dropped. Which Delta Live Tables configuration accomplishes this?

Accepted Answer

Define the table with a CONSTRAINT valid_role EXPECT (role IS NOT NULL), and create a separate streaming table that selects records where role IS NULL from the source.. Quarantining malformed records requires keeping them somewhere queryable. An EXPECT constraint without an ON VIOLATION clause preserves violating rows in the primary table and logs the violation, while a second streaming table can select the malformed records into a dedicated quarantine dataset. Drop and fail behaviors both remove the ability to review the bad data.

Answer

Define the table with a CONSTRAINT valid_role EXPECT (role IS NOT NULL) ON VIOLATION DROP ROW, and inspect the event log for dropped records.

Answer

Define the table with a CONSTRAINT valid_role EXPECT (role IS NOT NULL) ON VIOLATION DROP ROW, and configure the pipeline to write dropped rows to a Unity Catalog volume automatically.

Answer

Define the table with a CONSTRAINT valid_role EXPECT (role IS NOT NULL) ON VIOLATION FAIL UPDATE, and rely on pipeline retries to reprocess the batch.

Question 13

A GenAI engineer is building a Databricks RAG application where the retrieval step returns the top-5 chunks for each user question. The engineer wants to add a second LLM call that evaluates whether each retrieved chunk contains enough information to answer the question, and then filters out chunks that fail this evaluation before passing the remaining chunks to the final answer-generation prompt. Which design pattern is the engineer implementing?

Accepted Answer

Retrieval-augmented generation with relevance grading. The defining behavior in the scenario is a second LLM call that grades each retrieved chunk for answerability and drops the ones that fail. That is relevance grading layered on top of retrieval-augmented generation, which raises precision by keeping only context that can actually support the answer. The other techniques alter retrieval ranking, prompt reasoning, or embeddings but do not insert an evaluative filtering step between retrieval and generation.

Answer

Chain-of-thought prompting

Answer

Fine-tuning the embedding model on domain data

Answer

Hybrid search combining vector similarity with BM25 keyword matching

Question 14

A team is preparing a Delta table of product descriptions for a RAG application. The table receives continuous upserts from a streaming pipeline, and the embedding job reads the table every hour. Engineers notice the embedding job reprocesses every row on each run even though only a few rows change. Which change should be made to the source table to let the embedding job process only new or updated rows?

Accepted Answer

Enable Change Data Feed on the Delta table and have the embedding job read the change feed since the last processed version.. Change Data Feed captures row-level changes with commit versions, allowing a downstream job to read only the inserts, updates, and deletes that occurred since its last checkpoint. This directly solves the problem of reprocessing unchanged rows. Partitioning, compaction, and timestamp filtering improve read characteristics but do not expose which rows changed, so they cannot deliver incremental processing.

Answer

Partition the Delta table by the product category column so the embedding job reads fewer files.

Answer

Convert the table to a view that filters rows by the current timestamp so only recent rows appear.

Answer

Run OPTIMIZE with Z-ORDER on the product identifier column to compact small files before each embedding run.

Question 15

An engineer is building a RAG application that must query a vector search index containing confidential financial documents. The application will be deployed as a Model Serving endpoint, and only authorized users should see retrieved content. Which combination of Databricks features should the engineer implement to enforce this?

Accepted Answer

Enable inference tables on the endpoint and configure row-level filters on the vector search index.. Row-level filters on the Vector Search index enforce that each query only retrieves documents the requesting user is permitted to see, which is essential for confidential content. Inference tables add request and response logging for auditing and monitoring, so combining these two features addresses both access control and observability in a production RAG deployment.

Answer

Store the documents in DBFS and apply a cluster-level init script that filters results before returning them.

Answer

Configure the endpoint with a service principal and set the vector search index to public read access.

Answer

Use a personal access token in the application code and rely on Unity Catalog table ACLs on the source documents.

Question 16

A healthcare company uses Databricks to build a RAG application over clinical notes stored in a Unity Catalog table. An auditor requires proof that only authorized personnel can view raw note text, while the RAG application must use embeddings generated from those notes. The team wants to avoid copying data outside Unity Catalog. Which approach best satisfies the auditor while preserving RAG functionality?

Accepted Answer

Create a column mask on the note text column that returns a redacted value for unauthorized users, and grant the RAG service principal access to a separate embeddings table derived from the notes.. A column mask on the sensitive note text column enforces redaction for unauthorized identities at query time, which is auditable. By having the RAG application consume a derived embeddings table instead of raw text, the application retains functionality while raw PHI remains protected. This keeps governance within Unity Catalog and satisfies the auditor's requirement.

Answer

Move the clinical notes to an external location outside Unity Catalog and grant access via cloud IAM roles.

Answer

Grant the RAG application's service principal SELECT on the table and rely on the application to redact note text in its responses.

Answer

Use a dynamic view that filters rows based on the user's group and grant the RAG service principal access to that view.

Question 17

A team is using MLflow LLM Evaluation with the built-in answer_correctness metric to compare two prompt templates for a question-answering application. They notice that answer_correctness scores are nearly identical, but manual review shows one template produces answers that are factually correct yet omit key supporting details. Which additional built-in metric should they add to their evaluation to surface this difference?

Accepted Answer

answer_completeness. When answers are factually correct but differ in how much supporting detail they include, a completeness metric is required. answer_completeness compares the response against the ground truth to check whether all key points are covered, making it the right built-in metric to expose the omission of supporting details that answer_correctness alone misses.

Answer

groundedness

Answer

toxicity

Answer

answer_similarity

Question 18

A GenAI engineer runs mlflow.evaluate() with model_type="databricks-agent" on a RAG chain registered as a Unity Catalog function. The run completes and the evaluation results table is written, but the engineer needs to compare this run against the previous production baseline and programmatically gate the next deployment on it inside a Databricks job. Which action should the engineer take?

Accepted Answer

Query the MLflow run's metrics with mlflow.search_runs() filtering on the run ID and compare the returned metric values to the baseline thresholds in the job.. MLflow LLM Evaluation records aggregate judge metrics on the evaluation run itself, so pulling them back with mlflow.search_runs() gives the job a stable numeric signal to compare against a stored baseline. Re-running evaluation, recomputing from per-row results, or mixing in serving telemetry all fail to provide that consistent, framework-defined run-level metric needed for an automated promotion gate.

Answer

Read the Delta table that stores the evaluation results and compute a mean over the per-row correctness column to derive a run-level score.

Answer

Re-run mlflow.evaluate() with the same evaluation dataset and inspect the printed summary in the notebook output to confirm the metrics match the baseline.

Answer

Enable inference tables on the serving endpoint and compare the latest logged request payloads to the baseline metric values stored in the model registry.

Question 19

A GenAI engineer is designing a retrieval pipeline that uses Databricks Vector Search with hybrid search enabled. Users report that semantically similar but keyword-distinct queries return irrelevant chunks, and that results vary between runs of the same query. Which TWO configuration choices should the engineer make to improve result relevance and consistency? (Choose two.)

Accepted Answer

Configure the index to return only results whose similarity score exceeds a high static threshold, discarding everything below it.. Hybrid retrieval activated through query_text blends keyword and vector signals, and a reranker plus a similarity threshold tightens and stabilizes the final candidate set. Returning unfiltered extra results, seeding embeddings, or dropping keyword matching either fails to address ranking quality or misidentifies the cause of inconsistent results.

Answer

Disable hybrid search and rely solely on dense vector similarity, since keyword matching introduces nondeterministic scoring.

Answer

Set a fixed random seed on the embedding model so that repeated queries always produce identical similarity scores.

Answer

Increase the number of results returned by the index and pass all of them, unchanged, directly into the LLM prompt as context.

Question 20

Refer to the exhibit. An engineer is automating the deployment of a model to an endpoint using the Databricks CLI. Based on the error log provided, what is the most appropriate action to resolve this deployment failure?

Accepted Answer

Grant the service principal 'USE' and 'READ' permissions on the catalog, schema, and model.. Deployment failures due to 'INSUFFICIENT_PERMISSIONS' often stem from the service principal used by the CI/CD pipeline lacking the necessary grants on the registered model in Unity Catalog. The engineer must ensure the principal has 'USE CATALOG', 'USE SCHEMA', and 'READ' permissions on the model version. Addressing this at the Unity Catalog level is the standard procedure for cross-service authorization in Databricks.

Answer

Upgrade the model serving endpoint to a higher capacity cluster size.

Answer

Delete the model serving endpoint and recreate it using a personal access token.

Answer

Restart the Databricks workspace to force an update of the internal permission cache.