Scenario practice questions

Which TWO of the following are mandatory requirements for developing an AI application using the Databricks Mosaic AI Model Serving environment?

Trap 1: The model must be stored in a legacy Databricks Workspace folder.

Legacy workspace paths are not compatible with the modern Unity Catalog serving architecture. All models must be registered within Unity Catalog to leverage the unified governance model, lineage tracking, and permission structures that Mosaic AI services require to function effectively within a secure enterprise data environment.

Trap 2: The model must be served using a shared-access mode interactive…

Model serving endpoints use dedicated, managed inference infrastructure, not interactive clusters. Interactive clusters are designed for development and exploratory analysis, whereas inference services require specific, optimized runtime environments that provide scalability, high availability, and auto-scaling capabilities tailored specifically for model serving workloads rather than general-purpose data science tasks.

Trap 3: The application code must perform manual model sharding across…

Databricks Mosaic AI handles model sharding and distribution automatically within the managed serving environment. Developers do not need to manually manage hardware-level parallelism or model sharding, as the infrastructure layer abstracts these complexities to focus on ease of deployment, scalability, and simplified management of inference workloads.

A
The model must be stored in a legacy Databricks Workspace folder.
Why it fails: Legacy workspace paths are not compatible with the modern Unity Catalog serving architecture. All models must be registered within Unity Catalog to leverage the unified governance model, lineage tracking, and permission structures that Mosaic AI services require to function effectively within a secure enterprise data environment.
B
The model must be registered as a model version within a Unity Catalog schema.
Unity Catalog acts as the central repository for model artifacts and versions in Databricks. Registering the model here provides the necessary metadata, lineage, and access control required by the serving infrastructure to deploy the model securely and ensure it remains reachable by authorized internal or external applications.
C
The model must be served using a shared-access mode interactive cluster.
Why it fails: Model serving endpoints use dedicated, managed inference infrastructure, not interactive clusters. Interactive clusters are designed for development and exploratory analysis, whereas inference services require specific, optimized runtime environments that provide scalability, high availability, and auto-scaling capabilities tailored specifically for model serving workloads rather than general-purpose data science tasks.
D
The serving endpoint must be configured with a defined compute resource.
An inference endpoint needs specific compute resources (like GPU instances) to execute model logic. Defining this configuration is a mandatory step in the deployment process, allowing the platform to provision the necessary infrastructure to meet the required throughput and latency demands of the model being deployed for users.
E
The application code must perform manual model sharding across nodes.
Why it fails: Databricks Mosaic AI handles model sharding and distribution automatically within the managed serving environment. Developers do not need to manually manage hardware-level parallelism or model sharding, as the infrastructure layer abstracts these complexities to focus on ease of deployment, scalability, and simplified management of inference workloads.

Question 2hardmultiple choice

A developer deploys a new model version as shown in the exhibit. What is the purpose of this configuration?

Exhibit

Refer to the exhibit.

# Model Serving Policy Configuration
{
  "traffic_config": {
    "routes": [
      {
        "served_model_name": "model-v1",
        "traffic_percentage": 90
      },
      {
        "served_model_name": "model-v2",
        "traffic_percentage": 10
      }
    ]
  }
}

Trap 1: To increase compute capacity for the primary model.

Traffic splitting does not manage compute capacity. The number of instances and the resources allocated to each model version are determined by the scaling policy and workload types, not the traffic distribution settings, which simply control the routing of user requests between the available versions.

Trap 2: To load balance between two different cloud regions.

The 'served_model_name' refers to specific versions of a model deployed on the same serving endpoint. Traffic routing within a serving endpoint is local to the service and does not involve multi-region load balancing, which is a separate infrastructure-level configuration that is not managed through this endpoint routing policy.

Trap 3: To force all traffic to model-v1 when model-v2 errors.

This configuration represents a static split, not an automatic failover policy. If 'model-v2' returns errors, the endpoint will continue to route 10% of traffic to it. The system does not possess built-in intelligence to detect model-level errors and automatically redirect traffic back to the primary model.

A
To increase compute capacity for the primary model.
Why it fails: Traffic splitting does not manage compute capacity. The number of instances and the resources allocated to each model version are determined by the scaling policy and workload types, not the traffic distribution settings, which simply control the routing of user requests between the available versions.
B
To conduct A/B testing or a canary rollout.
Configuring traffic percentages allows for controlled, incremental rollouts, which are essential for A/B testing or canary deployments. This allows developers to validate the new model's performance on live traffic while minimizing the impact if the new model version exhibits unexpected behavior or suboptimal accuracy metrics.
C
To load balance between two different cloud regions.
Why it fails: The 'served_model_name' refers to specific versions of a model deployed on the same serving endpoint. Traffic routing within a serving endpoint is local to the service and does not involve multi-region load balancing, which is a separate infrastructure-level configuration that is not managed through this endpoint routing policy.
D
To force all traffic to model-v1 when model-v2 errors.
Why it fails: This configuration represents a static split, not an automatic failover policy. If 'model-v2' returns errors, the endpoint will continue to route 10% of traffic to it. The system does not possess built-in intelligence to detect model-level errors and automatically redirect traffic back to the primary model.

Question 3hardmultiple choice

A GenAI engineer is building a Databricks RAG application where the retrieval step returns the top-5 chunks for each user question. The engineer wants to add a second LLM call that evaluates whether each retrieved chunk contains enough information to answer the question, and then filters out chunks that fail this evaluation before passing the remaining chunks to the final answer-generation prompt. Which design pattern is the engineer implementing?

Trap 1: Chain-of-thought prompting

Chain-of-thought prompting asks the model to expose intermediate reasoning steps so that a complex task is solved more reliably. In this scenario, the engineer is not trying to elicit a reasoning trace from the answering model; they are inserting an independent LLM call whose only job is to judge and filter retrieved chunks. Chain-of-thought would not remove irrelevant chunks from the context window, so it does not solve the precision problem described.

Trap 2: Fine-tuning the embedding model on domain data

Fine-tuning an embedding model changes how vectors are produced and can improve recall for domain vocabulary. However, it does not add a separate evaluation step that inspects each retrieved chunk and discards those that fail a relevance check. The scenario explicitly describes a second LLM call that evaluates chunks, which is a runtime filtering pattern rather than a training-time embedding adjustment.

Trap 3: Hybrid search combining vector similarity with BM25 keyword matching

Hybrid search merges dense vector results with sparse keyword results to improve recall for exact terms. It still returns a ranked list of chunks and does not ask an LLM to judge whether each chunk contains enough information to answer the question. The scenario's second LLM call that filters chunks is the defining element, and hybrid search alone would not perform that filtering.

A
Chain-of-thought prompting
Why it fails: Chain-of-thought prompting asks the model to expose intermediate reasoning steps so that a complex task is solved more reliably. In this scenario, the engineer is not trying to elicit a reasoning trace from the answering model; they are inserting an independent LLM call whose only job is to judge and filter retrieved chunks. Chain-of-thought would not remove irrelevant chunks from the context window, so it does not solve the precision problem described.
B
Retrieval-augmented generation with relevance grading
The engineer is adding an LLM-based relevance grader between retrieval and generation. Each retrieved chunk is scored for whether it can actually answer the question, and only chunks that pass the grade are forwarded. This pattern improves answer precision by removing semantically similar but non-answering passages before the final prompt is assembled, which is exactly what the scenario describes.
C
Fine-tuning the embedding model on domain data
Why it fails: Fine-tuning an embedding model changes how vectors are produced and can improve recall for domain vocabulary. However, it does not add a separate evaluation step that inspects each retrieved chunk and discards those that fail a relevance check. The scenario explicitly describes a second LLM call that evaluates chunks, which is a runtime filtering pattern rather than a training-time embedding adjustment.
D
Hybrid search combining vector similarity with BM25 keyword matching
Why it fails: Hybrid search merges dense vector results with sparse keyword results to improve recall for exact terms. It still returns a ranked list of chunks and does not ask an LLM to judge whether each chunk contains enough information to answer the question. The scenario's second LLM call that filters chunks is the defining element, and hybrid search alone would not perform that filtering.

Question 4mediummultiple choice

An engineer is packaging a GenAI agent application with Databricks Asset Bundles so that the same bundle deploys to a development and a production workspace. The agent's serving endpoint name must differ per target, and the production endpoint needs more concurrent capacity. Which mechanism in the bundle configuration should the engineer use?

Trap 1: Use a post-deployment notebook that calls the serving endpoints API…

Renaming a serving endpoint after deployment is not a supported way to manage naming, and mutating resources outside the bundle causes drift between the declared and actual state. The bundle would not know about the changes, so subsequent deployments could fail or revert them. This is fragile and does not leverage the bundle's native parameterization.

Trap 2: Store the endpoint name and replica count as secrets in a…

Secret scopes hold sensitive values and are not designed to parameterize resource definitions such as endpoint names or replica counts. Bundles cannot interpolate secret values directly into resource fields at deploy time, and replica counts are not secrets. This adds complexity without solving the environment-specific configuration requirement.

Trap 3: Hard-code the production endpoint name and replica count, then…

Duplicating the bundle creates two divergent sources of truth that will drift and multiply maintenance work. Hard-coding values also prevents the development target from using different names or smaller capacity, and it bypasses the bundle's built-in target override mechanism. This approach defeats the purpose of using a single deployable bundle across workspaces.

A
Use a post-deployment notebook that calls the serving endpoints API to rename the endpoint and adjust replicas after each bundle deployment.
Why it fails: Renaming a serving endpoint after deployment is not a supported way to manage naming, and mutating resources outside the bundle causes drift between the declared and actual state. The bundle would not know about the changes, so subsequent deployments could fail or revert them. This is fragile and does not leverage the bundle's native parameterization.
B
Store the endpoint name and replica count as secrets in a Databricks secret scope and read them at deployment time with the Databricks CLI.
Why it fails: Secret scopes hold sensitive values and are not designed to parameterize resource definitions such as endpoint names or replica counts. Bundles cannot interpolate secret values directly into resource fields at deploy time, and replica counts are not secrets. This adds complexity without solving the environment-specific configuration requirement.
C
Define target-specific variables in databricks.yml and reference them with ${var.endpoint_name} and ${var.min_replicas} in the resource definition.
Databricks Asset Bundles support variables declared at the bundle level and overridden per target, which is the supported way to vary values such as endpoint names and replica counts across development and production. Referencing them with the ${var.} syntax keeps one resource definition while letting each target supply different values, matching the requirement precisely.
D
Hard-code the production endpoint name and replica count, then maintain a separate copy of the bundle for development.
Why it fails: Duplicating the bundle creates two divergent sources of truth that will drift and multiply maintenance work. Hard-coding values also prevents the development target from using different names or smaller capacity, and it bypasses the bundle's built-in target override mechanism. This approach defeats the purpose of using a single deployable bundle across workspaces.

Question 5mediummultiple choice

When preparing a dataset for fine-tuning an LLM, you need to ensure the data is representative of the target domain. What is the most effective approach to detect and mitigate sampling bias in your training set using Databricks?

Trap 1: Increase the batch size during the fine-tuning process.

Batch size affects training convergence and hardware utilization but has no impact on data sampling bias. Bias is a structural issue within the dataset itself, not a training parameter. Adjusting the batch size will not correct the underlying lack of representativeness in the training data provided.

Trap 2: Change the model architecture to a larger parameter count.

Increasing model size might allow the model to memorize more data, but it does not address the fundamental issue of sampling bias. If the training data is inherently biased, a larger model will simply learn those biases more effectively, potentially leading to worse outcomes in production scenarios.

Trap 3: Use a random seed to shuffle the data before splitting.

Shuffling helps with training stability but does not resolve structural sampling bias. If the entire dataset is skewed, shuffling only ensures that the skew is evenly distributed across training and validation splits. It does not introduce the missing data or diversity required to mitigate the actual bias.

A
Increase the batch size during the fine-tuning process.
Why it fails: Batch size affects training convergence and hardware utilization but has no impact on data sampling bias. Bias is a structural issue within the dataset itself, not a training parameter. Adjusting the batch size will not correct the underlying lack of representativeness in the training data provided.
B
Perform statistical profiling on feature distributions and re-sample if necessary.
Statistical profiling reveals imbalances, allowing engineers to apply techniques like oversampling or undersampling to correct them. This quantitative approach is objective and standard for ensuring high-quality machine learning training sets. It prevents the model from favoring specific patterns simply because they were more frequent in the dataset.
C
Change the model architecture to a larger parameter count.
Why it fails: Increasing model size might allow the model to memorize more data, but it does not address the fundamental issue of sampling bias. If the training data is inherently biased, a larger model will simply learn those biases more effectively, potentially leading to worse outcomes in production scenarios.
D
Use a random seed to shuffle the data before splitting.
Why it fails: Shuffling helps with training stability but does not resolve structural sampling bias. If the entire dataset is skewed, shuffling only ensures that the skew is evenly distributed across training and validation splits. It does not introduce the missing data or diversity required to mitigate the actual bias.

Question 6mediummultiple choice

Which of the following describes the purpose of using a 'Feature Store' when preparing data for Generative AI applications?

Trap 1: To store raw text files for long-term archival.

Feature stores are designed for structured, processed features, not for storing raw, unstructured text logs. Archiving raw files is typically handled by object storage like S3 or ADLS, not a feature store, which is optimized for fast lookup and versioned feature management during model cycles.

Trap 2: To serve as a high-performance database for LLM model weights.

Feature stores do not store model weights; they store data features used as inputs for models. Storing model weights is the job of the Model Registry. Confusing these two components leads to improper architectural design and incorrect utilization of the Databricks platform's specialized ML lifecycle tools.

Trap 3: To replace the need for data cleaning and preprocessing entirely.

Feature stores store features that have already been cleaned; they do not perform the cleaning themselves. Data engineers must still build pipelines to clean the data before it is ingested into the feature store. This misconception underestimates the work required to prepare quality data for machine learning.

A
To store raw text files for long-term archival.
Why it fails: Feature stores are designed for structured, processed features, not for storing raw, unstructured text logs. Archiving raw files is typically handled by object storage like S3 or ADLS, not a feature store, which is optimized for fast lookup and versioned feature management during model cycles.
B
To provide a unified interface for consistent feature computation during training and inference.
The primary goal of a feature store is to eliminate inconsistencies in feature logic. By centralizing this, organizations ensure that the features the model was trained on are computed in exactly the same way when deployed. This is foundational for building stable, repeatable machine learning pipelines in Databricks.
C
To serve as a high-performance database for LLM model weights.
Why it fails: Feature stores do not store model weights; they store data features used as inputs for models. Storing model weights is the job of the Model Registry. Confusing these two components leads to improper architectural design and incorrect utilization of the Databricks platform's specialized ML lifecycle tools.
D
To replace the need for data cleaning and preprocessing entirely.
Why it fails: Feature stores store features that have already been cleaned; they do not perform the cleaning themselves. Data engineers must still build pipelines to clean the data before it is ingested into the feature store. This misconception underestimates the work required to prepare quality data for machine learning.

Question 7hardmultiple choice

A team is using `databricks.agents.deploy()` to publish a Mosaic AI Agent to a serving endpoint. They must expose an environment-specific Vector Search index name and the endpoint name to the deployment without hardcoding values in the notebook, and the same notebook must run in dev and prod. Which approach should the engineer use?

Trap 1: Store the values in a Delta table and have the notebook query that…

Using a Delta table as a configuration store adds a runtime dependency and requires the deploying identity to read that table, yet it still does not integrate with the deployment tooling that selects environments. It also duplicates configuration that Databricks bundles and job parameters already manage. The mechanism is indirect and brittle compared with native parameterization, and it complicates CI promotion.

Trap 2: Detect the environment by reading the current workspace URL inside…

Inferring the environment from the workspace host embeds fragile string logic into the notebook and breaks when workspaces are renamed or when staging shares a host. It also scatters environment knowledge through the code instead of centralizing it in deployment configuration. This approach works only by accident and does not provide a clean parameterization mechanism for index and endpoint names.

Trap 3: Hardcode the prod values and use a separate copied notebook for dev…

Duplicating notebooks with hardcoded values creates drift: fixes must be applied twice, and the two copies can diverge silently. It also violates the explicit requirement that the same notebook run in both environments. While it technically produces working deployments, it is a maintenance anti-pattern that the scenario rules out and that undermines CI/CD reliability.

A
Store the values in a Delta table and have the notebook query that table at deploy time.
Why it fails: Using a Delta table as a configuration store adds a runtime dependency and requires the deploying identity to read that table, yet it still does not integrate with the deployment tooling that selects environments. It also duplicates configuration that Databricks bundles and job parameters already manage. The mechanism is indirect and brittle compared with native parameterization, and it complicates CI promotion.
B
Read the values from `dbutils.widgets` defined at the top of the notebook and pass them into `databricks.agents.deploy()`.
Notebook widgets provide parameterized inputs that a Databricks job or bundle can supply per environment, so the same notebook runs in dev and prod with different index and endpoint names. The values flow into `databricks.agents.deploy()` at runtime without editing code. This is the standard Databricks pattern for environment-parameterized deployments and satisfies the no-hardcoding requirement.
C
Detect the environment by reading the current workspace URL inside the notebook and branch on it.
Why it fails: Inferring the environment from the workspace host embeds fragile string logic into the notebook and breaks when workspaces are renamed or when staging shares a host. It also scatters environment knowledge through the code instead of centralizing it in deployment configuration. This approach works only by accident and does not provide a clean parameterization mechanism for index and endpoint names.
D
Hardcode the prod values and use a separate copied notebook for dev with the dev values.
Why it fails: Duplicating notebooks with hardcoded values creates drift: fixes must be applied twice, and the two copies can diverge silently. It also violates the explicit requirement that the same notebook run in both environments. While it technically produces working deployments, it is a maintenance anti-pattern that the scenario rules out and that undermines CI/CD reliability.

Question 8easymultiple choice

A team has deployed a RAG application on Databricks and wants to monitor the quality of responses in production. They have enabled inference table logging. Which built-in Databricks capability allows them to periodically evaluate the logged requests and responses for quality metrics like groundedness?

Trap 1: Databricks SQL dashboards that display the count of requests per…

SQL dashboards can visualize request volume but do not compute quality metrics like groundedness. They are useful for operational monitoring but lack the semantic evaluation needed for RAG quality. The team needs to assess the content of responses, not just counts. Therefore, SQL dashboards alone are insufficient.

Trap 2: Delta Live Tables expectations that validate response length.

Delta Live Tables expectations are for data quality checks on streaming or batch pipelines, such as schema validation or null checks. They cannot compute semantic quality metrics like groundedness. Using them for this purpose would be a misuse of the feature. They do not provide LLM-specific evaluation.

Trap 3: MLflow Tracking with custom metrics logged during model serving.

MLflow Tracking logs metrics and parameters during experiments, but it does not automatically evaluate inference tables for quality. Custom metrics would need to be computed and logged by the application code, which is not built-in. This approach requires significant engineering effort and is not a turnkey monitoring solution. It is not the built-in capability referenced.

A
Databricks SQL dashboards that display the count of requests per hour.
Why it fails: SQL dashboards can visualize request volume but do not compute quality metrics like groundedness. They are useful for operational monitoring but lack the semantic evaluation needed for RAG quality. The team needs to assess the content of responses, not just counts. Therefore, SQL dashboards alone are insufficient.
B
Delta Live Tables expectations that validate response length.
Why it fails: Delta Live Tables expectations are for data quality checks on streaming or batch pipelines, such as schema validation or null checks. They cannot compute semantic quality metrics like groundedness. Using them for this purpose would be a misuse of the feature. They do not provide LLM-specific evaluation.
C
Mosaic AI Agent Evaluation, which can be run on the inference table to compute quality metrics.
Mosaic AI Agent Evaluation is designed to evaluate agent and RAG applications. It can be run on logged inference tables to compute metrics such as groundedness and relevance. This allows continuous monitoring of production quality without manual intervention. It integrates with MLflow and Databricks workflows, making it the appropriate built-in capability for this scenario.
D
MLflow Tracking with custom metrics logged during model serving.
Why it fails: MLflow Tracking logs metrics and parameters during experiments, but it does not automatically evaluate inference tables for quality. Custom metrics would need to be computed and logged by the application code, which is not built-in. This approach requires significant engineering effort and is not a turnkey monitoring solution. It is not the built-in capability referenced.

Question 9mediummultiple choice

An engineer is preparing to deploy a Mosaic AI Agent application with Databricks Asset Bundles. The bundle defines the agent, the serving endpoint, and a job that refreshes the vector index. The engineer wants the deployment to target a staging workspace and a production workspace with different endpoint names and different Unity Catalog catalog names, without editing files between deployments. Which approach should the engineer use?

Trap 1: Hard-code the production endpoint and catalog names and use a…

Hard-coding production values and mutating them after deployment makes the deployed state diverge from the declared state, so the next bundle deploy may revert or conflict with the renames. It also requires extra custom code that is hard to test. This approach fights the declarative model instead of using its built-in parameterization.

Trap 2: Maintain two separate bundle projects, one per workspace, and keep…

Duplicating the bundle per environment creates drift, since every change must be applied twice and the two copies inevitably diverge. It also defeats the purpose of a declarative bundle as a single source of truth. The scenario explicitly wants to avoid editing files between deployments, and manual synchronization is error-prone.

Trap 3: Store the environment-specific values in a secrets scope and read…

Secrets are for sensitive values such as tokens, not for resource names that the bundle must resolve at deploy time to create the correct endpoint and reference the correct catalog. The bundle needs these values during deployment, before any agent code runs. Using secrets here adds runtime complexity and still leaves the deployment-time naming problem unsolved.

A
Hard-code the production endpoint and catalog names and use a post-deployment notebook to rename resources in staging.
Why it fails: Hard-coding production values and mutating them after deployment makes the deployed state diverge from the declared state, so the next bundle deploy may revert or conflict with the renames. It also requires extra custom code that is hard to test. This approach fights the declarative model instead of using its built-in parameterization.
B
Define bundle variables for the endpoint name and catalog name, and set their values per target in the bundle's databricks.yml targets section.
Databricks Asset Bundles support variables that can be overridden per target, so the same bundle definition can deploy to staging and production with different endpoint and catalog names. This keeps a single source of truth while allowing environment-specific values, which is exactly what the scenario requires. The engineer selects a target at deploy time and no files are edited.
C
Maintain two separate bundle projects, one per workspace, and keep them synchronized manually.
Why it fails: Duplicating the bundle per environment creates drift, since every change must be applied twice and the two copies inevitably diverge. It also defeats the purpose of a declarative bundle as a single source of truth. The scenario explicitly wants to avoid editing files between deployments, and manual synchronization is error-prone.
D
Store the environment-specific values in a secrets scope and read them at runtime inside the agent code.
Why it fails: Secrets are for sensitive values such as tokens, not for resource names that the bundle must resolve at deploy time to create the correct endpoint and reference the correct catalog. The bundle needs these values during deployment, before any agent code runs. Using secrets here adds runtime complexity and still leaves the deployment-time naming problem unsolved.

Question 10hardmultiple choice

An engineer is using Mosaic AI Agent Evaluation to score a conversational agent that calls tools. The agent sometimes answers correctly but with fabricated citations. Which evaluation approach best surfaces this specific failure mode?

Trap 1: Measure only token usage and latency to detect when the agent is…

Token usage and latency are operational metrics that reveal cost and performance, not factual grounding. A hallucinated citation may consume the same tokens and time as a legitimate one. These metrics cannot distinguish supported from fabricated claims, so they are unsuitable for this diagnostic goal.

Trap 2: Run the agent without tools so it cannot produce citations at all.

Removing tools eliminates the agent's ability to retrieve and cite sources, which changes the application's behavior rather than evaluating it. The goal is to detect fabricated citations in the real system, not to prevent citation generation. This approach sidesteps the requirement instead of measuring it.

Trap 3: Rely solely on the correctness judge, since a correct answer cannot…

A response can be factually correct yet cite a source that was never retrieved, which the correctness judge does not catch because it evaluates the answer against a reference rather than against the retrieved context. Fabricated citations are a grounding problem, so a correctness-only evaluation misses the failure mode entirely.

A
Measure only token usage and latency to detect when the agent is hallucinating citations.
Why it fails: Token usage and latency are operational metrics that reveal cost and performance, not factual grounding. A hallucinated citation may consume the same tokens and time as a legitimate one. These metrics cannot distinguish supported from fabricated claims, so they are unsuitable for this diagnostic goal.
B
Run the agent without tools so it cannot produce citations at all.
Why it fails: Removing tools eliminates the agent's ability to retrieve and cite sources, which changes the application's behavior rather than evaluating it. The goal is to detect fabricated citations in the real system, not to prevent citation generation. This approach sidesteps the requirement instead of measuring it.
C
Enable the groundedness judge and supply the retrieved context so the judge can verify claims against sources.
The groundedness judge checks whether statements in the response are supported by the provided context. Fabricated citations are exactly the failure mode it detects, because the cited source either does not exist in the context or does not contain the claim. Supplying the retrieved context is required for the judge to perform this verification accurately.
D
Rely solely on the correctness judge, since a correct answer cannot contain fabricated citations.
Why it fails: A response can be factually correct yet cite a source that was never retrieved, which the correctness judge does not catch because it evaluates the answer against a reference rather than against the retrieved context. Fabricated citations are a grounding problem, so a correctness-only evaluation misses the failure mode entirely.

Question 11hardmultiple choice

An engineer has a Databricks Vector Search index that stores chunk embeddings for a technical manual. Users report that queries containing exact part numbers return irrelevant chunks because the embedding model blurs numeric tokens. The engineer wants retrieval to consider both dense vector similarity and exact keyword matching on the same Delta table without building a separate search system. Which Databricks Vector Search feature should the engineer enable?

Trap 1: A larger embedding model dimension

Increasing embedding dimensions changes the representation capacity of the dense vector but does not introduce exact token matching, so numeric part numbers can still be blurred by tokenization and pooling. Re-embedding the entire corpus also adds cost and latency without addressing the keyword recall gap that users are reporting.

Trap 2: A higher number of returned results per query

Increasing the number of returned results widens the candidate set but does not change how those candidates are scored, so exact part numbers remain underrepresented in the ranking. It also inflates downstream prompt size and cost, and the additional chunks are likely to be even less relevant than the original top results.

Trap 3: Delta table change data feed on the source table

Change data feed tracks row-level changes in a Delta table so a vector index can be incrementally synced, but it governs index freshness rather than retrieval ranking. It does nothing to add keyword matching for exact part numbers, so queries would still rely solely on dense similarity and return the same irrelevant chunks.

A
A larger embedding model dimension
Why it fails: Increasing embedding dimensions changes the representation capacity of the dense vector but does not introduce exact token matching, so numeric part numbers can still be blurred by tokenization and pooling. Re-embedding the entire corpus also adds cost and latency without addressing the keyword recall gap that users are reporting.
B
Hybrid search on the index
Hybrid search in Databricks Vector Search combines dense vector similarity with keyword-based matching over the indexed text, so exact tokens such as part numbers contribute to ranking alongside semantic similarity. Enabling it on the existing index lets the engineer improve recall for exact identifiers without standing up a separate keyword search system or duplicating the source data.
C
A higher number of returned results per query
Why it fails: Increasing the number of returned results widens the candidate set but does not change how those candidates are scored, so exact part numbers remain underrepresented in the ranking. It also inflates downstream prompt size and cost, and the additional chunks are likely to be even less relevant than the original top results.
D
Delta table change data feed on the source table
Why it fails: Change data feed tracks row-level changes in a Delta table so a vector index can be incrementally synced, but it governs index freshness rather than retrieval ranking. It does nothing to add keyword matching for exact part numbers, so queries would still rely solely on dense similarity and return the same irrelevant chunks.