Databricks-GenAI-Assoc Practice Test 2 — 30 Questions

Question 1

A team stores raw documents in a Unity Catalog volume and needs to build a training set for fine-tuning a large language model. The raw files are in mixed formats including PDF, DOCX, and plain text. The team wants a single Delta table where every row is one document with its extracted text and source path, and wants the extraction to run in parallel across the cluster. Which approach should the team use?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use binaryFile data source to read the volume, then apply a UDF that extracts text per file and writes results to a Delta table.. The binaryFile data source is designed to read arbitrary files as rows with path and content, enabling distributed extraction with a UDF. This yields one row per document with text and source path in a Delta table. Driver loops and plain text readers cannot handle mixed binary formats at scale or in parallel.

Answer

Mount the volume as a local filesystem path and use a Python for-loop to read and extract each file on the driver.

Answer

Use spark.read.text on the volume, which automatically parses PDF and DOCX content into text columns.

Answer

Create an external table over the volume with a schema that includes columns for each document format's fields.

Question 2

What is the primary function of the 'Prompt' object in a Databricks RAG architecture?

Accepted Answer

To format instructions and retrieved context for LLM consumption.. The prompt acts as the interface between the data retrieved from the vector index and the LLM's generative capability. It organizes the context, instructions, and user request into a coherent message structure. Proper prompt design is essential to focus the LLM's attention, reduce extraneous output, and ensure that the generative output remains grounded in the provided evidence.

Answer

To store the user's chat history for session persistence.

Answer

To index documents for faster similarity search.

Answer

To calculate the latency of the RAG pipeline.

Question 3

A developer is using MLflow to track experiments for a RAG application. They want to log the retrieval step's parameters, such as the number of documents retrieved (k) and the embedding model used. Which MLflow API should they use?

Accepted Answer

mlflow.log_param(). MLflow parameters are meant for logging configuration settings that are constant for a run. The number of documents retrieved and the embedding model are such settings. mlflow.log_param() records them as key-value pairs, enabling easy comparison across runs in the MLflow UI. Metrics, artifacts, and tags serve different purposes and are not suitable for this use case.

Answer

mlflow.log_artifact()

Answer

mlflow.set_tag()

Answer

mlflow.log_metric()

Question 4

A developer deploys a new model version as shown in the exhibit. What is the purpose of this configuration?

Accepted Answer

To conduct A/B testing or a canary rollout.. This configuration implements a canary deployment strategy by splitting traffic between two model versions. By routing 90% of requests to the stable version and 10% to the new version, the team can monitor performance and accuracy in a real-world scenario with minimal risk. This is a standard practice for safely rolling out model improvements, allowing teams to collect data on the new version's behavior before a full-scale transition.

Answer

To increase compute capacity for the primary model.

Answer

To load balance between two different cloud regions.

Answer

To force all traffic to model-v1 when model-v2 errors.

Question 5

A GenAI engineer is building a Databricks RAG application where the retrieval step returns the top-5 chunks for each user question. The engineer wants to add a second LLM call that evaluates whether each retrieved chunk contains enough information to answer the question, and then filters out chunks that fail this evaluation before passing the remaining chunks to the final answer-generation prompt. Which design pattern is the engineer implementing?

Accepted Answer

Retrieval-augmented generation with relevance grading. The defining behavior in the scenario is a second LLM call that grades each retrieved chunk for answerability and drops the ones that fail. That is relevance grading layered on top of retrieval-augmented generation, which raises precision by keeping only context that can actually support the answer. The other techniques alter retrieval ranking, prompt reasoning, or embeddings but do not insert an evaluative filtering step between retrieval and generation.

Answer

Chain-of-thought prompting

Answer

Fine-tuning the embedding model on domain data

Answer

Hybrid search combining vector similarity with BM25 keyword matching

Question 6

When developing a feature engineering pipeline using Feature Store, which practice ensures maximum code reusability across training and inference?

Accepted Answer

Use the Feature Store client to log feature definitions and retrieve them.. Defining features as code using the Databricks Feature Store ensures that the same logic is applied during both batch training and real-time inference. By encapsulating feature calculations in Feature Tables, developers avoid the 'training-serving skew' where features are calculated differently in production. This practice is critical in GenAI development to ensure model performance consistency and reduce technical debt caused by disjointed preprocessing pipelines across different stages of the ML lifecycle.

Answer

Calculate features in the application code and pass them as raw inputs.

Answer

Hardcode feature calculation logic inside the training notebook.

Answer

Export features to a static CSV file after every training run.

Question 7

A team is preparing a Delta table of product descriptions for a RAG application. The table receives continuous upserts from a streaming pipeline, and the embedding job reads the table every hour. Engineers notice the embedding job reprocesses every row on each run even though only a few rows change. Which change should be made to the source table to let the embedding job process only new or updated rows?

Accepted Answer

Enable Change Data Feed on the Delta table and have the embedding job read the change feed since the last processed version.. Change Data Feed captures row-level changes with commit versions, allowing a downstream job to read only the inserts, updates, and deletes that occurred since its last checkpoint. This directly solves the problem of reprocessing unchanged rows. Partitioning, compaction, and timestamp filtering improve read characteristics but do not expose which rows changed, so they cannot deliver incremental processing.

Answer

Partition the Delta table by the product category column so the embedding job reads fewer files.

Answer

Convert the table to a view that filters rows by the current timestamp so only recent rows appear.

Answer

Run OPTIMIZE with Z-ORDER on the product identifier column to compact small files before each embedding run.

Question 8

Which Databricks feature is specifically designed to facilitate the rapid development and deployment of LLM applications by providing a managed environment for hosting and testing prompts?

Accepted Answer

Databricks Mosaic AI Playground. Databricks Mosaic AI Playground provides a low-code interface for developers to experiment with different LLMs and system prompts. It allows for quick iteration and testing of model behavior before moving into full production deployment. This environment is crucial because it bridges the gap between experimentation and application development, ensuring that developers can validate prompt engineering strategies within the same governed environment where their data and production models reside.

Answer

Databricks SQL Warehouse

Answer

Unity Catalog Volumes

Answer

Delta Live Tables

Question 9

A GenAI engineer has a RAG application whose retrieval step uses Databricks Vector Search. Users report that answers are sometimes irrelevant because the retriever pulls chunks from documents the user is not authorized to see. The engineer must enforce per-user document ACLs at query time without re-indexing the corpus. Which approach should the engineer take?

Accepted Answer

Add a metadata column that stores the document's ACL principals, then pass a filter string with the caller's identity to the Vector Search query.. Vector Search indexes can carry metadata columns that are queryable through the filters parameter, letting the application pass the caller's identity so only permitted chunks are returned. This satisfies per-user ACL enforcement at query time without rebuilding the index when permissions change, and it avoids leaking unauthorized content into the application tier. The other approaches either do not affect index results or are operationally impractical at scale.

Answer

Post-filter the retrieved chunks in the application layer by querying Unity Catalog for the user's group memberships and dropping chunks that fail the check.

Answer

Create a separate Vector Search index per user and have the application route each request to the user-specific index.

Answer

Enable Delta Sharing on the source Delta table and rely on the workspace's existing table ACLs to filter Vector Search results.

Question 10

An application requires low-latency retrieval of RAG metadata stored in Databricks. Which storage approach balances performance and cost while ensuring seamless integration with Unity Catalog?

Accepted Answer

Deploy a Databricks Vector Search index on a Delta table with automatic synchronization.. Vector Search indexes are designed for low-latency similarity search, which is critical for RAG applications. By leveraging Databricks Vector Search, you avoid the overhead of custom search implementations. This approach ensures that embeddings are automatically synchronized with Delta tables, providing a scalable solution that integrates directly with Unity Catalog's security and governance framework, which is essential for maintaining consistent data access policies across enterprise environments.

Answer

Store embeddings in a flat Parquet file on DBFS accessed via direct Spark reads.

Answer

Utilize a third-party managed vector database outside of the Databricks environment.

Answer

Implement a custom KNN algorithm using standard Python libraries on a single node cluster.

Question 11

Refer to the exhibit. An engineer observes an unexpected drop in "relevance" for the latest deployment. What is the most likely cause related to the evaluation process itself?

Accepted Answer

The evaluation dataset is misaligned with current production query trends.. A drop in relevance often suggests that the evaluation dataset (eval_set_v4) may have become outdated or misaligned with the current production use cases. If the user base or query patterns shift, the original evaluation set may no longer accurately reflect the actual performance requirements. Regularly refreshing the evaluation dataset to match current production trends is essential to ensure that metrics remain valid indicators of system performance.

Answer

The model was trained on too much data.

Answer

The inference cluster has too much compute capacity.

Answer

The faithfulness score increased too much.

Question 12

You need to ingest data from an external JSON source into a Delta table. The source schema is inconsistent. Which strategy is most effective for preparing this data?

Accepted Answer

Implement a bronze-silver medallion pattern.. The 'bronze-to-silver' pattern is the industry standard in Databricks for handling inconsistent data. By loading raw JSON into a 'Bronze' table with a schema of type 'string' (or a single JSON column), you preserve all information for audit. You then use 'Silver' tables to perform schema enforcement, data cleaning, and type casting. This separation of concerns allows for robust error handling and iterative refinement of the data cleaning logic.

Answer

Use inferSchema = true in the read configuration.

Answer

Define a rigid schema at the ingestion point.

Answer

Convert the JSON to CSV before loading into Delta.

Question 13

A healthcare company uses Databricks to build a RAG application over clinical notes stored in a Unity Catalog table. An auditor requires proof that only authorized personnel can view raw note text, while the RAG application must use embeddings generated from those notes. The team wants to avoid copying data outside Unity Catalog. Which approach best satisfies the auditor while preserving RAG functionality?

Accepted Answer

Create a column mask on the note text column that returns a redacted value for unauthorized users, and grant the RAG service principal access to a separate embeddings table derived from the notes.. A column mask on the sensitive note text column enforces redaction for unauthorized identities at query time, which is auditable. By having the RAG application consume a derived embeddings table instead of raw text, the application retains functionality while raw PHI remains protected. This keeps governance within Unity Catalog and satisfies the auditor's requirement.

Answer

Move the clinical notes to an external location outside Unity Catalog and grant access via cloud IAM roles.

Answer

Grant the RAG application's service principal SELECT on the table and rely on the application to redact note text in its responses.

Answer

Use a dynamic view that filters rows based on the user's group and grant the RAG service principal access to that view.

Question 14

A data engineer needs to prepare a Delta table of customer reviews for embedding generation. The reviews contain HTML tags, inconsistent whitespace, and mixed casing that hurt embedding quality. Which preparation step should be applied before generating embeddings?

Accepted Answer

Normalize the text by stripping HTML tags, collapsing whitespace, and applying consistent casing.. Embedding models encode the exact tokens they receive, so HTML markup, irregular whitespace, and inconsistent casing add noise that dilutes semantic signal. Normalizing the text before embedding removes that noise and produces vectors that better reflect meaning. Storage format and vector dimension do not change the input text, so they cannot solve the quality problem.

Answer

Apply a hash of the raw review text as the embedding input to guarantee deterministic vectors.

Answer

Increase the embedding vector dimension to capture the additional characters in the raw text.

Answer

Store the reviews in a Parquet file instead of Delta to improve text compression before embedding.

Question 15

A fintech company runs a customer-support RAG assistant on Databricks. Before promoting a new prompt template from staging to production, the ML team must demonstrate that the change does not regress answer quality. Which TWO evaluation practices should they apply in Mosaic AI Agent Evaluation to make this promotion decision defensible? (Choose two.)

Accepted Answer

Pin the evaluation dataset, judge model version, and retrieval configuration so both templates are scored under identical conditions and results are reproducible.. A defensible promotion gate requires a controlled comparison where the prompt template is the only variable, which means running both templates over the same dataset with the judge model and retrieval configuration pinned, and inspecting per-question deltas instead of relying on averages that can mask localized regressions. Together these practices attribute any score movement to the change under test and make the decision reproducible.

Answer

Deploy the candidate template to a small percentage of production traffic and rely on user thumbs feedback alone as the promotion signal.

Answer

Increase the judge model size until its scores on the candidate template exceed the production template's scores.

Answer

Evaluate only the questions where the candidate template produced different answers, skipping the rest to save judge tokens.

Question 16

An engineer is packaging a GenAI agent application with Databricks Asset Bundles so that the same bundle deploys to a development and a production workspace. The agent's serving endpoint name must differ per target, and the production endpoint needs more concurrent capacity. Which mechanism in the bundle configuration should the engineer use?

Accepted Answer

Define target-specific variables in databricks.yml and reference them with ${var.endpoint_name} and ${var.min_replicas} in the resource definition.. Databricks Asset Bundles let you declare variables and override them per target, so a single resource definition can deploy the agent with different endpoint names and replica counts in development and production. Referencing variables with the ${var.} syntax keeps the bundle DRY while honoring environment differences. Hard-coded values, secret scopes, and post-deployment mutation scripts all fail to provide the clean, declarative per-target configuration the scenario requires.

Answer

Use a post-deployment notebook that calls the serving endpoints API to rename the endpoint and adjust replicas after each bundle deployment.

Answer

Store the endpoint name and replica count as secrets in a Databricks secret scope and read them at deployment time with the Databricks CLI.

Answer

Hard-code the production endpoint name and replica count, then maintain a separate copy of the bundle for development.

Question 17

When preparing a dataset for fine-tuning an LLM, you need to ensure the data is representative of the target domain. What is the most effective approach to detect and mitigate sampling bias in your training set using Databricks?

Accepted Answer

Perform statistical profiling on feature distributions and re-sample if necessary.. Using exploratory data analysis (EDA) with Databricks SQL or Spark to analyze distribution statistics is the most effective way to identify bias. By comparing the distribution of the training set to a representative sample of real-world production data, engineers can identify under-represented categories. This ensures that the fine-tuned model performs reliably across all expected input scenarios, preventing the model from developing blind spots.

Answer

Increase the batch size during the fine-tuning process.

Answer

Change the model architecture to a larger parameter count.

Answer

Use a random seed to shuffle the data before splitting.

Question 18

A developer is building a RAG application using Mosaic AI Model Serving. They need to ensure that the embedding model endpoint is strictly accessed only by specific service principals within the workspace. Which feature should the developer configure to enforce this security requirement?

Accepted Answer

Model Serving Permissions. To restrict access to Mosaic AI Model Serving endpoints, developers should use Model Serving Permissions. By navigating to the Permissions tab of the specific endpoint, they can grant 'Can query' access exclusively to authorized service principals. This ensures that only authenticated and authorized applications can retrieve embeddings, protecting the model from unauthorized inference calls while maintaining a secure development lifecycle in Databricks.

Answer

Network Security Groups

Answer

Unity Catalog External Locations

Answer

Cluster Access Control Lists

Question 19

A data scientist has developed a GenAI application locally and now needs to share it with stakeholders as an interactive web app inside the Databricks workspace. The app must use the workspace's existing authentication and compute. Which Databricks capability should be used?

Accepted Answer

Databricks Apps, which host the application within the workspace and integrate with workspace identity and compute.. Databricks Apps is designed for hosting interactive applications directly in the workspace, using the workspace's identity and compute. This lets stakeholders access the GenAI app through a URL with existing authentication, avoiding the need to build custom hosting or manage separate credentials.

Answer

A Databricks job with a notebook task scheduled to run every five minutes.

Answer

An MLflow experiment tracking run that logs the application as an artifact.

Answer

A Databricks notebook with the %run magic command to execute the application logic on demand.

Question 20

A team is using MLflow LLM Evaluation with the built-in answer_correctness metric to compare two prompt templates for a question-answering application. They notice that answer_correctness scores are nearly identical, but manual review shows one template produces answers that are factually correct yet omit key supporting details. Which additional built-in metric should they add to their evaluation to surface this difference?

Accepted Answer

answer_completeness. When answers are factually correct but differ in how much supporting detail they include, a completeness metric is required. answer_completeness compares the response against the ground truth to check whether all key points are covered, making it the right built-in metric to expose the omission of supporting details that answer_correctness alone misses.

Answer

groundedness

Answer

toxicity

Answer

answer_similarity