Databricks-GenAI-Assoc Practice Test 24 — 15 Questions

Question 1

A team wants every MLflow evaluation run of their RAG chain to be reproducible months later, including the exact prompt template, model parameters, and retrieved context used. Which practice best ensures this reproducibility?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Log the evaluation as an MLflow run with the model parameters, prompt template, and input dataset version recorded as run metadata and artifacts.. Reproducibility depends on capturing inputs and configuration, not just scores. Logging the run with parameters, the prompt template, and a versioned dataset as artifacts and metadata ties every result to the exact conditions that produced it. That lets anyone rerun or audit the evaluation later, even after library upgrades or dataset changes.

Answer

Export the evaluation results to a CSV file stored on a personal laptop so the engineer can compare future runs manually.

Answer

Rely on the built-in default judge prompts so that no prompt configuration needs to be tracked between runs.

Answer

Store only the final aggregate metric values in a shared spreadsheet, since the underlying outputs can always be regenerated on demand.

Question 2

A team is deploying a RAG chatbot that answers questions from internal policy documents. The documents change frequently, and the team wants retrieval to reflect updates within minutes without re-running a full embedding job. The source Delta table already has change data feed enabled. Which approach should the team use to keep the Databricks Vector Search index current?

Accepted Answer

Configure the index to sync from the Delta table so changes are propagated incrementally. A Delta-synced vector search index consumes change data feed events to apply inserts, updates, and deletes incrementally, keeping embeddings aligned with the source table. Rebuilding the index or tuning the endpoint addresses cost and throughput rather than freshness, and metadata filters only narrow an already-indexed set.

Answer

Increase the cluster size of the vector search endpoint

Answer

Drop and recreate the vector search index on a schedule

Answer

Add a metadata filter for the document modification timestamp

Question 3

Refer to the exhibit. A developer wants to update this serving endpoint configuration to ensure it handles high-concurrency requests with consistent latency. Which change should be applied to the configuration?

Accepted Answer

Set 'scale_to_zero_enabled' to 'false' and define 'min_provisioned_concurrency'.. To handle high-concurrency with consistent latency, the configuration must move away from 'scale_to_zero_enabled: true' and utilize 'min_provisioned_concurrency'. Scaling to zero introduces cold-start latency that disrupts consistent response times required for high-concurrency production applications. By pinning minimum resources, the model remains active and ready to handle incoming traffic immediately, ensuring that performance remains predictable even during intermittent bursts of user activity within the production environment.

Answer

Change 'workload_type' to 'GPU' and remove 'scale_to_zero_enabled'.

Answer

Increase the 'model_version' to 'latest' to trigger automatic load balancing.

Answer

Add a 'timeout' parameter to the JSON configuration block.

Question 4

Which THREE practices are recommended when using MLflow for managing LLM experiments in Databricks?

Accepted Answer

Log model parameters such as temperature, top_p, and system prompts.. Effective experiment management requires traceability, reproducibility, and structured documentation. Logging parameters like temperature and system prompts ensures results can be replicated. Using signature definitions allows the system to validate inputs, reducing integration errors. Finally, tagging models aids in lifecycle management, allowing teams to distinguish between prototypes and production-ready versions. These practices ensure that teams can maintain high standards of rigor, enabling scalable AI operations within the enterprise environment.

Answer

Store all model weights directly in the MLflow run object.

Answer

Always delete old MLflow experiments to save storage space.

Question 5

A generative AI engineer is designing a multi-stage RAG application on Databricks. The application first retrieves documents using Vector Search, then reranks them with a cross-encoder model, and finally calls a foundation model endpoint to generate an answer. The engineer wants to ensure that the entire pipeline is reproducible and that each stage can be independently versioned and deployed. Which design approach best meets these requirements?

Accepted Answer

Package each stage as an MLflow model with its own signature and dependencies, and compose them into a single MLflow pipeline that can be logged and served as one model.. Packaging each stage as an MLflow model with its own signature and dependencies allows independent versioning. Composing them into a single MLflow pipeline enables logging and serving the entire multi-stage RAG application as one model, ensuring reproducibility. This approach is supported by Databricks Model Serving and MLflow, and it simplifies deployment and tracing compared to separate endpoints or scripts.

Answer

Implement the pipeline as a single Python script that calls all stages sequentially, and version the script in Git.

Answer

Use MLflow Projects to define each stage as a separate project, and orchestrate them with a Databricks Job that passes artifacts between stages.

Answer

Deploy each stage as a separate Databricks Model Serving endpoint and have the client application call them in sequence.

Question 6

A GenAI engineer is designing a Databricks application that must ground answers in a large corpus of internal policy documents. The corpus is updated by a nightly Delta job, and the application must cite the source document for every answer. The engineer is deciding how to structure the retrieval and generation stages. Which TWO design choices best satisfy the grounding and citation requirements? (Choose two.)

Accepted Answer

Instruct the model to answer only from the retrieved context and to include the provided document identifiers in its response. Grounding and citation depend on two things working together: retrieval that returns source identifiers with each chunk, and a generation prompt that restricts the model to that retrieved context while requiring it to surface the identifiers. Metadata columns supply verifiable provenance, and the constrained instruction keeps the answer tied to retrieved policy text instead of the model's own memory.

Answer

Rely on the model's pretrained knowledge of internal policies to fill gaps when retrieval returns no relevant chunks

Answer

Cache the full corpus in the prompt on every request so the model always has complete policy context

Answer

Increase the model's temperature setting so the generation stage explores a wider range of possible answers

Question 7

A generative AI engineer trains a model on a Delta table that contains customer support transcripts. Before registering the model, security requires that the training data be classified so that policies can be applied consistently across the lakehouse. The engineer wants to attach a governed label to the transcript column indicating it contains sensitive personal data. Which Unity Catalog feature should be used?

Accepted Answer

A governed tag applied to the column. Governed tags provide a structured, searchable classification layer in Unity Catalog that can be attached to columns and referenced by policy and discovery tooling. Applying a sensitivity tag to the transcript column gives security a consistent label for the training data, enabling uniform policy enforcement and auditability rather than relying on informal comments or access mechanisms.

Answer

A storage credential scoped to the transcript path

Answer

A comment added to the column definition

Answer

A row filter that excludes rows containing personal data

Question 8

A Generative AI engineer at a retail company has deployed a RAG chatbot on a Databricks Mosaic AI Model Serving endpoint. The team wants to capture per-request evaluation data, including the user's question, the retrieved context chunks, and the model's response, so they can analyze response quality over time. Which Databricks feature should they use to collect this data for downstream evaluation?

Accepted Answer

Inference tables. Inference tables are the Databricks mechanism that logs request and response payloads for Mosaic AI Model Serving endpoints. Because the logged payloads include the user question, retrieved context, and generated answer, the team gains the raw material needed to run MLflow LLM Evaluation or custom monitoring jobs later against real production traffic.

Answer

MLflow Model Registry

Answer

Delta Live Tables expectations

Answer

Unity Catalog lineage

Question 9

A team has deployed a model that is experiencing high latency. How should they identify if the bottleneck is the model inference or the preprocessing code?

Accepted Answer

Use MLflow Tracing to inspect the execution pipeline.. Distinguishing between inference time and preprocessing time is vital for performance tuning. By using MLflow's tracing capabilities or custom timing logs, engineers can isolate specific segments of the request pipeline. This granular visibility allows for targeted optimization, such as optimizing the vector search or reducing prompt overhead, rather than blindly attempting to speed up the model itself, which is often significantly more resource-intensive.

Answer

Check the cluster cost in the Databricks billing console.

Answer

Re-train the model on a larger dataset.

Answer

Increase the memory limit for the inference cluster.

Question 10

You need to store embeddings generated by an LLM in a Delta table. Which data type is most efficient for storing these high-dimensional vector arrays in Databricks?

Accepted Answer

Store as an array of doubles.. Storing embeddings as an 'Array' of 'Floats' (or 'Doubles') in a Delta table is the most efficient and native way to handle them. This allows Spark to utilize vectorized operations for similarity search calculations and enables compatibility with various indexing libraries. Using this approach ensures high performance during retrieval, minimizing the latency of finding the most relevant context for the LLM during generation.

Answer

Store as a Base64 encoded string.

Answer

Store as separate columns for each dimension (e.g., dim1, dim2, ...).

Answer

Store as a compressed binary blob (e.g., serialized byte array).

Question 11

A GenAI engineer is preparing a corpus of HTML product pages stored in a Unity Catalog volume for a RAG application. The pages contain navigation bars, script tags, and boilerplate footers that add noise to embeddings. Which Databricks-native approach best removes this noise while preserving the main article text before chunking?

Accepted Answer

Use an ai_parse_document or BeautifulSoup-style parser in a PySpark UDF to extract the main content, then write the cleaned text as a Delta table with a content column.. Cleaning HTML before chunking requires a parser that understands document structure, not a regex or a numeric feature transformer. Extracting the main content with a document-aware parser and persisting the cleaned text to Delta produces a reliable, lineage-tracked input for chunking and embedding, which is the goal of data preparation for RAG on Databricks.

Answer

Register the volume as a Delta table and run a SQL query using regexp_replace to strip all tags, then tokenize the result with the built-in Databricks tokenizer.

Answer

Increase the chunk size to 4096 tokens so that boilerplate is a smaller proportion of each chunk, and rely on the embedding model to ignore the noise.

Answer

Load the HTML files with spark.read.text and apply a VectorAssembler to convert the raw strings into dense vectors, then cluster and drop outliers.

Question 12

A team is building a RAG application using Databricks Vector Search with a Delta table as the source. They need the index to automatically reflect new and updated chunks as the source table changes, without rebuilding the entire index each time. Which configuration should they use?

Accepted Answer

Create a Vector Search index with sync mode set to TRIGGERED or CONTINUOUS and specify a Delta table with Change Data Feed enabled as the source.. Delta Sync indexes in Databricks Vector Search rely on Change Data Feed to pick up inserts, updates, and deletes incrementally. Choosing a triggered or continuous sync mode keeps embeddings current without full recomputation. Enabling CDF on the source Delta table is a prerequisite, and this pattern is the documented way to keep a RAG index fresh as documents change.

Answer

Enable Delta Lake time travel on the source table and point the Vector Search index at a specific version using VERSION AS OF.

Answer

Use an external HNSW index built on a Databricks cluster and register it in Unity Catalog as a model, then query it with the model serving endpoint.

Answer

Create a standard Vector Search index and schedule a nightly job that drops the index and recreates it from the full Delta table.

Question 13

You are deploying a RAG application. You need to ensure the model uses the most recent vector data without redeploying the model. What should you use?

Accepted Answer

Implement a RAG pattern with a vector database index.. Decoupling the model logic from the data retrieval mechanism is a best practice. By using a retrieval-augmented generation pattern where the retriever fetches data dynamically from a vector store, the model remains static while the knowledge base evolves. This approach is essential for applications requiring real-time information, as it avoids the expensive and slow process of retraining or fine-tuning the base model whenever the underlying data changes.

Answer

Fine-tune the model with new data daily.

Answer

Re-register the model in MLflow with every update.

Answer

Embed all data into the model weights at deployment.

Question 14

Which of the following metrics is most effective for evaluating a RAG-based chatbot's ability to retrieve relevant context from a vector database during production monitoring?

Accepted Answer

Context Relevance. Context relevance and faithfulness are key RAG metrics. Specifically, evaluating the retrieved context's alignment with the user query is essential for identifying retrieval failures. If the context is irrelevant, the model cannot generate a correct answer even with high-quality generative capabilities. Monitoring this metric helps distinguish between errors caused by the retriever versus errors caused by the generator model during the production lifecycle.

Answer

Model Training Loss

Answer

Inference Latency

Answer

Parameter Count

Question 15

Which security configuration is essential when deploying a model that accesses sensitive data stored in Unity Catalog?

Accepted Answer

Use a Service Principal with scoped permissions for the model.. Accessing sensitive data requires strict adherence to the principle of least privilege. Assigning a specific Service Principal to the model serving endpoint ensures that it only has access to the data necessary for inference. This configuration is critical for security, as it limits the potential blast radius if the model endpoint is ever compromised, maintaining data privacy in line with organizational governance policies.

Answer

Grant the endpoint access to the entire Unity Catalog.

Answer

Store credentials in plaintext in the model code.

Answer

Bypass Unity Catalog to use local file system access.