Databricks-GenAI-Assoc Design Applications — Timed 25 Questions

Question 1

Refer to the exhibit. An engineer is configuring a model serving endpoint. Based on the configuration, which outcome is expected during a sudden surge in traffic?

Accepted Answer

The endpoint will scale up to 16 CPU cores to handle the increased load.. The configuration enables auto-scaling, which allows the endpoint to dynamically adjust resources between the specified CPU limits. This capability is vital for managing variable workloads, ensuring that the application remains responsive during traffic spikes while optimizing infrastructure costs during idle periods. This is a core component of production-ready Databricks generative AI model serving design.

Answer

The endpoint will crash due to lack of static resource allocation.

Answer

The endpoint will force requests to queue indefinitely until traffic subsides.

Answer

The configuration is invalid because it lacks a GPU definition.

Question 2

Which THREE practices are recommended when designing a robust Databricks Model Serving endpoint for a high-traffic generative AI application?

Accepted Answer

Enabling auto-scaling to handle variable traffic demand.. Deploying a high-traffic generative AI application requires careful endpoint configuration, including resource limits, security, and observability. By setting appropriate scaling limits, using private endpoints, and enabling monitoring, engineers ensure the system is both performant and secure. These practices are essential for managing the unpredictable nature of AI traffic and ensuring that the service meets enterprise standards for availability and data protection.

Answer

Configuring the endpoint to be publicly accessible from the internet.

Answer

Allocating infinite CPU resources to prevent any latency.

Question 3

Which TWO architectural patterns are best practices when designing a scalable RAG application on Databricks to ensure data security and performance?

Accepted Answer

Use Unity Catalog to manage access control on the source Delta tables powering the vector index.. By decoupling the embedding generation from the retrieval process and enforcing Unity Catalog permissions, you ensure that the application is secure and performs efficiently. These patterns are essential for enterprise applications where data privacy is paramount and high concurrency is expected. Proper indexing strategies combined with granular access control ensure that users only retrieve information they are authorized to view during the generative process.

Answer

Store all sensitive data in cleartext within the vector index for faster retrieval.

Answer

Perform embedding generation as a batch process at the time of retrieval.

Answer

Expose the LLM endpoint directly to the public internet without authentication.

Question 4

Which THREE components are required to monitor the quality and performance of a RAG application hosted on Databricks?

Accepted Answer

MLflow Tracing for capturing the entire chain of inference requests.. Monitoring RAG applications requires a holistic view of the retrieval, generation, and infrastructure layers. By tracking retrieval metrics, LLM performance, and system latency, engineers can identify bottlenecks and optimize the system. These components are vital for maintaining the accuracy and reliability of the RAG pipeline, ensuring the model's output remains relevant and helpful for the end users over time.

Answer

A manual spreadsheet to log every user interaction for offline review.

Answer

A hardcoded threshold for total token usage per day.

Question 5

Which Databricks feature allows you to securely manage LLM prompts and keep them versioned as code?

Accepted Answer

MLflow Models. MLflow is the standard tool in Databricks for managing the lifecycle of machine learning models and associated assets, including prompts. By versioning prompts in MLflow, teams can collaborate effectively, track changes, and roll back if a new prompt version causes quality issues. This is essential for maintaining a high-quality, reproducible generative AI application in a production environment.

Answer

Databricks Repos

Answer

Unity Catalog Volumes

Answer

Delta Lake Constraints

Question 6

Which THREE strategies should be implemented to mitigate hallucination in a RAG application?

Accepted Answer

Incorporate a 'self-correction' step where the model reviews its own output.. Hallucination is a common challenge in LLM applications. By grounding the model with high-quality retrieval, setting strict temperature parameters, and implementing systematic evaluation, developers can significantly reduce inaccurate outputs. These strategies ensure the LLM remains constrained by the provided context and does not deviate into factual errors, which is critical for trustworthy enterprise deployments.

Answer

Set the LLM temperature to a high value like 1.0 or greater.

Answer

Increase the number of retrieved chunks to include more irrelevant noise.

Question 7

An engineer is designing a Databricks RAG chain using LangChain on a Databricks cluster. The chain must call an external embedding model through a Databricks Model Serving endpoint and must persist intermediate retrieval results for debugging failed conversations. Compliance requires that no raw user question text leave the workspace boundary except through the approved serving endpoint. Which design decision best satisfies the persistence and boundary requirements together?

Accepted Answer

Use MLflow Tracing with the Databricks tracking server to capture the chain spans, including retrieved documents and prompts. MLflow Tracing is the native Databricks mechanism for capturing the internal steps of a GenAI chain, producing structured spans for retrieval, prompt construction, and model calls. Because traces land in the workspace's MLflow experiment, engineers get detailed debugging data without routing raw user text to an external system, satisfying the compliance boundary while preserving observability.

Answer

Write raw question and answer pairs to a Delta table using a custom callback handler registered in the chain

Answer

Log the final prompt string to the driver's stdout and rely on cluster log delivery for retention

Answer

Stream all chain events to an external observability SaaS platform using its LangChain integration

Question 8

A GenAI engineer is designing a retrieval pipeline that uses Databricks Vector Search with hybrid search enabled. Users report that semantically similar but keyword-distinct queries return irrelevant chunks, and that results vary between runs of the same query. Which TWO configuration choices should the engineer make to improve result relevance and consistency? (Choose two.)

Accepted Answer

Configure the index to return only results whose similarity score exceeds a high static threshold, discarding everything below it.. Hybrid retrieval activated through query_text blends keyword and vector signals, and a reranker plus a similarity threshold tightens and stabilizes the final candidate set. Returning unfiltered extra results, seeding embeddings, or dropping keyword matching either fails to address ranking quality or misidentifies the cause of inconsistent results.

Answer

Disable hybrid search and rely solely on dense vector similarity, since keyword matching introduces nondeterministic scoring.

Answer

Set a fixed random seed on the embedding model so that repeated queries always produce identical similarity scores.

Answer

Increase the number of results returned by the index and pass all of them, unchanged, directly into the LLM prompt as context.

Question 9

A GenAI engineer is building a RAG chatbot on Databricks. The application must answer questions over a 40 GB internal knowledge base and must return the first token to users within one second. Retrieval latency is currently dominated by network round trips between the serving endpoint and the vector store. Which design change best reduces end-to-end latency while keeping retrieval quality intact?

Accepted Answer

Co-locate the Vector Search index and the model serving endpoint in the same Databricks workspace and region, and enable the endpoint's built-in query optimization.. The latency problem is caused by network round trips between the serving endpoint and the vector index. Placing both resources in the same Databricks workspace and region removes those hops, and the endpoint's optimized query path to the local index keeps retrieval quality intact. Scaling, embedding size, or billing model changes do not remove the network bottleneck.

Answer

Enable Foundation Model APIs pay-per-token for the chat model and keep the Vector Search index in a separate workspace.

Answer

Configure the serving endpoint to use scale-to-zero so idle replicas are eliminated during low-traffic periods.

Answer

Increase the embedding dimensions of the Vector Search index from 1024 to 3072 to improve retrieval precision.

Question 10

A GenAI engineer is designing a RAG chatbot on Databricks that must answer questions from a 2 million-document knowledge base with sub-second retrieval latency. The team wants to avoid re-embedding the entire corpus on every model upgrade and wants the index to stay synchronized with a Delta table that receives near-real-time updates. Which design choice best satisfies these requirements?

Accepted Answer

Create a Databricks Vector Search Delta Sync index on the source Delta table and configure a continuous sync mode.. The Delta Sync index in Databricks Vector Search is the only option that provides a managed, low-latency vector index that automatically stays synchronized with a Delta table. Continuous sync mode addresses near-real-time updates without custom pipelines, and the index decouples embedding generation from query serving so model upgrades can be handled by re-embedding or rebuilding the index rather than changing application code.

Answer

Store embeddings in a Unity Catalog volume and use a Python dictionary cached on the driver for lookups.

Answer

Export the corpus to Parquet files and use a Spark UDF to compute cosine similarity at query time.

Answer

Use MLflow Model Registry to version the embedding model and call the model serving endpoint directly for each document retrieval.

Question 11

A team is designing a GenAI application that uses Databricks Model Serving to host a fine-tuned Llama model. The application must support bursty traffic that can spike from 2 to 200 requests per second within minutes, and the team wants to avoid provisioning for peak capacity. Which Model Serving configuration should the team choose?

Accepted Answer

Serverless GPU compute with autoscaling enabled, setting a minimum of zero and a maximum that covers the burst, and configuring scale-to-zero for idle periods.. Bursty traffic that swings from 2 to 200 requests per second is best handled by serverless GPU compute with autoscaling, because replicas can be added and removed dynamically to match demand. Provisioned throughput overprovisions for peak, fixed low concurrency throttles spikes, and CPU inference cannot meet latency or scale requirements for a Llama model.

Answer

A single-node CPU cluster with the model loaded from a Unity Catalog volume at request time.

Answer

Pay-per-token Foundation Model APIs with a fixed concurrency limit of two requests.

Answer

Provisioned throughput with a fixed number of concurrent requests sized to the 200 requests-per-second peak.

Question 12

A data science team is building a generative AI application that must answer questions based on a large collection of PDF documents stored in a Unity Catalog volume. They want to automatically extract text, chunk it, compute embeddings, and keep the Vector Search index up to date as new documents arrive. Which Databricks component should they use to orchestrate this ingestion and indexing pipeline?

Accepted Answer

Delta Live Tables with a streaming pipeline that ingests new documents, applies transformations, and triggers Vector Search index updates.. Delta Live Tables provides a declarative framework for building reliable data pipelines with streaming ingestion, data quality checks, and lineage. It can automatically detect new documents in a Unity Catalog volume, process them, and trigger Vector Search index updates. This ensures the index stays current with minimal manual intervention, making it the best choice for orchestrating the ingestion and indexing workflow.

Answer

A custom Python script running on a single-node cluster that polls the volume for new files and updates the index.

Answer

Databricks SQL with a scheduled query that reads new files and writes embeddings to a Delta table.

Answer

Databricks Jobs with a notebook that runs on a schedule to process new files and update the Vector Search index.

Question 13

An organization needs to build a RAG application on Databricks that minimizes data egress and maximizes security by keeping all data within the workspace perimeter. Which architectural pattern best satisfies this requirement?

Accepted Answer

Deploy an embedding model on Mosaic AI Model Serving and utilize Databricks Vector Search.. Utilizing Mosaic AI Model Serving with private endpoints and leveraging Vector Search indexes ensures that both the embedding model and the retrieval process occur within the Databricks control plane. By avoiding external API calls to third-party providers, the organization maintains strict governance, data residency compliance, and lower latency for inference, which is critical for enterprise-grade generative AI applications handling sensitive corporate documents.

Answer

Call external LLM APIs from a standard Python notebook without VPC constraints.

Answer

Export data to an external vector database and use a cloud-hosted LLM.

Answer

Use a public LLM endpoint with a public bucket to store the vector index.

Question 14

Which TWO factors should be prioritized when selecting an embedding model for a domain-specific RAG application on Databricks?

Accepted Answer

The semantic relevance of the model to the target domain's terminology.. Selecting the right embedding model requires balancing semantic accuracy within the specific domain and the operational cost of maintaining the model. By prioritizing domain-specific performance and resource efficiency, teams ensure that the RAG pipeline provides relevant search results without incurring excessive latency or compute costs, which are foundational for sustaining long-term generative AI production workloads.

Answer

The total number of parameters in the model regardless of the domain.

Answer

The availability of the model on the public Hugging Face repository.

Answer

The color scheme of the model's documentation page.

Question 15

Which THREE strategies improve the quality of retrieval in a Databricks Vector Search-based RAG application?

Accepted Answer

Implementing hybrid search using both semantic vectors and keyword-based filtering.. Improving retrieval quality requires a combination of high-quality data chunking, metadata filtering, and optimized search parameters. These strategies directly influence the relevance and accuracy of retrieved context, which is the foundation of a successful RAG application. Mastering these techniques allows engineers to fine-tune system behavior to meet specific user needs and reduce hallucinations during the generation phase.

Answer

Increasing the chunk size to include the entire dataset in a single vector.

Answer

Removing all stop words from the documents during the ingestion phase.

Question 16

What is the primary benefit of using Unity Catalog when designing generative AI applications in Databricks?

Accepted Answer

It provides centralized governance, lineage, and access control for data assets.. Unity Catalog provides a unified governance and security layer, ensuring that data used for training, fine-tuning, or RAG is access-controlled and lineage-tracked. This is crucial for compliance and reproducibility in AI projects. By centralizing permissions and audit logs, teams can securely manage access to sensitive data across their entire generative AI pipeline, which is a fundamental requirement for enterprise AI adoption.

Answer

It automatically generates Python code for model training.

Answer

It converts unstructured text into vectors automatically.

Answer

It eliminates the need for data preprocessing before RAG.

Question 17

When designing a production RAG application, which technique is most effective for preventing the LLM from hallucinating based on outdated information?

Accepted Answer

Enforcing a streaming data pipeline to keep the vector index updated.. Implementing a robust data pipeline that refreshes the vector index in real-time or near-real-time ensures that the retrieved context is current. When coupled with source attribution, this allows the system to verify findings against the latest data. This approach is essential for maintaining accuracy, as stale information in the vector database directly leads to hallucinations that can damage user trust in the AI application.

Answer

Increasing the temperature parameter of the model to maximum.

Answer

Restricting the LLM to a specific list of keywords for its output.

Answer

Adding a long system prompt instructing the model not to lie.

Question 18

Which Databricks feature is specifically designed to monitor model quality and drift in production?

Accepted Answer

MLflow Model Monitoring.. MLflow Model Monitoring (now part of Mosaic AI) provides the necessary tooling to track model performance metrics and identify drift over time. For generative AI, this is critical because language models can exhibit performance degradation or shifts in behavior when input distributions change. Proactive monitoring allows engineers to detect these shifts early and trigger retraining or fine-tuning, ensuring consistent performance in the production environment.

Answer

Unity Catalog audit logs.

Answer

The Databricks SQL query history.

Answer

Cluster event logs.

Question 19

When designing an application that requires fine-tuning a small model (like Llama-3-8B) on Databricks, which THREE factors must be considered to ensure a successful training job?

Accepted Answer

Ensuring the training data is sufficiently representative of the target task.. Training small models on Databricks requires careful consideration of compute, data quality, and hyperparameter tuning. By ensuring sufficient GPU memory, clean data, and properly tuned learning rates, engineers can successfully adapt the model to new domains. These factors are critical to avoid common pitfalls like catastrophic forgetting or overfitting, which can render the fine-tuned model less useful than its base version.

Answer

Disabling checkpointing to save storage costs.

Answer

Increasing the batch size to the maximum possible for all datasets.

Question 20

Which design pattern is best for protecting the LLM from prompt injection attacks when building a customer-facing chatbot on Databricks?

Accepted Answer

Using a guardrail service to validate and sanitize user inputs.. Implementing an input-filtering layer using a separate, lightweight classification model or guardrail service effectively detects and blocks malicious prompts before they reach the LLM. This proactive defense is vital for securing AI applications, as prompt injection can lead to unauthorized data access or malicious behaviors. This layering approach creates a defense-in-depth architecture that keeps the generative model safe from adversarial inputs.

Answer

Embedding all user inputs into the vector database.

Answer

Relying on the LLM’s internal safety training to reject injections.

Answer

Sending all inputs through a standard SQL query first.