Databricks-GenAI-Assoc Multiple Select — 20 Questions

Question 1

An organization is deploying a GenAI application using Databricks Model Serving. Which TWO steps are required to ensure the deployment environment handles model governance and observability effectively?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Enable Unity Catalog for the registered model and its versions.. Effective model governance and observability require integrating Unity Catalog for centralized access control and MLflow for tracking inference payloads. These components allow organizations to monitor model performance, lineage, and bias over time. Mastering these tools ensures that production deployments remain compliant, transparent, and auditable, which is essential for regulated industries using generative AI to make data-driven decisions while minimizing operational risks associated with model drift and unauthorized access.

Answer

Hardcode API credentials within the model inference script.

Answer

Disable automatic scaling to maintain consistent latency.

Answer

Manually deploy the model via the standard cluster user interface.

Question 2

A data team is developing a Mosaic AI Agent application. Which TWO of the following are mandatory for deploying this application using the Databricks Model Serving infrastructure?

Accepted Answer

A logged model signature defining input and output schemas. Deploying Mosaic AI Agents requires defining a signature for the model input/output and ensuring the agent code is packaged with its dependencies. These steps are crucial because the Serving environment needs to understand how to route requests and instantiate the environment. Properly defining the signature ensures type safety during inference, while dependency management prevents runtime errors, ensuring the agent operates as expected within the serverless environment.

Answer

A manual provisioning of a dedicated GPU worker pool

Answer

A hardcoded API key for external vector store access

Answer

A pre-configured Kubernetes manifest file

Question 3

When evaluating LLM outputs, which TWO metrics are most appropriate for measuring the 'quality' of a response in a RAG system? (Choose two)

Accepted Answer

Faithfulness. Answering quality in RAG requires looking at two distinct dimensions: the correctness of the content (faithfulness) and the relevance of the answer to the user's intent. Faithfulness ensures the model remains grounded in the provided evidence, while answer relevance captures whether the model actually addressed the specific user question. These metrics provide a balanced view, ensuring the system is both accurate and useful, which is essential for end-user satisfaction.

Answer

Inference Latency

Answer

GPU Utilization

Answer

Training Iterations

Question 4

Which TWO of the following are mandatory requirements for developing an AI application using the Databricks Mosaic AI Model Serving environment?

Accepted Answer

The model must be registered as a model version within a Unity Catalog schema.. Mosaic AI Model Serving requires specific configurations to ensure secure and performant access. Firstly, the model must be registered in Unity Catalog to maintain governance and lineage. Secondly, the serving endpoint requires defined compute resources, typically GPU-accelerated for LLMs, to handle inference requests. These requirements are essential for productionizing models, as they ensure that models are discoverable, governed, and have the necessary hardware to meet low-latency performance targets in real-time scenarios.

Answer

The model must be stored in a legacy Databricks Workspace folder.

Answer

The model must be served using a shared-access mode interactive cluster.

Answer

The application code must perform manual model sharding across nodes.

Question 5

An engineer is designing a Databricks RAG application that must support multi-turn conversations where follow-up questions refer to earlier turns. They want the retrieval step to remain accurate as the conversation progresses. Which TWO design elements should they include? (Choose two.)

Accepted Answer

Maintain a conversation history buffer and condense prior turns into a standalone query before invoking Vector Search.. Conversational RAG needs two coordinated elements: a rewritten, standalone query for retrieval, and conversation history for generation. Rewriting resolves references so Vector Search returns on-topic chunks, while including prior turns helps the model produce coherent follow-ups. Simply enlarging the retrieval window or removing filters does not address the ambiguity that multi-turn questions introduce.

Answer

Increase the number of retrieved chunks for every turn to compensate for conversational ambiguity.

Answer

Store the conversation history in the prompt and pass it unchanged to the retriever as the search query.

Answer

Disable metadata filtering during multi-turn conversations so more historical documents are eligible.

Question 6

Which TWO factors are most important when selecting a chunking strategy for text data prior to vectorization?

Accepted Answer

The maximum input token length of the target embedding model.. Selecting a chunking strategy requires balancing semantic context and technical limitations like context window size. If chunks are too small, they lack meaning; if too large, they exceed the limits of the embedding model and introduce irrelevant noise. Proper chunking is vital for maximizing the accuracy of RAG systems, as it determines the granularity of the information available for the retriever to present to the LLM.

Answer

The file format of the source documents (e.g., PDF vs. Word).

Answer

The total number of documents in the corpus.

Answer

The storage cost of the resulting vector database.

Question 7

Which TWO of the following are benefits of using Databricks Asset Bundles for deploying AI applications?

Accepted Answer

Enables automated testing through CI/CD pipeline integration. Databricks Asset Bundles standardize the deployment process, making it repeatable, version-controlled, and easier to integrate into CI/CD pipelines. This reduces the risk of configuration drift, where the production environment deviates from the development environment due to manual changes in the UI. By treating infrastructure as code, teams can maintain a clear history of changes and improve the reliability and auditability of their AI application releases.

Answer

Provides a graphical drag-and-drop interface for deployment

Answer

Automatically handles data labeling and cleaning tasks

Answer

Allows users to bypass Unity Catalog governance

Question 8

When designing a production-ready Databricks notebook for model inference, which TWO practices improve maintainability and performance?

Accepted Answer

Use Databricks Widgets for parameterizing inputs like model paths.. Modular code design allows for easier testing and unit-level debugging, which is crucial for complex inference logic. Using parameterization enables the notebook to be reused across different environments and models without changing the core code. These practices reduce technical debt and simplify the lifecycle of ML models, making the transition from development to production much smoother and ensuring consistent performance across different stages of the CI/CD pipeline.

Answer

Embed all logic including data preprocessing in a single, large notebook.

Answer

Hardcode all file paths and configuration settings for consistency.

Answer

Always run inference on the driver node to avoid network latency.

Question 9

A fintech company runs a customer-support RAG assistant on Databricks. Before promoting a new prompt template from staging to production, the ML team must demonstrate that the change does not regress answer quality. Which TWO evaluation practices should they apply in Mosaic AI Agent Evaluation to make this promotion decision defensible? (Choose two.)

Accepted Answer

Pin the evaluation dataset, judge model version, and retrieval configuration so both templates are scored under identical conditions and results are reproducible.. A defensible promotion gate requires a controlled comparison where the prompt template is the only variable, which means running both templates over the same dataset with the judge model and retrieval configuration pinned, and inspecting per-question deltas instead of relying on averages that can mask localized regressions. Together these practices attribute any score movement to the change under test and make the decision reproducible.

Answer

Deploy the candidate template to a small percentage of production traffic and rely on user thumbs feedback alone as the promotion signal.

Answer

Increase the judge model size until its scores on the candidate template exceed the production template's scores.

Answer

Evaluate only the questions where the candidate template produced different answers, skipping the rest to save judge tokens.

Question 10

Which TWO actions are necessary to ensure that a model serving endpoint in Databricks remains available and performant during peak traffic hours?

Accepted Answer

Enable auto-scaling for the serving endpoint.. Maintaining performance requires both vertical and horizontal scaling strategies. First, configuring auto-scaling on the endpoint allows the infrastructure to adjust to fluctuating demand automatically. Second, ensuring the endpoint uses appropriate instance types—such as GPU-accelerated instances for LLMs—ensures the compute capacity is sufficient for inference tasks. These two actions are foundational for achieving high availability and low latency, preventing request timeouts and ensuring a smooth experience during heavy usage periods.

Answer

Disable the serving endpoint's logging to save on compute cycles.

Answer

Configure the endpoint to run on a single, fixed-size node to minimize latency.

Answer

Manually restart the endpoint every hour to clear the cache.

Question 11

A GenAI engineer is designing a retrieval-augmented generation application whose source documents are long PDFs. Early testing shows that answers are vague because retrieved chunks contain several unrelated topics, and the language model frequently cites content that does not support its claims. The engineer wants to improve chunk quality before indexing. Which TWO changes should the engineer make to the ingestion pipeline? (Choose two.)

Accepted Answer

Split documents along semantic boundaries such as sections and headings instead of a fixed character count. Chunk quality is determined at ingestion: splitting on semantic boundaries keeps each chunk topically focused, and overlap preserves meaning across boundaries. Together they raise the precision of retrieved context so the model has grounded material to cite. Embedding dimension, temperature, and oversized single chunks do not correct incoherent chunking.

Answer

Store the full PDF text in a single chunk per document

Answer

Increase the embedding dimension of the model used for the index

Answer

Lower the temperature setting on the language model

Question 12

A GenAI engineer is deploying a RAG application that uses Databricks Vector Search and a Foundation Model API. The solution must comply with governance policies that require all data access and model invocations to be auditable and access-controlled at a fine-grained level. Which two Unity Catalog features should the engineer leverage to meet these requirements? (Choose two.)

Accepted Answer

Unity Catalog privileges on the Vector Search index and the source Delta table.. Unity Catalog privileges on the Vector Search index and source table provide fine-grained access control, ensuring only authorized users can retrieve data. Inference tables for the Foundation Model API automatically log requests and responses, including user identity, creating an audit trail. Together, these features meet the access-control and auditability requirements for the RAG application.

Answer

Unity Catalog volumes to store the model weights and configuration files.

Answer

Cluster policies to restrict the types of clusters that can access the data.

Answer

Databricks SQL dashboards to visualize access patterns and model usage.

Question 13

When designing an application that requires fine-tuning a small model (like Llama-3-8B) on Databricks, which THREE factors must be considered to ensure a successful training job?

Accepted Answer

Ensuring the training data is sufficiently representative of the target task.. Training small models on Databricks requires careful consideration of compute, data quality, and hyperparameter tuning. By ensuring sufficient GPU memory, clean data, and properly tuned learning rates, engineers can successfully adapt the model to new domains. These factors are critical to avoid common pitfalls like catastrophic forgetting or overfitting, which can render the fine-tuned model less useful than its base version.

Answer

Disabling checkpointing to save storage costs.

Answer

Increasing the batch size to the maximum possible for all datasets.

Question 14

Which TWO factors should be prioritized when selecting an embedding model for a domain-specific RAG application on Databricks?

Accepted Answer

The semantic relevance of the model to the target domain's terminology.. Selecting the right embedding model requires balancing semantic accuracy within the specific domain and the operational cost of maintaining the model. By prioritizing domain-specific performance and resource efficiency, teams ensure that the RAG pipeline provides relevant search results without incurring excessive latency or compute costs, which are foundational for sustaining long-term generative AI production workloads.

Answer

The total number of parameters in the model regardless of the domain.

Answer

The availability of the model on the public Hugging Face repository.

Answer

The color scheme of the model's documentation page.

Question 15

Which THREE strategies improve the quality of retrieval in a Databricks Vector Search-based RAG application?

Accepted Answer

Implementing hybrid search using both semantic vectors and keyword-based filtering.. Improving retrieval quality requires a combination of high-quality data chunking, metadata filtering, and optimized search parameters. These strategies directly influence the relevance and accuracy of retrieved context, which is the foundation of a successful RAG application. Mastering these techniques allows engineers to fine-tune system behavior to meet specific user needs and reduce hallucinations during the generation phase.

Answer

Increasing the chunk size to include the entire dataset in a single vector.

Answer

Removing all stop words from the documents during the ingestion phase.

Question 16

A team is evaluating their RAG application using Mosaic AI Model Evaluation. Which TWO metrics are most relevant for assessing the quality of the generated responses?

Accepted Answer

Faithfulness.. Model evaluation requires quantitative metrics to measure both the accuracy of retrieved context and the quality of the final output. Faithfulness ensures the model sticks to the provided context, while answer relevance checks if the model actually addresses the user's intent. Using these metrics allows developers to iterate on their prompt engineering and retrieval strategy, ensuring the application remains accurate and useful for end-users while minimizing the risk of hallucinations.

Answer

CPU utilization percentage.

Answer

Network latency in milliseconds.

Answer

Total storage cost of the index.

Question 17

Which THREE components are critical to include in a comprehensive evaluation strategy for a RAG-based Generative AI application?

Accepted Answer

Retrieval metrics (e.g., Hit Rate, MRR). A robust RAG evaluation requires a holistic view that covers the entire pipeline. Evaluating just the generation or just the retrieval is insufficient because the quality of the final response is dependent on both stages. By measuring retrieval performance, generation faithfulness, and end-to-end answer quality, engineers obtain a full picture of where the application is succeeding and where it needs tuning to meet user requirements.

Answer

Hardware temperature monitoring for GPUs

Answer

Network packet loss percentage

Question 18

A team is designing an LLM application that requires strict data privacy. Which TWO approaches ensure that sensitive data is not leaked during the model inference process or training?

Accepted Answer

Redact PII from prompts using local libraries before sending data to the model.. Data privacy in GenAI necessitates both input sanitization and secure environment configuration. By using PII redaction libraries before sending tokens to the model and enforcing Unity Catalog access controls on training datasets, developers create a robust defense-in-depth posture. These techniques protect against prompt injection and unauthorized data exposure, which are critical security considerations when deploying large language models within enterprise-grade environments where compliance and data sovereignty are top priorities.

Answer

Enable public access on all input data buckets for faster retrieval.

Answer

Store all API keys as plain text in notebook variables.

Answer

Use the model provider's default logging for all user queries.

Question 19

An AI engineer is designing a scalable customer support application on Databricks that integrates custom vector search indexes with a fine-tuned LLM. Which TWO architectural components are essential for enabling efficient similarity search and low-latency retrieval within the Databricks ecosystem? (Choose TWO)

Accepted Answer

Databricks Vector Search. Databricks Vector Search provides serverless vector database capabilities to store and query embeddings efficiently without managing external infrastructure. Combined with Databricks Model Serving for hosting the LLM, these native services ensure high throughput, enterprise-grade security, and low-latency inference required for modern retrieval-augmented generation applications built on the Databricks platform.

Answer

An external third-party Hadoop cluster managed via SSH tunnels

Answer

A manual cron job running on a local developer laptop

Answer

A static CSV file stored on local driver disk storage

Question 20

When deploying a model to Databricks Model Serving, which THREE of the following are best practices to ensure production reliability?

Accepted Answer

Configuring autoscaling policies based on traffic patterns.. Production reliability for model serving requires careful attention to environment configuration, scalability, and security. Best practices include using specific model versions to prevent unexpected updates, configuring autoscaling to handle variable traffic, and utilizing private connectivity to keep data transmission secure. These steps minimize downtime and ensure that the serving endpoint behaves predictably under various load conditions, which is critical for enterprise-grade generative AI applications.

Answer

Deploying the 'latest' alias for all production endpoints.

Answer

Using static credentials embedded directly in the deployment YAML.