Databricks-GenAI-Assoc Design Applications Practice Test 2 — 15 Questions

Question 1

What is the primary benefit of using Unity Catalog when designing generative AI applications in Databricks?

Accepted Answer

It provides centralized governance, lineage, and access control for data assets.. Unity Catalog provides a unified governance and security layer, ensuring that data used for training, fine-tuning, or RAG is access-controlled and lineage-tracked. This is crucial for compliance and reproducibility in AI projects. By centralizing permissions and audit logs, teams can securely manage access to sensitive data across their entire generative AI pipeline, which is a fundamental requirement for enterprise AI adoption.

Answer

It automatically generates Python code for model training.

Answer

It converts unstructured text into vectors automatically.

Answer

It eliminates the need for data preprocessing before RAG.

Question 2

When designing a production RAG application, which technique is most effective for preventing the LLM from hallucinating based on outdated information?

Accepted Answer

Enforcing a streaming data pipeline to keep the vector index updated.. Implementing a robust data pipeline that refreshes the vector index in real-time or near-real-time ensures that the retrieved context is current. When coupled with source attribution, this allows the system to verify findings against the latest data. This approach is essential for maintaining accuracy, as stale information in the vector database directly leads to hallucinations that can damage user trust in the AI application.

Answer

Increasing the temperature parameter of the model to maximum.

Answer

Restricting the LLM to a specific list of keywords for its output.

Answer

Adding a long system prompt instructing the model not to lie.

Question 3

Which Databricks feature is specifically designed to monitor model quality and drift in production?

Accepted Answer

MLflow Model Monitoring.. MLflow Model Monitoring (now part of Mosaic AI) provides the necessary tooling to track model performance metrics and identify drift over time. For generative AI, this is critical because language models can exhibit performance degradation or shifts in behavior when input distributions change. Proactive monitoring allows engineers to detect these shifts early and trigger retraining or fine-tuning, ensuring consistent performance in the production environment.

Answer

Unity Catalog audit logs.

Answer

The Databricks SQL query history.

Answer

Cluster event logs.

Question 4

When designing an application that requires fine-tuning a small model (like Llama-3-8B) on Databricks, which THREE factors must be considered to ensure a successful training job?

Accepted Answer

Ensuring the training data is sufficiently representative of the target task.. Training small models on Databricks requires careful consideration of compute, data quality, and hyperparameter tuning. By ensuring sufficient GPU memory, clean data, and properly tuned learning rates, engineers can successfully adapt the model to new domains. These factors are critical to avoid common pitfalls like catastrophic forgetting or overfitting, which can render the fine-tuned model less useful than its base version.

Answer

Disabling checkpointing to save storage costs.

Answer

Increasing the batch size to the maximum possible for all datasets.

Question 5

Which design pattern is best for protecting the LLM from prompt injection attacks when building a customer-facing chatbot on Databricks?

Accepted Answer

Using a guardrail service to validate and sanitize user inputs.. Implementing an input-filtering layer using a separate, lightweight classification model or guardrail service effectively detects and blocks malicious prompts before they reach the LLM. This proactive defense is vital for securing AI applications, as prompt injection can lead to unauthorized data access or malicious behaviors. This layering approach creates a defense-in-depth architecture that keeps the generative model safe from adversarial inputs.

Answer

Embedding all user inputs into the vector database.

Answer

Relying on the LLM’s internal safety training to reject injections.

Answer

Sending all inputs through a standard SQL query first.

Question 6

Refer to the exhibit. An engineer is testing a model endpoint. The outputs are too brief and often stop mid-sentence. What is the most likely cause?

Accepted Answer

The max_new_tokens limit is too low for the expected response length.. The 'max_new_tokens' parameter is set to 512, which imposes a hard limit on the length of the generated output. If the model needs more tokens to complete its thought, it will be abruptly cut off. Increasing this limit is the logical troubleshooting step when encountering truncation issues in generative model responses, ensuring the model has sufficient space to finalize its output properly.

Answer

The temperature is too low for the model to generate full sentences.

Answer

The top_p parameter is too high, leading to early termination.

Answer

The model version is incompatible with the specified parameters.

Question 7

When evaluating the performance of a RAG application, which metric is most useful for measuring the quality of the retrieved context?

Accepted Answer

Context Precision.. Context Precision measures how much of the retrieved information is actually relevant to the user query. This is a critical metric for RAG systems because high retrieval precision directly reduces the noise in the LLM input, leading to more accurate and focused answers. Measuring this allows engineers to iteratively improve the retrieval pipeline, which is essential for maximizing the utility of the application.

Answer

Total number of documents retrieved.

Answer

The total latency of the RAG pipeline.

Answer

The number of parameters in the LLM.

Question 8

Which Databricks component is recommended for orchestrating the end-to-end RAG pipeline, including data ingestion, transformation, and vector indexing?

Accepted Answer

Databricks Workflows.. Databricks Workflows allows for the scheduled and triggered execution of tasks, such as notebook runs or Delta Live Tables pipelines. This is the recommended way to orchestrate complex RAG pipelines where data must be cleaned, chunked, and indexed systematically. By using Workflows, engineers ensure that the entire pipeline is reproducible, monitorable, and resilient to failures, which is essential for maintaining a production-grade AI application.

Answer

Databricks SQL.

Answer

The Databricks File System (DBFS).

Answer

Unity Catalog.

Question 9

An engineer needs to ensure that only authorized users can access the RAG chatbot. Which Databricks security feature should be used to enforce this access control?

Accepted Answer

Unity Catalog access controls.. Unity Catalog's access control lists (ACLs) provide a robust framework for managing permissions at various levels, including catalog, schema, table, and function. By integrating identity management with these controls, engineers can define granular access to the data and the endpoints behind the chatbot. This ensures that sensitive information is only accessible by verified users, maintaining compliance and enterprise-level data security throughout the system.

Answer

Notebook tags.

Answer

Cluster environment variables.

Answer

The public internet firewall.

Question 10

An application requires low-latency retrieval of RAG metadata stored in Databricks. Which storage approach balances performance and cost while ensuring seamless integration with Unity Catalog?

Accepted Answer

Deploy a Databricks Vector Search index on a Delta table with automatic synchronization.. Vector Search indexes are designed for low-latency similarity search, which is critical for RAG applications. By leveraging Databricks Vector Search, you avoid the overhead of custom search implementations. This approach ensures that embeddings are automatically synchronized with Delta tables, providing a scalable solution that integrates directly with Unity Catalog's security and governance framework, which is essential for maintaining consistent data access policies across enterprise environments.

Answer

Store embeddings in a flat Parquet file on DBFS accessed via direct Spark reads.

Answer

Utilize a third-party managed vector database outside of the Databricks environment.

Answer

Implement a custom KNN algorithm using standard Python libraries on a single node cluster.

Question 11

Refer to the exhibit. The Databricks Vector Search index is currently in an 'ONLINE' status with incremental sync. If a developer performs a DELETE operation on the source Delta table, what happens to the vector index?

Accepted Answer

The record is automatically removed from the index during the next sync cycle.. When using incremental synchronization, the Databricks Vector Search service monitors the Delta table's transaction log for changes. A delete operation recorded in the Delta log will trigger a corresponding removal in the index. This ensures the index remains a true representation of the source data, which is critical for compliance and data quality, ensuring the RAG application does not retrieve outdated or removed sensitive information.

Answer

The record remains in the index until a full re-index is manually triggered.

Answer

The system throws an error because vector indexes do not support deletes.

Answer

The index enters a 'PENDING_DELETE' state and stops accepting queries.

Question 12

You are building an application that uses Model Serving to host a fine-tuned LLM. Which configuration is required to optimize for high-concurrency request throughput?

Accepted Answer

Enable 'Provisioned Throughput' on the serving endpoint configuration.. Enabling Provisioned Throughput for Model Serving allows for dedicated resources that can handle high-concurrency workloads efficiently. This is crucial for applications serving many users simultaneously. By optimizing resource allocation, you prevent bottlenecks during peak usage, ensuring consistent performance and minimizing request latency, which directly impacts the user experience and the scalability of the overall application architecture.

Answer

Set the serving endpoint to use CPU-only instances for all LLM inference tasks.

Answer

Disable auto-scaling and fix the number of replicas to one.

Answer

Configure the endpoint to use an external API key for every request.

Question 13

Refer to the exhibit. What is the correct way to log a custom RAG chain so it can be loaded using the provided code?

Accepted Answer

mlflow.pyfunc.log_model(artifact_path="my_model", python_model=my_chain_object). To load a custom chain using the 'models:/' URI, the chain must be logged as an MLflow model. Using 'mlflow.pyfunc.log_model' with the appropriate artifact parameters allows the entire RAG pipeline—retrieval logic, prompt handling, and LLM inference—to be captured as a single deployable unit. This ensures consistency between local development and production serving environments.

Answer

mlflow.log_artifact(local_path="chain.py")

Answer

mlflow.register_model("path/to/chain")

Answer

mlflow.sklearn.log_model(my_chain_object)

Question 14

When designing a RAG application for a regulated industry, what is the most important Unity Catalog feature for ensuring data compliance during retrieval?

Accepted Answer

Row-level security and column masking.. Unity Catalog's attribute-based access control (ABAC) or row-level security allows for granular control over which users can access specific pieces of data. In regulated industries, it is critical that the RAG model only retrieves information that the querying user is explicitly authorized to view, preventing sensitive data leakage and ensuring auditability at the catalog level.

Answer

Catalog-level tagging for organization.

Answer

Personal Access Tokens (PATs) for service principals.

Answer

Automatic data lineage tracking.

Question 15

Which approach is recommended for managing secrets, such as API keys for external LLM providers, in Databricks?

Accepted Answer

Storing keys in Databricks Secrets.. Databricks Secrets provides a secure, encrypted storage mechanism for sensitive information, preventing credentials from being hardcoded in notebooks or scripts. This is a foundational security best practice. By using scope-based secret management, you ensure that only authorized users or service principals can access the credentials, keeping your RAG application secure and compliant with enterprise security standards.

Answer

Hardcoding keys in a configuration file within a Repo.

Answer

Passing keys as plain text arguments in a job parameter.

Answer

Storing keys in a public S3 bucket with restricted access.