Databricks-GenAI-Assoc Practice Test 4 — 25 Questions

Question 1

Your organization requires an auditable record of all LLM evaluation results for compliance. Which Databricks feature provides the best centralized storage for these evaluation runs?

Accepted Answer

MLflow Experiments. MLflow Experiments act as the centralized repository for all model development and evaluation runs. By logging evaluation results to MLflow, teams create a permanent, version-controlled audit trail. This allows stakeholders to compare different iterations of models, review evaluation metrics, and verify that quality gates were met before deployment, ensuring compliance with internal AI governance policies and external regulatory frameworks for model transparency.

Answer

Databricks File System (DBFS) text logs

Answer

Unity Catalog volume for raw CSV storage

Answer

Git commit history of the inference code

Question 2

A developer needs to store prompt templates, model parameters, and evaluation results for a GenAI application so that each iteration can be compared and reproduced later. Which Databricks capability should they use?

Accepted Answer

MLflow Tracking with experiments and runs. MLflow Tracking provides experiments and runs that capture parameters, metrics, and artifacts, making it the right tool to version prompts, record model settings, and store evaluation results for comparison and reproduction. Data pipelines, volumes, and dashboards serve different purposes and do not offer run-based experiment tracking.

Answer

Delta Live Tables pipelines

Answer

Databricks SQL dashboards

Answer

Unity Catalog volumes

Question 3

A team is preparing data for a RAG system and needs to remove duplicates from a large collection of PDF text extracts. What is the most efficient way to perform de-duplication in Databricks?

Accepted Answer

Execute a distributed 'dropDuplicates' operation in Spark.. Using Spark's 'dropDuplicates()' on the text content column or calculating a hash (e.g., MD5) of the text to identify duplicates is highly scalable. This is crucial because redundant information in the vector database can cause the retriever to prioritize duplicate entries, leading to biased results and inefficiency. Removing duplicates ensures that the search index remains focused and that the retrieved context is diverse and informative.

Answer

Use a Python 'set' in a local loop on the driver node.

Answer

Manually compare every document against every other document using nested loops.

Answer

Ignore duplicates, as vector databases automatically handle them during insertion.

Question 4

You are building an application that uses Model Serving to host a fine-tuned LLM. Which configuration is required to optimize for high-concurrency request throughput?

Accepted Answer

Enable 'Provisioned Throughput' on the serving endpoint configuration.. Enabling Provisioned Throughput for Model Serving allows for dedicated resources that can handle high-concurrency workloads efficiently. This is crucial for applications serving many users simultaneously. By optimizing resource allocation, you prevent bottlenecks during peak usage, ensuring consistent performance and minimizing request latency, which directly impacts the user experience and the scalability of the overall application architecture.

Answer

Set the serving endpoint to use CPU-only instances for all LLM inference tasks.

Answer

Disable auto-scaling and fix the number of replicas to one.

Answer

Configure the endpoint to use an external API key for every request.

Question 5

Which TWO factors should be prioritized when selecting an embedding model for a domain-specific RAG application on Databricks?

Accepted Answer

The semantic relevance of the model to the target domain's terminology.. Selecting the right embedding model requires balancing semantic accuracy within the specific domain and the operational cost of maintaining the model. By prioritizing domain-specific performance and resource efficiency, teams ensure that the RAG pipeline provides relevant search results without incurring excessive latency or compute costs, which are foundational for sustaining long-term generative AI production workloads.

Answer

The total number of parameters in the model regardless of the domain.

Answer

The availability of the model on the public Hugging Face repository.

Answer

The color scheme of the model's documentation page.

Question 6

Which THREE strategies improve the quality of retrieval in a Databricks Vector Search-based RAG application?

Accepted Answer

Implementing hybrid search using both semantic vectors and keyword-based filtering.. Improving retrieval quality requires a combination of high-quality data chunking, metadata filtering, and optimized search parameters. These strategies directly influence the relevance and accuracy of retrieved context, which is the foundation of a successful RAG application. Mastering these techniques allows engineers to fine-tune system behavior to meet specific user needs and reduce hallucinations during the generation phase.

Answer

Increasing the chunk size to include the entire dataset in a single vector.

Answer

Removing all stop words from the documents during the ingestion phase.

Question 7

Which Databricks feature allows developers to track experiment parameters, model artifacts, and evaluation metrics in a structured way?

Accepted Answer

MLflow Tracking. MLflow Tracking is the industry-standard component integrated within Databricks for recording experiments. It allows developers to log parameters (e.g., learning rate), metrics (e.g., accuracy), and artifacts (e.g., model weights) during the development process. This structured approach is vital for reproducibility and comparing model versions, ensuring that the most effective model is identified before promotion to production, which is a key requirement for any serious ML engineering project.

Answer

Unity Catalog

Answer

Delta Lake

Answer

Databricks SQL

Question 8

Refer to the exhibit. A developer encounters this error when trying to call a Model Serving endpoint from a job. Which action should the developer take to resolve this authorization failure?

Accepted Answer

Grant 'Can query' permissions to the service principal in the endpoint settings.. The error indicates a lack of 'Can query' permission for the service principal or user attempting to access the endpoint. To resolve this, the developer must update the permissions for the specific endpoint within the Databricks workspace. By granting the required 'Can query' privilege to the service principal running the job, the application will regain authorization to perform inference against the model, satisfying the security policy requirements.

Answer

Increase the 'workload_size' in the model configuration.

Answer

Update the workspace token expiration policy.

Answer

Restart the serving endpoint to refresh security policies.

Question 9

Which of the following describes the purpose of using a 'Feature Store' when preparing data for Generative AI applications?

Accepted Answer

To provide a unified interface for consistent feature computation during training and inference.. The Feature Store acts as a centralized repository for standardized, reusable features. It ensures that the same logic is used for data preparation during both training and inference. This consistency is critical for preventing training-serving skew, where the model's performance in production differs from its training performance because the input data was transformed differently, ensuring the model remains accurate and reliable in real-world scenarios.

Answer

To store raw text files for long-term archival.

Answer

To serve as a high-performance database for LLM model weights.

Answer

To replace the need for data cleaning and preprocessing entirely.

Question 10

A developer is creating a custom model serving endpoint that requires an external API call for data enrichment. What is the recommended way to handle sensitive API keys within the Databricks environment?

Accepted Answer

Use the Databricks Secrets API to manage and retrieve keys.. Secrets management is the cornerstone of secure application development in Databricks. By using the Databricks Secrets API, developers can decouple sensitive credentials from source code. This practice prevents the accidental disclosure of keys in version control systems and ensures that credentials are injected into the runtime environment securely, which is mandatory for maintaining a secure and audit-compliant AI development lifecycle in enterprise cloud environments.

Answer

Store keys as environment variables in the notebook code.

Answer

Hardcode the keys directly into the model serving inference function.

Answer

Use the DBFS root directory to store key-value text files.

Question 11

When integrating an external LLM via a Databricks Model Serving endpoint, how should the API credentials be managed to ensure they are not exposed in the application code?

Accepted Answer

Use Databricks Secrets to reference credentials at runtime.. Databricks Secrets provide a secure way to reference sensitive information like API keys without hardcoding them in notebooks or source files. By using the 'dbutils.secrets.get' function, the application pulls the key at runtime from a secure vault. This is a best practice for developers to ensure security and prevent credentials from being accidentally committed to version control systems or visible to unauthorized users within the workspace.

Answer

Store credentials as environment variables in the notebook directly.

Answer

Hardcode the credentials in a hidden Python module.

Answer

Encrypt credentials and store them in a JSON file within the repo.

Question 12

A developer is building a RAG application. Which step is essential to prevent the model from hallucinating or providing outdated information?

Accepted Answer

Implement a real-time data ingestion pipeline for the vector database.. Ensuring data freshness and providing high-quality, relevant context to the LLM during the retrieval step is critical. By implementing a robust data ingestion pipeline that updates vector indices in near real-time, the application ensures that the model is always informed by the most recent documents. This practice mitigates the risk of hallucinations by grounding the model's responses in verified, up-to-date business data, which is essential for building trustworthy GenAI solutions.

Answer

Increase the temperature parameter of the model to 1.5.

Answer

Use a larger model to automatically detect and correct hallucinations.

Answer

Force the model to always answer using a specific tone of voice.

Question 13

A team is evaluating their RAG application using Mosaic AI Model Evaluation. Which TWO metrics are most relevant for assessing the quality of the generated responses?

Accepted Answer

Faithfulness.. Model evaluation requires quantitative metrics to measure both the accuracy of retrieved context and the quality of the final output. Faithfulness ensures the model sticks to the provided context, while answer relevance checks if the model actually addresses the user's intent. Using these metrics allows developers to iterate on their prompt engineering and retrieval strategy, ensuring the application remains accurate and useful for end-users while minimizing the risk of hallucinations.

Answer

CPU utilization percentage.

Answer

Network latency in milliseconds.

Answer

Total storage cost of the index.

Question 14

When developing a Generative AI application in Databricks, which tool provides a collaborative environment for engineers to write code, visualize data, and document their experiments using Markdown?

Accepted Answer

Databricks Notebooks. Databricks Notebooks provide a versatile, collaborative environment designed for data science and AI development. They allow developers to combine code, SQL queries, visualizations, and rich text documentation (Markdown) in a single document. This makes them ideal for building GenAI applications, as engineers can document their model tuning processes, test retrieval logic, and share findings with stakeholders directly within the platform, facilitating a highly efficient and iterative development workflow.

Answer

Databricks File System (DBFS)

Answer

Unity Catalog Explorer

Answer

Databricks SQL Editor

Question 15

When evaluating an LLM for deployment, what is a crucial 'non-functional' requirement that must be addressed?

Accepted Answer

The inference latency and throughput under expected load.. Non-functional requirements like latency, throughput, and cost are as important as the model's accuracy. In production, an accurate model that is too slow to provide a response is useless. Similarly, a high-performing model that is prohibitively expensive to run is not viable. Balancing these factors is essential for ensuring that the generative AI application is both technically feasible and financially sustainable at scale.

Answer

The model's ability to learn from user feedback automatically.

Answer

The number of parameters in the model architecture.

Answer

The language in which the model was initially trained.

Question 16

A data engineer needs to ensure that sensitive PII columns are masked for specific groups while remaining visible to analysts. Which Unity Catalog feature should be used to implement this requirement?

Accepted Answer

Dynamic data masking. Unity Catalog dynamic data masking allows administrators to apply functions to columns that redact or obfuscate data based on the user's role or group membership. By using SQL functions like mask_email or custom UDFs within a masking policy, the data remains consistent at the physical layer while presenting transformed values at query time. This ensures compliance with privacy regulations without creating multiple copies of datasets.

Answer

Row-level security filters

Answer

Credential passthrough

Answer

Attribute-based access control (ABAC)

Question 17

Which action must an administrator perform to allow a user to use Databricks SQL to query a table that is stored in an external storage location?

Accepted Answer

Grant the user the READ FILES privilege on the external location.. To query an external table, the user needs access to the storage location, the catalog, the schema, and the table. The administrator must grant the USAGE privilege on the catalog and schema, the SELECT privilege on the table, and the READ FILES privilege on the external location. This multi-layered approach ensures that data access is both logically and physically controlled.

Answer

Grant the user the OWNER role for the storage credential.

Answer

Grant the user the ALL PRIVILEGES privilege on the metastore.

Answer

Add the user to the workspace-level admin group.

Question 18

Which Unity Catalog feature is best suited for sharing data assets with users outside of your Databricks account?

Accepted Answer

Delta Sharing. Delta Sharing is an open-protocol standard that allows organizations to share data with users regardless of whether they have a Databricks account. It enables secure, read-only data sharing across different platforms and cloud environments. This is essential for organizations that need to collaborate with partners, vendors, or external clients while maintaining centralized control and auditability within Unity Catalog.

Answer

Unity Catalog cross-account roles

Answer

External locations

Answer

Workspace federation

Question 19

Which THREE components are critical to include in a comprehensive evaluation strategy for a RAG-based Generative AI application?

Accepted Answer

Retrieval metrics (e.g., Hit Rate, MRR). A robust RAG evaluation requires a holistic view that covers the entire pipeline. Evaluating just the generation or just the retrieval is insufficient because the quality of the final response is dependent on both stages. By measuring retrieval performance, generation faithfulness, and end-to-end answer quality, engineers obtain a full picture of where the application is succeeding and where it needs tuning to meet user requirements.

Answer

Hardware temperature monitoring for GPUs

Answer

Network packet loss percentage

Question 20

A data engineer is preparing a dataset for fine-tuning a chat model. The dataset contains conversations with alternating user and assistant messages. They need to format each conversation into a single string with special tokens indicating roles. Which approach is most appropriate in Databricks?

Accepted Answer

Use the concat_ws function to join messages with a delimiter, and prefix each message with a role token.. The most appropriate approach is to use concat_ws to join messages after prefixing each with a role token. This creates a single formatted string per conversation efficiently and deterministically, which is ideal for fine-tuning chat models.

Answer

Use the format_string function to insert role tokens into a template string for each message.

Answer

Use the explode function to flatten the messages and then collect_list to reassemble them with role tokens.

Answer

Use the ai_query function to call an LLM that formats the conversation into a single string.