Databricks-GenAI-Assoc Practice Test 9 — 10 Questions

Question 1

Which THREE strategies improve the quality of retrieval in a Databricks Vector Search-based RAG application?

Accepted Answer

Implementing hybrid search using both semantic vectors and keyword-based filtering.. Improving retrieval quality requires a combination of high-quality data chunking, metadata filtering, and optimized search parameters. These strategies directly influence the relevance and accuracy of retrieved context, which is the foundation of a successful RAG application. Mastering these techniques allows engineers to fine-tune system behavior to meet specific user needs and reduce hallucinations during the generation phase.

Answer

Increasing the chunk size to include the entire dataset in a single vector.

Answer

Removing all stop words from the documents during the ingestion phase.

Question 2

Which Databricks feature allows developers to track experiment parameters, model artifacts, and evaluation metrics in a structured way?

Accepted Answer

MLflow Tracking. MLflow Tracking is the industry-standard component integrated within Databricks for recording experiments. It allows developers to log parameters (e.g., learning rate), metrics (e.g., accuracy), and artifacts (e.g., model weights) during the development process. This structured approach is vital for reproducibility and comparing model versions, ensuring that the most effective model is identified before promotion to production, which is a key requirement for any serious ML engineering project.

Answer

Unity Catalog

Answer

Delta Lake

Answer

Databricks SQL

Question 3

Refer to the exhibit. A developer encounters this error when trying to call a Model Serving endpoint from a job. Which action should the developer take to resolve this authorization failure?

Accepted Answer

Grant 'Can query' permissions to the service principal in the endpoint settings.. The error indicates a lack of 'Can query' permission for the service principal or user attempting to access the endpoint. To resolve this, the developer must update the permissions for the specific endpoint within the Databricks workspace. By granting the required 'Can query' privilege to the service principal running the job, the application will regain authorization to perform inference against the model, satisfying the security policy requirements.

Answer

Increase the 'workload_size' in the model configuration.

Answer

Update the workspace token expiration policy.

Answer

Restart the serving endpoint to refresh security policies.

Question 4

Which of the following describes the purpose of using a 'Feature Store' when preparing data for Generative AI applications?

Accepted Answer

To provide a unified interface for consistent feature computation during training and inference.. The Feature Store acts as a centralized repository for standardized, reusable features. It ensures that the same logic is used for data preparation during both training and inference. This consistency is critical for preventing training-serving skew, where the model's performance in production differs from its training performance because the input data was transformed differently, ensuring the model remains accurate and reliable in real-world scenarios.

Answer

To store raw text files for long-term archival.

Answer

To serve as a high-performance database for LLM model weights.

Answer

To replace the need for data cleaning and preprocessing entirely.

Question 5

A developer is creating a custom model serving endpoint that requires an external API call for data enrichment. What is the recommended way to handle sensitive API keys within the Databricks environment?

Accepted Answer

Use the Databricks Secrets API to manage and retrieve keys.. Secrets management is the cornerstone of secure application development in Databricks. By using the Databricks Secrets API, developers can decouple sensitive credentials from source code. This practice prevents the accidental disclosure of keys in version control systems and ensures that credentials are injected into the runtime environment securely, which is mandatory for maintaining a secure and audit-compliant AI development lifecycle in enterprise cloud environments.

Answer

Store keys as environment variables in the notebook code.

Answer

Hardcode the keys directly into the model serving inference function.

Answer

Use the DBFS root directory to store key-value text files.

Question 6

When integrating an external LLM via a Databricks Model Serving endpoint, how should the API credentials be managed to ensure they are not exposed in the application code?

Accepted Answer

Use Databricks Secrets to reference credentials at runtime.. Databricks Secrets provide a secure way to reference sensitive information like API keys without hardcoding them in notebooks or source files. By using the 'dbutils.secrets.get' function, the application pulls the key at runtime from a secure vault. This is a best practice for developers to ensure security and prevent credentials from being accidentally committed to version control systems or visible to unauthorized users within the workspace.

Answer

Store credentials as environment variables in the notebook directly.

Answer

Hardcode the credentials in a hidden Python module.

Answer

Encrypt credentials and store them in a JSON file within the repo.

Question 7

A developer is building a RAG application. Which step is essential to prevent the model from hallucinating or providing outdated information?

Accepted Answer

Implement a real-time data ingestion pipeline for the vector database.. Ensuring data freshness and providing high-quality, relevant context to the LLM during the retrieval step is critical. By implementing a robust data ingestion pipeline that updates vector indices in near real-time, the application ensures that the model is always informed by the most recent documents. This practice mitigates the risk of hallucinations by grounding the model's responses in verified, up-to-date business data, which is essential for building trustworthy GenAI solutions.

Answer

Increase the temperature parameter of the model to 1.5.

Answer

Use a larger model to automatically detect and correct hallucinations.

Answer

Force the model to always answer using a specific tone of voice.

Question 8

A team is evaluating their RAG application using Mosaic AI Model Evaluation. Which TWO metrics are most relevant for assessing the quality of the generated responses?

Accepted Answer

Faithfulness.. Model evaluation requires quantitative metrics to measure both the accuracy of retrieved context and the quality of the final output. Faithfulness ensures the model sticks to the provided context, while answer relevance checks if the model actually addresses the user's intent. Using these metrics allows developers to iterate on their prompt engineering and retrieval strategy, ensuring the application remains accurate and useful for end-users while minimizing the risk of hallucinations.

Answer

CPU utilization percentage.

Answer

Network latency in milliseconds.

Answer

Total storage cost of the index.

Question 9

When developing a Generative AI application in Databricks, which tool provides a collaborative environment for engineers to write code, visualize data, and document their experiments using Markdown?

Accepted Answer

Databricks Notebooks. Databricks Notebooks provide a versatile, collaborative environment designed for data science and AI development. They allow developers to combine code, SQL queries, visualizations, and rich text documentation (Markdown) in a single document. This makes them ideal for building GenAI applications, as engineers can document their model tuning processes, test retrieval logic, and share findings with stakeholders directly within the platform, facilitating a highly efficient and iterative development workflow.

Answer

Databricks File System (DBFS)

Answer

Unity Catalog Explorer

Answer

Databricks SQL Editor

Question 10

When evaluating an LLM for deployment, what is a crucial 'non-functional' requirement that must be addressed?

Accepted Answer

The inference latency and throughput under expected load.. Non-functional requirements like latency, throughput, and cost are as important as the model's accuracy. In production, an accurate model that is too slow to provide a response is useless. Similarly, a high-performing model that is prohibitively expensive to run is not viable. Balancing these factors is essential for ensuring that the generative AI application is both technically feasible and financially sustainable at scale.

Answer

The model's ability to learn from user feedback automatically.

Answer

The number of parameters in the model architecture.

Answer

The language in which the model was initially trained.