Databricks-GenAI-Assoc Application Development Practice Test 2 — 15 Questions

Question 1

A developer is building a RAG application using Databricks Mosaic AI Model Serving. They need to log the model with MLflow and deploy it to a serving endpoint. The model must accept a JSON payload with a 'query' field and return a response from an LLM, while also retrieving relevant documents from a Vector Search index. Which approach correctly implements this?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Log a custom PyFunc model that uses the Vector Search client to retrieve documents, then calls the LLM endpoint, and deploy it with MLflow.. The correct approach is to use a custom PyFunc model because it gives full control over the RAG workflow. You can integrate the Databricks Vector Search client to fetch relevant documents and then call an LLM endpoint. MLflow logs the model with its dependencies, and Mosaic AI Model Serving deploys it as a scalable endpoint. This pattern is flexible and production-ready.

Answer

Use MLflow's built-in langchain flavor to log a chain that includes Vector Search retrieval and LLM calls, then deploy to a serving endpoint.

Answer

Create a Databricks Job that runs a Python script to retrieve documents and call the LLM, and expose it via a REST API using Flask.

Answer

Log the LLM using mlflow.transformers and deploy it directly, relying on the model to retrieve documents internally.

Question 2

An AI engineer is deploying a RAG application on Databricks Model Serving. The application uses a Vector Search index to retrieve context and a foundation model endpoint to generate answers. During load testing, they observe that end-to-end latency is high, with Vector Search queries taking up to 2 seconds. Which configuration change is most likely to reduce latency while maintaining retrieval quality?

Accepted Answer

Switch from a direct vector index to a managed vector index with optimized embedding storage and quantization.. Managed vector indexes in Databricks Vector Search are engineered for low-latency queries, with features like quantization and optimized storage. Switching from a direct index to a managed index can reduce query latency without sacrificing retrieval quality, making it the most effective change.

Answer

Reduce the embedding dimension by using a smaller model, such as a 384-dimensional model instead of a 768-dimensional one.

Answer

Enable the Vector Search index's automatic sync and increase the sync frequency to every minute.

Answer

Increase the number of nearest neighbors returned by the Vector Search query from 5 to 20.

Question 3

A data scientist is building a GenAI application that uses the OpenAI SDK to call a foundation model hosted on Databricks Mosaic AI Model Serving. They want to authenticate the SDK calls securely without hardcoding credentials. Which authentication method should they use?

Accepted Answer

OAuth token from a service principal. OAuth tokens from a service principal are the most secure and recommended way to authenticate SDK calls to Databricks Mosaic AI Model Serving. They are short-lived, can be scoped to specific resources, and are automatically rotated. This method avoids hardcoding credentials and provides better security than PATs or secrets, making it ideal for production GenAI applications.

Answer

Azure Active Directory token

Answer

Personal access token (PAT) stored in environment variables

Answer

Databricks secrets with the API token

Question 4

A team is building a generative AI application that uses Databricks Model Serving to host a fine-tuned LLM. They need to implement safeguards to prevent the model from generating harmful or biased content. Which TWO approaches should they use to mitigate this risk? (Choose two.)

Accepted Answer

Add a pre-processing step that filters user prompts for toxic language before sending them to the model.. Pre-processing prompts to filter toxic inputs and fine-tuning on safe data are both proactive measures that reduce the chance of harmful outputs. Pre-processing enforces input policies, while fine-tuning shapes the model's behavior. Together, they provide a robust defense.

Answer

Increase the temperature parameter to make the model's outputs more diverse and less likely to repeat harmful patterns.

Answer

Implement a post-processing filter that scans model outputs for toxic or biased language and blocks them.

Answer

Use a larger model with more parameters, as larger models are inherently less biased.

Question 5

A GenAI engineer has a retrieval-augmented generation chain built with LangChain and hosted on Databricks. The chain's retriever is a Databricks Vector Search index, and the LLM is a Foundation Model API endpoint. The engineer wants to use MLflow's evaluate function to score the chain on a labeled evaluation dataset and then compare runs across prompt revisions. Which approach should the engineer take to instrument the chain so MLflow can automatically capture retriever outputs and generate evaluation metrics?

Accepted Answer

Wrap the chain with mlflow.langchain.log_model and pass the model URI to mlflow.evaluate as the model argument.. Logging the LangChain chain with the LangChain flavor produces an MLflow model that mlflow.evaluate recognizes, letting the evaluator run the chain end to end and compute retrieval and generation metrics against the labeled dataset. This preserves the retriever and LLM components so prompt revisions can be compared across runs in the Experiments UI without custom metric code.

Answer

Create a Databricks Vector Search endpoint and pass its name to mlflow.evaluate as the model argument.

Answer

Write a custom Python function that calls the chain and pass it to mlflow.evaluate as a pyfunc with a signature.

Answer

Register the chain in Unity Catalog as a model version and pass the fully qualified model name to mlflow.evaluate.

Question 6

A developer is building a RAG application using Mosaic AI Model Serving. They need to ensure that the model endpoint logs inference requests and responses for audit purposes. Which configuration parameter should they enable?

Accepted Answer

Configure the 'inference_table_config' block in the endpoint request JSON.. Enabling 'inference_table_config' in the model serving endpoint configuration is the standard Databricks approach for capturing telemetry data. This feature automatically writes request and response payloads to a Delta table, enabling compliance auditing, model monitoring, and drift detection. Understanding this integration is critical for production-grade AI deployments where transparency, debugging, and regulatory logging are mandatory requirements for enterprise-scale machine learning operations.

Answer

Enable 'auto_capture_logs' in the endpoint environment variables.

Answer

Set the 'log_level' to 'DEBUG' in the Serving Endpoint settings.

Answer

Enable 'external_logging' in the Unity Catalog schema settings.

Question 7

Refer to the exhibit. A developer wants to update this serving endpoint configuration to ensure it handles high-concurrency requests with consistent latency. Which change should be applied to the configuration?

Accepted Answer

Set 'scale_to_zero_enabled' to 'false' and define 'min_provisioned_concurrency'.. To handle high-concurrency with consistent latency, the configuration must move away from 'scale_to_zero_enabled: true' and utilize 'min_provisioned_concurrency'. Scaling to zero introduces cold-start latency that disrupts consistent response times required for high-concurrency production applications. By pinning minimum resources, the model remains active and ready to handle incoming traffic immediately, ensuring that performance remains predictable even during intermittent bursts of user activity within the production environment.

Answer

Change 'workload_type' to 'GPU' and remove 'scale_to_zero_enabled'.

Answer

Increase the 'model_version' to 'latest' to trigger automatic load balancing.

Answer

Add a 'timeout' parameter to the JSON configuration block.

Question 8

When developing a feature engineering pipeline using Feature Store, which practice ensures maximum code reusability across training and inference?

Accepted Answer

Use the Feature Store client to log feature definitions and retrieve them.. Defining features as code using the Databricks Feature Store ensures that the same logic is applied during both batch training and real-time inference. By encapsulating feature calculations in Feature Tables, developers avoid the 'training-serving skew' where features are calculated differently in production. This practice is critical in GenAI development to ensure model performance consistency and reduce technical debt caused by disjointed preprocessing pipelines across different stages of the ML lifecycle.

Answer

Calculate features in the application code and pass them as raw inputs.

Answer

Hardcode feature calculation logic inside the training notebook.

Answer

Export features to a static CSV file after every training run.

Question 9

Which Databricks asset is best suited for scheduling and orchestrating a multi-step GenAI pipeline that includes data ingestion, vector index updating, and model evaluation?

Accepted Answer

Databricks Workflows. Databricks Workflows is the unified tool for orchestrating multi-step pipelines. It allows developers to define dependencies between tasks, manage retries, and monitor job execution. For GenAI, this is crucial because pipelines often involve sequential dependencies—such as ensuring vector database updates complete before a model is evaluated. Leveraging Workflows ensures reliability and observability in automated production pipelines, which are essential for maintaining the integrity of data and models in an enterprise environment.

Answer

Databricks Feature Store

Answer

Databricks SQL Warehouse

Answer

Unity Catalog

Question 10

An organization requires that all GenAI models deployed in Databricks be tracked and managed with a unified registry for compliance. Which feature should the developer use?

Accepted Answer

Unity Catalog Model Registry. Unity Catalog Model Registry serves as the centralized hub for governing, versioning, and deploying machine learning models across an organization. By using the Unity Catalog, developers ensure that every model has a documented lineage, access control, and deployment status. This is critical for regulatory compliance and enterprise security, as it prevents unvetted models from being deployed and provides a transparent audit trail of every model version currently in use.

Answer

Use Git branches to manage different model versions.

Answer

Store model artifacts as Pickle files in a shared workspace folder.

Answer

Create a custom Python class to track model metadata in a Delta table.

Question 11

When integrating an external LLM via a Databricks Model Serving endpoint, how should the API credentials be managed to ensure they are not exposed in the application code?

Accepted Answer

Use Databricks Secrets to reference credentials at runtime.. Databricks Secrets provide a secure way to reference sensitive information like API keys without hardcoding them in notebooks or source files. By using the 'dbutils.secrets.get' function, the application pulls the key at runtime from a secure vault. This is a best practice for developers to ensure security and prevent credentials from being accidentally committed to version control systems or visible to unauthorized users within the workspace.

Answer

Store credentials as environment variables in the notebook directly.

Answer

Hardcode the credentials in a hidden Python module.

Answer

Encrypt credentials and store them in a JSON file within the repo.

Question 12

A team is deploying a fine-tuned LLM using Mosaic AI Model Serving. To reduce the cost of serving the model while maintaining acceptable performance, which strategy should be prioritized?

Accepted Answer

Implement model quantization to reduce memory footprint and hardware requirements.. Using smaller, optimized model architectures or quantization techniques significantly reduces the compute and memory footprint required for serving. By shrinking the model size, you can effectively use smaller instance types for serving, leading to direct cost savings. Furthermore, optimizing the underlying serving configuration by choosing the right hardware type ensures you are not paying for expensive GPU capacity when CPU-based serving would suffice for your throughput needs.

Answer

Use the largest available instance type to ensure zero request failures.

Answer

Switch to a Multi-Model Serving endpoint regardless of throughput.

Answer

Disable all logging to save on storage costs.

Question 13

Which Databricks feature allows developers to track experiment parameters, model artifacts, and evaluation metrics in a structured way?

Accepted Answer

MLflow Tracking. MLflow Tracking is the industry-standard component integrated within Databricks for recording experiments. It allows developers to log parameters (e.g., learning rate), metrics (e.g., accuracy), and artifacts (e.g., model weights) during the development process. This structured approach is vital for reproducibility and comparing model versions, ensuring that the most effective model is identified before promotion to production, which is a key requirement for any serious ML engineering project.

Answer

Unity Catalog

Answer

Delta Lake

Answer

Databricks SQL

Question 14

A developer is building a RAG application. Which step is essential to prevent the model from hallucinating or providing outdated information?

Accepted Answer

Implement a real-time data ingestion pipeline for the vector database.. Ensuring data freshness and providing high-quality, relevant context to the LLM during the retrieval step is critical. By implementing a robust data ingestion pipeline that updates vector indices in near real-time, the application ensures that the model is always informed by the most recent documents. This practice mitigates the risk of hallucinations by grounding the model's responses in verified, up-to-date business data, which is essential for building trustworthy GenAI solutions.

Answer

Increase the temperature parameter of the model to 1.5.

Answer

Use a larger model to automatically detect and correct hallucinations.

Answer

Force the model to always answer using a specific tone of voice.

Question 15

When designing a production-ready Databricks notebook for model inference, which TWO practices improve maintainability and performance?

Accepted Answer

Use Databricks Widgets for parameterizing inputs like model paths.. Modular code design allows for easier testing and unit-level debugging, which is crucial for complex inference logic. Using parameterization enables the notebook to be reused across different environments and models without changing the core code. These practices reduce technical debt and simplify the lifecycle of ML models, making the transition from development to production much smoother and ensuring consistent performance across different stages of the CI/CD pipeline.

Answer

Embed all logic including data preprocessing in a single, large notebook.

Answer

Hardcode all file paths and configuration settings for consistency.

Answer

Always run inference on the driver node to avoid network latency.