Databricks-GenAI-Assoc Exam Questions and Answers

Utilize Unity Catalog column-level masking functions.

Unity Catalog enables defining dynamic masks on columns using SQL functions. This centralizes security policy enforcement, applying masking rules consistently across all users and compute clusters. It is the most robust method for PII protection because it prevents unauthorized visibility without altering the underlying raw data storage files.

Implement a bronze-silver medallion pattern.

The bronze-silver pattern enables ingestion of raw, unstructured data into a staging area, followed by rigorous cleaning and validation in a downstream silver table. This approach prevents pipeline failures during ingestion and ensures that cleaning logic is decoupled from the data acquisition layer, allowing for better maintainability.

ACID compliance for reliable write operations.

ACID compliance ensures that concurrent reads and writes are managed correctly, preventing data corruption during simultaneous operations. In standard Parquet, a failing write operation could leave orphan files or partial data, whereas Delta Lake ensures an 'all-or-nothing' consistency model that is critical for production pipelines.

Time travel capability for auditing and debugging.

Time travel allows users to query older versions of a table using snapshot IDs or timestamps. This is invaluable for debugging data preparation issues, recovering from accidental overwrites, or maintaining audit trails, which is not natively possible with standard Parquet files on object storage.

Unity Catalog

Unity Catalog captures fine-grained lineage information as data moves from raw ingestion to final training sets. It offers a unified view of dependencies, which is essential for auditing the training process. This visibility is crucial for ensuring that training data meets safety and compliance standards in enterprise environments.

Perform statistical profiling on feature distributions and re-sample if necessary.

Statistical profiling reveals imbalances, allowing engineers to apply techniques like oversampling or undersampling to correct them. This quantitative approach is objective and standard for ensuring high-quality machine learning training sets. It prevents the model from favoring specific patterns simply because they were more frequent in the dataset.

To provide a unified interface for consistent feature computation during training and inference.

The primary goal of a feature store is to eliminate inconsistencies in feature logic. By centralizing this, organizations ensure that the features the model was trained on are computed in exactly the same way when deployed. This is foundational for building stable, repeatable machine learning pipelines in Databricks.

Configure the 'inference_table_config' block in the endpoint request JSON.

The 'inference_table_config' parameter is the specific configuration key required to enable automated request and response logging in Databricks Model Serving. This maps incoming requests directly to a specified Delta table, providing a structured, queryable record of all model interactions, which is essential for ongoing performance monitoring.

Set 'scale_to_zero_enabled' to 'false' and define 'min_provisioned_concurrency'.

Disabling scale-to-zero prevents the endpoint from shutting down, which eliminates cold-start latency. By defining 'min_provisioned_concurrency', the developer reserves a set number of replicas that are always running, ensuring that the system can handle concurrent requests immediately without waiting for infrastructure initialization, thus stabilizing latency under heavy load.

Use the Feature Store client to log feature definitions and retrieve them.

Utilizing the Feature Store API allows developers to define features once and publish them to a Feature Table. This centralized repository acts as a single source of truth, allowing both training pipelines and online serving endpoints to fetch consistent, pre-computed feature values using the same lookup key.

Databricks Workflows

Databricks Workflows provides a robust orchestration engine to schedule and run multi-step pipelines. It supports various task types, including notebooks, JARs, and SQL queries, and allows for complex branching and dependencies, making it the ideal tool for orchestrating the end-to-end lifecycle of a GenAI data and model pipeline.

Unity Catalog Model Registry

Unity Catalog Model Registry allows for central management, governance, and deployment of models. It enforces consistent access policies, tracks the lineage of model artifacts, and manages the lifecycle stages of models, ensuring that only approved models reach production while maintaining a clear, auditable trail for compliance requirements.

Use Databricks Secrets to reference credentials at runtime.

Databricks Secrets are designed to securely store and manage sensitive credentials. Using the secret utility API, developers can inject keys into their code at runtime without the keys ever being written to the source code or persisted in plain text, maintaining high security standards for application integrations.

Enable Unity Catalog for the registered model and its versions.

Unity Catalog acts as the central governance layer for all data and AI assets. Enabling it for registered models provides unified access control, lineage tracking, and auditability, ensuring that only authorized users can deploy or modify models, which is a foundational requirement for robust enterprise AI security and governance.

Use MLflow to enable inference logging for model endpoints.

Enabling inference logging via MLflow captures request and response data, which is vital for monitoring model performance and identifying potential hallucinations. This data helps engineers analyze drift, debug production failures, and optimize the generative AI model, providing the necessary visibility into how the model behaves when deployed in production.

MLflow Model Registry

The MLflow Model Registry provides a centralized store for managing the full lifecycle of a model. It allows teams to version models, transition them between lifecycle stages like 'Staging' and 'Production', and maintain a comprehensive history of model development, which is essential for reliable and reproducible deployments in production environments.

Implement a RAG pattern with a vector database index.

The RAG pattern retrieves relevant context from a vector database and feeds it into the model's prompt. Since the vector store can be updated independently of the model, this provides a scalable way to ensure the application always has access to the latest data without needing to redeploy models.

Use MLflow Tracing to inspect the execution pipeline.

MLflow Tracing provides detailed visibility into the duration of every step in the request flow, from input preprocessing to model inference and output post-processing. This allows engineers to pinpoint exactly where the latency is occurring, enabling them to optimize the specific components that are causing the performance delays.

Network connectivity between Model Serving and the vector store is misconfigured.

The timeout error explicitly points to a failure in establishing a connection to the vector store. This suggests that the network routing, VPC peering, or security group rules are blocking the communication path, which is a common deployment issue that must be addressed to restore RAG service functionality.

Serving endpoints provide better isolation and predictable performance.

Serving endpoints ensure that inference requests are not competing with interactive workloads or data science tasks for CPU and GPU resources. This isolation is critical for maintaining predictable, low-latency performance in production, which is a non-negotiable requirement for high-quality generative AI user experiences.

Mosaic AI Inference Tables

Inference tables automatically log inference requests and responses to a Delta table in Unity Catalog. This enables seamless integration with monitoring tools for assessing model performance, latency, and throughput. It is the standardized method for capturing production data required to perform comprehensive model evaluation and drift detection.

Context Relevance

Context relevance measures how much of the retrieved information is actually necessary to answer the user query. Low scores indicate a failure in the embedding or vector search configuration, providing actionable insights for tuning the retriever component, which is a critical part of RAG evaluation.

Enable Inference Tables on the serving endpoint.

Enabling inference tables is the mandatory first step to store production request/response logs. Without this, you lack the raw material needed for post-hoc evaluation, analysis of failure modes, or detection of data drift. It creates a persistent data asset in Unity Catalog for ongoing oversight.

Implement an automated evaluation pipeline to score inference logs.

An automated evaluation pipeline uses the captured logs to run metrics such as faithfulness or relevance. This process transforms raw logs into actionable insights about model performance. By scoring these logs, the team can identify regressions or quality degradation before they severely impact end-user experience.

To provide a reference for comparative performance analysis.

A baseline allows developers to quantitatively compare current model performance against known good results. This comparison is vital for validating that new model versions or updated RAG configurations maintain or exceed performance standards, facilitating data-driven decisions about whether to promote a new version to production.

Non-deterministic output behavior.

LLMs can produce different outputs for the same prompt due to temperature settings or inherent stochasticity. This makes simple equality checks useless for evaluation. Monitoring must account for this variance, requiring probabilistic or semantic similarity metrics rather than static, exact-match validation common in traditional classification tasks.

Difficulty in defining a single 'ground truth' for responses.

In generative tasks, there are often many valid ways to phrase a correct answer. Unlike a classification model where a label is either right or wrong, LLM outputs require qualitative judgment, which is inherently more subjective and difficult to automate without robust evaluation frameworks.

High cost of manual evaluation at scale.

Human evaluation is the gold standard but is prohibitively expensive and slow at production scale. Monitoring strategies must therefore balance automated proxy metrics (like model-based evaluation) with periodic human spot-checks to ensure the automated systems remain aligned with human-perceived quality over the long term.

Faithfulness

Faithfulness specifically assesses whether the answer is logically derived from the provided context. High faithfulness indicates the model is respecting the context; low faithfulness suggests the model is generating responses based on its own training data, which leads to hallucinations and incorrect information in RAG systems.

Dynamic data masking

Dynamic data masking is specifically designed to redact or transform sensitive column data in real-time based on the user's identity or group. By attaching a masking function to a column in Unity Catalog, data engineers can ensure that analysts see masked results without modifying the underlying data files.

The user lacks the USAGE privilege on the 'sales' schema.

Accessing any object in Unity Catalog requires the USAGE privilege on all containing objects, including the schema. Even if a user has explicit SELECT rights on the tables, the query will fail if they have not been granted USAGE on the parent schema container.

External Location

The external location object defines the specific path in cloud storage and associates it with a storage credential. It is the primary mechanism for accessing data in Unity Catalog that is stored in user-managed cloud accounts, providing a governed interface for data engineers to register data assets.

The user must have the USAGE privilege on the catalog.

Access to any object within a catalog starts with the USAGE privilege at the catalog level. Without this fundamental permission, a user cannot navigate into the catalog to access schemas or perform any operations like creating new tables, even if they have other schema-level permissions.

The user must have the USAGE privilege on the schema.

The USAGE privilege on a schema is mandatory for any interaction with objects contained within that schema. It acts as the gateway to the schema's contents, and without it, a user cannot perform DDL operations, including table creation, regardless of other privileges assigned to them.

The user must have the CREATE TABLE privilege on the schema.

Beyond simple navigation, the CREATE TABLE privilege on the schema specifically grants the user the right to define new table objects. This granular control is a core feature of Unity Catalog, allowing administrators to delegate table management responsibilities to specific users or groups securely.

Grant the user the READ FILES privilege on the external location.

Querying an external table requires the user to have the READ FILES privilege on the external location object that manages the underlying storage path. This is in addition to the standard catalog, schema, and table privileges, ensuring that storage access is explicitly governed.

The new table is a different object, so the GRANT statement must be repeated.

Unity Catalog treats the newly created table as a entirely new resource with a fresh identity. Any permissions applied to the previous version of the table do not carry over to the replacement, requiring administrators to re-issue GRANT commands for the new object.

Deploy an embedding model on Mosaic AI Model Serving and utilize Databricks Vector Search.

This approach keeps all data processing, storage, and inference within the Databricks workspace perimeter. By using internal serving for embeddings and native vector search capabilities, the architecture minimizes egress traffic and simplifies security policy enforcement, ensuring that data never leaves the protected environment during the RAG retrieval process.

The semantic relevance of the model to the target domain's terminology.

Embedding models must map domain-specific terms to accurate vector representations to ensure relevant document retrieval. A model that has not been trained or fine-tuned on the specific domain’s jargon will produce poor vector alignments, leading to inaccurate RAG responses, regardless of the model's performance on general-purpose benchmarks.

The computational resource requirements for inference latency.

Inference latency directly impacts the user experience and the scalability of the RAG application. Choosing a model that fits within the allotted compute resources ensures that the retrieval step does not become a bottleneck, allowing for high-throughput interactions while maintaining cost efficiency during periods of high user demand.

Implementing hybrid search using both semantic vectors and keyword-based filtering.

Hybrid search combines the strengths of semantic understanding with the precision of keyword matching. This ensures that specific technical terms or IDs are found accurately while still capturing the intent behind user queries, significantly improving retrieval quality for complex domain-specific datasets where semantic similarity alone might lead to irrelevant results.

Adding relevant metadata tags to documents to enable targeted filtering.

Metadata filtering allows the search process to narrow down the candidate document set before or during vector similarity matching. This reduces the noise in retrieval results, ensuring that only information relevant to a specific department, date, or document category is considered, thereby enhancing the relevance of retrieved content.

Optimizing the chunking strategy to maintain context boundaries.

Maintaining context boundaries, such as headers or paragraph breaks, ensures that retrieved chunks are self-contained and semantically coherent. Poor chunking can split a sentence or concept, leading to fragments that lack the necessary information for the LLM to provide a correct answer, directly undermining the quality of RAG.

It provides centralized governance, lineage, and access control for data assets.

Unity Catalog enables secure, governed access to all data used in the AI lifecycle. By tracking data lineage from the source to the model, it ensures transparency and compliance. This centralized approach simplifies security management and audit readiness, which are essential when handling proprietary data in generative AI applications.

Enforcing a streaming data pipeline to keep the vector index updated.

Keeping the vector index synchronized with the source data via a streaming pipeline ensures that the context retrieved during RAG is always current. This minimizes the risk of the model using outdated information, which is a major source of hallucinations in production systems that rely on rapidly changing business data.

MLflow Model Monitoring.

MLflow Model Monitoring is the dedicated Databricks component for tracking the health of deployed models. It allows engineers to monitor metrics, detect performance degradation, and identify drift in model outputs. This is essential for maintaining high-quality generative AI applications and responding effectively to changes in data or user behavior.