Governance practice questions

It can be managed independently for each workspace without account-level synchronization.

Why it fails: Unity Catalog requires a centralized metastore per region that is linked to a Databricks account. It is not designed to be managed independently per workspace, as this would prevent cross-workspace governance, sharing, and centralized auditing, which are core value propositions of the Unity Catalog architecture.

It enforces access control policies exclusively at the file-system level.

Why it fails: Unity Catalog enforces access control primarily at the logical object level (catalogs, schemas, tables, and views), not just the file-system level. By abstracting permissions away from the underlying cloud storage, it simplifies administration and ensures that security policies remain consistent regardless of the underlying storage implementation.

It forces data storage into the root of the metastore's default storage bucket.

Why it fails: While a metastore has a root storage location, Unity Catalog allows for external locations and storage credentials. Data can be stored in various locations across cloud providers, and the metastore simply registers and manages the metadata, providing flexibility in storage architecture without forcing all data into one location.

Workspace-level network policies

Why it fails: Network policies manage traffic flow and connectivity, but they do not enforce data residency requirements for storage. While they are useful for security, they cannot prevent a user from creating a table in a storage path located in a different cloud region if access is allowed.

Unity Catalog metastore region pinning

Why it fails: There is no 'region pinning' feature for the metastore itself in Unity Catalog. The metastore must be created in a specific region, but this does not automatically govern the storage locations of tables or restrict them to that same region unless external locations are configured accordingly.

Credential passthrough

Why it fails: Credential passthrough is a legacy security feature that relies on passing user credentials directly to storage. It does not provide any governance capabilities for enforcing data residency or limiting the cloud regions where data is stored by the users.

Grant MODIFY on the Delta table that the index is built from to the 'genai_team' group, and rely on the index inheriting those permissions.

Why it fails: Permissions on the source Delta table do not automatically propagate to the Vector Search index. The index is a separate securable object in Unity Catalog, and access must be granted explicitly on the index itself. Granting MODIFY on the table may allow data changes but does not control query access to the index.

Set the index's owner to the 'genai_team' group and revoke all privileges from the metastore admin.

Why it fails: Changing ownership and revoking privileges from the metastore admin is not a supported or advisable access control pattern. The metastore admin typically retains ultimate control. Ownership alone does not grant query access to other group members, and revoking admin privileges can cause operational issues without achieving the desired isolation.

Attach a cluster-scoped library to the workspace and grant CAN ATTACH TO permissions on that cluster to the 'genai_team' group.

Why it fails: Cluster permissions control who can attach to a cluster, not who can query a Vector Search index. Even if the group can attach to a cluster, they still need explicit SELECT on the index. This approach does not restrict access to the index for other workspace users, so it fails to meet the requirement.

Create a custom logging handler in the model's Python code that writes to a Unity Catalog table with manually encrypted columns.

Why it fails: While a custom logging handler is possible, it requires modifying the model code and managing encryption manually, which is error-prone and not a native Databricks feature. Inference tables provide automatic, server-side logging without code changes. The manual approach also may not capture all metadata like user identity and model version consistently.

Use MLflow Model Registry to log model signatures and rely on its built-in audit logging for inference requests.

Why it fails: MLflow Model Registry tracks model versions and metadata but does not log individual inference requests or responses. Its audit logs are for registry operations, not for serving traffic. It cannot provide per-request user identity or model version at inference time, nor does it offer column-level encryption for request text.

Configure a Databricks SQL alert on the serving endpoint's metrics table and export the results to an encrypted Delta table.

Why it fails: Databricks SQL alerts monitor metrics and can trigger notifications, but they do not capture per-request payloads like request text or user identity. Exporting metrics does not provide the detailed inference logs required for auditing. This approach lacks the granularity and the ability to apply column-level encryption to the request content.

Enable inference tables on the serving endpoint used by the application.

Why it fails: Inference tables capture request and response payloads for monitoring and debugging model serving endpoints, but they are designed for payload logging, not for capturing the identity of the user making a call for audit. They would not satisfy the requirement to log user identity for every embedding call without additional custom code, and they are not the mechanism that provides identity-based audit logging.

Configure system tables for audit logging and query them to see model invocation events.

Why it fails: System tables do contain audit logs, but enabling them does not automatically record user identity for Foundation Model API calls made through the SDK in a way that ties to the application user without additional configuration. The engineer would still need to ensure that the calls are attributed to the correct user, which requires a different mechanism. System tables alone do not fulfill the requirement without code changes.

Add a custom header to each API request containing the user's email address.

Why it fails: Adding a custom header would require modifying the application code to extract and include the user identity, which violates the requirement of no code changes. Moreover, custom headers are not automatically logged by the platform's audit system, so the identity would not be captured for audit unless additional logging is configured. This approach is manual and error-prone, and does not leverage built-in governance features.

Column masking policies applied to the embedding column

Why it fails: Column masking policies in Unity Catalog replace column values with masked data for unauthorized users, but they are not applied during Vector Search operations. Masking the embedding column would corrupt the vector representation and break similarity search. It also does not provide row-level isolation, so all users would see the same masked or unmasked vectors.

Delta Sharing with a recipient defined per end user

Why it fails: Delta Sharing is designed for sharing data across organizations or Databricks workspaces, not for enforcing per-user row access within a single Vector Search index. Creating a recipient per user would require duplicating shares and does not filter results at query time. It also cannot be attached to a Vector Search index endpoint.

Row-level filtering using user-defined functions on the underlying Delta table

Why it fails: Row-level filtering with UDFs applies to SQL queries against the Delta table, not to Vector Search similarity queries. Vector Search does not evaluate UDF-based row filters at query time, so confidential embeddings would still be returned to unauthorized users. This approach also adds latency and does not integrate with the Vector Search access model.

Create a view that excludes sensitive columns and grant access to the view only.

Why it fails: A view that excludes sensitive columns would still expose the remaining data to the Assistant. Moreover, the Assistant could infer information from the included columns. This approach also requires changing the data scientist's workflow to use the view, which may not be desirable. It does not specifically prevent the Assistant from seeing data values in the original table.

Apply a column mask to all sensitive columns in the table.

Why it fails: Column masks dynamically redact data for users, but they do not prevent Databricks Assistant from seeing the data. The Assistant operates with the user's permissions and may still access underlying values unless the user lacks SELECT. Column masks are for user-facing queries, not for controlling Assistant's access to metadata.

Revoke SELECT permission on the table from the data scientist.

Why it fails: Revoking SELECT would prevent the data scientist from querying the table at all, which is not the goal. The requirement is to allow querying while preventing the Assistant from seeing data values. The data scientist needs SELECT to run queries, so this approach would block legitimate work rather than selectively protect data from the Assistant.

Apply row-level security policies to the source Delta table and rebuild the Vector Search index.

Why it fails: Row-level security on the source table affects direct queries to the table, but Vector Search indexes are built by reading the source data and creating embeddings. The index itself does not enforce row-level security at query time, so rebuilding the index would not dynamically filter results based on the querying user's permissions. The index would still contain all documents, and the chatbot could retrieve unauthorized content unless additional filtering is applied at query time.

Enable attribute-based access control on the Vector Search index itself.

Why it fails: Attribute-based access control (ABAC) is not a native feature of Vector Search indexes in Unity Catalog. While Unity Catalog supports fine-grained access control on tables and other objects, Vector Search indexes do not have their own ABAC policies for row-level filtering. The engineer must instead rely on filters applied at query time using Unity Catalog functions or similar mechanisms to enforce row-level security on retrieved documents.

Create a separate Vector Search index for each user group and route queries accordingly.

Why it fails: Creating separate indexes per group would be operationally complex and would not scale well as groups change. It also duplicates data and embeddings, increasing cost and maintenance. More importantly, it does not automatically enforce row-level security based on Unity Catalog policies; it requires manual routing logic and does not handle dynamic changes in group membership or fine-grained access rules. This approach is not aligned with Unity Catalog's centralized governance model.

A storage credential scoped to the index's cloud storage location

Why it fails: Storage credentials authorize Databricks to access a cloud storage path when creating external locations and external tables. They control whether the platform can read bytes from storage, not which rows or columns a particular user can see in query results. Applying a credential would not exclude internal-only transcript rows from retrieved chunks.

A Unity Catalog volume mounted at the index storage path

Why it fails: Volumes govern access to non-tabular files stored under a managed or external location. A Vector Search index is a Databricks-managed serving artifact, and its internal storage is not something a user reads through a volume path. Wrapping the index in a volume does not filter rows or mask columns at query time, so internal escalation notes would still be retrievable.

A Delta table constraint that enforces NOT NULL on the transcript column

Why it fails: Delta constraints such as NOT NULL or CHECK validate data on write and reject violating rows. They say nothing about which rows a reader is allowed to see, and they apply uniformly to every principal. A NOT NULL constraint on the transcript column would not hide internal escalation notes from the RAG endpoint while exposing them to data scientists.

Configure the AI Gateway to enforce rate limits and log usage metrics.

Why it fails: The AI Gateway provides rate limiting, usage tracking, and guardrails, but it does not by itself log full prompt and completion payloads to a Unity Catalog table. While it can complement auditing, the requirement for content-level logging is met by inference tables, not the gateway's metrics.

Create a personal access token for each user and share it only with the group.

Why it fails: Sharing personal access tokens is insecure and does not provide group-based access control. Tokens are tied to individual identities and cannot be scoped to a group in Unity Catalog. This approach also makes auditing difficult and violates the principle of least privilege, failing the access restriction requirement.

Use Unity Catalog audit logs to capture inference requests and responses.

Why it fails: Unity Catalog audit logs capture access events such as table reads and permission changes, but they do not log the content of prompts and completions sent to a model serving endpoint. Inference payload logging requires a separate mechanism, such as inference tables or the AI Gateway, not Unity Catalog audit logs.

Enable inference tables on the serving endpoint to log all requests and responses.

Why it fails: Inference tables log payloads for monitoring, but they do not govern data or track user permissions. They are useful for debugging and quality monitoring, but they do not satisfy the requirement that data is governed by Unity Catalog or that permissions are tracked. This action is about observability, not governance.

Configure the serving endpoint to use Unity Catalog for authentication and authorization.

Why it fails: While serving endpoints can be integrated with Unity Catalog for authentication, simply configuring the endpoint does not by itself ensure that data sent to the model is governed by Unity Catalog. The data governance requirement is about the data used in the RAG pipeline, not just the endpoint authentication. This action alone does not provide tracking of user permissions on the model or the data.

Use a personal access token (PAT) for each user to authenticate to the serving endpoint.

Why it fails: Using individual PATs can help attribute requests to users, but it does not govern the data or model through Unity Catalog. PATs are authentication tokens, and while they can be used for auditing, they do not enforce Unity Catalog permissions on the model or data. This approach also introduces management overhead and does not centralize governance.

Enable inference tables on the fine-tuning cluster to log all training data

Why it fails: Inference tables are for logging requests and responses of model serving endpoints, not for fine-tuning clusters. Enabling them on a fine-tuning cluster is not supported. Even if it were, logging training data would create a new copy of PII, potentially violating the policy against storing PII in plaintext.

Use Databricks secrets to encrypt the model weights at rest

Why it fails: Databricks secrets are for storing credentials, not for encrypting model weights. Model weights are stored in Unity Catalog and are encrypted at rest by the cloud provider's storage encryption. Using secrets for this purpose is not possible and does not address the PII in training data or model access control.

Store the training data in a Unity Catalog volume with public read access

Why it fails: Storing training data in a volume with public read access would expose the data to all users, violating the requirement to protect PII. Volumes do not automatically mask data; they are simply storage containers. This action would increase the risk of PII exposure and is contrary to governance best practices.

Create the Vector Search index in a Unity Catalog-enabled workspace and specify the index's owner as a group with access to the source table.

Why it fails: While creating the index in a Unity Catalog-enabled workspace is necessary, setting the index owner to a group does not enforce per-user permissions. Unity Catalog permissions are enforced based on the querying user's identity, not the owner's. The owner's permissions determine who can manage the index, but query access is controlled by grants on the index itself. This action alone does not ensure that only authorized users can query the index.

Grant USE SCHEMA and SELECT privileges on the index to the appropriate users or groups.

Why it fails: Granting USE SCHEMA and SELECT on the index is part of managing access to the index, but it is not sufficient to enforce Unity Catalog permissions on the underlying data. The index's permissions control who can query the index, but Databricks also checks the user's permissions on the source table. Without SELECT on the source table, the query will fail. Therefore, this action alone does not satisfy the requirement.

Configure the Vector Search endpoint to use a service principal with access to the source table.

Why it fails: Configuring the endpoint with a service principal is not how Unity Catalog permissions are enforced for Vector Search. The endpoint is a compute resource; permissions are evaluated based on the identity of the user or service principal that queries the index, not the endpoint's identity. Using a service principal for the endpoint might be relevant for other purposes, but it does not ensure that only authorized users can query the index.

Hardcode the key in the model's environment variables inside the MLflow model signature

Why it fails: The MLflow model signature records input and output schema, not credentials, and environment variables set at model build time travel with the artifact. Anyone who can download the model can extract the key. This violates the requirement that the secret never appear in code or artifacts and provides no Unity Catalog management surface.

Store the key in a Databricks secret scope and reference it with dbutils.secrets.get in the notebook, then pass the literal value to the serving endpoint

Why it fails: Databricks secret scopes do hide the value from notebook output, but a secret scope is a workspace-level construct, not a Unity Catalog object, so it does not meet the requirement that the key be manageable from Unity Catalog. Passing the literal value into the serving endpoint also exposes it in the request payload and logs.

Write the key into a Delta table in Unity Catalog and have the agent SELECT it at runtime

Why it fails: Storing a secret as table data exposes it to every principal with SELECT on that table and leaves it visible in query history and table history. It also turns a credential into data that can be copied or exported. Although the table lives in Unity Catalog, this approach fails the core security requirement that the key never be readable in plaintext.

Create a Unity Catalog function that wraps the endpoint, grant EXECUTE on the function to `users`, and have the function run with a definer's rights security context.

Why it fails: A definer's-rights function executes with the owner's privileges, so every caller effectively reads the source table as the owner. That defeats per-user row filtering and makes audit records show the function owner rather than the actual end user, failing the attribution requirement even though the function itself is properly governed in Unity Catalog.

Grant SELECT on the source Delta table to the `users` group and leave the serving endpoint unrestricted so any workspace user can invoke it.

Why it fails: Leaving the endpoint unrestricted removes the access control boundary entirely; any principal in the workspace, including ones with no SELECT on the source table, could invoke it. Direct SELECT on the table also does not by itself make the serving endpoint honor per-user identity, so the attribution and parity goals are not met by this combination.

Grant EXECUTE on the serving endpoint to the `users` group and attach a personal access token of a service principal to the notebook that calls it.

Why it fails: A shared service-principal token collapses all callers into one identity, so retrieval and completion logs can no longer be attributed to the individual end user. It also breaks the parity requirement because the endpoint would read the source table with the service principal's privileges rather than each user's, potentially exposing rows a given analyst is not entitled to see.

Row-level security filters

Why it fails: Row-level security restricts access to specific records within a table based on user attributes. It does not modify column-level content or apply masking functions to sensitive data fields. While it controls visibility of rows, it does not address the need to mask specific columns while showing others.

Credential passthrough

Why it fails: Credential passthrough is an older authentication mechanism that allows users to authenticate to Azure Data Lake Storage directly using their identity. It is not designed for data masking or granular column-level control within a Unity Catalog environment and is generally discouraged in favor of Unity Catalog access control.

Attribute-based access control (ABAC)

Why it fails: ABAC uses tags and metadata attributes to determine access levels to objects. While useful for broader policy management, it does not provide the specific masking functionality required to transform column values (like redacting emails) during query execution. Masking requires a specific SQL function-based policy applied to the table.

The user lacks the SELECT privilege on the schema object.

Why it fails: The SELECT privilege is not applicable to schema objects in Unity Catalog; permissions for schemas are governed by the USAGE privilege. The failure occurs because the user lacks USAGE on the schema, not because they are missing SELECT permissions on the schema container itself.

The user requires the MODIFY privilege to perform joins.

Why it fails: A join operation is a read-only activity that only requires the SELECT privilege on the underlying tables. The MODIFY privilege is required for DML operations like INSERT, UPDATE, or DELETE, and it is not a requirement for performing complex read operations or joining datasets.

The user needs to be an owner of the tables to perform joins.

Why it fails: Table ownership is not required for users to query tables. As long as the user has been granted the necessary SELECT and USAGE privileges via the Databricks SQL GRANT command, they can access and join tables regardless of who originally created or currently owns the table objects.

Storage Credential

Why it fails: A storage credential is the long-term secret (like an IAM role ARN or Service Principal) used by Databricks to access cloud storage. While it is associated with an external location, it is not the object used to map the specific storage path to the Unity Catalog hierarchy.

Managed Table

Why it fails: Managed tables are created within the metastore's root storage location. While Unity Catalog manages the lifecycle and lifecycle of the underlying files, the object itself is the table, not the mechanism used to link disparate external storage paths to the governance model.

Unity Catalog Volume

Why it fails: Volumes are used to manage non-tabular data (files) in cloud storage. While they do map storage paths, they are intended for unstructured or semi-structured data files rather than acting as the primary registration point for tabular data governance and cataloging within the metastore's hierarchy.

The user must have the MODIFY privilege on the catalog.

Why it fails: The MODIFY privilege is not required at the catalog level to create a table within a schema. Privileges are checked hierarchically; as long as the user has the specific CREATE TABLE privilege on the schema, they do not need broader modification rights on the parent catalog.

The user must have the OWNER role for the entire metastore.

Why it fails: Owning the entire metastore is not required for creating tables. Unity Catalog is designed for distributed management where specific users are granted privileges on schemas. Requiring metastore ownership would violate the principle of least privilege and prevent effective delegation of data management duties.

Grant the user the OWNER role for the storage credential.

Why it fails: The owner role for a storage credential provides administrative rights to the credential object itself, not access to the data. It is a high-level privilege that should not be assigned to general users and does not grant the necessary read permissions for querying table data.

Grant the user the ALL PRIVILEGES privilege on the metastore.

Why it fails: Granting ALL PRIVILEGES on the metastore is excessive and violates the principle of least privilege. Users only require specific permissions on the relevant objects—catalog, schema, table, and external location—to perform their jobs, not administrative control over the entire metastore.

Add the user to the workspace-level admin group.

Why it fails: Workspace-level admin rights grant administrative control over the workspace configuration but do not automatically grant permissions on Unity Catalog objects. Access to data assets in Unity Catalog is managed through specific GRANT statements, regardless of the user's role within the local Databricks workspace.