Data Sharing and Federation practice questions

The partition filter is ignored because the column is not a clustered column.

Why it fails: Delta Sharing supports partition filters regardless of clustering. The restriction is not due to the physical layout of the table but rather the method of applying the filter. Unity Catalog requires specific SQL commands to bind the partition filter to the recipient's access path, which is missing here.

The partner organization does not support Delta Sharing partition filters.

Why it fails: Delta Sharing is an open protocol, and partition filtering is a standard feature supported by all compliant clients. The issue is not the recipient's capability but rather the lack of an applied filter in the provider's Unity Catalog share configuration, resulting in the exposure of all table data.

The table must be converted to a Liquid-clustered table first.

Why it fails: Liquid clustering is a storage optimization feature and is not required for partition filtering in Delta Sharing. The failure to filter data is purely a governance configuration issue within Unity Catalog. The Data Engineer needs to execute the correct SQL statement to ensure the filter is registered.

The shared table has been moved to a different metastore.

Why it fails: Moving a table within a metastore might break the share path, but it would typically trigger a path-not-found error rather than a credential provider authentication failure. The error message explicitly points to authentication, indicating that the token is the primary issue rather than the data location path.

The recipient is using an outdated Delta Sharing client library.

Why it fails: While using an old client is possible, it would usually result in a protocol version mismatch or an unsupported feature error. The specific error message about the credential provider failing to authenticate strongly points towards a token or permission issue rather than a client-side version compatibility problem.

The provider has revoked the recipient's access to the share.

Why it fails: If access were revoked, the error would typically be an 'Access Denied' or 'Permission Denied' message. The 'credential provider failed' message specifically identifies a failure in the authentication handshake process, which is most commonly caused by an invalid or expired token rather than a simple removal of access rights.

The recipient must be using a Databricks account.

Why it fails: Delta Sharing is designed to be agnostic of the recipient's environment. A recipient does not need to have a Databricks account. They can use any compatible client—such as Power BI, Tableau, or an open-source Delta Sharing library—to consume the data, making it a highly versatile data sharing protocol.

The recipient must have the provider's private encryption keys.

Why it fails: Delta Sharing handles security via HTTPS and token-based authentication. Recipients do not need or receive the provider's private encryption keys. The security model relies on the protocol's built-in authentication and the secure transmission of data, maintaining confidentiality without needing to share sensitive internal key management infrastructure.

GRANT READ ON CONNECTION postgres_conn TO `data_analysts`;

Why it fails: There is no 'READ' privilege for connections in Unity Catalog. The correct privilege is 'USAGE', which allows the user to access the connection object to establish connectivity. Since the 'USAGE' privilege was already granted in the exhibit, granting an incorrect 'READ' privilege would not solve the access issue.

GRANT SELECT ON SCHEMA postgres_sales TO `data_analysts`;

Why it fails: While granting access to schemas is standard practice, granting 'SELECT' on the catalog (as shown in the exhibit) covers the necessary access to schemas if the user has the required 'USAGE' privileges. The primary blocker is the missing 'USAGE' privilege on the catalog itself, not schema-level permissions.

GRANT MODIFY ON FOREIGN CATALOG postgres_sales TO `data_analysts`;

Why it fails: The 'MODIFY' privilege is for users who need to alter the metadata or properties of the catalog. It is not required for data consumers who only need to run 'SELECT' queries. Granting 'MODIFY' would violate the principle of least privilege and would not fix the underlying access traversal issue.

The share does not include the table's history or the recipient has not been granted SELECT on the table.

Why it fails: The share includes the table, and the recipient has been granted SELECT on the share. However, in Databricks-to-Databricks sharing, the recipient must also be granted SELECT on the table itself, not just the share. This is a common oversight.

The recipient's metastore is not in the same region as the share.

Why it fails: Delta Sharing does not require the recipient's metastore to be in the same region as the share. Cross-region sharing is supported, and the recipient's metastore location does not affect their ability to query shared tables. The issue is more likely related to permissions or share configuration.

The provider has not granted the recipient's Unity Catalog metastore permission to access the share.

Why it fails: In Databricks-to-Databricks sharing, the provider grants the recipient access to the share by creating a recipient object and granting SELECT on the share. There is no separate permission on the recipient's metastore that the provider must set. The recipient's metastore is managed by the recipient's account.

The recipient must be granted USE SCHEMA on the schema containing the shared table.

Why it fails: For Delta Sharing, the recipient does not need USE SCHEMA on the underlying schema in Unity Catalog. The share grants access to the table directly; the recipient's permissions are managed at the share level, not through schema-level grants. Adding USE SCHEMA would not resolve the read failure because the recipient never interacts with the source schema.

The shared table's data files are stored in a location that the recipient cannot access.

Why it fails: In Delta Sharing, the recipient never accesses the underlying cloud storage directly. The sharing server provides pre-signed URLs or proxies data access, so storage permissions are not the issue. The recipient's inability to read data is due to client-side protocol configuration, not storage access.

The partition filter was applied incorrectly, excluding all rows from the shared table.

Why it fails: If the partition filter excluded all rows, the recipient would see an empty result set, not a failure to read data. The symptom is inability to read any data, which points to an access or configuration issue rather than an empty partition. A misapplied filter would not prevent the read operation itself.

Users must have SELECT privileges on the foreign catalog and the underlying MySQL tables to query the data.

Why it fails: While users need privileges in Unity Catalog to access the foreign catalog, the underlying MySQL table privileges are managed by the external database's authentication. The connection uses a service account or credentials stored in Unity Catalog. Users do not need direct MySQL privileges; they only need Unity Catalog permissions on the foreign catalog.

The foreign catalog can be used to write data back to the MySQL database using INSERT, UPDATE, and DELETE statements.

Why it fails: Lakehouse Federation is primarily designed for read-only access. While some federated sources may support writes, MySQL federation in Unity Catalog does not support write operations. Attempting to write will result in an error. The focus is on querying external data without moving it.

Query performance is always identical to querying a native Delta table because all data is cached in Databricks.

Why it fails: Lakehouse Federation does not cache all data by default. Performance depends on the external database's speed and network latency. While some optimizations like predicate pushdown occur, it is not equivalent to querying a native Delta table. Caching may be used for specific queries, but it is not automatic for all data.

Enable audit logging on the shared table to track recipient access.

Why it fails: Audit logging is useful for monitoring, but it is not required to enable the recipient to access the data or to revoke access. The question asks for actions to ensure the recipient can access securely and that the share can be revoked. Audit logging does not directly fulfill either requirement, though it is a good practice.

Set up a private network connection between the provider and recipient.

Why it fails: Delta Sharing uses a secure token-based authentication over HTTPS. It does not require a private network connection like VNet peering or Private Link. The recipient accesses the share over the internet using the provided URL and token. While network security can be enhanced, it is not a requirement for basic Delta Sharing with an open-source client.

Configure a Unity Catalog metastore in the recipient's environment.

Why it fails: The recipient is using the open-source Delta Sharing client, not Databricks. They do not have a Unity Catalog metastore. The provider does not configure anything in the recipient's environment. The recipient simply uses the token and the share URL to access data. This option is not required and is not possible for an external recipient without Databricks.

Use Delta Sharing's built-in column-level encryption to encrypt the 'email' column.

Why it fails: Delta Sharing does not provide built-in column-level encryption. While Delta tables can be encrypted at rest using cloud storage encryption, that does not prevent recipients from reading the data once shared. There is no feature in Delta Sharing to encrypt specific columns for recipients, so this approach is not feasible and would not meet the masking requirement.

Create a view that masks the 'email' column and share the view instead of the table.

Why it fails: Delta Sharing does not support sharing views; only tables and partitions can be shared. Attempting to share a view will result in an error because the protocol is designed around Delta tables. Even if views were supported, the masking logic would need to be applied at the source, but the sharing mechanism does not allow view sharing, making this approach invalid.

Apply a column mask function to the 'email' column using Unity Catalog and then share the table.

Why it fails: Column masks in Unity Catalog are enforced for queries executed within the Databricks workspace, but they are not applied when data is shared via Delta Sharing. Recipients access the data through the Delta Sharing protocol, which bypasses Unity Catalog's fine-grained access controls. Therefore, the original email values would still be exposed to recipients, violating the requirement.

The recipient's IP address is not included in the allowed IP ranges for the share.

Why it fails: Delta Sharing does not have an IP allowlist feature at the share level. While network security can be enforced at the cloud provider level, the 403 error indicates an authorization issue, not a network block. If the IP were blocked, the error would typically be a connection timeout or a different HTTP status. The recipient can list tables, which means network connectivity is fine, so IP restrictions are not the cause.

The shared tables are partitioned, and the recipient is querying a partition that is not included in the share.

Why it fails: If a share includes only specific partitions, the recipient can still read those partitions. Querying a non-included partition would result in an empty result set or a 'partition not found' error, not a 403 Forbidden. The 403 error is related to authentication or authorization, not to partition selection. Therefore, partition filtering is not the cause of the reported error.

The recipient has not been granted the SELECT privilege on the shared tables.

Why it fails: In Delta Sharing, the recipient's access is governed by the share's contents and the recipient's token. There is no separate SELECT privilege on shared tables; the act of adding a table to a share grants read access to all recipients of that share. If the recipient can list the tables, they have been added to the share. The 403 error on data read suggests a different issue, such as an expired token or a missing credential file.

Use column-level and row-level filters directly on the shared table in the share definition.

Why it fails: Delta Sharing does not support defining column-level or row-level filters directly on a table within a share. While Unity Catalog supports column and row-level security on tables, these are not propagated through Delta Sharing. To share a subset, you must use a view.

Grant the recipient SELECT privileges only on specific columns of the table.

Why it fails: Granting SELECT privileges on specific columns is a Unity Catalog feature, but it does not apply to Delta Sharing recipients. External recipients do not have Unity Catalog accounts, so column-level grants are not possible. The share must contain a view or a table with the desired columns.

Create a separate Delta table containing only the subset of data and add it to the share.

Why it fails: Creating a separate table would duplicate data and require maintenance to keep it updated. While it would achieve the goal, it is not the recommended approach. Delta Sharing supports views, which provide a dynamic subset without data duplication. The view approach is more efficient and ensures the recipient always sees the latest data.

Share the table and instruct the recipient to filter the data themselves.

Why it fails: This approach relies on the recipient to enforce the filter, which is not secure. The recipient could ignore the instruction and access all data. The engineer needs a server-side enforcement mechanism to restrict data to the last 30 days. Trusting the recipient is not acceptable for sensitive data.

Create a view on the table that filters for the last 30 days and add the view to the share.

Why it fails: Delta Sharing does not support sharing views. Only tables and partitions can be shared. A view cannot be added to a share, so this approach would fail. The engineer must use a different method, such as sharing a filtered table or using partition filtering.

Create a separate table containing only the last 30 days of data and share that table.

Why it fails: Creating a separate table requires additional ETL to maintain and update daily. While it would work, it is not the most appropriate approach because it duplicates data and adds complexity. Partition filtering is a native Delta Sharing feature designed for this use case, making it more efficient and less error-prone.

Create a materialized view of the external MySQL tables in Unity Catalog to improve query performance.

Why it fails: Creating a materialized view would copy the data into Databricks storage, which defeats the purpose of federation. Lakehouse Federation is designed for querying data in place without duplication. Materialized views are suitable for frequently accessed data that can tolerate some staleness, but they are not part of the federation setup. The requirement is to query the external database directly, so materializing the data is not appropriate here.

Grant the USAGE privilege on the foreign catalog to all users in the workspace.

Why it fails: Granting USAGE on the foreign catalog to all users may be excessive and violates the principle of least privilege. The engineer should grant access only to specific users or groups who require it. While USAGE is necessary for querying, it should be scoped appropriately. The question focuses on optimization and security, and broad access does not contribute to either; it actually weakens security.

Configure the foreign catalog to use a dedicated SQL warehouse for all queries to ensure consistent performance.

Why it fails: While using a dedicated SQL warehouse can provide performance isolation, it is not a specific requirement for Lakehouse Federation optimization or security. Federation queries can run on any SQL warehouse or cluster. The key optimizations are predicate pushdown and secure credential management. A dedicated warehouse might be beneficial for workload isolation but is not a mandatory action for the federation setup itself.

The shared table must be in a Unity Catalog metastore that is accessible to the partner.

Why it fails: Delta Sharing allows sharing across metastores. The partner does not need direct access to the provider's metastore. The sharing mechanism handles cross-metastore access. The problem is not metastore accessibility but the missing grant of the share to the recipient.

The partner must create a foreign catalog pointing to the share.

Why it fails: For Databricks-to-Databricks sharing, the recipient accesses the share through a provider object, not a foreign catalog. Foreign catalogs are used for Lakehouse Federation, not Delta Sharing. The partner would use the provider to list shares and create a catalog from the share, but the immediate issue is likely missing grant.

The recipient must activate the share using a bearer token before accessing the data.

Why it fails: Bearer tokens are used for non-Databricks recipients. For Databricks-to-Databricks sharing, authentication is handled through the Databricks organization ID and sharing identifier, not a bearer token. Activation is not required, so this is not the cause of the partner's inability to see the table.

A shared Unity Catalog metastore link

Why it fails: Sharing a metastore link is not a feature of Delta Sharing. Unity Catalog metastores are regional and internal to a Databricks account. Access to external data is managed via shares and recipients, not by linking entire metastores across organizational boundaries, which would pose significant security and governance risks.

A personal access token with REST API permissions

Why it fails: Personal access tokens are intended for programmatic access to Databricks REST APIs by authenticated users within the workspace. They are not used for Delta Sharing recipients, as they do not provide the necessary handshake mechanism required by the Delta Sharing protocol to establish a secure, ephemeral session for data retrieval.

A cross-account IAM role in the recipient's cloud

Why it fails: Cross-account IAM roles are used for cloud-level infrastructure access, not for Delta Sharing. Delta Sharing is an open protocol that operates at the data layer, abstracting away the underlying cloud IAM complexity. The recipient does not need to configure cloud-level roles to read the shared Delta tables.

The SQL Warehouse performs the data ingestion into the Unity Catalog metastore.

Why it fails: Lakehouse Federation does not ingest data. It enables 'query-in-place' functionality, where the SQL Warehouse executes the query directly against the external database. Ingestion would require building ETL pipelines, which is a different architectural pattern compared to the real-time, direct access provided by Lakehouse Federation's query pushdown capabilities.

Unity Catalog must store the external data in a managed Delta table.

Why it fails: Unity Catalog does not store the external data itself in managed tables when using Federation. It only stores the connection metadata, such as the JDBC URL and authentication details. The actual data remains stored in the external database system, maintaining the original source of truth and avoiding data duplication.

All federated queries are executed solely within the Databricks control plane.

Why it fails: Queries are executed within the data plane on SQL Warehouses, not the control plane. The control plane manages the orchestration and metadata, but the compute intensive tasks of connecting to external sources and retrieving data occur on the assigned SQL Warehouse resources within the customer's cloud account.

Ingest all PostgreSQL data into a S3 bucket first.

Why it fails: Ingestion is not required for Lakehouse Federation. The purpose of federation is to avoid the cost and latency of data movement. By querying directly, users access the source system in real-time, which is the primary value proposition of the federation feature compared to traditional batch-based ETL ingestion.

Install a custom JDBC driver on every user's local machine.

Why it fails: JDBC drivers are handled by the Databricks SQL Warehouse or cluster runtime environments. Users do not need to install drivers on their local machines. All interaction with the external database is mediated by the Databricks compute resources, ensuring a consistent and secure environment for all authorized workspace users.

Configure a VPC peering connection to the external database.

Why it fails: While network connectivity is necessary, VPC peering is only one possible method and is not a requirement for Unity Catalog configuration. Other methods like Private Link or even public internet (with firewalls) can work. The focus of the question is on Unity Catalog configuration steps, not infrastructure networking configuration.

Delta Sharing is only for real-time streaming data.

Why it fails: Delta Sharing is not restricted to streaming data. It is primarily designed for sharing static Delta tables or snapshots of data. While it can support incremental updates, it is not a real-time message bus, and streaming capabilities are handled by other components like Delta Lake Change Data Feed.

Databricks-to-Databricks sharing requires the recipient to download a credential file.

Why it fails: Databricks-to-Databricks sharing is designed to avoid the need for credential files. It uses internal Unity Catalog metastore identities to handle authentication and authorization automatically. This simplifies the experience for users within the Databricks ecosystem, as they can access shared data as if it were local to their metastore.

Databricks-to-Databricks sharing is limited to within the same cloud region.

Why it fails: Databricks-to-Databricks sharing supports cross-region and even cross-cloud scenarios. The restriction is not geographical but rather organizational, requiring both sides to be within the Databricks platform. It is highly flexible and designed to enable cross-enterprise data collaboration without the limitations found in other cloud-specific data sharing technologies.

It eliminates the need for any network configuration.

Why it fails: Unity Catalog is a metadata and governance layer, not a networking solution. You still need to configure appropriate networking—such as VPC peering, Private Link, or firewall rules—to ensure that your Databricks compute resources can physically reach the external data source, regardless of the governance layer used.

It automatically converts all external data to Delta format.

Why it fails: Lakehouse Federation does not convert external data. The data remains in its native format in the source system. If users want to use Delta features (like Time Travel or optimized file layouts), they would need to ingest the data into Databricks, which is a separate process from federation.

It allows the SQL Warehouse to run entirely on the external database.

Why it fails: The SQL Warehouse is a Databricks compute resource that runs on your cloud infrastructure. While it pushes queries to the external database, it does not 'run on' the external database. It orchestrates the query, processes results, and performs any necessary post-processing, maintaining Databricks as the primary execution engine.

The provider must recreate the share object.

Why it fails: Recreating the share object is unnecessary. Delta Sharing is built on Delta Lake, and it tracks the current version of the table. Once a table is added to a share, any subsequent writes to that table are automatically visible to the recipient upon their next query execution.

The recipient must request a new credential file.

Why it fails: Credential files are for authentication, not data refresh. As long as the credential file has not expired, the recipient can continue to query the shared data. The updates are handled at the storage layer, meaning the recipient will see the latest data without needing to re-authenticate.

The provider must run an 'REFRESH SHARE' command.

Why it fails: There is no 'REFRESH SHARE' command in Databricks. The share metadata is maintained automatically by Unity Catalog. The data access is live, and the sharing service handles the underlying Delta table versioning, ensuring that the recipient always accesses the correct, most up-to-date version of the shared data.