Databricks-Spark-Assoc Developing DataFrame/DataSet API Applications Practice Question
A data engineer has two DataFrames: `orders` with columns `order_id` and `customer_id`, and `customers` with columns `customer_id` and `region`. They must produce a result containing only orders whose `customer_id` exists in `customers`, keeping every matching order row exactly once with no customer columns added. Which operation should be used?
⚠ Common exam trap
The trap here is defaulting to an inner join for existence filtering, ignoring that it brings along right-side columns and can duplicate rows when the right side has repeated keys.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
orders.join(customers, 'customer_id', 'left_semi')
A left semi join keeps only left-side rows that have a matching key on the right, returns just the left columns, and never duplicates rows regardless of duplicate keys on the right. An inner join would add the region column and could duplicate rows, a left anti join inverts the logic, and a union mixes incompatible data. The semi join matches the requirement exactly.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
orders.join(customers, 'customer_id', 'left_anti')
Why it's wrong here
A left anti join returns orders that have no match in `customers`, which is the exact opposite of what is needed. The engineer wants only orders whose customer exists, so this would return the complement of the desired set. It directly contradicts the requirement and would silently produce the wrong population.
- ✓
orders.join(customers, 'customer_id', 'left_semi')
Why this is correct
A left semi join returns rows from `orders` that have a match in `customers`, includes only the left DataFrame's columns, and does not duplicate rows even when the right side has multiple matches. This precisely satisfies the requirement of filtering orders by existence without adding customer columns or multiplying rows.
- ✗
orders.join(customers, 'customer_id', 'inner')
Why it's wrong here
An inner join returns only matching orders, but it also adds the `region` column from `customers`, which the requirement explicitly forbids. Beyond the extra column, if `customers` had duplicate `customer_id` values the join would multiply order rows, violating the exactly-once condition. It over-delivers columns and risks row duplication.
- ✗
orders.union(customers.select('customer_id').withColumnRenamed('customer_id', 'order_id'))
Why it's wrong here
A union stacks rows rather than filtering them by existence, and aligning unrelated columns through renaming is semantically meaningless here. It would combine orders with customer identifiers as if they were orders, producing a nonsensical dataset. This approach neither filters nor preserves the intended schema, so it fails entirely.
About these practice questions
This Databricks-Spark-Assoc question is part of Courseiva's 295-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official Databricks exam blueprint
This Databricks-Spark-Assoc practice question is part of Courseiva's free Databricks certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the Databricks-Spark-Assoc exam.