DP-203 Develop data processing Practice Question
A data engineer is tasked with optimizing a Spark job in Azure Synapse Analytics that processes 10 TB of data daily. The job currently uses 50 executors with 4 cores each. The performance is bottlenecked by shuffle operations. The engineer wants to reduce shuffle data size. Which technique should be applied?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use column pruning to select only required columns before shuffle.
Column pruning (selecting only required columns) reduces the amount of data shuffled, decreasing shuffle size. Option A is incorrect because coalescing reduces the number of partitions but does not reduce the data volume shuffled. Option B is incorrect because increasing the number of shuffle partitions can increase shuffle overhead, not reduce it. Option C is incorrect because broadcasting all tables is impractical for large datasets (10 TB) and does not reduce shuffle data.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Coalesce the number of partitions before the shuffle.
Why it's wrong here
Reduces partitions but not shuffle data size.
- ✗
Increase the number of shuffle partitions.
Why it's wrong here
May increase shuffle overhead.
- ✗
Broadcast all tables to avoid shuffles.
Why it's wrong here
Not feasible for large datasets.
- ✓
Use column pruning to select only required columns before shuffle.
Why this is correct
Reduces data volume.
Go deeper
Related to this question
About these practice questions
This DP-203 question is part of Courseiva's 760-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This DP-203 practice question is part of Courseiva's free Microsoft certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DP-203 exam.