Develop data processing →mediumMultiple Choice

DP-203 Develop data processing Practice Question

Exhibit

Refer to the exhibit.

{
  "name": "AggregateProductSales",
  "properties": {
    "folder": {
      "name": "Sales"
    },
    "content": {
      "jobType": "SparkJob",
      "jobDefinition": {
        "file": "abfss://container@storage.dfs.core.windows.net/synapse/workspaces/workspace/sparkjobdefinitions/aggregate_sales.py",
        "conf": {
          "spark.dynamicAllocation.enabled": "false",
          "spark.executor.instances": 10,
          "spark.executor.cores": 4,
          "spark.executor.memory": "8g"
        }
      }
    }
  }
}

You are reviewing a Spark job definition in Azure Synapse Analytics. The job aggregates sales data. The job runs successfully but takes longer than expected. You notice that dynamic allocation is disabled and the executor instances are fixed at 10. The cluster has a maximum of 20 nodes. What is the most likely reason for the slow performance?

⚠ Common exam trap

The trap here is that candidates may overlook the explicit configuration detail (dynamic allocation disabled, fixed 10 executors) and instead focus on generic performance issues like memory or partitioning, missing the direct scaling limitation.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

The job cannot scale out beyond 10 executors because dynamic allocation is disabled.

With dynamic allocation disabled and executor instances fixed at 10, the Spark job cannot utilize additional cluster resources even though the cluster supports up to 20 nodes. This means the job is artificially constrained to 10 executors, limiting parallelism and causing slower performance despite available compute capacity.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
The file path is incorrect, causing data read errors.
Why it's wrong here
An incorrect path would surface read errors or job failure, not a successful run that merely exceeds expected duration. It is tempting because path faults are a common Spark troubleshooting step, but here the job completes, so the bottleneck lies in fixed executor count against available cluster capacity.
✓
The job cannot scale out beyond 10 executors because dynamic allocation is disabled.
Why this is correct
With dynamic allocation disabled, Spark holds the executor count at the fixed value of 10, so it cannot request additional executors from the cluster's 20-node maximum. Enabling dynamic allocation lets Spark add executors during the aggregation's shuffle-heavy stages, which is the scaling constraint causing the slow runtime.
✗
The job is not parallelized because of a single partition.
Why it's wrong here
A single partition would serialise work into one task regardless of executor count, yet the stem's fixed ten executors against twenty available nodes points to unallocated capacity. It is tempting because skew and partitioning genuinely cause slowness, but dynamic allocation is the specific setting named as disabled.
✗
The executor memory is too low for the aggregation.
Why it's wrong here
Insufficient executor memory typically manifests as spills, out-of-memory failures or executor loss, not a clean successful run. It is tempting because aggregation is memory-hungry, but the stem identifies fixed executor instances at ten with twenty nodes available, so unused cluster capacity is the concrete constraint.

Go deeper

DP-203 Develop data processing Practice Question

Exhibit

Why each option matters

Option-by-option breakdown

Related to this question