- A
Use n1-highmem-4 instances for the cluster to get 26 GB RAM per node and increase executor memory to 12g.
Why wrong: Increasing node memory is expensive and might not solve the issue if memory is wasted due to partition size.
- B
Migrate the PySpark jobs to Cloud Dataflow with the Apache Beam SDK to benefit from auto-scaling.
Why wrong: Migration is a large effort and does not directly address memory issues; Dataflow might also encounter similar problems.
- C
Increase the number of executors and reduce the executor memory to 4g, then add preemptible secondary workers to lower cost.
Adding more executor instances distributes memory and reduces per executor load; preemptible workers lower costs.
- D
Enable cluster autoscaling and set minimum to 5 workers, maximum to 20 workers.
Why wrong: Autoscaling adds nodes but executor memory allocation per node remains unchanged; still may get OOM.
Quick Answer
The correct answer is to increase the number of executors and reduce executor memory back to 4g, then add preemptible secondary workers to lower cost. This resolves the Dataproc PySpark OOM executor optimization issue because the persistent OutOfMemoryError after raising memory to 8g indicates a parallelism bottleneck, not a per-executor memory shortage—the total cluster memory is insufficient for the 3x data volume. By scaling out with more executors at 4g each, you distribute the shuffle and processing load across additional JVMs, reducing memory pressure per executor while preemptible workers provide cost-efficient compute capacity. On the Google Professional Data Engineer exam, this scenario tests your understanding that OOM errors often stem from inadequate parallelism or skewed data distribution, not just raw memory size; a common trap is to keep increasing executor memory without adjusting the executor count. Remember the mnemonic: “More executors, not more memory—scale out, not up.”
PDE Practice Question: Building and operationalizing data processing systems
This PDE practice question tests your understanding of building and operationalizing data processing systems. The scenario asks you to isolate a root cause — eliminate options that address a different problem before choosing. After answering, compare your reasoning against the explanation and wrong-answer breakdown below. Once you have made your selection, read the full explanation to reinforce the concept and understand why each distractor is designed to mislead on exam day.
You are a data engineer at a financial services company. You manage a batch pipeline that processes daily trade settlement reports. The pipeline runs on Cloud Dataproc using PySpark jobs triggered by Cloud Composer (Airflow). Recent trades have increased by 3x, and the pipeline now frequently fails with 'OutOfMemoryError' in the executor logs. You have already increased the executor memory from 4g to 8g, but the problem persists. The cluster uses standard worker nodes (n1-standard-4) with 15 GB RAM per node. You need to make the pipeline stable and cost-efficient. What should you do?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
Increase the number of executors and reduce the executor memory to 4g, then add preemptible secondary workers to lower cost.
Option C is correct because the OutOfMemoryError persists even after increasing executor memory to 8g, indicating that the issue is not simply insufficient memory per executor but rather that the total memory across all executors is insufficient for the 3x data volume. By increasing the number of executors (parallelism) and reducing executor memory back to 4g, you distribute the data processing load across more JVMs, reducing the memory pressure per executor. Adding preemptible secondary workers lowers cost while providing the additional compute capacity needed to handle the increased data volume efficiently.
Key principle: Answer the scenario, not the keyword: identify the specific constraint before choosing the most familiar-sounding option.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Use n1-highmem-4 instances for the cluster to get 26 GB RAM per node and increase executor memory to 12g.
Why it's wrong here
Increasing node memory is expensive and might not solve the issue if memory is wasted due to partition size.
- ✗
Migrate the PySpark jobs to Cloud Dataflow with the Apache Beam SDK to benefit from auto-scaling.
Why it's wrong here
Migration is a large effort and does not directly address memory issues; Dataflow might also encounter similar problems.
- ✓
Increase the number of executors and reduce the executor memory to 4g, then add preemptible secondary workers to lower cost.
Why this is correct
Adding more executor instances distributes memory and reduces per executor load; preemptible workers lower costs.
Related concept
Read the scenario before looking for a memorised answer.
- ✗
Enable cluster autoscaling and set minimum to 5 workers, maximum to 20 workers.
Why it's wrong here
Autoscaling adds nodes but executor memory allocation per node remains unchanged; still may get OOM.
Common exam traps
Common exam trap: answer the scenario, not the keyword
Google Cloud often tests the misconception that increasing executor memory alone solves OutOfMemoryErrors, when in reality the issue is often insufficient parallelism or misconfigured memory overhead, and the correct solution involves balancing executor count, memory, and cost-efficient instance types like preemptible VMs.
Trap categories for this question
Similar concept trap
Migration is a large effort and does not directly address memory issues; Dataflow might also encounter similar problems.
Detailed technical explanation
How to think about this question
In Spark, the total memory available for executors on a node is limited by the node's RAM minus overhead for the OS and YARN container overhead (typically 10-15%). On an n1-standard-4 with 15 GB RAM, setting executor memory to 8g leaves only ~7 GB for overhead and other containers, which can cause container failures or OOM. By reducing executor memory to 4g, you can run multiple executors per node (e.g., 3 executors with 4g each, leaving 3 GB for overhead), increasing parallelism and reducing the risk of OOM. Preemptible secondary workers are cost-effective because they are up to 80% cheaper than regular instances, but they can be terminated at any time, so the pipeline must be designed to handle preemptions gracefully (e.g., using checkpointing or idempotent writes).
KKey Concepts to Remember
- Read the scenario before looking for a memorised answer.
- Find the constraint that changes the correct option.
- Eliminate answers that are true in general but not in this case.
TExam Day Tips
- Watch for words such as best, first, most likely and least administrative effort.
- Review why wrong options are wrong, not only why the correct option is correct.
Key takeaway
Answer the scenario, not the keyword: identify the specific constraint before choosing the most familiar-sounding option.
Real-world example
How this comes up in practice
A startup's cloud architect reviews their monthly bill and notices costs are higher than expected for a long-running batch job. Switching from on-demand instances to Reserved Instances — or using Spot/Preemptible VMs — can reduce compute costs by up to 72 %. Questions like this test whether you understand the tradeoffs between commitment, flexibility, and cost across cloud pricing models.
What to study next
Got this wrong? Here's your next step.
Identify which exam domain this question belongs to, review the core concept, then practise similar questions from the same domain.
- →
Building and operationalizing data processing systems — study guide chapter
Learn the concepts, then practise the questions
- →
Building and operationalizing data processing systems practice questions
Targeted practice on this topic area only
- →
All PDE questions
499 questions across all exam domains
- →
Google Professional Data Engineer study guide
Full concept coverage aligned to exam objectives
- →
PDE practice test guide
How to use practice tests most effectively before exam day
Related practice questions
Related PDE practice-question pages
Use these pages to review the topic behind this question. This is how one missed question becomes focused revision.
Designing data processing systems practice questions
Practise PDE questions linked to Designing data processing systems.
Building and operationalizing data processing systems practice questions
Practise PDE questions linked to Building and operationalizing data processing systems.
Operationalizing machine learning models practice questions
Practise PDE questions linked to Operationalizing machine learning models.
Ensuring solution quality practice questions
Practise PDE questions linked to Ensuring solution quality.
PDE fundamentals practice questions
Practise PDE questions linked to PDE fundamentals.
PDE scenario practice questions
Practise PDE questions linked to PDE scenario.
PDE troubleshooting practice questions
Practise PDE questions linked to PDE troubleshooting.
Practice this exam
Start a free PDE practice session
Short sessions build daily habit. Longer sessions build exam-day stamina. Try a timed session to simulate real conditions.
FAQ
Questions learners often ask
What does this PDE question test?
Building and operationalizing data processing systems — This question tests Building and operationalizing data processing systems — Read the scenario before looking for a memorised answer..
What is the correct answer to this question?
The correct answer is: Increase the number of executors and reduce the executor memory to 4g, then add preemptible secondary workers to lower cost. — Option C is correct because the OutOfMemoryError persists even after increasing executor memory to 8g, indicating that the issue is not simply insufficient memory per executor but rather that the total memory across all executors is insufficient for the 3x data volume. By increasing the number of executors (parallelism) and reducing executor memory back to 4g, you distribute the data processing load across more JVMs, reducing the memory pressure per executor. Adding preemptible secondary workers lowers cost while providing the additional compute capacity needed to handle the increased data volume efficiently.
What should I do if I get this PDE question wrong?
Identify which exam domain this question belongs to, review the core concept, then practise similar questions from the same domain.
What is the key concept behind this question?
Read the scenario before looking for a memorised answer.
About these practice questions
Courseiva creates original exam-style practice questions with explanations and wrong-answer analysis. It does not publish real exam questions, exam dumps, or protected exam content. Learn why practice questions differ from exam dumps →
Last reviewed: Jun 30, 2026
This PDE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PDE exam.
Question Discussion
Share a tip, memory trick, or ask about the reasoning behind this question. Do not post real exam questions, leaked content, braindumps, or copyrighted exam material. Comments are moderated and may be removed without notice.
Sign in to join the discussion.