The answer is to cap the maximum number of workers by setting --maxNumWorkers to a value between 5 and 10. This resolves the increasing Dataflow streaming latency because the pipeline’s default autoscaling range (2–20 workers) is too wide for the 3-node Bigtable cluster, causing worker scaling to lag behind the incoming Pub/Sub load; by limiting workers to 5–10, you ensure sufficient parallelism without overwhelming the dimension store, allowing the fixed windows to process and emit before the system lag grows. On the Google Professional Data Engineer exam, this scenario tests your understanding of how worker scaling directly impacts streaming latency—a common trap is assuming Bigtable is the bottleneck when CPU is low, but the real issue is that autoscaling’s default max is too high, leading to slow scale-up and window expiration. Remember the “3-5-10” rule: a 3-node Bigtable cluster pairs best with a max of 5–10 workers to keep streaming latency in check.
PDE Designing data processing systems Practice Question
This PDE practice question tests your understanding of designing data processing systems. The scenario asks you to isolate a root cause — eliminate options that address a different problem before choosing. After answering, compare your reasoning against the explanation and wrong-answer breakdown below. Once you have made your selection, read the full explanation to reinforce the concept and understand why each distractor is designed to mislead on exam day.
A company runs a Cloud Dataflow streaming pipeline that reads from Cloud Pub/Sub, performs a fixed window of 10 seconds, joins with a slowly-changing dimension table stored in Cloud Bigtable, and writes results to BigQuery. The pipeline has been running for months but recently started exhibiting increasing latency and occasional data loss. The pipeline uses default settings with autoscaling enabled (min 2, max 20 workers). The Bigtable cluster has 3 nodes. The dimensions are updated infrequently. The latency has grown from seconds to minutes. Examining the Dataflow monitoring UI, you see that the 'System Lag' metric is increasing, and some windows are not being emitted. The CPU utilization on Bigtable nodes is below 50%. There are no errors in the logs. Which action is most likely to resolve the issue?
Clue words in this question
Noticing these words before you look at the options changes how you read each choice.
Clue: "most likely"
Why it matters: Probability qualifier — the question wants the most probable cause or outcome, not a guaranteed one. Eliminate low-probability options.
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Set the pipeline option --maxNumWorkers to a value between 5 and 10.
The increasing system lag and unemitted windows in a streaming pipeline with autoscaling (2–20 workers) and a 3-node Bigtable cluster indicate that the pipeline is bottlenecked by the number of workers, not by Bigtable performance. With default autoscaling, Dataflow may not scale up aggressively enough to handle the sustained load, causing backlog and window expiration. Capping maxNumWorkers to 5–10 ensures sufficient parallelism without over-provisioning, allowing the pipeline to catch up and emit windows reliably.
Key principle: Answer the scenario, not the keyword: identify the specific constraint before choosing the most familiar-sounding option.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
✓
Set the pipeline option --maxNumWorkers to a value between 5 and 10.
Why this is correct
Prevents over-scaling and shuffle overhead.
Clue confirmation
The clue word "most likely" in the question point toward this answer.
Related concept
Read the scenario before looking for a memorised answer.
✗
Increase the window duration to 30 seconds to reduce the number of windows.
Redesign the pipeline to use a side input for the dimension table instead of a lookup.
Why it's wrong here
Side input may cause similar issues.
✗
Increase the number of Bigtable nodes to reduce lookup latency.
Why it's wrong here
Bigtable CPU is low, so not the bottleneck.
Common exam traps
Common exam trap: answer the scenario, not the keyword
Google Cloud often tests the misconception that Bigtable or side inputs are the bottleneck when the real issue is insufficient worker parallelism, leading candidates to choose scaling Bigtable or redesigning the join strategy instead of adjusting autoscaling limits.
Trap categories for this question
Similar concept trap
Side input may cause similar issues.
Detailed technical explanation
How to think about this question
Dataflow's default autoscaling algorithm uses a 'recommended workers' metric based on backlog and throughput, but it can be conservative, especially with stateful operations like fixed windows and joins. The system lag metric reflects the time between event time and processing time; when it grows monotonically, it signals that workers are overwhelmed, causing windows to be dropped if their watermark exceeds the allowed lateness (default 0). In practice, capping maxNumWorkers prevents runaway scaling costs while ensuring the pipeline has enough parallelism to clear the backlog, as Dataflow will scale up to the cap if needed.
KKey Concepts to Remember
Read the scenario before looking for a memorised answer.
Find the constraint that changes the correct option.
Eliminate answers that are true in general but not in this case.
TExam Day Tips
→Watch for words such as best, first, most likely and least administrative effort.
→Review why wrong options are wrong, not only why the correct option is correct.
Key takeaway
Answer the scenario, not the keyword: identify the specific constraint before choosing the most familiar-sounding option.
Real-world example
How this comes up in practice
An e-commerce site experiences heavy traffic on Black Friday and near-zero traffic during off-peak weeks. Rather than provisioning permanent large VMs, the team uses auto-scaling groups that add capacity automatically under load and reduce it overnight. Questions like this test whether you understand elasticity, availability zones, and cloud compute scaling patterns.
What to study next
Got this wrong? Here's your next step.
Identify which exam domain this question belongs to, review the core concept, then practise similar questions from the same domain.
Designing data processing systems — This question tests Designing data processing systems — Read the scenario before looking for a memorised answer..
What is the correct answer to this question?
The correct answer is: Set the pipeline option --maxNumWorkers to a value between 5 and 10. — The increasing system lag and unemitted windows in a streaming pipeline with autoscaling (2–20 workers) and a 3-node Bigtable cluster indicate that the pipeline is bottlenecked by the number of workers, not by Bigtable performance. With default autoscaling, Dataflow may not scale up aggressively enough to handle the sustained load, causing backlog and window expiration. Capping maxNumWorkers to 5–10 ensures sufficient parallelism without over-provisioning, allowing the pipeline to catch up and emit windows reliably.
What should I do if I get this PDE question wrong?
Identify which exam domain this question belongs to, review the core concept, then practise similar questions from the same domain.
Are there clue words in this question I should notice?
Yes — watch for: "most likely". Probability qualifier — the question wants the most probable cause or outcome, not a guaranteed one. Eliminate low-probability options.
What is the key concept behind this question?
Read the scenario before looking for a memorised answer.
About these practice questions
Courseiva creates original exam-style practice questions with explanations and wrong-answer analysis. It does not publish real exam questions, exam dumps, or protected exam content. Learn why practice questions differ from exam dumps →
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A company uses Cloud Dataflow to process streaming data. They notice that the pipeline's throughput is lower than expected and the system is experiencing high latency. What is the most likely cause?
easy
A.Using batch mode instead of streaming mode
B.Too many workers
✓ C.Too few workers
D.Incorrect watermark setting
Why C: Option A is correct because insufficient workers are a common cause of low throughput and high latency. Autoscaling may not be enabled or workers are too few. Option B is wrong because batch mode is not used in streaming. Option C is incorrect; watermark settings affect late data, not throughput. Option D is wrong; too many workers would not cause high latency.
Share a tip, memory trick, or ask about the reasoning behind this question. Do not post real exam questions, leaked content, braindumps, or copyrighted exam material. Comments are moderated and may be removed without notice.
This PDE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PDE exam.
Question Discussion
Share a tip, memory trick, or ask about the reasoning behind this question. Do not post real exam questions, leaked content, braindumps, or copyrighted exam material. Comments are moderated and may be removed without notice.
Sign in to join the discussion.