hardMultiple Choice
PDE Practice Question: A data engineer at a global e-commerce company
You are a data engineer at a global e-commerce company. Your team manages a real-time recommendation system that ingests user clickstream events from a Pub/Sub topic (topic-clickstream). The pipeline uses Dataflow to read events, join with user profile data from Cloud Bigtable, compute recommendations using a machine learning model hosted on Cloud Run, and write results to a BigQuery table for analytics. The pipeline has been running smoothly for months, but recently the Dataflow job started failing with the error: "Workflow failed. Causes: S01:ReadPubSub/Read+Transform/ParDo(ExtractUserID)+ ... (5a3b2c1d) The job failed because a worker encountered an out-of-memory error." The Dataflow job uses the Streaming Engine feature with a worker type of n2-standard-8 (8 vCPU, 32 GB memory) and autoscaling from 2 to 20 workers. The clickstream event rate has increased from 500 events/second to 5000 events/second over the past week. The user profile data in Bigtable has also grown, with average row size increasing from 1 KB to 10 KB due to additional fields. You need to resolve the out-of-memory errors without completely redesigning the pipeline. What should you do?
⚠ Common exam trap
Google Cloud often tests the misconception that scaling out (more workers) solves memory issues, when in fact the per-worker memory limit is the bottleneck and must be increased via a higher-memory machine type.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Change the worker machine type to n2-highmem-8 (8 vCPU, 64 GB memory) in the Dataflow job configuration.
The out-of-memory error is caused by the increased per-worker memory load from larger Bigtable rows (1 KB to 10 KB) and higher event throughput (500 to 5000 events/sec). Switching to n2-highmem-8 doubles the memory from 32 GB to 64 GB, giving each worker more headroom to cache user profiles and process larger batches without OOM. This directly addresses the root cause without redesigning the pipeline.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Increase the maximum number of workers in autoscaling from 20 to 50.
Why it's wrong here
Raising the worker ceiling lets autoscaling add more n2-standard-8 workers, but each worker still holds the enlarged 10 KB profile rows in memory, so individual workers keep exhausting their 32 GB. It tempts because horizontal scaling relieves throughput bottlenecks, which would be correct if workers were CPU-bound rather than memory-bound.
- ✓
Change the worker machine type to n2-highmem-8 (8 vCPU, 64 GB memory) in the Dataflow job configuration.
Why this is correct
Doubling worker memory to 64 GB directly addresses the out-of-memory error, since the larger 10 KB Bigtable rows inflate per-element state during the join. Streaming Engine already offloads shuffle, so the constraint is worker heap, not pipeline design.
- ✗
Reduce the batch size in the Dataflow pipeline by setting the `max_batch_size` parameter to a lower value.
Why it's wrong here
Setting max_batch_size reduces per-request grouping in Bigtable or Pub/Sub client calls, but the out-of-memory arises from enlarged 10 KB profile rows held during the join, not batch grouping. It tempts because batching parameters do tune memory in some pipelines, yet here the per-element footprint grew.
- ✗
Increase the number of Bigtable nodes to improve read throughput.
Why it's wrong here
Adding Bigtable nodes raises read throughput and reduces latency, but worker memory exhaustion stems from holding larger 10 KB profile rows during the join, not from insufficient read capacity. It tempts because Bigtable scaling fixes slow reads, which would be correct if the job were throttled rather than OOM-killed.
Visual reference
Go deeper
Related to this question
About these practice questions
Courseiva writes every PDE question from scratch — 747 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This PDE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PDE exam.