Google PCA Ensure solution and operations reliability Practice Question

Your company runs a data pipeline on Google Cloud using Cloud Dataflow for streaming processing from Pub/Sub to BigQuery. The pipeline writes to a BigQuery table partitioned by day. The data is used for real-time dashboards. Recently, a spike in traffic caused the Dataflow pipeline to fall behind, and the dashboard displayed stale data. You need to design the pipeline to handle traffic spikes without data loss or long delays. The pipeline must be cost-efficient and use defaults where possible. Which solution should you implement?

⚠ Common exam trap

Google Cloud often tests the misconception that manual scaling (Option C) or static resource changes (Option D) are sufficient for handling spikes, when in fact Dataflow's built-in autoscaling and Streaming Engine are the designed, cost-efficient solutions for dynamic workloads.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Enable autoscaling in the Dataflow pipeline and use Streaming Engine to handle larger throughput

Enabling autoscaling in Dataflow allows the pipeline to dynamically adjust the number of workers based on the processing backlog, while Streaming Engine offloads the shuffle and state storage to Google-managed resources, reducing the impact of traffic spikes. This combination ensures the pipeline can scale up quickly to handle increased throughput without data loss or long delays, and it remains cost-efficient by scaling down when demand decreases.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✓
Enable autoscaling in the Dataflow pipeline and use Streaming Engine to handle larger throughput
Why this is correct
Autoscaling adds workers dynamically as Pub/Sub backlog grows, while Streaming Engine offloads pipeline state and shuffle processing to a managed service, raising throughput without the cost of permanently over-provisioned workers. Together they absorb traffic spikes with default settings, avoiding stale dashboards and data loss.
✗
Modify the pipeline to use a batch (non-streaming) approach, writing hourly batches from Pub/Sub to BigQuery
Why it's wrong here
Hourly batch processing introduces latency incompatible with real-time dashboards and cannot absorb spikes without delay. It is tempting because batch reduces cost, but streaming with autoscaling and default settings preserves freshness while handling variable load.
✗
Create a Cloud Scheduler job that increases the number of Dataflow workers every 5 minutes based on Pub/Sub subscription backlog
Why it's wrong here
Cloud Scheduler polling backlog every five minutes reacts slowly to sudden spikes and adds operational complexity, while Dataflow's default autoscaling already adds workers from backlog signals. Scheduler suits scheduled, predictable scaling actions rather than sub-minute streaming demand.
✗
Change the Dataflow worker machine type from n1-standard-4 to n1-highmem-8
Why it's wrong here
A larger worker machine type raises per-worker capacity but does not increase worker count, so a traffic spike still queues in Pub/Sub; horizontal autoscaling is what absorbs variable load. High-memory workers suit memory-bound transforms, such as large side inputs or grouping.

Go deeper

Google PCA Ensure solution and operations reliability Practice Question

Why each option matters

Option-by-option breakdown

Related to this question