PDE Ingesting and Processing the Data Practice Question
A streaming pipeline ingests events from Pub/Sub, enriches them via a slow REST API call, and writes the result to BigQuery. The API has a limit of 10 requests per second per client. The pipeline processes 1000 messages per second. Which approach minimizes latency while respecting API limits?
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use a global window with a trigger that fires every second, and inside the DoFn limit concurrent API calls to 10.
Using a global window with a trigger every second groups 1000 messages into a batch, and then throttling concurrent API calls to 10 within the DoFn (e.g., using a fixed-size thread pool) respects the API limit while minimizing latency by processing messages in parallel up to the limit. Option B is wrong because fanning out to multiple API instances doesn't help if the limit is per client; the total requests per second across all instances would still exceed the client limit. Option C is wrong because Dataflow Flex Templates are used to run parameterized pipelines, not to solve throttling issues. Option D is wrong because assigning a random key and using a sliding window distributes messages across workers, but without explicit throttling, the API limit could still be exceeded.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Use a global window with a trigger that fires every second, and inside the DoFn limit concurrent API calls to 10.
Why this is correct
Groups messages into batches per second, then controls concurrency to stay within the 10 req/s limit.
- ✗
Fan out the stream to multiple REST API instances using Pub/Sub topic splitting.
Why it's wrong here
If the API limit is per client, splitting does not help; each client still has a limit.
- ✗
Use a Dataflow Flex Template to run multiple pipelines, each processing a subset of messages.
Why it's wrong here
Multiple pipelines would each be separate clients, increasing total allowed calls, but adds operational complexity and does not guarantee fair distribution.
- ✗
Assign each message a random key and use a sliding window of 10 seconds; the API call will be distributed across workers.
Why it's wrong here
Random keys will not control concurrency; total API calls per second may still exceed limit.
Go deeper
Related to this question
About these practice questions
Courseiva writes every PDE question from scratch — 890 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This PDE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PDE exam.