PDE Ingesting and Processing the Data Practice Question
You are building a Dataflow pipeline in Python that reads messages from Pub/Sub, enriches them with data from a BigQuery table, and writes the results to BigQuery. The enrichment lookup table is large and changes infrequently. Which approach minimizes cost and latency?
⚠ Common exam trap
Google often tests the misconception that querying BigQuery per message is acceptable in streaming pipelines, but the trap here is that candidates overlook the cost and latency implications of per-element I/O, especially with BigQuery's pricing model and query latency.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use a side input that reads the BigQuery table periodically and caches it.
Using a side input that periodically reads the BigQuery table and caches it avoids querying BigQuery for every incoming message, which would be prohibitively expensive and high-latency. The side input is refreshed at a configurable interval (e.g., every 10 minutes) via a pipeline option, and the cached data is broadcast to all workers, enabling fast, in-memory lookups without per-element I/O. This approach minimizes cost by reducing BigQuery API calls and minimizes latency by avoiding synchronous queries for each message.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Use a CoGroupByKey transform to join the incoming stream with a stream from BigQuery.
Why it's wrong here
CoGroupByKey requires both sides to be in a stream; the lookup table is static, not a stream.
- ✗
Use BigQuery IO to query the table for every incoming message.
Why it's wrong here
This would be very slow and expensive due to many queries.
- ✓
Use a side input that reads the BigQuery table periodically and caches it.
Why this is correct
Side inputs are ideal for distributing a static lookup table to all workers. The data can be refreshed on a schedule.
- ✗
Use a stateful DoFn and store the lookup in state per key.
Why it's wrong here
Stateful DoFn is for per-key state across events, not for a global lookup table.
Go deeper
Related to this question
About these practice questions
Courseiva writes every PDE question from scratch — 890 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This PDE practice question is part of Courseiva's free Google Cloud certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the PDE exam.