PMLE Scaling Prototypes into ML Models Practice Question

You have a prototype model trained on a small sample of data. You now want to train on the full dataset using Vertex AI, but the dataset is stored in BigQuery and is several terabytes. You need to minimize data movement and avoid exporting the full dataset to Cloud Storage. What should you do?

⚠ Common exam trap

The trap here is assuming that training data must always be staged in Cloud Storage, when Vertex AI training can read directly from BigQuery.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Use the BigQuery reader in the training application to read data directly from BigQuery during training.

When training data lives in BigQuery and the goal is to avoid moving it, the training application should read from BigQuery directly using the available client libraries or Storage API. This keeps the data in place and avoids exporting terabytes to Cloud Storage. Exporting to CSV or Parquet, or routing through Feature Store, adds unnecessary copies or is not designed for bulk training reads.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Export the BigQuery table to Cloud Storage as CSV and pass the Cloud Storage path to the training job.
Why it's wrong here
Exporting several terabytes to Cloud Storage creates a large intermediate copy, consumes time and storage cost, and moves data unnecessarily. The requirement is to minimize data movement, so materializing the dataset outside BigQuery contradicts the goal. This approach also requires managing the exported files and their format compatibility with the training code.
✓
Use the BigQuery reader in the training application to read data directly from BigQuery during training.
Why this is correct
Vertex AI custom training supports reading data directly from BigQuery using client libraries or the BigQuery Storage API. This avoids exporting the dataset and keeps data in place, which minimizes movement and storage duplication. The training container can stream or batch records from BigQuery, and you can control the query to select only needed columns or rows, reducing I/O further.
✗
Create a Dataproc cluster to convert the BigQuery table to Parquet in Cloud Storage, then train from that path.
Why it's wrong here
Converting to Parquet still writes a full copy of the dataset to Cloud Storage, which is the data movement the scenario wants to avoid. It also adds a Dataproc cluster and a conversion step that increase cost and complexity. While Parquet is efficient for training reads, the export itself is unnecessary when the training job can read from BigQuery directly.
✗
Load the BigQuery table into a Vertex AI Feature Store and train from the feature store online serving endpoint.
Why it's wrong here
Feature Store is designed for serving features to online predictions and for managing feature consistency, not for bulk training reads of a multi-terabyte dataset. Online serving endpoints have latency and throughput characteristics suited to small requests, not full-dataset scans. Using it here would add complexity and would not efficiently replace direct BigQuery reads for training.

Go deeper

PMLE Scaling Prototypes into ML Models Practice Question

Why each option matters

Option-by-option breakdown

Related to this question