PDE Designing Data Processing Systems Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✓

  Export the Parquet files to encrypted portable drives and use the Transfer Appliance to ship them to a Google ingest location, then load the objects into BigQuery.

  Why this is correct

  Transfer Appliance is purpose-built for bulk offline migration when network bandwidth makes online transfer impractical. Shipping encrypted appliances avoids the limited outbound link entirely and comfortably moves 30 TB within the window. After Google uploads the appliance contents to Cloud Storage, a BigQuery load job reads the Parquet objects into native tables, after which dashboards query managed storage with full columnar performance.

• ✗

  Use the Storage Transfer Service with a transfer job configured against the on-premises source, then load the resulting Cloud Storage objects into BigQuery using a load job.

  Why it's wrong here

  Storage Transfer Service is designed for cloud-to-cloud and URL-list transfers or transfers from an agent-based on-premises source; it is a valid path but requires deploying and maintaining a transfer agent in the data center and does not exploit the offline shipping option. With only 48 hours and constrained outbound bandwidth, this still hinges on network throughput and may not finish in time. It is workable but not the best fit.

• ✗

  Create a Dataproc cluster in the same region and use a Spark job to read from the on-premises Hadoop cluster over the network and write directly into BigQuery.

  Why it's wrong here

  A Spark job still has to pull 30 TB across the constrained on-premises uplink, so the bottleneck is unchanged and the 48-hour window is unlikely to be met. It also adds cluster provisioning and shuffle overhead. Dataproc is a strong choice for transformation, but for a pure one-time bulk move where bandwidth is the limiting factor, an offline transfer is the correct tool.

• ✗

  Run gcloud storage cp from a Compute Engine VM to upload the files into a BigQuery-managed bucket, then query them with an external table.

  Why it's wrong here

  Copying files into a bucket and querying with an external table leaves the data outside BigQuery's managed storage, so dashboards pay per-scan query bytes against Cloud Storage and lose BigQuery's columnar storage and caching benefits. It also depends entirely on the limited outbound bandwidth of the source, which conflicts with the 48-hour window. The goal is a one-time load into native tables, not permanent external querying.

Go deeper

Related to this question