You need to ingest large amounts of unstructured data into Cloud Storage from an on-premises data center with limited bandwidth. Which service should you choose to ensure the most cost-effective and secure transfer?
You have a large CSV file in Cloud Storage that needs to be loaded into BigQuery. The file contains a nested JSON structure in one column. How should you best prepare this data?
You are configuring a Cloud Data Fusion pipeline to ingest data from an external SQL database. You need to ensure that only rows modified within the last hour are ingested. Which feature should you use?
Your company ingests a continuously growing stream of application logs into Pub/Sub. Analysts need to query the most recent 30 days of logs in BigQuery with standard SQL, but they do not want to manage any pipeline code or infrastructure. You must implement the ingestion with the least operational overhead. What should you do?
You are ingesting a 2 TB CSV file from Cloud Storage into BigQuery using the bq load command. The file has a header row and uses comma delimiters, but some fields contain embedded commas within double quotes. You need to ensure the data loads correctly without errors. What should you specify in your load command?
You need to load a 4 TB uncompressed CSV file from Cloud Storage into BigQuery. The file has a header row and uses comma delimiters. You want the load to complete as quickly as possible and minimize cost. What should you do first?
You need to copy a 500 GB dataset from an on-premises Hadoop cluster to Cloud Storage for a one-time analytics project. The data consists of many small files. Which Google Cloud tool should you use to transfer the data efficiently?
You are preparing a Dataflow pipeline that reads from a Pub/Sub subscription and writes to BigQuery. The pipeline must handle late-arriving data that can arrive up to 24 hours after the event timestamp. You need to ensure that all data is processed exactly once and that late data is included in the correct window. Which approach should you take?
You are designing a data pipeline that ingests data from an on-premises Oracle database into BigQuery. The database has strict network egress controls and cannot be accessed directly from the public internet. You need to ensure secure and reliable data transfer with minimal latency. Which TWO approaches should you use? (Choose two.)
A streaming pipeline writes raw JSON events to a Cloud Storage bucket every few minutes. Analysts query this data with BigQuery external tables, but each query scans the entire history and returns duplicate-looking records because the same event identifier appears in multiple files. You want queries to read only new files and to avoid counting an event twice when a corrected file overwrites an earlier one. Which combination of features should you configure?
You are ingesting streaming events into Pub/Sub. Downstream, you need to write each event to Cloud Storage in Parquet format with exactly-once semantics and minimal latency. You want to avoid managing infrastructure. What should you use?
You need to copy a 500 GB dataset from an on-premises NFS server to Cloud Storage. Your network bandwidth is limited to 100 Mbps, and the transfer must complete within one week. You want a fully managed service that requires minimal setup. What should you use?
You need to copy a 500 GB dataset from an on-premises NFS server to Cloud Storage. The server has a 1 Gbps network connection, and the transfer must be completed within a week. You want a fully managed service that handles retries and parallelism automatically. Which Google Cloud service should you use?
You are designing a data ingestion pipeline that reads CSV files from Cloud Storage, applies transformations, and writes to BigQuery. The files arrive every hour, and you need to ensure that only new files are processed and that the pipeline can handle late-arriving data. Which Dataflow feature should you use to track which files have been processed?
You are using Cloud Data Fusion to ingest data from an on-premises Oracle database into BigQuery. The database is behind a firewall and only allows outbound connections to specific Google Cloud IP ranges. Which component should you deploy to enable secure connectivity?
You need to ingest data from an on-premises Oracle database into BigQuery daily. The database is behind a firewall and cannot accept inbound connections. You want a fully managed, serverless solution that minimizes operational overhead. (Choose two.)
A retail company wants to continuously copy new objects from a Cloud Storage bucket into a BigQuery table for near-real-time dashboards. The source files are written by an external partner and are not guaranteed to be consistently named. The team prefers a fully managed, configuration-driven solution with no custom code. What should they use?
You need to ingest data from a local file system into Cloud Storage on a recurring basis. The data is about 10 GB per day and must be encrypted in transit. Which tool should you use?
You have a Cloud Storage bucket with millions of small JSON files. You need to query this data using SQL in BigQuery without loading it, and you want to minimize cost. What should you do?
A healthcare analytics group receives daily CSV extracts from an on-premises billing system. The files land in a Cloud Storage bucket, and a Dataprep by Trifacta job is used to clean and standardize them before loading into BigQuery. Compliance requires that the original CSV files be retained unchanged for seven years, and the team wants to minimize storage cost for those originals. What should they do?
You are using Cloud Data Fusion to build a pipeline that reads from Cloud Storage and writes to BigQuery. You need to ensure that the pipeline can handle schema evolution, where new columns are added to the source files over time. Which TWO approaches should you use? (Choose two.)
You are preparing data for analytics in BigQuery. You have a Cloud Storage bucket containing JSON files with inconsistent schemas and nested arrays. You need to load this data into BigQuery while preserving the nested structure and minimizing manual schema definition. Which TWO approaches should you use? (Choose two.)
A logistics company ingests GPS pings from delivery vehicles into Pub/Sub. A Dataflow streaming pipeline writes the raw messages to BigQuery using Storage Write API with exactly-once semantics, and a separate pipeline aggregates them. The team notices duplicate rows in the raw BigQuery table during pipeline restarts. They confirm the pipeline uses the Storage Write API with exactly-once enabled. What is the most likely cause?
You are designing a data ingestion pipeline that must handle late-arriving data in a streaming pipeline. Events can arrive up to 2 hours after their event timestamp. You need to ensure that windows are triggered only after all data for a given window has arrived, but you also want to limit resource usage. What should you configure in Dataflow?
A financial analytics team needs to move several terabytes of on-premises Oracle tables into BigQuery each night. The source database cannot accept inbound connections from Google Cloud, and the team wants minimal operational overhead for the transfer. They plan to use the BigQuery Data Transfer Service. Which connector should they configure?
You are designing a streaming pipeline in Dataflow. You need to ensure that data is processed in the order it was generated, even if it arrives late. What do you use?
Your team uses Dataprep by Trifacta to clean data before loading it into BigQuery. You notice that the column header names contain inconsistent casing and special characters. Which Dataprep transformation should you use to standardize these headers globally?
You are building a streaming pipeline using Dataflow to process sensor data arriving via Pub/Sub. You need to handle out-of-order data by allowing events to arrive late. Which Dataflow concept must you configure?
You are troubleshooting a Dataflow job that is running slower than expected when writing data to BigQuery. You suspect the issue is related to hot keys. What is the recommended strategy to mitigate this?
Your organization requires that all data ingested into Cloud Storage be encrypted at rest using keys managed by you, not Google. Which feature should you implement?
You need to load a large CSV file from Cloud Storage into BigQuery. The file has a header row and uses comma delimiters, but the data contains commas within quoted fields. Which BigQuery load option should you use to correctly parse the file?
Your team runs an Apache Airflow environment on Google Cloud Managed Service for Apache Airflow. A daily ingestion DAG must wait for a partner team's separate DAG to finish loading a landing table before it can begin its own transformation. The partner DAG is in the same Airflow environment but runs on an unpredictable schedule. You want the dependency handled natively by Airflow rather than with a fixed sleep or time sensor. What should you configure?
You are loading a 4 TB compressed CSV file from Cloud Storage into BigQuery using a load job. The file has a header row and uses pipe delimiters. You need the load to complete as fast as possible and minimize cost. What should you do?
Your team ingests JSON event files into Cloud Storage, and you must load them into BigQuery daily. The files contain a field named `user_info` that holds a nested JSON object with varying keys. You want BigQuery to store this field as a native STRUCT so analysts can query subfields with dot notation. Which schema type should you define for the `user_info` column when creating the table?
You are designing a data ingestion pipeline that uses Pub/Sub to capture real-time events. You need to ensure that messages are processed exactly once and in order. Which two features should you configure? (Choose two.)
Your team is ingesting streaming clickstream events from Pub/Sub into BigQuery. You need to ensure that the pipeline can handle duplicate message deliveries without creating duplicate rows in the destination table. What should you do?
You are building a Dataflow pipeline that reads from Pub/Sub and writes to BigQuery. Some messages contain malformed JSON that causes the pipeline to fail repeatedly. You want the pipeline to continue processing valid messages and send the bad records to a dead-letter location for later analysis. Which Dataflow feature should you use?
Your team needs to ingest data from an on-premises Oracle database into BigQuery with minimal latency and without writing code. The database is behind a firewall and cannot be accessed directly from the internet. You want to use a fully managed Google Cloud service. What should you use?
You are using Dataflow to process a stream of JSON messages from Pub/Sub. The messages contain a timestamp field, but the event time is sometimes delayed by up to 30 minutes. You need to handle these late-arriving events correctly. What should you configure in your pipeline?
You need to load a 5 TB uncompressed CSV file from Cloud Storage into BigQuery. The file is too large to load in a single job. What is the recommended approach?
You are using Cloud Dataflow to process a batch pipeline that reads from Cloud Storage, transforms data, and writes to BigQuery. You notice that the pipeline is running slowly and want to optimize it. Which of the following should you do to improve performance?
A financial analytics team ingests transaction events into Pub/Sub, then uses a Dataflow streaming pipeline with a GroupByKey transform to aggregate per-account totals. The team reports that late-arriving events—sometimes hours behind due to mobile connectivity—are being dropped and never counted in the aggregates. You need the pipeline to include those late events in the correct window. What should you configure?
A colleague must load a 200 GB uncompressed CSV file from a Cloud Storage bucket into a BigQuery table once, with the lowest possible load cost and no transformation. They want the simplest option that BigQuery handles natively. What should they use?
You are building a Dataflow pipeline that reads from Pub/Sub and writes to BigQuery. The pipeline must handle late-arriving data up to 1 hour after the event time and still include it in the correct window. Which Dataflow feature should you configure?
You are designing a data ingestion pipeline that must handle late-arriving data in a streaming source. The data arrives with timestamps that may be up to 10 minutes behind the current time. You need to ensure that all data is processed correctly and that windowed aggregations include late data. Which Dataflow feature should you use?
Your team runs a nightly Apache Spark job on a Dataproc cluster to cleanse raw event logs stored in Cloud Storage. The job currently reads the raw data, transforms it, and writes Parquet files back to Cloud Storage for downstream BigQuery external tables. The job duration has grown from 20 minutes to over 3 hours as data volume increased, and the cluster is a fixed-size 10-node cluster. You want to reduce runtime without rewriting the Spark code. What should you do?
A data engineer must ingest change data capture streams from an on-premises PostgreSQL database into BigQuery with minimal latency and without writing custom extraction code. The source table has a primary key that updates frequently. Which Google Cloud service should be used to replicate these changes continuously?
You are designing a Dataflow pipeline that reads from Cloud Storage and writes to BigQuery. The input files are in Avro format and contain a field with a timestamp that is often null. You need to ensure that records with null timestamps are not dropped but are assigned a default timestamp of the pipeline start time. Which approach should you use?
You need to prepare data for analysis in BigQuery. You have a CSV file in Cloud Storage that contains personally identifiable information (PII) such as email addresses and phone numbers. You must obfuscate this PII before loading it into BigQuery. Which two Google Cloud services or features can help you achieve this? (Choose two.)
You are ingesting CSV files into BigQuery using the bq load command. The CSV files have a header row, and you want BigQuery to automatically detect the schema and skip the header. Which flags should you include?
You need to load a 4 TB uncompressed CSV file from Cloud Storage into BigQuery. The file has 200 columns, and you want the load to complete as quickly as possible while minimizing cost. The data is already clean and does not require transformation. Which approach should you take?
You need to ingest data from a MySQL database into BigQuery. The database is updated continuously, and you need near-real-time replication with minimal impact on the source database. You also want to avoid writing custom code. What should you use?
You are designing a Dataflow streaming pipeline that reads from Pub/Sub and writes to BigQuery. The pipeline must handle late-arriving data and you want to emit correct results for each 5-minute window even when events arrive up to 10 minutes late. You also want to minimize cost by not keeping windows open indefinitely. Which combination of Dataflow features should you use?
You are using Cloud Data Fusion to build a batch pipeline that reads from an on-premises Oracle database and writes to BigQuery. The pipeline must run daily and you need to ensure that only new or changed records are ingested since the last run. Which plugin feature should you configure in the source stage?
A media company ingests clickstream events into Pub/Sub and processes them with Dataflow before loading into BigQuery. They need to reduce the number of duplicate records that reach BigQuery while keeping the pipeline streaming and low-latency. Which two approaches should they use? (Choose two.)
Your team stores daily clickstream data as newline-delimited JSON files in a Cloud Storage bucket. The ingestion pipeline must load each day's files into BigQuery, and the JSON structure occasionally gains new top-level fields between days. You want BigQuery to infer the schema automatically at load time and add any newly observed columns without failing the load. Which load configuration should you use?