Data Preparation And Ingestion

Which TWO of the following are common Dataflow pipeline patterns?

Your team stores daily clickstream data as newline-delimited JSON files in a Cloud Storage bucket. The ingestion pipeline must load each day's files into BigQuery, and the JSON structure occasionally gains new top-level fields between days. You want BigQuery to infer the schema automatically at load time and add any newly observed columns without failing the load. Which load configuration should you use?

You are building a Dataflow pipeline that reads from Pub/Sub and writes to BigQuery. Some messages contain malformed JSON that causes the pipeline to fail repeatedly. You want the pipeline to continue processing valid messages and send the bad records to a dead-letter location for later analysis. Which Dataflow feature should you use?

You are designing a streaming pipeline in Dataflow. You need to ensure that data is processed in the order it was generated, even if it arrives late. What do you use?

You are designing a Dataflow streaming pipeline that reads from Pub/Sub and writes to BigQuery. The pipeline must handle late-arriving data and you want to emit correct results for each 5-minute window even when events arrive up to 10 minutes late. You also want to minimize cost by not keeping windows open indefinitely. Which combination of Dataflow features should you use?

What is the primary benefit of using partitioned tables in BigQuery?

You need to provide temporary access to a GCS file for a third party. What should you use?

How do you restrict access to a specific GCS bucket to only members of a specific team?

You need to load a large CSV file from Cloud Storage into BigQuery. The file has a header row and uses comma delimiters, but the data contains commas within quoted fields. Which BigQuery load option should you use to correctly parse the file?

Your team needs to ingest data from an on-premises Oracle database into BigQuery with minimal latency and without writing code. The database is behind a firewall and cannot be accessed directly from the internet. You want to use a fully managed Google Cloud service. What should you use?

You are designing a data ingestion pipeline that must handle late-arriving data in a streaming source. The data arrives with timestamps that may be up to 10 minutes behind the current time. You need to ensure that all data is processed correctly and that windowed aggregations include late data. Which Dataflow feature should you use?

You are using Dataflow to process a stream of JSON messages from Pub/Sub. The messages contain a timestamp field, but the event time is sometimes delayed by up to 30 minutes. You need to handle these late-arriving events correctly. What should you configure in your pipeline?

You are using BigQuery and need to load data from an external file that is not in GCS. Which method is recommended?

You are planning to ingest log data into Google Cloud. Which THREE of the following services support streaming ingestion?

Which THREE features does Dataprep provide?

Which tool is used to monitor the performance of your Dataflow pipelines?

You are loading a 4 TB compressed CSV file from Cloud Storage into BigQuery using a load job. The file has a header row and uses pipe delimiters. You need the load to complete as fast as possible and minimize cost. What should you do?

Which GCP tool provides a managed way to run Apache Airflow workflows?

You need to transfer files from an Amazon S3 bucket to GCS. What is the most efficient method?

Which THREE are features of Pub/Sub?

You are building a Dataflow pipeline that reads from Pub/Sub and writes to BigQuery. The pipeline must handle late-arriving data up to 1 hour after the event time and still include it in the correct window. Which Dataflow feature should you configure?

Which THREE of the following are necessary for a production-ready Dataflow pipeline?

You need to anonymize PII (Personally Identifiable Information) before loading data into BigQuery. Which tool is best?

Which TWO of the following are recommended practices when designing a Dataflow pipeline for high-throughput streaming ingestion?

You are migrating a high-throughput on-premises database to BigQuery. You need to ensure zero downtime. What is your best strategy?

Your organization requires that all data ingested into Cloud Storage be encrypted at rest using keys managed by you, not Google. Which feature should you implement?

You are using BigQuery and need to query a large dataset that is partitioned by day. Which clause should you always include for efficiency?

You need to load a 5 TB uncompressed CSV file from Cloud Storage into BigQuery. The file is too large to load in a single job. What is the recommended approach?

A media company ingests clickstream events into Pub/Sub and processes them with Dataflow before loading into BigQuery. They need to reduce the number of duplicate records that reach BigQuery while keeping the pipeline streaming and low-latency. Which two approaches should they use? (Choose two.)

You are running a Dataflow job that joins two large datasets. Which join strategy should you avoid to prevent OOM errors?

Which TWO of the following are valid ways to monitor Dataflow performance?

You need to delete all files in a GCS bucket older than 90 days. What is the most efficient approach?

You are optimizing a Dataflow job. Which THREE of the following actions can help improve job performance and reduce costs?

You need to track who accessed which GCS bucket. Which service provides this audit trail?

Your team runs a nightly Apache Spark job on a Dataproc cluster to cleanse raw event logs stored in Cloud Storage. The job currently reads the raw data, transforms it, and writes Parquet files back to Cloud Storage for downstream BigQuery external tables. The job duration has grown from 20 minutes to over 3 hours as data volume increased, and the cluster is a fixed-size 10-node cluster. You want to reduce runtime without rewriting the Spark code. What should you do?

Your team ingests JSON event files into Cloud Storage, and you must load them into BigQuery daily. The files contain a field named `user_info` that holds a nested JSON object with varying keys. You want BigQuery to store this field as a native STRUCT so analysts can query subfields with dot notation. Which schema type should you define for the `user_info` column when creating the table?

Which TWO of the following are key benefits of using Dataflow?

You are processing streaming data in Pub/Sub and need to archive every message into GCS without writing custom code. What should you use?

Which TWO are common causes of Dataflow pipeline failures?

You are designing a data ingestion pipeline that uses Pub/Sub to capture real-time events. You need to ensure that messages are processed exactly once and in order. Which two features should you configure? (Choose two.)

You need to set up an alert when a GCS bucket exceeds a certain size. Which tool do you use?

You are processing streaming data in Dataflow and notice 'stuck' elements causing pipeline latency. Which feature helps debug this?

You are using Dataflow to read from Pub/Sub. The pipeline is failing due to malformed messages. How can you handle these without crashing?

A colleague must load a 200 GB uncompressed CSV file from a Cloud Storage bucket into a BigQuery table once, with the lowest possible load cost and no transformation. They want the simplest option that BigQuery handles natively. What should they use?

You are designing a Dataflow pipeline that reads from Cloud Storage and writes to BigQuery. The input files are in Avro format and contain a field with a timestamp that is often null. You need to ensure that records with null timestamps are not dropped but are assigned a default timestamp of the pipeline start time. Which approach should you use?

Which TWO are valid methods to trigger a Dataflow job?

You are cleaning data using Dataprep. You want to save the final dataset in BigQuery. What do you do?

You are troubleshooting a Dataflow job that is running slower than expected when writing data to BigQuery. You suspect the issue is related to hot keys. What is the recommended strategy to mitigate this?

You need to perform a rolling update on a Dataflow job without losing current state. How do you do this?

You need to load a 4 TB uncompressed CSV file from Cloud Storage into BigQuery. The file has 200 columns, and you want the load to complete as quickly as possible while minimizing cost. The data is already clean and does not require transformation. Which approach should you take?

Which GCS storage class is most cost-effective for data accessed only once per year?

You are using Dataflow with a custom container. Which command do you use to specify the container image?

Your team runs an Apache Airflow environment on Google Cloud Managed Service for Apache Airflow. A daily ingestion DAG must wait for a partner team's separate DAG to finish loading a landing table before it can begin its own transformation. The partner DAG is in the same Airflow environment but runs on an unpredictable schedule. You want the dependency handled natively by Airflow rather than with a fixed sleep or time sensor. What should you configure?

Which THREE of the following are valid GCS storage classes?

You need to prepare data for analysis in BigQuery. You have a CSV file in Cloud Storage that contains personally identifiable information (PII) such as email addresses and phone numbers. You must obfuscate this PII before loading it into BigQuery. Which two Google Cloud services or features can help you achieve this? (Choose two.)

Your team uses Dataprep by Trifacta to clean data before loading it into BigQuery. You notice that the column header names contain inconsistent casing and special characters. Which Dataprep transformation should you use to standardize these headers globally?

You need to ingest data from a MySQL database into BigQuery. The database is updated continuously, and you need near-real-time replication with minimal impact on the source database. You also want to avoid writing custom code. What should you use?

In Dataflow, what is the impact of using a high number of workers for a small dataset?

Your team is ingesting streaming clickstream events from Pub/Sub into BigQuery. You need to ensure that the pipeline can handle duplicate message deliveries without creating duplicate rows in the destination table. What should you do?

You need to verify the integrity of a large file uploaded to GCS. What is the standard way to do this?

You are building a streaming pipeline using Dataflow to process sensor data arriving via Pub/Sub. You need to handle out-of-order data by allowing events to arrive late. Which Dataflow concept must you configure?

Which TWO factors should you consider when choosing a GCS storage class?

Which TWO of the following are GCP ingestion tools?

You are ingesting CSV files into BigQuery using the bq load command. The CSV files have a header row, and you want BigQuery to automatically detect the schema and skip the header. Which flags should you include?

You need to store sensitive data in GCS. Which feature ensures data is encrypted at rest?

You need to perform data cleansing on a large dataset in Dataprep. Which execution engine should you choose for scalability?

Which TWO are methods to secure GCS data?

Which service should you use to ingest streaming data from mobile devices?

A data engineer must ingest change data capture streams from an on-premises PostgreSQL database into BigQuery with minimal latency and without writing custom extraction code. The source table has a primary key that updates frequently. Which Google Cloud service should be used to replicate these changes continuously?

You need to ensure that deleted files in GCS can be restored for 30 days. What should you configure?

You are running a Dataflow job and notice that it is consuming too much memory and crashing. How do you optimize this?

You need to ingest log data into BigQuery from GCS files that arrive sporadically. Which service is best to trigger this?

You are using Cloud Dataflow to process a batch pipeline that reads from Cloud Storage, transforms data, and writes to BigQuery. You notice that the pipeline is running slowly and want to optimize it. Which of the following should you do to improve performance?

You need to perform a complex windowing operation on streaming data in Dataflow. Which windowing strategy is best for session-based activity?

You are using Cloud Data Fusion to build a batch pipeline that reads from an on-premises Oracle database and writes to BigQuery. The pipeline must run daily and you need to ensure that only new or changed records are ingested since the last run. Which plugin feature should you configure in the source stage?

You are using Dataflow with autoscaling. What metric determines whether the worker pool scales up or down?

You need to ingest IoT data into BigQuery. You need to handle messages arriving in millions per second. Which service acts as the buffer?

A financial analytics team ingests transaction events into Pub/Sub, then uses a Dataflow streaming pipeline with a GroupByKey transform to aggregate per-account totals. The team reports that late-arriving events—sometimes hours behind due to mobile connectivity—are being dropped and never counted in the aggregates. You need the pipeline to include those late events in the correct window. What should you configure?

Which THREE of the following are Dataflow windowing types?

Which THREE steps are involved in an effective Dataflow pipeline development?

You need to ingest data from an external API that requires an OAuth token. Which tool is best for executing this periodically?

You are using Dataprep to prepare data. How can you share your work with a team?