DEA-C01 Data Ingestion and Transformation Practice Test 35 — 15 Questions

Question 1

A company has a large volume of CSV files in S3 that need to be transformed into Parquet using AWS Glue. The files are partitioned by date. The engineer wants to minimize costs by processing only new files each day. Which approach should be used?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Enable job bookmarks in the Glue job.. Using a Job Bookmark in Glue tracks processed data and skips already processed files, processing only new ones. Option A is wrong because it would reprocess all files. Option B is wrong because partitioning alone does not prevent reprocessing. Option D is wrong because an SQS event can trigger a job, but without bookmarks, it may still reprocess.

Answer

Use S3 partition discovery to automatically read new partitions.

Answer

Schedule the Glue job to run daily and process all files.

Answer

Use S3 Event Notifications to trigger the Glue job on each new file.

Question 2

A data engineer is building a streaming pipeline using Amazon Kinesis Data Streams. The data must be enriched with reference data from a DynamoDB table before being written to S3. The engineer wants to minimize latency. Which architecture is BEST?

Accepted Answer

Use Kinesis Data Analytics for Apache Flink to enrich and output to Firehose.. (Kinesis Data Analytics for Apache Flink) is the best choice because it supports low-latency enrichment using external sources like DynamoDB via asynchronous I/O, meeting the requirement to minimize latency. Option A (AWS Glue streaming ETL) is designed for batch-oriented processing and introduces higher latency. Option C (Kinesis Data Firehose with a Lambda function) may experience cold starts and limited concurrency, increasing latency. Option D (Lambda polling the stream) also suffers from cold starts and scalability issues, making it less suitable for low-latency enrichment.

Answer

Use AWS Glue streaming ETL to read from Kinesis, enrich, and write to S3.

Answer

Use Kinesis Data Firehose with a Lambda function for enrichment.

Answer

Use a Lambda function to poll the stream, enrich, and write to Firehose.

Question 3

A data engineer is designing a data ingestion pipeline for real-time clickstream data. Which TWO services can be used to ingest the data into Amazon Kinesis Data Streams?

Accepted Answer

Kinesis Producer Library (KPL). Options B and D are correct. The Kinesis Producer Library (KPL) is a library for producers to send data to Kinesis Data Streams. AWS SDK can also be used directly. Option A is wrong because Amazon S3 is a storage service, not a producer. Option C is wrong because Kinesis Data Firehose is a downstream consumer or delivery service, not a producer. Option E is wrong because AWS Glue is an ETL service, not a producer.

Answer

Amazon S3

Answer

Kinesis Data Firehose

Answer

AWS Glue

Question 4

A company uses AWS Glue to transform data in S3. The Glue job fails with memory errors. Which THREE actions can help resolve this?

Accepted Answer

Optimize the transformation to use pushdown predicates.. Options A, B, and C are correct. Using pushdown predicates (A) reduces the amount of data read by filtering at the data source, which can alleviate memory pressure. Using a larger worker type (B), such as G.2X, increases the memory available per worker, directly addressing out-of-memory errors. Increasing the number of DPUs (C) adds more workers, distributing the memory load. Option D (increasing job timeout) does not solve memory issues, and Option E (decreasing DPUs) would reduce available memory, making the problem worse.

Answer

Increase the job timeout.

Answer

Decrease the number of DPUs.

Question 5

A data engineer is implementing a CDC (Change Data Capture) pipeline from a relational database to Amazon S3 using AWS Database Migration Service (DMS). Which TWO configurations are required for continuous replication?

Accepted Answer

Enable binary logging on the source database.. For continuous CDC replication with AWS DMS, two key configurations are required: enabling binary logging on the source database (option B) and selecting both 'Full load' and 'Ongoing replication' in the DMS task (option D). Binary logging (e.g., binlog for MySQL) captures ongoing changes, while enabling both full load and ongoing replication ensures that the initial data load is completed before continuous change capture begins. Option A (transformation rules) is optional, not required. Option C (VPC endpoint) is not necessary because DMS can connect to the source database over the internet, VPN, or VPC peering without a VPC endpoint. Option E (pre-creating the target table in S3) is not needed because DMS can create the target table automatically or you can let DMS handle the schema.

Answer

Define transformation rules in the DMS task.

Answer

Configure a VPC endpoint for DMS.

Answer

Pre-create the target table in S3.

Question 6

A company ingests JSON data from an S3 bucket into a Glue ETL job. The data contains nested structures and arrays. The team wants to flatten the data into a tabular format for analysis in Athena. Which Glue transformation is appropriate?

Accepted Answer

Relationalize. The Relationalize transformation is specifically designed to flatten nested JSON and arrays into a tabular format suitable for Athena. Option A (Map) applies a function to each record but does not flatten structures. Option C (Filter) selects rows based on a condition. Option D (DropNullFields) removes null fields but does not address nested structures.

Answer

Map

Answer

Filter

Answer

DropNullFields

Question 7

A data pipeline uses AWS Glue to process data from an S3 data lake. The pipeline fails intermittently with a 'ThrottlingException' when writing to a DynamoDB table. What is the MOST likely cause?

Accepted Answer

The DynamoDB table's write capacity is insufficient for the workload.. A ThrottlingException from DynamoDB indicates that the request rate to the table has exceeded the provisioned write capacity. AWS Glue jobs can generate high-throughput writes, and if the DynamoDB table's write capacity units (WCUs) are not sufficient to handle the burst, DynamoDB will throttle the requests. This is the most direct cause of the intermittent failure described.

Answer

The network connection between Glue and DynamoDB is unstable.

Answer

The Glue job's timeout setting is too low.

Answer

The Glue job does not have sufficient IAM permissions to write to DynamoDB.

Question 8

A company uses Amazon Kinesis Data Firehose to deliver streaming data to an S3 bucket. The data must be transformed from JSON to Parquet format before delivery. Which approach should be used?

Accepted Answer

Configure Kinesis Data Firehose to invoke a Lambda function for data transformation.. Kinesis Data Firehose can invoke a Lambda function as a transformation step before data is delivered to S3. This allows you to convert JSON records to Parquet format inline, without needing an intermediate storage or separate processing pipeline. The Lambda function receives batches of records, transforms them (e.g., using PyArrow or similar libraries), and returns them to Firehose for delivery.

Answer

Use an AWS Glue ETL job to read from S3 and write Parquet back to S3.

Answer

Use Amazon EMR to process the data and output Parquet.

Answer

Use Kinesis Data Analytics to convert the data to Parquet.

Question 9

A data engineer uses AWS Glue to catalog data from an S3 bucket. The data is partitioned by year, month, day. After adding new partitions, the Glue Crawler does not detect them. What is the MOST likely reason?

Accepted Answer

The crawler is configured to only add new partitions to existing tables, but the table schema has changed.. The Glue Crawler, by default, is configured to add new partitions only if the table schema remains unchanged. When new partitions are added to an S3 bucket, if the underlying data schema has changed (e.g., new columns, different data types), the crawler will not add those partitions to the existing table. This is a common safeguard to prevent schema drift from corrupting the cataloged table structure.

Answer

The crawler runs only once and does not schedule subsequent runs.

Answer

The IAM role lacks permission to write to the Glue Data Catalog.

Answer

The partition depth exceeds the crawler's default limit.

Question 10

A company needs to ingest data from an on-premises database to Amazon S3 with minimal impact on the source database. The data volume is several TB. Which AWS service is best suited for this task?

Accepted Answer

AWS Database Migration Service (DMS). AWS Database Migration Service (DMS) is best suited because it can continuously replicate data from an on-premises database to Amazon S3 with minimal impact on the source. DMS uses change data capture (CDC) to capture only incremental changes after an initial full load, avoiding heavy read loads on the source database. This makes it ideal for migrating several TB of data while keeping the source operational.

Answer

AWS Direct Connect

Answer

AWS Snowball Edge

Answer

Amazon S3 Transfer Acceleration

Question 11

A real-time analytics application uses Amazon Kinesis Data Streams. The consumer application falls behind, causing increased latency. Which action would MOST effectively improve throughput?

Accepted Answer

Increase the number of shards in the data stream.. Increasing the number of shards in the Kinesis Data Stream directly increases the stream's read capacity (each shard supports up to 2 MB/s read and 5 transactions per second for shared throughput). This allows the consumer application to process more data in parallel, reducing the backlog and latency. The question specifies a consumer application falling behind, which is a read-throughput bottleneck, and scaling shards is the most effective way to address it.

Answer

Reduce the RecordMaxBufferedTime parameter in the Firehose delivery stream.

Answer

Increase the batch size in the Kinesis Producer Library.

Answer

Use enhanced fan-out to dedicate a shard per consumer.

Question 12

A data pipeline uses AWS Glue ETL to process data from an S3 bucket and write results to a Redshift cluster. The job fails with a 'DiskFull' error on the Glue worker nodes. What is the best way to resolve this issue?

Accepted Answer

Increase the number of Glue DPUs or use G.1X worker type.. The 'DiskFull' error on Glue worker nodes indicates that the local storage allocated per worker is insufficient for the data being processed. Increasing the number of DPUs or switching to a G.1X worker type (which provides more disk space per worker) directly addresses this by either distributing the workload across more workers or upgrading to a worker type with higher storage capacity.

Answer

Decrease the number of partitions in the output.

Answer

Use a different file format like Parquet to reduce storage.

Answer

Increase the job timeout setting.

Question 13

Which TWO services can be used to ingest streaming data into Amazon S3? (Choose two.)

Accepted Answer

Amazon Kinesis Data Streams. Amazon Kinesis Data Streams is a real-time streaming service that can ingest and store streaming data, which can then be consumed and written to Amazon S3 using a Kinesis Data Analytics or a custom consumer application. Amazon Kinesis Data Firehose is a fully managed service that can directly load streaming data into Amazon S3, Amazon Redshift, or Amazon Elasticsearch Service, with optional data transformation and compression.

Answer

Amazon Athena

Answer

AWS Glue

Answer

AWS Database Migration Service (DMS)

Question 14

Which THREE factors should be considered when choosing between AWS Glue and Amazon EMR for data transformation? (Choose three.)

Accepted Answer

EMR allows fine-grained control over cluster configuration and software.. Amazon EMR provides full control over cluster configuration, including the ability to customize software, install libraries, and tune Spark, Hadoop, or Hive parameters. This fine-grained control is essential for complex or specialized data transformation pipelines that require specific versions or custom configurations.

Answer

Glue automatically stores data in S3 after transformation.

Answer

EMR supports real-time stream processing with Spark Streaming.

Question 15

Which TWO AWS services can be used as sources for AWS Glue ETL jobs? (Choose two.)

Accepted Answer

Amazon S3. Amazon S3 is a fully managed object storage service that serves as a common source for AWS Glue ETL jobs. Glue can read data from S3 using its built-in crawlers and connectors, supporting formats like Parquet, JSON, CSV, and Avro. The Glue Data Catalog can reference S3 locations, and ETL scripts can directly read from S3 buckets via the s3:// protocol.

Answer

Amazon Route 53

Answer

Amazon CloudFront

Answer

Amazon API Gateway