DEA-C01 Data Ingestion and Transformation Practice Test 18 — 15 Questions

Question 1

A data engineering team needs to ingest CSV files from an S3 bucket into a Redshift cluster on a daily basis. The files are large (up to 100 GB each). Which approach is MOST cost-effective and efficient?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use the Redshift COPY command with an IAM role to load data directly from S3.. Using the Redshift COPY command with an IAM role is the most cost-effective and efficient approach for loading large CSV files from S3 into Redshift on a daily basis. The COPY command is optimized for bulk data loading and can handle files up to 100 GB efficiently. Option A (Kinesis Data Firehose) is designed for streaming data, not batch loading of large files. Option B (AWS Data Pipeline) adds unnecessary cost and complexity for a simple scheduled copy job. Option C (Lambda function) has execution time limits (15 minutes) and memory constraints, making it unsuitable for processing 100 GB files atomically. Therefore, the COPY command is the best choice.

Answer

Use Amazon Kinesis Data Firehose to load data into Redshift.

Answer

Use AWS Data Pipeline to copy data from S3 to Redshift.

Answer

Use a Lambda function to read each file and insert rows individually.

Question 2

A company uses Amazon Kinesis Data Firehose to deliver streaming data to an S3 bucket. The data must be transformed from JSON to Parquet format before delivery. Which approach should be used?

Accepted Answer

Configure Kinesis Data Firehose to invoke a Lambda function for data transformation.. Kinesis Data Firehose can invoke a Lambda function as a transformation step before data is delivered to S3. This allows you to convert JSON records to Parquet format inline, without needing an intermediate storage or separate processing pipeline. The Lambda function receives batches of records, transforms them (e.g., using PyArrow or similar libraries), and returns them to Firehose for delivery.

Answer

Use an AWS Glue ETL job to read from S3 and write Parquet back to S3.

Answer

Use Amazon EMR to process the data and output Parquet.

Answer

Use Kinesis Data Analytics to convert the data to Parquet.

Question 3

A data engineer uses AWS Glue to catalog data from an S3 bucket. The data is partitioned by year, month, day. After adding new partitions, the Glue Crawler does not detect them. What is the MOST likely reason?

Accepted Answer

The crawler is configured to only add new partitions to existing tables, but the table schema has changed.. The Glue Crawler, by default, is configured to add new partitions only if the table schema remains unchanged. When new partitions are added to an S3 bucket, if the underlying data schema has changed (e.g., new columns, different data types), the crawler will not add those partitions to the existing table. This is a common safeguard to prevent schema drift from corrupting the cataloged table structure.

Answer

The crawler runs only once and does not schedule subsequent runs.

Answer

The IAM role lacks permission to write to the Glue Data Catalog.

Answer

The partition depth exceeds the crawler's default limit.

Question 4

A real-time analytics application uses Amazon Kinesis Data Streams. The consumer application falls behind, causing increased latency. Which action would MOST effectively improve throughput?

Accepted Answer

Increase the number of shards in the data stream.. Increasing the number of shards in the Kinesis Data Stream directly increases the stream's read capacity (each shard supports up to 2 MB/s read and 5 transactions per second for shared throughput). This allows the consumer application to process more data in parallel, reducing the backlog and latency. The question specifies a consumer application falling behind, which is a read-throughput bottleneck, and scaling shards is the most effective way to address it.

Answer

Reduce the RecordMaxBufferedTime parameter in the Firehose delivery stream.

Answer

Increase the batch size in the Kinesis Producer Library.

Answer

Use enhanced fan-out to dedicate a shard per consumer.

Question 5

A data pipeline uses AWS Glue ETL to process data from an S3 bucket and write results to a Redshift cluster. The job fails with a 'DiskFull' error on the Glue worker nodes. What is the best way to resolve this issue?

Accepted Answer

Increase the number of Glue DPUs or use G.1X worker type.. The 'DiskFull' error on Glue worker nodes indicates that the local storage allocated per worker is insufficient for the data being processed. Increasing the number of DPUs or switching to a G.1X worker type (which provides more disk space per worker) directly addresses this by either distributing the workload across more workers or upgrading to a worker type with higher storage capacity.

Answer

Decrease the number of partitions in the output.

Answer

Use a different file format like Parquet to reduce storage.

Answer

Increase the job timeout setting.

Question 6

Which TWO services can be used to ingest streaming data into Amazon S3? (Choose two.)

Accepted Answer

Amazon Kinesis Data Streams. Amazon Kinesis Data Streams (C) is correct because it is a real-time streaming service that can deliver records to Amazon S3 as a destination via Kinesis Data Firehose or custom consumers, enabling continuous ingestion of streaming data into S3. Amazon Kinesis Data Firehose (E) is correct because it is a fully managed service designed to reliably load streaming data directly into Amazon S3 (as well as Redshift, OpenSearch, and Splunk), making it a primary AWS solution for streaming ingestion into S3. Amazon Athena (A) is incorrect because it is an interactive query service for analyzing data already in S3, not a streaming ingestion mechanism. AWS Glue (B) is incorrect because it is a serverless ETL and data catalog service that processes batch or micro-batch jobs, not a native streaming ingestion service into S3. AWS Database Migration Service (D) is incorrect because it migrates and replicates databases, which is not the same as ingesting streaming data into S3.

Answer

Amazon Athena

Answer

AWS Glue

Answer

AWS Database Migration Service (DMS)

Question 7

A company uses Amazon Kinesis Data Analytics to process real-time data. The application needs to aggregate data over a 10-minute window. The team notices that late-arriving events are being dropped. Which configuration should they adjust?

Accepted Answer

Set the allowed_lateness parameter in the application's windowed aggregation.. In Amazon Kinesis Data Analytics (KDA), late-arriving events are handled by the allowed_lateness parameter in windowed aggregations. Setting this parameter allows the application to accept and process events that arrive after the window has closed, up to the specified lateness threshold. This directly addresses the issue of dropped late events by extending the window's acceptance period.

Answer

Configure a Kinesis Firehose delivery stream to buffer the late events.

Answer

Increase the shard count of the source Kinesis stream.

Answer

Increase the RecordColumn count in the input stream mapping.

Question 8

A data pipeline uses AWS Glue to read CSV files from an S3 bucket, transform them, and write Parquet back to S3. The pipeline runs daily and processes about 500 GB per run. The team wants to reduce costs without increasing runtime. Which approach is most effective?

Accepted Answer

Optimize the Glue script to select only required columns and filter rows early.. Optimizing the Glue script to use column pruning and predicate pushdown reduces the amount of data scanned, lowering costs without increasing runtime. Option A (pre-converting CSV to Parquet) adds an extra processing step and cost, making it less effective. Option B (enabling job bookmarks) tracks already processed data but does not reduce the cost per run. Option C (increasing DPUs) may actually increase costs and can cause unpredictable runtime.

Answer

Pre-convert the CSV files to Parquet in S3 using a separate process.

Answer

Enable job bookmarks to skip already processed data.

Answer

Increase the number of DPUs for the Glue job to improve parallelism.

Question 9

A company needs to ingest data from an on-premises Oracle database into Amazon S3 for analytics. The data volume is about 1 TB initially, with daily incremental updates of about 10 GB. Which TWO services can be combined to achieve this with minimal custom code?

Accepted Answer

Amazon S3. Option E, AWS Database Migration Service (DMS), is correct because it can connect directly to an on-premises Oracle database as a source and replicate full-load plus ongoing change data capture (CDC) into Amazon S3 as a target, handling the 1 TB initial load and 10 GB daily incremental updates with minimal custom code. Option D, Amazon S3, is correct because it is the required destination for the ingested data and serves as the landing zone that DMS writes to, enabling downstream analytics. Together, DMS (extract/replicate) plus S3 (store) form the minimal-code ingestion pipeline. Option A, AWS Glue, is not needed here because the scenario asks for ingestion rather than ETL/cataloging, and Glue would add custom job logic. Option B, Amazon Kinesis Data Streams, is for real-time streaming data, not batch/CDC ingestion from an Oracle database. Option C, Amazon Athena, is a query service over S3 data and does not perform ingestion from Oracle.

Answer

AWS Glue

Answer

Amazon Kinesis Data Streams

Answer

Amazon Athena

Question 10

A data engineer is designing a streaming pipeline using Amazon Kinesis Data Streams. The data must be transformed in real-time and then stored in Amazon S3 for long-term retention. Which THREE services can be used together to achieve this?

Accepted Answer

Amazon Kinesis Data Analytics. Options A, B, and E are correct: Kinesis Data Streams ingests data, Kinesis Data Analytics performs real-time transformations, and Kinesis Data Firehose delivers the transformed data to S3. Option C (AWS Glue) is a batch ETL service, not suitable for real-time streaming. Option D (Amazon Athena) is a query service, not for data ingestion or transformation in a streaming pipeline.

Answer

AWS Glue

Answer

Amazon Athena

Question 11

Refer to the exhibit. A Glue ETL job failed. What is the most likely cause?

Accepted Answer

Some source files have a different schema than others.. The error 'Cannot merge incompatible schemas' indicates that some source files have a different schema than others, preventing the Glue ETL job from merging them. Option D is wrong because insufficient DPUs cause memory or performance errors, not schema conflicts. Option B is wrong because partition pruning is about filtering data, not schema mismatches. Option C is wrong because job bookmarks track processed files to avoid reprocessing, not schema issues.

Answer

The job bookmarks are misconfigured.

Answer

The source data has inconsistent partitioning.

Answer

The job ran out of memory due to insufficient DPUs.

Question 12

Refer to the exhibit. A Lambda function with this IAM policy is used to process records from a Kinesis stream and write to S3. The function is failing with access denied errors when writing to S3. What is the issue?

Accepted Answer

The S3 bucket name in the policy does not match the actual bucket used by the function.. The IAM policy grants s3:PutObject on a specific bucket ARN, but the Lambda function is writing to a different bucket name than the one specified in the policy's Resource element. IAM evaluates the resource ARN exactly, so any mismatch (even a typo or different environment bucket) results in AccessDenied. Correcting the bucket name in the policy to match the actual target bucket resolves the error.

Answer

The function needs to use Kinesis Data Analytics for transformation.

Answer

The Kinesis stream ARN is incorrect.

Answer

The Lambda function does not have permission to read from the Kinesis stream.

Question 13

Refer to the exhibit. A data engineer is troubleshooting a Glue job that reads objects from this S3 bucket. The job runs successfully but produces no output. The Glue catalog table points to the same S3 path. What is the most likely cause?

Accepted Answer

The S3 key does not follow Hive-style partitioning (e.g., year=2024/month=01).. AWS Glue Data Catalog relies on Hive-style partitioning (e.g., year=2024/month=01) to automatically discover and register partitions. The S3 key in the exhibit does not follow this pattern, so Glue cannot identify the partitions, resulting in no data being read even though the job runs. Option B is incorrect because object metadata size does not affect Glue's ability to read data. Option C is incorrect because the STANDARD storage class is fully supported by Glue. Option D is incorrect because ContentType is not a factor in Glue catalog table definitions.

Answer

The object metadata is too large.

Answer

The StorageClass is not supported by Glue.

Answer

The ContentType is not supported by Glue.

Question 14

A company wants to move data from an Amazon RDS for MySQL database to Amazon Redshift for analytics. The data needs to be refreshed daily. Which AWS service is best suited for this?

Accepted Answer

AWS Glue. AWS Glue is the best-suited service because it provides serverless ETL capabilities, including crawlers to discover schema, jobs to transform data, and built-in connectors for RDS and Redshift. It can schedule daily refreshes via triggers, making it ideal for recurring analytics data pipelines without managing infrastructure.

Answer

AWS Database Migration Service (DMS)

Answer

Amazon EMR

Answer

Amazon Athena

Question 15

A company wants to ingest data from thousands of IoT devices into AWS for real-time analytics. The data is in JSON format and each device sends about 1 KB every second. Which service should be used as the primary ingestion point?

Accepted Answer

Amazon Kinesis Data Streams. Amazon Kinesis Data Streams (D) is the correct choice because it is designed for real-time streaming of large amounts of data from many producers, such as thousands of IoT devices. Each device sends 1 KB per second, resulting in ~1 MB/s total throughput, which Kinesis Data Streams can handle with sharding. It supports multiple consumers for real-time analytics. AWS IoT Core (A) is for device management and MQTT messaging, not a general-purpose ingestion point for analytics. Kinesis Data Firehose (B) is for loading streaming data into storage, but it does not support multiple real-time consumers and has a minimum 60-second buffer latency. Amazon SQS (C) is a message queue for decoupled applications, not built for high-throughput streaming analytics.

Answer

AWS IoT Core

Answer

Amazon Kinesis Data Firehose

Answer

Amazon SQS