DEA-C01 Data Ingestion and Transformation Practice Test 8 — 15 Questions

Question 1

A company needs to transfer 10 TB of historical data from an on-premises HDFS cluster to Amazon S3. The data is stored on a single 20 TB disk. The network link to AWS has a bandwidth of 1 Gbps. The transfer must be completed within 2 days. Which solution meets these requirements?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use AWS Snowball Edge to transfer the data physically.. AWS Snowball Edge is the correct solution because it can physically transfer 10 TB of data from a single 20 TB disk within the 2-day window, bypassing the network bandwidth limitation. With a 1 Gbps link, the theoretical maximum transfer of 10 TB would take approximately 22.2 hours under ideal conditions, but real-world overhead, latency, and potential congestion make it unreliable to complete within 2 days. Snowball Edge provides a petabyte-scale physical transport solution that avoids network constraints entirely.

Answer

Use Amazon Kinesis Data Streams to stream data to S3.

Answer

Use AWS DMS to migrate data from HDFS to S3.

Answer

Use AWS CLI to copy data directly to S3 over the network.

Question 2

A data engineer notices that an AWS Glue job writing to Amazon S3 in Parquet format creates many small files (less than 1 MB each). This leads to poor query performance in Amazon Athena. What is the BEST way to reduce the number of output files?

Accepted Answer

Enable 'groupFiles' in the Glue job's S3 target configuration.. Enabling 'groupFiles' in the AWS Glue job's S3 target configuration instructs Glue to coalesce small files into larger ones (default target size ~128 MB) during the write phase. This directly reduces the number of small Parquet files written to S3, improving Athena query performance by minimizing S3 LIST and GET overhead.

Answer

Use 'coalesce(1)' at the end of the ETL script.

Answer

Use 'repartition(100)' to increase parallelism.

Answer

Configure an S3 lifecycle policy to delete small files.

Question 3

A data engineer needs to ingest real-time clickstream data from a website into Amazon S3 for analytics. The data arrives as JSON records, each under 1 KB. The engineer wants to use a serverless solution with automatic scaling and minimal operational overhead. Which AWS service should be used as the ingestion endpoint?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is a fully managed, serverless service designed to ingest streaming data and automatically load it into Amazon S3 with no ongoing administration. It handles automatic scaling, converts incoming JSON records to formats like Parquet or ORC if needed, and can batch data into S3 based on time or size intervals, making it ideal for real-time clickstream ingestion with minimal operational overhead.

Answer

Amazon S3 with presigned URLs

Answer

Amazon Kinesis Data Analytics

Answer

AWS Lambda function behind an API Gateway

Question 4

A company uses AWS DMS to migrate data from an on-premises Oracle database to Amazon RDS for PostgreSQL. The migration is ongoing with continuous replication. The data engineer notices that some changes are not being captured in the target database. What is the MOST likely cause?

Accepted Answer

The source Oracle database is not configured to retain archived redo logs for a sufficient period.. In AWS DMS continuous replication (CDC), the service reads changes from the source Oracle database's archived redo logs. If the logs are rotated or deleted before DMS has a chance to process them, changes are lost. Option D is correct because insufficient retention of archived redo logs is the most likely cause of missing changes in the target.

Answer

The VPC peering connection between on-premises and AWS is down.

Answer

The DMS task's table mapping is incorrectly configured.

Answer

DMS is not publishing task logs to CloudWatch Logs.

Question 5

A data engineer is designing a data ingestion pipeline to load JSON files from Amazon S3 into Amazon Redshift. Which TWO methods can be used to load the data efficiently?

Accepted Answer

Use the Redshift COPY command to load from S3.. The Redshift COPY command is specifically designed to efficiently load large datasets from Amazon S3 by automatically parallelizing the data across cluster nodes, leveraging the cluster's compute resources for high-throughput ingestion. It supports JSON data natively via the 'json' option, making it ideal for loading JSON files directly from S3 without intermediate transformations.

Answer

Use Amazon Kinesis Data Firehose to directly load into Redshift.

Answer

Use AWS DMS to replicate from S3 to Redshift.

Answer

Use individual INSERT statements in a loop.

Question 6

Refer to the exhibit. A data engineer is creating an IAM policy for an application that sends data to a Kinesis stream and stores processed data in S3. The policy is attached to an IAM role used by an EC2 instance. The application fails to write to S3 with an access denied error. What is the cause?

Accepted Answer

The policy does not allow s3:ListBucket on the bucket.. The error occurs because the IAM policy grants s3:PutObject but not s3:ListBucket on the target S3 bucket. When the application writes to S3, the AWS SDK often performs a ListBucket operation first to verify bucket existence or to handle multipart uploads, and without s3:ListBucket permission, the request is denied with an access denied error.

Answer

The IAM role is not attached to the EC2 instance profile.

Answer

The policy does not allow kinesis:PutRecord on the stream.

Answer

The EC2 instance does not have an internet gateway to reach S3.

Question 7

Refer to the exhibit. A data engineer is running an AWS Glue job that reads data from an S3 source. The job fails with the error shown. What is the MOST likely cause?

Accepted Answer

One of the source files is empty or corrupted.. The error message indicates that the Glue job encountered a 'NullPointerException' or similar parsing failure when reading from S3. This typically occurs when a source file is empty or corrupted, causing the Spark DataFrame reader to fail during schema inference or data parsing. AWS Glue jobs rely on Spark's ability to read files; an empty or malformed file triggers a runtime error because Spark cannot extract any records or infer a valid schema from it.

Answer

The IAM role does not have s3:GetObject permission.

Answer

The file is in JSON format but the schema expects Parquet.

Answer

The Glue job has insufficient memory allocated.

Question 8

A data engineer is ingesting streaming data from an IoT fleet into Amazon Kinesis Data Streams. The data must be transformed in real-time and loaded into an Amazon Redshift cluster. Which solution minimizes operational overhead?

Accepted Answer

Use Kinesis Data Firehose with a Lambda transformation function. Kinesis Data Firehose is the fully managed service for loading streaming data into Redshift with near-real-time latency. By attaching a Lambda transformation function, you can perform lightweight data transformations (e.g., JSON flattening, field masking) without managing any compute infrastructure. This combination eliminates the need to provision or tune any servers, clusters, or consumer applications, minimizing operational overhead.

Answer

Use AWS Glue ETL jobs running continuously

Answer

Use Kinesis Client Library (KCL) to consume and transform data, then write to Redshift using COPY

Answer

Use AWS Direct Connect to stream data directly into Redshift

Question 9

A company ingests clickstream data into Amazon S3 via Kinesis Data Firehose. The data arrives in 20 MB files every 2 minutes. The data engineering team needs to transform nested JSON into a flat structure before loading into Amazon Redshift. Which approach is most cost-effective and scalable?

Accepted Answer

Create an AWS Glue ETL job that runs on a schedule, using dynamic frames to flatten the data and write to S3 in Parquet. AWS Glue ETL jobs are designed for serverless, scalable data transformation, and using dynamic frames to flatten nested JSON and write to Parquet is both cost-effective (pay per DPU-hour) and scalable (automatically handles data volume). The 20 MB files arriving every 2 minutes are well-suited for Glue's batch processing, and Parquet output optimizes Redshift loading via COPY commands.

Answer

Run an Amazon EMR cluster with Spark to flatten the data and write back to S3

Answer

Use AWS Lambda to transform each file as it arrives in S3

Answer

Use Amazon Redshift Spectrum to query the nested JSON directly and create a view

Question 10

A company uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data is compressed with GZIP and partitioned by year, month, day, and hour. The delivery stream is configured to buffer up to 5 MB or 60 seconds. Some records are missing from S3. What is the most likely cause?

Accepted Answer

The Lambda transformation function timed out and records were skipped. When a Lambda transformation function times out, Kinesis Data Firehose will skip the affected records by default. The delivery stream configuration (5 MB or 60 seconds) only controls buffering, not Lambda invocation failures. If the Lambda function exceeds its timeout limit, Firehose treats the invocation as failed and, depending on the error handling configuration, may drop the records entirely, leading to missing data in S3.

Answer

The S3 bucket does not have sufficient permissions

Answer

The data compression format is incompatible with S3

Answer

The partition key configuration is incorrect

Question 11

Refer to the exhibit. A data engineer runs the AWS CLI command and observes the output. The stream has two shards. A producer sends a record with a partition key that hashes to 150000000000000000000000000000000000000. To which shard will the record be written?

Accepted Answer

shardId-000000000001. The record will be written to shardId-000000000001 because the hash key range for shardId-000000000000 is [0, 170141183460469231731687303715884105727) and for shardId-000000000001 is [170141183460469231731687303715884105727, 340282366920938463463374607431768211455). The partition key hash value of 150000000000000000000000000000000000000 falls within the second shard's range, so the record is routed to shardId-000000000001.

Answer

The record will be rejected because it does not match any shard

Answer

shardId-000000000000

Answer

The record will be written to both shards

Question 12

Refer to the exhibit. A data engineer creates an AWS Glue job using this CloudFormation template. The job processes new data files in S3 and uses job bookmarks to track processed files. After initial success, the job runs again but processes all files again instead of only new ones. What is the most likely cause?

Accepted Answer

The S3 input path does not have a partitioning scheme or timestamp to identify new files. AWS Glue job bookmarks rely on the structure of the input data to identify new files. Without a partitioning scheme or a timestamp-based naming convention in the S3 path, Glue cannot determine which files are new; it falls back to reprocessing all files. The job bookmark feature tracks processed files by examining the S3 path and file metadata, so a flat or non-partitioned structure prevents it from distinguishing new files from old ones.

Answer

The job bookmark option is set to 'job-bookmark-disable'

Answer

The enable-metrics parameter is set to true

Answer

The MaxRetries parameter is set to 0

Question 13

A company is ingesting streaming data from IoT devices into Amazon Kinesis Data Streams. The data must be transformed into Parquet format and stored in Amazon S3. Which AWS service can perform the transformation in near real-time with minimal operational overhead?

Accepted Answer

Amazon Kinesis Data Firehose with a transformation Lambda function. Amazon Kinesis Data Firehose is the fully managed service designed to load streaming data into S3 with built-in data format conversion. By attaching a Lambda transformation function, you can convert incoming records to Parquet format in near real-time without managing any infrastructure, making it the lowest-operational-overhead choice for this task.

Answer

Amazon EMR cluster running Spark Streaming

Answer

AWS Glue ETL job triggered by Kinesis stream

Answer

Amazon Kinesis Data Analytics for Apache Flink

Question 14

A data engineer is designing a data ingestion pipeline for clickstream data. The data arrives in batches of 10-50 MB every 5 seconds. The engineer needs to buffer the data, perform simple transformations (e.g., add timestamp, remove PII), and land it in S3 within 10 minutes. Which TWO services should be combined? (Choose TWO.)

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service designed to ingest streaming data, buffer it, perform simple transformations (such as adding timestamps or removing PII) via built-in Lambda functions, and automatically deliver the data to S3. It can handle the 10-50 MB batches every 5 seconds and meet the 10-minute delivery window without requiring custom code for buffering or delivery.

Answer

Amazon Simple Queue Service (SQS)

Answer

AWS Glue ETL

Answer

Amazon Kinesis Data Streams

Question 15

A data engineer needs to transform CSV files in S3 to Parquet format using a serverless solution. The files are large (up to 5 GB each) and arrive irregularly. Which TWO services can accomplish this with minimal operational overhead? (Choose TWO.)

Accepted Answer

AWS Glue ETL job. AWS Glue ETL job is correct because it is a fully managed, serverless service that can automatically convert CSV to Parquet without provisioning infrastructure. It handles large files (up to 5 GB) by scaling Spark executors dynamically, and can be triggered by S3 events for irregular arrivals, minimizing operational overhead.

Answer

Amazon EC2 with a script

Answer

Amazon EMR cluster

Answer

Amazon Redshift Spectrum