DEA-C01 Data Ingestion and Transformation Practice Test 14 — 15 Questions

Question 1

A company uses Amazon EMR to process large datasets stored in Amazon S3. The data is in Parquet format and partitioned by date. The EMR cluster uses Spark SQL for transformations. Recently, the job has been slow and some tasks are failing due to 'java.lang.OutOfMemoryError'. The cluster has 10 core nodes of type m5.xlarge. Which configuration change would MOST improve performance and stability?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Change the core node instance type to r5.xlarge (memory-optimized).. The error 'java.lang.OutOfMemoryError' indicates that the Spark executors are running out of memory during processing. The m5.xlarge instance type provides 16 GiB of memory, but the workload likely requires more memory per task. Switching to r5.xlarge (32 GiB of memory) doubles the available memory per node, reducing memory pressure and preventing task failures, which directly improves stability and performance for memory-intensive transformations.

Answer

Increase the number of Spark partitions using repartition(), but keep the same nodes.

Answer

Increase the number of executor cores in the Spark configuration.

Answer

Enable Kryo serialization in the Spark configuration.

Question 2

A data engineer is designing a data ingestion pipeline for real-time user activity logs. The logs are generated by a web application and must be ingested into Amazon S3 with minimal latency (under 1 minute). The logs also need to be queried in Amazon Athena. The engineer considers using Amazon Kinesis Data Firehose. Which TWO configurations are required to achieve near-real-time delivery to S3? (Choose TWO.)

Accepted Answer

Set the BufferIntervalInSeconds to 60 seconds.. Setting BufferIntervalInSeconds to 60 seconds forces Kinesis Data Firehose to deliver data to S3 every 60 seconds, meeting the sub-1-minute latency requirement. Option E is correct because setting BufferSizeInMBs to 1 MB ensures that the buffer fills quickly and triggers delivery when the buffer reaches 1 MB, which, combined with the time-based trigger, minimizes latency.

Answer

Enable S3 compression (e.g., GZIP) on the delivery stream.

Answer

Enable Amazon CloudWatch error logging for the delivery stream.

Answer

Enable data format conversion to Parquet using AWS Glue.

Question 3

A company uses AWS Glue to run ETL jobs that transform data from Amazon S3 (Parquet) into a denormalized format for Amazon Redshift. The Glue job uses the DynamicFrame API. The job is failing with a 'MemoryError' when performing a join operation. The data is skewed on the join key. Which THREE actions can reduce memory usage and improve job stability? (Choose THREE.)

Accepted Answer

Use a broadcast join if one of the tables is small enough.. A broadcast join (using `join` with `broadcast` hint or `DynamicFrame.join(..., transformation_ctx='...')` with broadcast enabled) avoids shuffling the larger table across the cluster by copying the small table to every executor. This eliminates the memory pressure from skewed key distribution during the shuffle phase, reducing the risk of a MemoryError.

Answer

Increase the number of DPUs for the Glue job.

Answer

Repartition the data on the join key before the join operation.

Question 4

A company wants to ingest streaming data from thousands of IoT devices into AWS for real-time analytics. Which AWS service is best suited for this purpose?

Accepted Answer

Amazon Kinesis Data Streams. Amazon Kinesis Data Streams is purpose-built for ingesting and processing streaming data at scale from thousands of sources. It can capture and store terabytes of data per hour from IoT devices, enabling real-time analytics with millisecond latencies. The service provides durable, ordered data streams that can be consumed by multiple applications simultaneously.

Answer

Amazon S3

Answer

AWS Lambda

Answer

Amazon RDS

Question 5

A data engineer needs to transform CSV files arriving in an S3 bucket into Parquet format and store them in another S3 bucket. The transformation is simple and on-demand, triggered by data arrival. Which solution is the MOST cost-effective and requires the least operational overhead?

Accepted Answer

Use S3 Events to trigger an AWS Lambda function that transforms the data. Using S3 Events to trigger an AWS Lambda function is the most cost-effective and operationally lightweight solution for simple, on-demand CSV-to-Parquet transformations. Lambda scales automatically with each S3 PUT event, incurs no idle cost, and requires no cluster management, making it ideal for event-driven, low-volume transformations.

Answer

Use Amazon EMR with Spark streaming

Answer

Use Amazon Athena to create a new table with Parquet format

Answer

Use AWS Glue ETL jobs scheduled to run every hour

Question 6

A company uses Kinesis Data Analytics for SQL-based real-time analytics on streaming data. They notice that the application is processing data slower than the incoming rate, causing increased latency. Which action is MOST likely to improve the throughput?

Accepted Answer

Increase the number of Kinesis Processing Units (KPUs) for the application. Kinesis Data Analytics for SQL applications processes data using Kinesis Processing Units (KPUs), which define the compute and memory resources available. When the incoming data rate exceeds the processing capacity, increasing the number of KPUs directly scales the application's parallelism and throughput, allowing it to keep up with the stream. This is the most direct way to reduce latency caused by insufficient processing power.

Answer

Increase the number of shards in the Kinesis data stream

Answer

Enable auto-scaling on the Kinesis data stream

Answer

Decrease the retention period of the Kinesis data stream

Question 7

An organization needs to ingest data from on-premises databases into AWS S3 for archival purposes. The data volume is several TB per day, and the network has moderate bandwidth. Which AWS service is BEST suited for this bulk data transfer?

Accepted Answer

AWS DataSync. AWS DataSync is the best choice because it is designed for efficiently moving large volumes of data (several TB per day) from on-premises storage to AWS, including S3. It automates and accelerates data transfer over the existing network using a purpose-built protocol and parallel multi-threading, while handling encryption, validation, and scheduling. For moderate bandwidth, DataSync optimizes throughput and can resume interrupted transfers, making it ideal for ongoing archival ingestion.

Answer

AWS Direct Connect

Answer

Amazon S3 Transfer Acceleration

Answer

AWS Snowball

Question 8

A company uses AWS Glue for ETL jobs. The data engineer needs to ensure that the Glue job can access an S3 bucket in another account. What is the recommended approach?

Accepted Answer

Assign an IAM role to the Glue job with permissions to access the bucket, and configure the bucket policy to allow the role. For a Glue job in one account to access an S3 bucket in another account, the standard approach is to grant the Glue job's IAM role permissions to perform S3 actions on the bucket and configure the bucket policy in the target account to allow that role's ARN. Option A is technically possible if the target account's role trusts the Glue job's account and the job has sts:AssumeRole permissions, but this adds unnecessary complexity and is not the recommended method. Option C is incorrect because Glue jobs do not have resource-based policies. Option D is incorrect because Glue jobs cannot use static access keys.

Answer

Create an IAM role in the target account and have the Glue job assume that role

Answer

Configure the S3 bucket policy to allow the Glue job's IAM role and also set the Glue job's resource-based policy

Answer

Store AWS access keys for the target account in AWS Secrets Manager and have the Glue job retrieve them

Question 9

A data streaming application uses Kinesis Data Streams with 10 shards. The data producer is throttled frequently. Which action should be taken to resolve this issue?

Accepted Answer

Increase the number of shards. Throttling in Kinesis Data Streams occurs when the write throughput exceeds the shard limits. Each shard supports up to 1 MB/s or 1,000 records/s for writes. With 10 shards, the total write capacity is 10 MB/s or 10,000 records/s. Increasing the number of shards (Option D) directly increases the write capacity, resolving the throttling issue by distributing the load across more shards.

Answer

Decrease the data retention period

Answer

Use enhanced fan-out for consumers

Answer

Enable server-side encryption

Question 10

A company needs to transform JSON data from an S3 bucket into a structured format for Amazon Redshift. The transformation should be done serverlessly. Which service should be used?

Accepted Answer

AWS Glue. AWS Glue is the correct choice because it is a fully managed, serverless ETL service designed specifically for transforming and preparing data for analytics, including converting JSON to structured formats like Parquet or ORC for Amazon Redshift. It can crawl the S3 source, infer schemas, and run Spark-based transformation jobs without provisioning any infrastructure, aligning perfectly with the serverless requirement.

Answer

Amazon EMR

Answer

Amazon Athena

Answer

AWS Lambda

Question 11

A company is ingesting data from multiple sources into S3 using AWS Glue. The data engineer notices that the Glue job is failing with an OutOfMemory error. Which step should be taken to resolve this issue?

Accepted Answer

Increase the number of DPUs allocated to the Glue job. AWS Glue jobs run on Apache Spark, which distributes data processing across multiple executors. An OutOfMemory error typically indicates that the data being processed exceeds the memory available to the executors. Increasing the number of DPUs (Data Processing Units) allocates more memory and compute resources to the job, allowing it to handle larger datasets without running out of memory.

Answer

Reduce the volume of incoming data

Answer

Configure the job to use a larger memory setting

Answer

Use a smaller file size for input

Question 12

A company needs to ingest real-time clickstream data from a web application into Amazon Redshift with minimal latency. The data volume is high and requires processing before loading. Which architecture is MOST appropriate?

Accepted Answer

Kinesis Data Streams -> Kinesis Data Firehose -> Redshift. D is correct because Kinesis Data Streams captures high-volume clickstream data in real time, and Kinesis Data Firehose can buffer, transform (e.g., with Lambda), and load the data directly into Amazon Redshift with near-zero latency. This architecture is purpose-built for streaming ingestion with minimal overhead, unlike batch or intermediary storage approaches.

Answer

AWS Glue ETL jobs scheduled every 5 minutes -> Redshift

Answer

S3 -> Lambda -> Redshift

Answer

DynamoDB Streams -> Lambda -> Redshift

Question 13

A data engineer is designing a data ingestion pipeline for IoT sensor data. The data is generated at a high velocity and must be processed in near real-time. The pipeline must also handle bursty traffic. Which TWO AWS services should be combined to achieve this? (Choose TWO.)

Accepted Answer

Amazon Kinesis Data Analytics. Amazon Kinesis Data Streams is designed for real-time, high-velocity data ingestion, providing durable, ordered data streams that can handle bursty traffic by scaling shard capacity. Amazon Kinesis Data Analytics can process these streams in near real-time using SQL or Apache Flink, enabling immediate transformations and analytics without needing to store data first.

Answer

Amazon S3

Answer

Amazon Simple Queue Service (SQS)

Answer

AWS Glue

Question 14

A company uses AWS Glue to transform data stored in S3. The Glue job runs daily and processes data in the range of hundreds of GB. The data engineer wants to optimize the job for cost and performance. Which THREE actions should be taken? (Choose THREE.)

Accepted Answer

Increase the number of DPUs for the job. Increasing the number of DPUs (Data Processing Units) for an AWS Glue job can improve performance by enabling parallel processing of large datasets (hundreds of GB). AWS Glue allocates resources in increments of DPUs, where each DPU provides 4 vCPU and 16 GB of memory; scaling out DPUs reduces execution time, which can lower overall cost if the job runs fewer minutes, balancing cost and performance.

Answer

Store intermediate data in HDFS on Amazon EMR

Answer

Reduce the number of DPUs to save cost

Question 15

A data engineer needs to transfer 50 TB of data from an on-premises data center to Amazon S3 over a 1 Gbps network. The transfer must be completed within one week. Which TWO AWS services can be used for this task? (Choose TWO.)

Accepted Answer

AWS DataSync. AWS DataSync is correct because it is designed to efficiently transfer large datasets over the network using a purpose-built agent that parallelizes data transfer and optimizes network utilization. With a 1 Gbps link, DataSync can transfer 50 TB within a week by leveraging its built-in compression, encryption, and incremental transfer capabilities, making it suitable for this time-constrained migration.

Answer

AWS Glue

Answer

Amazon S3 Transfer Acceleration

Answer

AWS Direct Connect