DEA-C01 Data Ingestion and Transformation Practice Test 12 — 15 Questions

Question 1

A data engineering team is ingesting streaming data from IoT devices into Amazon Kinesis Data Streams. The data must be transformed in real-time and then loaded into an Amazon S3 bucket for long-term storage. Which AWS service should be used to perform the transformation and delivery to S3 with minimal operational overhead?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service designed to automatically load streaming data into Amazon S3, Redshift, Elasticsearch, and Splunk. It can invoke an AWS Lambda function for real-time data transformation before delivery, eliminating the need to manage any infrastructure or write custom code for the delivery pipeline. This directly meets the requirement of minimal operational overhead for transformation and S3 delivery.

Answer

AWS Glue

Answer

Amazon EMR

Answer

Amazon Kinesis Data Analytics

Question 2

A company uses AWS DMS to migrate a 2 TB PostgreSQL database to Amazon Aurora PostgreSQL. The migration is taking longer than expected due to the initial load. Which AWS service can be used to accelerate the initial load by transferring the database files directly?

Accepted Answer

AWS Snowball. AWS Snowball is a petabyte-scale data transport solution that uses physical storage devices to transfer large amounts of data into and out of AWS. For a 2 TB PostgreSQL database, the initial load via DMS over the network can be slow due to bandwidth constraints, especially for large datasets. By using Snowball, you can export the database files (e.g., using pg_dump or physical file copy) to the device, ship it to AWS, and have the data loaded directly into Amazon Aurora PostgreSQL, bypassing the network bottleneck and significantly accelerating the initial load.

Answer

Amazon S3 Transfer Acceleration

Answer

AWS Direct Connect

Answer

Amazon Kinesis Data Firehose

Question 3

A data engineering team is ingesting data from multiple sources into Amazon S3 using AWS Glue ETL jobs. The jobs are failing intermittently with the error: 'Task ran out of memory'. The input data size varies widely from 100 MB to 10 GB per job. Which configuration change would best mitigate this issue?

Accepted Answer

Enable job bookmarking to process only incremental data. Enabling job bookmarking allows the Glue ETL job to process only incremental (new or changed) data, which directly addresses the intermittent out-of-memory errors caused by widely varying input sizes (100 MB to 10 GB). By skipping previously processed data, the job consistently handles smaller data volumes per run, reducing memory pressure on the Spark executors.

Answer

Increase the number of workers in the Glue job

Answer

Reduce the batch size in the S3 source node

Answer

Change the job type from Spark to Python shell

Question 4

A data engineering team is designing a data ingestion pipeline for a social media analytics platform. The pipeline must handle up to 100,000 events per second with less than 1 second processing latency. Which TWO services should be used together to meet these requirements?

Accepted Answer

Amazon Kinesis Data Analytics for Apache Flink. Amazon Kinesis Data Streams (Option E) is designed for real-time data ingestion at scale, supporting up to 1,000 records per second per shard with sub-second latency, making it suitable for 100,000 events per second when provisioned with sufficient shards. Amazon Kinesis Data Analytics for Apache Flink (Option D) can consume data directly from Kinesis Data Streams and perform low-latency stream processing (e.g., aggregations, filtering) with exactly-once semantics, meeting the <1 second processing latency requirement. Together, they form a fully managed, scalable pipeline for high-throughput, low-latency streaming analytics.

Answer

AWS Glue streaming ETL

Answer

Amazon Kinesis Data Firehose

Answer

Amazon SQS

Question 5

A company is migrating its data warehouse from on-premises to Amazon Redshift. The migration involves copying 50 TB of data from an S3 bucket to Redshift. The network bandwidth is limited to 1 Gbps. Which TWO approaches should the team use to complete the transfer within 7 days?

Accepted Answer

Use Amazon S3 Transfer Acceleration. Amazon S3 Transfer Acceleration (option A) uses AWS edge locations to accelerate uploads over the public internet by routing traffic through optimized paths, which can significantly improve transfer speeds for large datasets when bandwidth is limited. With 1 Gbps bandwidth, the theoretical maximum transfer for 50 TB over 7 days is approximately 75.6 TB (1 Gbps * 7 days * 86400 seconds/day / 8 bits/byte), so the raw bandwidth is sufficient, but Transfer Acceleration helps overcome latency and packet loss issues that can reduce effective throughput. This makes it a valid approach to ensure the transfer completes within the time window.

Answer

Use AWS Direct Connect with 10 Gbps

Answer

Use AWS Lambda to copy data in parallel

Answer

Use Amazon Kinesis Data Firehose

Question 6

A data engineer needs to ingest JSON files from an Amazon S3 bucket into an Amazon DynamoDB table. The files are uploaded every hour. Which THREE services can be used together to build this ingestion pipeline?

Accepted Answer

Amazon SQS. Amazon SQS is correct because it decouples the ingestion pipeline, allowing S3 Event Notifications to send messages to an SQS queue when new JSON files arrive. AWS Lambda can then poll the SQS queue to process the files and write to DynamoDB, ensuring reliable, asynchronous ingestion without data loss.

Answer

AWS Step Functions

Answer

Amazon DynamoDB Streams

Question 7

A company receives streaming clickstream data from its website. The data must be ingested with low latency and transformed in real time before being stored in Amazon S3. Which AWS service combination is most suitable for this use case?

Accepted Answer

Amazon Kinesis Data Streams with Amazon Kinesis Data Analytics. Amazon Kinesis Data Streams ingests streaming clickstream data with low latency, and Amazon Kinesis Data Analytics performs real-time transformations using SQL or Apache Flink. The processed data can then be stored in Amazon S3 via a Kinesis Data Firehose delivery stream, meeting the requirement for low-latency ingestion and real-time transformation.

Answer

Amazon S3 with S3 Object Lambda

Answer

Amazon Kinesis Data Firehose with AWS Lambda for transformation

Answer

AWS Glue jobs triggered by Amazon S3 events

Question 8

A data engineer needs to ingest data from an Amazon RDS for PostgreSQL database into Amazon S3 on a daily basis. The data volume is approximately 500 GB per day. Which service is most appropriate for this task?

Accepted Answer

AWS Glue with a scheduled ETL job. AWS Glue with a scheduled ETL job is the most appropriate choice because it provides a fully managed, serverless ETL service that can efficiently extract 500 GB of data daily from Amazon RDS for PostgreSQL and write it to Amazon S3. Glue can handle large volumes via its distributed Spark-based execution, and scheduling ensures the daily cadence without manual intervention.

Answer

AWS Database Migration Service (DMS) with continuous replication

Answer

Amazon Athena with federated query to RDS

Answer

Amazon EMR with Spark job

Question 9

A company uses AWS Glue to transform data in Amazon S3. The transformation logic is complex and involves multiple steps. The data engineer wants to implement a workflow that handles dependencies and retries on failure. Which AWS service should be used to orchestrate the Glue jobs?

Accepted Answer

AWS Step Functions. AWS Step Functions is a serverless orchestration service that allows you to coordinate multiple AWS services, including AWS Glue jobs, into a workflow. It provides built-in error handling, retries, and dependency management, making it ideal for orchestrating complex ETL processes with multiple steps.

Answer

AWS Lambda

Answer

Amazon Managed Workflows for Apache Airflow (MWAA)

Answer

Amazon CloudWatch Events

Question 10

A data engineer is designing a data lake on Amazon S3. The data ingestion pipeline must handle both batch and streaming data. The engineer wants to use a single service to ingest both types of data. Which service should the engineer choose?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service that can ingest both batch and streaming data and deliver it directly to Amazon S3 without requiring custom code. It can receive streaming data from sources like Kinesis Data Streams or Amazon CloudWatch Logs, and also handle batch data via API calls, making it a single ingestion point for both patterns. While AWS Glue does support both batch and streaming ETL, it is primarily an ETL service that requires writing and managing jobs, and is not designed as a direct data ingestion service to S3 without additional configuration. Therefore, Kinesis Data Firehose is the most appropriate service for this requirement.

Answer

Amazon Athena

Answer

S3 Transfer Acceleration

Answer

AWS Glue

Question 11

A data engineer is designing a data ingestion pipeline for streaming data from IoT devices. The devices send JSON messages every second. The engineer needs to ingest the data with low latency and store it in Amazon S3 in Parquet format. Which TWO services should the engineer use together?

Accepted Answer

Amazon Kinesis Data Streams. Amazon Kinesis Data Firehose (E) is correct because it is a fully managed streaming delivery service that can ingest high-throughput JSON messages, buffer them, and automatically convert the data to Parquet format before delivering it to Amazon S3, satisfying both the low-latency ingestion and Parquet storage requirements. Amazon Kinesis Data Streams (C) is correct because it provides a low-latency, real-time streaming ingestion layer for the IoT device messages, allowing the data to be captured durably and processed before being delivered to S3. Together, Kinesis Data Streams can feed Kinesis Data Firehose, which handles the Parquet conversion and S3 delivery. AWS Lambda (A) is not marked correct here because although it can process streaming records, it does not natively convert data to Parquet or deliver it to S3 as a managed pipeline. Amazon Athena (B) is a query service for data in S3, not an ingestion service. AWS Glue (D) is a serverless ETL and catalog service that can convert formats, but it is not the primary low-latency streaming ingestion service for this scenario.

Answer

AWS Lambda

Answer

Amazon Athena

Answer

AWS Glue

Question 12

A company uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data engineer needs to transform the data before delivery. Which THREE options can be used to perform the transformation?

Accepted Answer

Amazon Kinesis Data Firehose data format conversion (e.g., JSON to Parquet). Amazon Kinesis Data Firehose can transform data natively using data format conversion (e.g., converting JSON to Parquet or ORC) without requiring external services. This is a built-in capability that applies schema-based conversion using AWS Glue tables, enabling efficient storage and querying in Amazon S3.

Answer

Amazon Athena queries

Answer

AWS Glue ETL job

Question 13

Refer to the exhibit. An IAM policy is attached to an AWS Glue ETL job. The job reads from the Kinesis stream 'input-stream' and writes to S3 bucket 'data-lake-bucket'. The job fails with an access denied error. Which missing permission is most likely the cause?

Accepted Answer

s3:ListBucket permission on the bucket. The AWS Glue ETL job fails with an access denied error because it lacks the s3:ListBucket permission on the 'data-lake-bucket'. When writing to S3, the job must first list the bucket to verify the target prefix exists and to handle multipart uploads; without this permission, the write operation fails even if s3:PutObject is granted.

Answer

kinesis:PutRecord permission on a wildcard stream ARN

Answer

kinesis:DescribeStream permission

Answer

s3:PutObject permission on a specific prefix

Question 14

A data engineering team is ingesting streaming data from IoT devices using AWS IoT Core and needs to process the data in near real-time with minimal code. Which AWS service should they use to transform the data before storing it in Amazon S3?

Accepted Answer

Amazon Kinesis Data Analytics. Amazon Kinesis Data Analytics (now part of Amazon Managed Service for Apache Flink) is the correct choice because it allows you to transform streaming data in near real-time using SQL or Apache Flink with minimal code. It can directly consume data from AWS IoT Core via Kinesis Data Streams or Amazon MSK, apply transformations like filtering, aggregation, or enrichment, and then output the processed data to Amazon S3 without requiring custom application servers.

Answer

AWS Glue

Answer

Amazon Redshift

Answer

Amazon Athena

Question 15

A company is ingesting data from multiple on-premises databases into AWS using AWS Database Migration Service (DMS). The data must be continuously replicated with minimal downtime. However, the source databases do not support native CDC. What should the data engineer do to enable continuous replication?

Accepted Answer

Use AWS DMS with log-based CDC if the source databases support it; otherwise, use DMS with batch replication and schedule frequent refreshes.. When source databases do not support native CDC, AWS DMS can still achieve continuous replication by using log-based CDC if the database engine supports it (e.g., Oracle, SQL Server, MySQL, PostgreSQL). If log-based CDC is not supported, DMS batch replication with frequent scheduled refreshes provides near-continuous replication by periodically extracting full or incremental changes, minimizing downtime. This approach leverages DMS's built-in task settings for change data capture and resumption without requiring external streaming services.

Answer

Use Amazon Kinesis Data Streams with a custom producer to capture database changes.

Answer

Use Amazon Redshift Spectrum to directly query the on-premises databases.

Answer

Set up AWS Glue jobs to run every minute to extract and load the data.