DEA-C01 Data Ingestion and Transformation Practice Test 25 — 15 Questions

Question 1

A data engineer is designing a data lake on Amazon S3. The data ingestion pipeline must handle both batch and streaming data. The engineer wants to use a single service to ingest both types of data. Which service should the engineer choose?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service that can ingest both batch and streaming data and deliver it directly to Amazon S3 without requiring custom code. It can receive streaming data from sources like Kinesis Data Streams or Amazon CloudWatch Logs, and also handle batch data via API calls, making it a single ingestion point for both patterns. While AWS Glue does support both batch and streaming ETL, it is primarily an ETL service that requires writing and managing jobs, and is not designed as a direct data ingestion service to S3 without additional configuration. Therefore, Kinesis Data Firehose is the most appropriate service for this requirement.

Answer

Amazon Athena

Answer

S3 Transfer Acceleration

Answer

AWS Glue

Question 2

A company uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data is transformed using an AWS Lambda function. Some records fail transformation and are lost because the Lambda function throws an exception. The data engineer needs to capture the failed records for analysis without affecting the pipeline. What should the engineer do?

Accepted Answer

Configure the Firehose delivery stream to send failed records to a backup S3 bucket. Amazon Kinesis Data Firehose natively supports a 'backup' or 'error output' configuration: when a Lambda transformation fails (e.g., throws an exception), Firehose can automatically route the failed records to a designated Amazon S3 bucket for failed data. This allows the engineer to capture and analyze the failed records without blocking or altering the main delivery pipeline, as the stream continues processing successfully transformed records.

Answer

Increase the buffer size of the Firehose stream

Answer

Disable the Lambda transformation and process all records in batch later

Answer

Modify the Lambda function to write failed records to Amazon DynamoDB

Question 3

A data engineer is designing a data ingestion pipeline for streaming data from IoT devices. The devices send JSON messages every second. The engineer needs to ingest the data with low latency and store it in Amazon S3 in Parquet format. Which TWO services should the engineer use together?

Accepted Answer

Amazon Kinesis Data Streams. The correct answers are C and E. Amazon Kinesis Data Streams (KDS) provides low-latency, real-time ingestion of streaming data from IoT devices, handling JSON messages sent every second. Amazon Kinesis Data Firehose (KDF) can then consume data from KDS, automatically convert it to Parquet format, and deliver it to Amazon S3. This combination meets the low-latency and Parquet conversion requirements without custom code. Option A (Lambda) could transform data but requires additional management and is not necessary for Parquet conversion as Firehose handles it natively. Option B (Athena) is a query service, not for ingestion. Option D (Glue) is batch-oriented and not suited for low-latency streaming.

Answer

AWS Lambda

Answer

Amazon Athena

Answer

AWS Glue

Question 4

A data engineer is building a pipeline to ingest data from an on-premises Oracle database into Amazon S3. The pipeline must capture change data (CDC) in near real-time and handle schema changes. Which TWO AWS services should the engineer use?

Accepted Answer

AWS Glue Schema Registry. AWS Glue Schema Registry (A) is correct because it enables schema discovery, validation, and evolution for streaming data, allowing the pipeline to handle schema changes from the Oracle CDC source. It integrates with Apache Kafka and Amazon Kinesis Data Streams to enforce schema compatibility rules (e.g., backward, forward, full) as data arrives, ensuring downstream consumers can adapt to evolving schemas without breaking.

Answer

AWS Snowball Edge

Answer

Amazon AppFlow

Answer

Amazon Kinesis Data Streams with Kinesis Agent

Question 5

A company uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data engineer needs to transform the data before delivery. Which THREE options can be used to perform the transformation?

Accepted Answer

Amazon Kinesis Data Firehose data format conversion (e.g., JSON to Parquet). Amazon Kinesis Data Firehose can transform data natively using data format conversion (e.g., converting JSON to Parquet or ORC) without requiring external services. This is a built-in capability that applies schema-based conversion using AWS Glue tables, enabling efficient storage and querying in Amazon S3.

Answer

Amazon Athena queries

Answer

AWS Glue ETL job

Question 6

Refer to the exhibit. An IAM policy is attached to an AWS Glue ETL job. The job reads from the Kinesis stream 'input-stream' and writes to S3 bucket 'data-lake-bucket'. The job fails with an access denied error. Which missing permission is most likely the cause?

Accepted Answer

s3:ListBucket permission on the bucket. The AWS Glue ETL job fails with an access denied error because it lacks the s3:ListBucket permission on the 'data-lake-bucket'. When writing to S3, the job must first list the bucket to verify the target prefix exists and to handle multipart uploads; without this permission, the write operation fails even if s3:PutObject is granted.

Answer

kinesis:PutRecord permission on a wildcard stream ARN

Answer

kinesis:DescribeStream permission

Answer

s3:PutObject permission on a specific prefix

Question 7

Refer to the exhibit. A data engineer is using a Kinesis Data Stream with one shard. The application writes 2000 records per second, each 1 KB. The put record calls are frequently throttled. What is the most likely cause?

Accepted Answer

The stream has only one shard, which limits writes to 1000 records per second. A Kinesis Data Stream shard has a write throughput limit of 1,000 records per second (or 1 MB per second). Since the application is writing 2,000 records per second (each 1 KB) to a single shard, it exceeds the shard's record-per-second quota, causing the PutRecord calls to be throttled. The solution is to increase the number of shards to at least two to distribute the load.

Answer

The retention period of 24 hours is too short

Answer

The stream uses KMS encryption, causing additional latency

Answer

Enhanced monitoring is not enabled, causing performance issues

Question 8

Refer to the exhibit. An S3 event notification is configured to trigger an AWS Lambda function when objects are created in 'my-bucket'. The Lambda function processes the JSON file and writes results to Amazon DynamoDB. The function fails with a timeout error. Which action should the engineer take to resolve the issue?

Accepted Answer

Increase the Lambda function timeout. The Lambda function is failing with a timeout error, which indicates that the function is taking longer to execute than the default timeout of 3 seconds. Increasing the Lambda function timeout allows the function to run longer and complete its processing of the JSON file and DynamoDB write operation without being prematurely terminated.

Answer

Modify the S3 event notification to use a different event type

Answer

Grant the Lambda function permission to access DynamoDB

Answer

Change the trigger to Amazon SQS instead of S3

Question 9

A data engineering team is ingesting streaming data from IoT devices using AWS IoT Core and needs to process the data in near real-time with minimal code. Which AWS service should they use to transform the data before storing it in Amazon S3?

Accepted Answer

Amazon Kinesis Data Analytics. Amazon Kinesis Data Analytics (now part of Amazon Managed Service for Apache Flink) is the correct choice because it allows you to transform streaming data in near real-time using SQL or Apache Flink with minimal code. It can directly consume data from AWS IoT Core via Kinesis Data Streams or Amazon MSK, apply transformations like filtering, aggregation, or enrichment, and then output the processed data to Amazon S3 without requiring custom application servers.

Answer

AWS Glue

Answer

Amazon Redshift

Answer

Amazon Athena

Question 10

A company uses Amazon S3 to store raw data and needs to transform it into Parquet format for analytics. The transformation job runs daily on a schedule. Which AWS service is BEST suited for this task?

Accepted Answer

AWS Glue. AWS Glue is a fully managed, serverless ETL service that can automatically convert data formats (e.g., from CSV to Parquet) and run on a schedule (e.g., daily). It is ideal for this use case because it is purpose-built for ETL transformations and handles schema discovery, data cataloging, and job scheduling without managing infrastructure. Option A (Amazon Redshift) is wrong because Redshift is a data warehouse for querying, not a transformation service; it could load Parquet but not convert raw data to Parquet directly. Option B (Amazon EMR) is wrong because EMR requires provisioning and managing clusters, adding operational overhead. Option C (AWS Lambda) is wrong because Lambda has a maximum execution timeout of 15 minutes, which is too short for daily large-scale data transformation jobs.

Answer

Amazon Redshift

Answer

Amazon EMR

Answer

AWS Lambda

Question 11

A company is ingesting data from multiple on-premises databases into AWS using AWS Database Migration Service (DMS). The data must be continuously replicated with minimal downtime. However, the source databases do not support native CDC. What should the data engineer do to enable continuous replication?

Accepted Answer

Use AWS DMS with log-based CDC if the source databases support it; otherwise, use DMS with batch replication and schedule frequent refreshes.. When source databases do not support native CDC, AWS DMS can still achieve continuous replication by using log-based CDC if the database engine supports it (e.g., Oracle, SQL Server, MySQL, PostgreSQL). If log-based CDC is not supported, DMS batch replication with frequent scheduled refreshes provides near-continuous replication by periodically extracting full or incremental changes, minimizing downtime. This approach leverages DMS's built-in task settings for change data capture and resumption without requiring external streaming services.

Answer

Use Amazon Kinesis Data Streams with a custom producer to capture database changes.

Answer

Use Amazon Redshift Spectrum to directly query the on-premises databases.

Answer

Set up AWS Glue jobs to run every minute to extract and load the data.

Question 12

A data engineering team is designing a data ingestion pipeline that will receive millions of small JSON files per hour from external partners via API. The files should be stored in Amazon S3 and then transformed into Parquet for querying. Which approach is MOST cost-effective and scalable?

Accepted Answer

Use Amazon Kinesis Data Firehose to buffer and deliver data to S3, then use AWS Glue to convert to Parquet.. Amazon Kinesis Data Firehose is the most cost-effective and scalable approach because it can buffer millions of small JSON files per hour, automatically batch them, and deliver them to S3 without requiring any server management. After delivery, AWS Glue can efficiently convert the JSON data to Parquet format for optimized querying, leveraging its serverless, pay-per-use model that scales with data volume.

Answer

Use AWS Lambda to process each file as it arrives and write to S3.

Answer

Use AWS Direct Connect to establish a dedicated network for file uploads.

Answer

Use Amazon EMR to process the files as they arrive in S3.

Question 13

A company wants to import data from an external FTP server into Amazon S3 on a daily basis. The data volumes are moderate. Which AWS service is MOST suitable for this task?

Accepted Answer

AWS Transfer Family. AWS Transfer Family is the most suitable service because it provides fully managed support for SFTP, FTPS, and FTP protocols, enabling direct, secure file transfers from an external FTP server to Amazon S3 without needing to manage any infrastructure. It integrates natively with S3, so files are automatically stored in the specified bucket upon transfer completion, making it ideal for daily imports of moderate data volumes.

Answer

Amazon S3 Transfer Acceleration

Answer

AWS DataSync

Answer

AWS Glue with a JDBC connection

Question 14

A data engineer is troubleshooting a slow AWS Glue ETL job that reads from Amazon S3 and writes to Amazon Redshift. The job processes 10 GB of CSV data. The engineer notices that the job runs with a single DPU and takes longer than expected. Which change would MOST likely improve performance?

Accepted Answer

Increase the number of DPUs and configure the job to use the S3 list implementation for parallel reads.. The job runs with a single DPU, which limits parallelism. Increasing the number of DPUs allows the job to process data in parallel across multiple workers. Additionally, configuring the S3 list implementation enables the job to list objects in parallel, reducing the overhead of discovering input files. This combination directly addresses the bottleneck of a single-threaded read from S3.

Answer

Replace Redshift with Amazon Redshift Spectrum.

Answer

Change the input format to Parquet and enable predicate pushdown.

Answer

Use a JDBC connection to read data directly from S3.

Question 15

A company uses Amazon Kinesis Data Streams to ingest clickstream data from a website. The data must be transformed (e.g., enrich with user location) before being stored in Amazon S3. Which architecture is MOST efficient for this transformation?

Accepted Answer

Use Amazon Kinesis Data Analytics to transform the stream and output to Amazon Kinesis Data Firehose, which writes to S3.. Amazon Kinesis Data Analytics (KDA) can perform real-time transformations (e.g., enriching clickstream data with user location via SQL or Flink) on the stream, then output the transformed data to Kinesis Data Firehose, which can batch and compress records before writing to S3. This architecture minimizes operational overhead and is purpose-built for streaming transformations, avoiding the latency and complexity of Lambda cold starts or the provisioning overhead of Glue/EMR.

Answer

Use AWS Glue to run a streaming ETL job.

Answer

Use Amazon EMR to consume the stream using Spark Streaming.

Answer

Use AWS Lambda to process each record from the stream and write to S3.