DEA-C01 Data Ingestion and Transformation Practice Test 31 — 15 Questions

Question 1

A company uses AWS Glue to transform data in S3. The transformation job reads Parquet files, filters rows, and writes to another S3 bucket. The job takes longer than expected. Which change would MOST likely reduce the job execution time?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Increase the number of DPUs allocated to the Glue job.. Increasing the number of DPUs (Data Processing Units) allocated to the Glue job directly increases the parallelism of the Apache Spark-based execution environment. Since the job reads Parquet files, filters rows, and writes output, a bottleneck in compute capacity is the most likely cause of prolonged execution time. More DPUs allow Spark to distribute the workload across more executors, reducing overall runtime.

Answer

Use a single large file instead of multiple small files.

Answer

Reduce the number of partitions in the output data.

Answer

Convert the input files from Parquet to CSV format.

Question 2

A data pipeline uses Amazon Kinesis Data Firehose to deliver data to an S3 bucket. The delivery stream is configured with a buffer interval of 60 seconds and a buffer size of 5 MB. The data arrives at an average rate of 2 MB per second. What is the expected time interval between S3 writes?

Accepted Answer

Approximately 2.5 seconds. Amazon Kinesis Data Firehose writes to S3 when either the buffer interval (60 seconds) or buffer size (5 MB) is reached first. With data arriving at 2 MB/s, the 5 MB buffer fills in 2.5 seconds (5 MB / 2 MB/s), triggering a write before the 60-second interval expires. Thus, the expected time between S3 writes is approximately 2.5 seconds.

Answer

Approximately 30 seconds

Answer

Approximately 60 seconds

Answer

Approximately 10 seconds

Question 3

A company wants to ingest real-time data from a social media API into Amazon S3 for analysis. The API provides data as JSON records. Which AWS service is best suited for this ingestion?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the best choice because it is a fully managed service designed to ingest real-time streaming data, such as JSON records from a social media API, and automatically load it into Amazon S3 with optional data transformation and compression. It handles scaling, buffering, and delivery without requiring custom code or infrastructure management, making it ideal for this use case.

Answer

AWS Glue

Answer

Amazon Simple Queue Service (SQS)

Answer

Amazon DataZone

Question 4

A data engineer needs to transform JSON data into Parquet format using AWS Glue. The input data has nested fields. Which Glue feature should be used to flatten the nested structure?

Accepted Answer

Relationalize transform. The Relationalize transform is the correct choice because it is specifically designed to flatten nested JSON structures (such as arrays and structs) into a set of related tables (DataFrames) that can be written as Parquet. This transform recursively extracts nested fields, creating separate DataFrames for each level of nesting, which is essential for converting complex JSON into a flat, columnar Parquet format suitable for analytics.

Answer

DropNullFields transform

Answer

FindMatches transform

Answer

Map transform

Question 5

A company needs to ingest streaming data from thousands of IoT devices. The data must be processed in real-time and stored in Amazon S3. Which TWO services should be used together?

Accepted Answer

Amazon Kinesis Data Streams. Amazon Kinesis Data Streams (A) is correct because it provides a highly scalable, real-time streaming ingestion layer that can continuously capture data from thousands of IoT devices with low latency and durable, ordered record storage across shards. Amazon Kinesis Data Firehose (B) is correct because it is the fully managed delivery service that can consume that streaming data and automatically batch, transform, and load it directly into Amazon S3 without writing custom consumer applications. Together they satisfy the requirement for real-time processing plus reliable storage in S3. AWS Glue (C) is a serverless ETL and data catalog service, not a streaming ingestion or delivery mechanism, so it does not fit this pipeline. Amazon SQS (D) is a message queue for decoupling applications, not a real-time streaming service designed for high-throughput IoT telemetry or direct S3 delivery. AWS Direct Connect (E) is a dedicated network connection from on-premises to AWS, not a data streaming or processing service.

Answer

AWS Glue

Answer

Amazon Simple Queue Service (SQS)

Answer

AWS Direct Connect

Question 6

A data engineer needs to transform data in Amazon S3 using AWS Glue. The job must handle schema evolution and partition pruning. Which THREE features should be used?

Accepted Answer

AWS Glue Data Catalog. AWS Glue Data Catalog (A) is correct because it stores the table definitions, schemas, and partition metadata that Glue ETL jobs reference, and it is the component that must be updated when schema evolution occurs so the job can read new columns. AWS Glue crawlers (D) are correct because they automatically scan S3 data, detect new or changed columns, and update the Data Catalog with revised schemas and newly discovered partitions, which is exactly how schema evolution is handled. Partition indexes (E) are correct because they accelerate partition pruning by allowing Glue and Athena to look up partitions without listing the entire catalog, dramatically reducing planning time for highly partitioned tables. AWS Glue job bookmarks (B) only track previously processed data to support incremental loads and do not address schema evolution or partition pruning. AWS Glue FindMatches transform (C) is a machine-learning transform for deduplicating and matching records, which is unrelated to schema evolution or partition pruning.

Answer

AWS Glue job bookmarks

Answer

AWS Glue FindMatches transform

Question 7

A data engineer is ingesting streaming data from thousands of IoT devices into AWS. The data is JSON-formatted and must be stored in Amazon S3 for long-term analytics. Which service is most appropriate for real-time ingestion and routing to S3?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the most appropriate service because it is designed for real-time ingestion of streaming data and can directly deliver data to Amazon S3 without requiring custom code. It automatically handles buffering, compression, and partitioning of JSON data, making it ideal for long-term analytics storage.

Answer

Amazon SQS

Answer

Amazon Kinesis Data Streams

Answer

AWS Glue

Question 8

A company runs a SQL Server transactional database on Amazon RDS. They need to capture change data (inserts, updates, deletes) in near real-time and replicate them to an Amazon S3 data lake. Which AWS service is most suitable?

Accepted Answer

AWS Database Migration Service (DMS) with change data capture. AWS DMS with change data capture (CDC) is the most suitable service because it can continuously capture and replicate incremental changes (inserts, updates, deletes) from a SQL Server transactional database on Amazon RDS to an S3 data lake in near real-time. DMS uses native SQL Server transaction logs (e.g., MS-CDC or log-based replication) to read changes without impacting source performance, and it supports target S3 in formats like Parquet or CSV. This directly meets the requirement for near-real-time CDC replication to a data lake.

Answer

AWS Glue DataBrew

Answer

Amazon Kinesis Data Streams with Kinesis Client Library

Answer

Amazon Redshift Spectrum

Question 9

A data engineer needs to ingest data from an on-premises Apache Kafka cluster into Amazon S3. The data volume is about 10 TB per day. The engineer wants to set up a managed Kafka connector. Which AWS service should they use?

Accepted Answer

Amazon MSK Connect. Amazon MSK Connect is a managed Kafka connector service that integrates with Amazon MSK or self-managed Apache Kafka clusters to stream data into Amazon S3 using Kafka Connect. It handles the 10 TB/day volume efficiently with auto-scaling and checkpointing, making it the correct choice for a managed connector setup.

Answer

AWS Database Migration Service

Answer

AWS Lambda with Kafka trigger

Answer

Amazon Kinesis Data Streams

Question 10

A data engineer needs to ingest data from an Amazon S3 bucket into an Amazon Redshift table on a daily schedule. The data is in CSV format and the schema matches. Which service is simplest for this batch ingestion?

Accepted Answer

Amazon Redshift COPY command. The Amazon Redshift COPY command is the simplest and most efficient method for batch loading data from Amazon S3 into Redshift when the schema matches and the data is in CSV format. It leverages Redshift's massively parallel processing (MPP) architecture to read data directly from S3, automatically handling compression, encryption, and error logging without requiring any intermediate services or custom code.

Answer

AWS Glue ETL job with JDBC connection

Answer

AWS Data Pipeline

Answer

Amazon Athena CREATE TABLE AS SELECT

Question 11

A company is ingesting Apache logs from multiple web servers into AWS. The logs are sent via Amazon CloudWatch Logs to a subscription filter that delivers to a Lambda function. The Lambda function parses the logs and writes to Amazon S3. However, there is a significant backlog. Which THREE actions can reduce the backlog?

Accepted Answer

Increase the Lambda function memory allocation. Increasing the Lambda function's memory allocation also increases its CPU allocation, allowing the function to process each log event faster. This reduces the per-invocation processing time, enabling the function to handle more log data per unit time and thus reduce the backlog.

Answer

Route the CloudWatch Logs subscription to an Amazon SQS queue first

Answer

Change the Lambda function runtime from Python to Node.js

Question 12

Refer to the exhibit. An IAM policy for an AWS Lambda function. The Lambda function is triggered by an S3 event (object created) and needs to read from a Kinesis stream. However, the function fails with access denied when trying to read from Kinesis. What is the most likely cause?

Accepted Answer

The Kinesis stream is encrypted with a customer managed KMS key, and the Lambda function lacks kms:Decrypt permission. When a Kinesis stream is encrypted with a customer managed KMS key, the Lambda function must have the `kms:Decrypt` permission on that key to read data from the stream. Without this permission, the Lambda function will receive an access denied error even if it has the necessary Kinesis actions (e.g., `kinesis:GetRecords`) allowed in its IAM policy. The S3 event trigger only invokes the function; it does not grant Kinesis access.

Answer

The Lambda function is not in the same region as the Kinesis stream

Answer

The Lambda function does not have permission to list S3 buckets

Answer

The S3 bucket policy denies access to the Lambda function

Question 13

Refer to the exhibit. A CloudFormation stack outputs the Glue job name and S3 bucket names. The Glue job transforms CSV files from the raw bucket to Parquet in the processed bucket. However, the Glue job is failing with an error that it cannot write to the processed bucket. What is the most likely cause?

Accepted Answer

The Glue job does not have permission to write to the processed bucket. The most likely cause is that the Glue job's IAM role lacks the necessary permissions (e.g., s3:PutObject, s3:ListBucket) on the processed bucket. AWS Glue jobs require an IAM role with policies that grant write access to the target S3 bucket; without these permissions, the job fails with a write error. This is a common misconfiguration when the role is scoped only to read from the raw bucket.

Answer

The raw data bucket is in a different region

Answer

The Glue job is not using the correct worker type

Answer

The Glue job is using an incorrect file format

Question 14

Refer to the exhibit. A Lambda function named 'IngestionProcessor' is failing. The engineer checks CloudWatch Logs and sees the log group exists but storedBytes is 0. Why might the logs show no data?

Accepted Answer

The Lambda execution role does not have permission to write logs to CloudWatch. The Lambda execution role must have the `logs:CreateLogStream` and `logs:PutLogEvents` permissions to write logs to CloudWatch Logs. If the role lacks these permissions, the log group will be created (if it doesn't exist) but no log events will be written, resulting in `storedBytes` being 0. This is a common misconfiguration when the IAM policy does not include the necessary CloudWatch Logs actions.

Answer

The Lambda function is configured with a dead letter queue

Answer

The Lambda function has not been invoked yet

Answer

The log group is encrypted with a KMS key and the Lambda function lacks decrypt permission

Question 15

A company needs to ingest real-time clickstream data from a web application into Amazon S3 for analytics. The data must be available within minutes of generation. Which AWS service should be used to capture and deliver this streaming data?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service designed to capture, transform, and load streaming data into Amazon S3, Redshift, Elasticsearch, or Splunk in near real-time (typically within 60 seconds). It directly addresses the requirement for ingesting real-time clickstream data and delivering it to S3 within minutes, without requiring custom code or manual scaling.

Answer

Amazon RDS

Answer

AWS Glue

Answer

Amazon Simple Queue Service (SQS)