DEA-C01 Data Ingestion and Transformation Practice Test 10 — 15 Questions

Question 1

A data engineer is ingesting streaming data from thousands of IoT devices into AWS. The data is JSON-formatted and must be stored in Amazon S3 for long-term analytics. Which service is most appropriate for real-time ingestion and routing to S3?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the most appropriate service because it is designed for real-time ingestion of streaming data and can directly deliver data to Amazon S3 without requiring custom code. It automatically handles buffering, compression, and partitioning of JSON data, making it ideal for long-term analytics storage.

Answer

Amazon SQS

Answer

Amazon Kinesis Data Streams

Answer

AWS Glue

Question 2

A company runs a SQL Server transactional database on Amazon RDS. They need to capture change data (inserts, updates, deletes) in near real-time and replicate them to an Amazon S3 data lake. Which AWS service is most suitable?

Accepted Answer

AWS Database Migration Service (DMS) with change data capture. AWS DMS with change data capture (CDC) is the most suitable service because it can continuously capture and replicate incremental changes (inserts, updates, deletes) from a SQL Server transactional database on Amazon RDS to an S3 data lake in near real-time. DMS uses native SQL Server transaction logs (e.g., MS-CDC or log-based replication) to read changes without impacting source performance, and it supports target S3 in formats like Parquet or CSV. This directly meets the requirement for near-real-time CDC replication to a data lake.

Answer

AWS Glue DataBrew

Answer

Amazon Kinesis Data Streams with Kinesis Client Library

Answer

Amazon Redshift Spectrum

Question 3

A data engineer is designing a data pipeline that ingests CSV files from an FTP server to Amazon S3. The files arrive hourly and each file is about 500 MB. The engineer wants to minimize operational overhead and cost. Which approach is best?

Accepted Answer

Use AWS Transfer for SFTP and point the endpoint to an S3 bucket. AWS Transfer for SFTP provides a fully managed FTP service that writes directly to S3, eliminating the need to manage servers. Lambda with boto3 is code-heavy; EC2 requires management; Snowball is for large offline transfers.

Answer

Write a Python script in AWS Lambda using boto3 to download from FTP and upload to S3

Answer

Use AWS Snowball Edge to transfer files weekly

Answer

Deploy an Amazon EC2 instance with a cron job to run wget and aws s3 cp

Question 4

A company needs to transform JSON data from Amazon Kinesis Data Streams into Parquet format and store it in Amazon S3. The transformation includes simple field mappings and type conversions. Which approach is most cost-effective and serverless?

Accepted Answer

Use Amazon Kinesis Data Firehose with an AWS Lambda function for transformation and output to Parquet. Amazon Kinesis Data Firehose can directly deliver streaming data to Amazon S3 in Parquet format. By attaching an AWS Lambda function for simple field mappings and type conversions, you achieve a fully serverless, cost-effective solution without managing any infrastructure. This approach minimizes costs because you pay only for the data volume processed by Firehose and the Lambda invocations, avoiding the overhead of always-on compute resources.

Answer

Use Amazon EC2 instances running Apache Spark Streaming

Answer

Use Amazon SageMaker for transformation

Answer

Use an AWS Glue ETL job triggered by a Kinesis stream

Question 5

A data engineer needs to ingest data from an on-premises Apache Kafka cluster into Amazon S3. The data volume is about 10 TB per day. The engineer wants to set up a managed Kafka connector. Which AWS service should they use?

Accepted Answer

Amazon MSK Connect. Amazon MSK Connect is a managed Kafka connector service that integrates with Amazon MSK or self-managed Apache Kafka clusters to stream data into Amazon S3 using Kafka Connect. It handles the 10 TB/day volume efficiently with auto-scaling and checkpointing, making it the correct choice for a managed connector setup.

Answer

AWS Database Migration Service

Answer

AWS Lambda with Kafka trigger

Answer

Amazon Kinesis Data Streams

Question 6

A data engineer needs to ingest data from an Amazon S3 bucket into an Amazon Redshift table on a daily schedule. The data is in CSV format and the schema matches. Which service is simplest for this batch ingestion?

Accepted Answer

Amazon Redshift COPY command. The Amazon Redshift COPY command is the simplest and most efficient method for batch loading data from Amazon S3 into Redshift when the schema matches and the data is in CSV format. It leverages Redshift's massively parallel processing (MPP) architecture to read data directly from S3, automatically handling compression, encryption, and error logging without requiring any intermediate services or custom code.

Answer

AWS Glue ETL job with JDBC connection

Answer

AWS Data Pipeline

Answer

Amazon Athena CREATE TABLE AS SELECT

Question 7

A company is ingesting streaming data from a fleet of weather sensors. Each sensor sends a JSON payload every second. The data is used for real-time dashboarding and also archived to S3. The pipeline should handle sudden bursts of data without data loss. Which architecture meets these requirements?

Accepted Answer

Amazon Kinesis Data Streams -> AWS Lambda for dashboard -> Amazon Kinesis Data Firehose -> S3. Kinesis Data Streams provides durable, scalable ingestion that can handle sudden bursts of data without loss, while Lambda processes records for real-time dashboarding and Kinesis Data Firehose reliably buffers and archives data to S3. This decoupled architecture ensures no data is lost even during traffic spikes, as Kinesis Data Streams retains data for up to 365 days and Firehose can buffer incoming records before writing to S3.

Answer

Amazon EC2 with Apache Kafka -> S3

Answer

Amazon Kinesis Data Firehose directly with no buffer

Answer

Amazon SQS -> AWS Lambda -> S3

Question 8

A data engineer is ingesting XML data from an external API into Amazon S3. The engineer needs to transform the XML to JSON using AWS Glue. The XML structure is deeply nested. Which Apache Spark method should be used in the Glue ETL script?

Accepted Answer

Use spark.read.format('xml') with Databricks XML library. The Databricks XML library (spark.read.format('xml')) provides native support for parsing deeply nested XML into a DataFrame, which is essential for AWS Glue ETL scripts running on Spark. AWS Glue does not have a built-in 'xml' data source, and the Databricks library handles complex nested structures, attributes, and arrays automatically, making it the standard approach for XML-to-JSON transformation in Spark.

Answer

Use the built-in AWS Glue 'xml' data source

Answer

Use Hadoop's XmlInputFormat

Answer

Use the Spark SQL function from_xml()

Question 9

A company is ingesting large volumes of sensor data into Amazon S3. The data must be encrypted at rest using an AWS KMS customer managed key. Which TWO actions are required to enable server-side encryption with AWS KMS (SSE-KMS) on the S3 bucket?

Accepted Answer

Set the default encryption on the S3 bucket to AWS-KMS and specify the KMS key. Setting the default encryption on the S3 bucket to AWS-KMS and specifying the KMS key ensures that all objects uploaded to the bucket are automatically encrypted with SSE-KMS using that customer managed key. This is the primary configuration step to enforce server-side encryption at rest with a customer managed key.

Answer

Enable S3 Versioning on the bucket

Answer

Enable Amazon CloudWatch Logs for the bucket

Answer

Add a bucket policy that denies uploads without encryption

Question 10

A data engineer is designing a data ingestion pipeline for real-time clickstream data. The data must be available for both real-time analytics and batch processing. The engineer wants to use Amazon Kinesis Data Streams. Which THREE components should be included in the architecture?

Accepted Answer

Amazon Kinesis Data Streams. Amazon Kinesis Data Streams is the core ingestion service for real-time clickstream data, providing low-latency, durable storage of data records that can be consumed by multiple applications simultaneously. It enables both real-time analytics (via Kinesis Data Analytics or KCL applications) and batch processing (by integrating with Kinesis Data Firehose to deliver data to Amazon S3).

Answer

Amazon Kinesis Data Analytics

Answer

Amazon SQS as a buffer

Question 11

A company is ingesting Apache logs from multiple web servers into AWS. The logs are sent via Amazon CloudWatch Logs to a subscription filter that delivers to a Lambda function. The Lambda function parses the logs and writes to Amazon S3. However, there is a significant backlog. Which THREE actions can reduce the backlog?

Accepted Answer

Increase the Lambda function memory allocation. Increasing the Lambda function's memory allocation also increases its CPU allocation, allowing the function to process each log event faster. This reduces the per-invocation processing time, enabling the function to handle more log data per unit time and thus reduce the backlog.

Answer

Route the CloudWatch Logs subscription to an Amazon SQS queue first

Answer

Change the Lambda function runtime from Python to Node.js

Question 12

Refer to the exhibit. An IAM policy for an AWS Lambda function. The Lambda function is triggered by an S3 event (object created) and needs to read from a Kinesis stream. However, the function fails with access denied when trying to read from Kinesis. What is the most likely cause?

Accepted Answer

The Kinesis stream is encrypted with a customer managed KMS key, and the Lambda function lacks kms:Decrypt permission. When a Kinesis stream is encrypted with a customer managed KMS key, the Lambda function must have the `kms:Decrypt` permission on that key to read data from the stream. Without this permission, the Lambda function will receive an access denied error even if it has the necessary Kinesis actions (e.g., `kinesis:GetRecords`) allowed in its IAM policy. The S3 event trigger only invokes the function; it does not grant Kinesis access.

Answer

The Lambda function is not in the same region as the Kinesis stream

Answer

The Lambda function does not have permission to list S3 buckets

Answer

The S3 bucket policy denies access to the Lambda function

Question 13

Refer to the exhibit. A CloudFormation stack outputs the Glue job name and S3 bucket names. The Glue job transforms CSV files from the raw bucket to Parquet in the processed bucket. However, the Glue job is failing with an error that it cannot write to the processed bucket. What is the most likely cause?

Accepted Answer

The Glue job does not have permission to write to the processed bucket. The most likely cause is that the Glue job's IAM role lacks the necessary permissions (e.g., s3:PutObject, s3:ListBucket) on the processed bucket. AWS Glue jobs require an IAM role with policies that grant write access to the target S3 bucket; without these permissions, the job fails with a write error. This is a common misconfiguration when the role is scoped only to read from the raw bucket.

Answer

The raw data bucket is in a different region

Answer

The Glue job is not using the correct worker type

Answer

The Glue job is using an incorrect file format

Question 14

Refer to the exhibit. A Lambda function named 'IngestionProcessor' is failing. The engineer checks CloudWatch Logs and sees the log group exists but storedBytes is 0. Why might the logs show no data?

Accepted Answer

The Lambda execution role does not have permission to write logs to CloudWatch. The Lambda execution role must have the `logs:CreateLogStream` and `logs:PutLogEvents` permissions to write logs to CloudWatch Logs. If the role lacks these permissions, the log group will be created (if it doesn't exist) but no log events will be written, resulting in `storedBytes` being 0. This is a common misconfiguration when the IAM policy does not include the necessary CloudWatch Logs actions.

Answer

The Lambda function is configured with a dead letter queue

Answer

The Lambda function has not been invoked yet

Answer

The log group is encrypted with a KMS key and the Lambda function lacks decrypt permission

Question 15

A company needs to ingest real-time clickstream data from a web application into Amazon S3 for analytics. The data must be available within minutes of generation. Which AWS service should be used to capture and deliver this streaming data?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service designed to capture, transform, and load streaming data into Amazon S3, Redshift, Elasticsearch, or Splunk in near real-time (typically within 60 seconds). It directly addresses the requirement for ingesting real-time clickstream data and delivering it to S3 within minutes, without requiring custom code or manual scaling.

Answer

Amazon RDS

Answer

AWS Glue

Answer

Amazon Simple Queue Service (SQS)