DEA-C01 Data Ingestion and Transformation Practice Test 14 — 15 Questions

Question 1

A data engineer is designing a pipeline to ingest daily CSV files from an SFTP server into Amazon S3. The files are large (up to 10 GB) and must be encrypted in transit. The pipeline should be fully managed and serverless where possible. Which TWO services should be used together to achieve this? (Choose TWO.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

AWS Glue. Options C and D are correct. AWS Transfer Family provides a fully managed SFTP service that can transfer files directly to S3 with encryption in transit, meeting the requirement for serverless, managed file ingestion. AWS Glue can then be used to process the CSV files in S3, as it is a serverless ETL service suitable for batch processing large files. Option A (Kinesis Data Firehose) is for streaming data, not batch file transfers. Option B (Lambda) has a 15-minute timeout and is not ideal for large files up to 10 GB. Option E (Athena) is a query service, not a transfer or processing service.

Answer

Amazon Kinesis Data Firehose

Answer

AWS Lambda

Answer

Amazon Athena

Question 2

A company is ingesting IoT sensor data into Amazon Kinesis Data Streams. Each sensor sends a JSON payload every second. The data must be transformed and aggregated in real-time before being stored in Amazon DynamoDB. Which THREE services should be used together in the pipeline? (Choose THREE.)

Accepted Answer

AWS Lambda. Amazon Kinesis Data Streams (D) is the ingestion layer where each IoT sensor's JSON payload is written every second, providing the durable, low-latency streaming backbone that the rest of the pipeline consumes. Amazon Kinesis Data Analytics (B) is the real-time processing engine that runs SQL or Apache Flink applications over the stream to transform and aggregate the data before it is stored. AWS Lambda (A) is used to consume the processed output (for example, as a Kinesis Data Analytics or Kinesis Data Streams event source) and write the transformed records into Amazon DynamoDB. Amazon S3 (C) is object storage and is not a real-time transformation or aggregation service, so it does not fit this streaming pipeline. Amazon Kinesis Data Firehose (E) is designed to deliver data to destinations such as S3, Redshift, or Splunk and, while it can invoke Lambda for transformation, it is not the aggregation/analytics engine required here and is not needed alongside Kinesis Data Analytics for this scenario.

Answer

Amazon S3

Answer

Amazon Kinesis Data Firehose

Question 3

Refer to the exhibit. An IAM policy is attached to an EC2 instance role that runs a data ingestion application. The application reads files from an S3 bucket 'data-lake-primary' and sends records to a Kinesis stream named 'clickstream'. The application is failing with an 'AccessDenied' error when trying to read from S3. What is the MOST likely cause?

Accepted Answer

The policy does not include the s3:ListBucket permission.. The application reads files from an S3 bucket, which requires both s3:GetObject (to read the object) and s3:ListBucket (to list objects in the bucket, typically needed for operations like listing or navigating the bucket). Without s3:ListBucket, the SDK may fail with an AccessDenied error when it tries to enumerate objects or validate the bucket path, even if s3:GetObject is present. Option D correctly identifies this missing permission as the most likely cause.

Answer

The actions are specified incorrectly; they should be s3:GetObject and s3:PutObject only.

Answer

The policy is not attached to the EC2 instance role.

Answer

The Kinesis stream name is incorrect.

Question 4

Refer to the exhibit. A data engineer runs the describe-stream command and sees this output. The application is writing records to the stream but is experiencing high write latency. The average record size is 50 KB, and the write rate is 1500 records per second. What is the MOST likely cause of the latency?

Accepted Answer

The stream does not have enough shards to handle the write throughput.. Kinesis Data Streams scales by shards. Each shard supports up to 1 MB/sec or 1,000 records/sec for writes. With 1,500 records/sec at 50 KB each, the total write throughput is 75 MB/sec. If the stream has fewer than 75 shards (or fewer than 2 shards if limited by records/sec, but here MB/sec dominates), the stream will throttle writes, causing high latency. The most likely cause is insufficient shards to handle the write throughput.

Answer

The application is running in a different AWS region.

Answer

The application is exceeding the DynamoDB provisioned throughput.

Answer

The Kinesis stream is throttling the application because of a hot shard.

Question 5

A company wants to ingest streaming data from thousands of IoT devices into AWS for real-time analytics. The data volume is variable and can spike unpredictably. The solution must be serverless and minimize operational overhead. Which AWS service should be used for ingestion?

Accepted Answer

Use Amazon Kinesis Data Streams to ingest and process data in real time.. Amazon Kinesis Data Streams is the correct choice because it is a serverless, real-time data ingestion service designed to handle variable and unpredictable data volumes from thousands of sources. It provides durable, low-latency streaming with the ability to process data in real time using consumers like Lambda or Kinesis Data Analytics, meeting the requirements for real-time analytics and minimal operational overhead.

Answer

Use Amazon Kinesis Data Firehose to load streaming data directly into Amazon S3.

Answer

Use Amazon SQS to queue messages and process them in batches.

Answer

Use AWS IoT Core to ingest data and route it to Amazon DynamoDB.

Question 6

A company is streaming data from an application to Amazon Kinesis Data Streams. The data must be transformed in real time and then stored in Amazon S3 in Parquet format. Which AWS service should be used for the transformation step?

Accepted Answer

Amazon Kinesis Data Analytics for Apache Flink.. Amazon Kinesis Data Analytics for Apache Flink is the correct service for real-time transformation of streaming data because it provides a fully managed Apache Flink environment designed for complex stream processing, including transformations, aggregations, and windowing. It can consume from Kinesis Data Streams, apply SQL or Flink applications for transformation, and output to S3 in Parquet format via a sink connector. This is the most appropriate service for real-time transformation before storage.

Answer

Amazon Kinesis Data Firehose with a Lambda transformation.

Answer

Amazon EMR running Apache Spark Streaming.

Answer

AWS Lambda with a Kinesis trigger.

Question 7

A company is using AWS Glue to run ETL jobs that process data from Amazon S3 and load it into Amazon Redshift. The jobs are failing with the error 'Unable to connect to Redshift cluster'. The Redshift cluster is in the same VPC as the Glue job. What is the MOST likely cause?

Accepted Answer

The Glue job is not configured to use the same VPC as the Redshift cluster.. The most likely cause is that the Glue job is not configured to use the same VPC as the Redshift cluster. By default, AWS Glue jobs run in a separate VPC managed by Glue. To connect to resources in a customer VPC (like a Redshift cluster), the Glue job must be explicitly associated with that VPC via a Glue connection or job configuration. Option C is correct. Option A is incorrect because even if the security group allows inbound traffic, the Glue job's network may not be routed to the Redshift cluster if it is in a different VPC. Option B is incorrect because the IAM role handles authentication, not network connectivity. Option D is incorrect because the Redshift cluster is in the same VPC, so public accessibility is not required; the error indicates a network connectivity issue, not a public access issue.

Answer

The Redshift cluster's security group does not allow inbound traffic from the Glue job's security group.

Answer

The IAM role associated with the Glue job does not have permission to access Redshift.

Answer

The Redshift cluster is not publicly accessible and Glue is trying to connect from outside the VPC.

Question 8

A company is using Amazon Kinesis Data Firehose to deliver streaming data to an Amazon S3 bucket. The data is delivered in JSON format. The company wants to convert the data to Apache Parquet format before delivery to reduce storage costs and improve query performance. How can this be achieved?

Accepted Answer

Use the AWS Glue Data Catalog to define a schema and configure Firehose to use it for Parquet conversion.. Kinesis Data Firehose can convert the input data to Parquet or ORC format using a schema from the AWS Glue Data Catalog. Option A is incorrect because delivering as JSON then converting with Athena is an extra step after storage, not before delivery, and does not reduce storage costs from the outset. Option C is incorrect because Lambda can be used for custom transformations, but Firehose natively supports Parquet conversion using Glue. Option D is incorrect because Firehose cannot convert to Parquet automatically without a schema; you must provide a schema (e.g., from Glue).

Answer

Deliver data to S3 as JSON, then use Amazon Athena to convert to Parquet.

Answer

Write an AWS Lambda function to transform the data to Parquet and deliver it to S3.

Answer

Configure the Firehose stream to convert data to Parquet automatically without any additional setup.

Question 9

A company is using AWS Glue to process data from an Amazon S3 data lake. The Glue job runs daily and transforms data into multiple output formats. Which TWO actions can the company take to optimize the Glue job's performance and reduce costs? (Choose TWO.)

Accepted Answer

Increase the number of DPUs allocated to the job.. Option A is correct because increasing the number of DPUs allocated to the Glue job adds more compute capacity, which can speed up processing of large datasets and reduce the total runtime, thereby lowering overall cost when the job is compute-bound. Option E is correct because enabling job bookmarking allows AWS Glue to track previously processed data and only process new or changed data on subsequent runs, avoiding redundant work and reducing both runtime and cost. Option B is not appropriate because reducing DPUs generally slows processing and can increase runtime, potentially raising costs for large workloads. Option C is wrong because disabling job bookmarking forces full reprocessing of all data every run, increasing runtime and cost. Option D is wrong because increasing the job timeout does not improve performance or reduce cost; it only allows a job to run longer before failing.

Answer

Reduce the number of DPUs (Data Processing Units) allocated to the job.

Answer

Disable job bookmarking to force full reprocessing every run.

Answer

Increase the job timeout to allow more time for processing.

Question 10

A company is running a 10-node Amazon EMR cluster to process data from Amazon S3. The cluster is using Apache Spark for transformations. The data processing is taking longer than expected. Which THREE actions can improve the performance of the Spark jobs on EMR? (Choose THREE.)

Accepted Answer

Enable dynamic allocation of executors.. Enabling dynamic allocation of executors allows Amazon EMR to automatically scale the number of executors up or down based on workload demand. This prevents resource underutilization or over-provisioning, which can significantly improve Spark job performance by ensuring that the cluster's resources are efficiently matched to the processing needs of the transformations.

Answer

Reduce the number of shuffle partitions.

Answer

Disable speculative execution to reduce redundant tasks.

Question 11

A data engineer is reviewing the S3 Lifecycle policy for a data lake bucket. The goal is to archive log data after 30 days and delete it after 365 days, and delete temporary data after 1 day. What is wrong with the current configuration?

Accepted Answer

The transition to GLACIER should be after 30 days, but the expiration should be after 365 days from the transition, not from creation.. S3 Lifecycle expiration actions are always calculated from the object's creation date (or the date it was last modified), not from the date of a storage class transition. In the described policy, the expiration is set to 365 days from creation, but the intent is to delete the data 365 days after it is archived (i.e., 30 + 365 = 395 days from creation). The correct configuration should set the expiration to 395 days from creation to achieve deletion 365 days after the transition to GLACIER. Option D correctly identifies this mismatch between the intended retention period and the actual lifecycle behavior.

Answer

The prefix filter for the first rule does not include a wildcard, so it may not match all log files.

Answer

The rule for temp data has no transition, so it will not expire objects.

Answer

The expiration for the first rule will not delete objects in GLACIER storage class unless they are restored first.

Question 12

A data engineer is troubleshooting an AWS Glue job that is failing with an Access Denied error when trying to read data from an S3 bucket. The IAM policy attached to the Glue job's IAM role is shown in the exhibit. What is the likely cause of the failure?

Accepted Answer

The policy does not include s3:ListBucket permission on the bucket.. The [CORRECT] answer is C because AWS Glue, like most AWS services, requires s3:ListBucket permission on the bucket itself (not just the objects) to perform operations like listing the bucket contents or checking for the existence of objects. Without s3:ListBucket, the Glue job cannot enumerate the objects in the bucket, leading to an Access Denied error even if s3:GetObject is granted. This is a common oversight when writing IAM policies for S3 access.

Answer

The policy does not include s3:GetObject or s3:PutObject permissions.

Answer

The policy does not include glue:StartJobRun permission.

Answer

The policy does not include glue:GetJobRun permission.

Question 13

A data engineering team needs to ingest real-time streaming data from thousands of IoT devices and transform the data before storing it in Amazon S3. Which AWS service is most suitable for performing the transformation step in near real-time?

Accepted Answer

AWS Lambda. AWS Lambda is the most suitable service for performing near real-time transformations on streaming data from thousands of IoT devices before storing it in Amazon S3. Lambda can be invoked by Amazon Kinesis Data Streams or AWS IoT Core to process each record as it arrives, apply transformation logic, and write the transformed data to S3. It scales automatically and charges only for compute time, making it ideal for variable workloads.

Answer

Amazon Kinesis Data Analytics

Answer

Amazon Kinesis Data Firehose

Answer

Amazon S3

Question 14

A data engineer is designing a data ingestion pipeline for JSON files landing in an Amazon S3 bucket. The pipeline must transform the data (e.g., flatten nested structures) and load it into Amazon Redshift. The transformation logic is complex and may evolve frequently. Which approach provides the MOST flexibility and ease of maintenance?

Accepted Answer

Use AWS Glue ETL jobs to transform the data and load into Redshift.. AWS Glue ETL jobs provide a serverless, code-based environment using Apache Spark, which offers flexibility for complex transformations like flattening nested JSON structures. Glue can be configured to support exactly-once semantics through Spark checkpointing and transactional writes to Redshift, making it reliable for critical data pipelines. It handles varying file sizes and can be easily updated as transformation logic evolves. Option A is incorrect because Lambda has execution time and memory limits, making it unsuitable for large JSON files or complex transformations, and achieving exactly-once requires careful idempotency design. Option B is incorrect because the Redshift COPY command loads raw JSON without transformation. Option D is incorrect because Athena is primarily for querying data in S3, not for performing ETL transformations and loading into Redshift.

Answer

Use AWS Lambda functions to transform each file and load into Redshift.

Answer

Use the Amazon Redshift COPY command to load raw JSON directly.

Answer

Use Amazon Athena to query the raw data and insert into Redshift.

Question 15

A data engineer needs to ingest data from an on-premises Oracle database into Amazon S3 using AWS DMS. The change data capture (CDC) must be enabled to capture ongoing changes. Which additional AWS service is required to store the transaction logs for CDC?

Accepted Answer

Amazon S3. AWS DMS change data capture (CDC) for Oracle databases does not require an additional AWS service to store transaction logs. DMS reads the redo logs directly from the source Oracle database, either using LogMiner or binary reader. Therefore, none of the listed services are needed for storing transaction logs. As none of the provided options are correct, this question contains no valid answer.

Answer

Amazon RDS

Answer

Amazon EBS

Answer

Amazon CloudWatch Logs