DEA-C01 Data Ingestion and Transformation Practice Test 6 — 15 Questions

Question 1

A data engineer is designing a data ingestion pipeline that uses AWS Lambda to process records from a Kinesis Data Stream and write to DynamoDB. Which TWO strategies can help handle increased throughput and prevent data loss? (Choose TWO.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Configure the Lambda event source mapping with a batch window and set the number of concurrent batches per shard. Option A is correct because configuring the Lambda event source mapping with a batch window and setting the number of concurrent batches per shard (via MaximumConcurrency / ParallelizationFactor) allows multiple batches from the same shard to be processed in parallel, increasing throughput while the batch window aggregates records to reduce invocation overhead. Option C is correct because increasing the number of shards in the Kinesis data stream raises the stream's overall ingest and read capacity, and since Lambda scales its pollers per shard, more shards directly enable higher parallel processing throughput. Option B is not appropriate because synchronous invocation from the producer couples the producer to Lambda latency and does not improve stream consumption throughput or prevent data loss. Option D, a dead-letter queue, helps capture failed invocations for later analysis but does not itself increase throughput or prevent data loss during high-volume ingestion. Option E, increasing the Lambda timeout, only gives a function more time to run and does not address throughput scaling or data loss prevention.

Answer

Use synchronous invocation of Lambda from the producer

Answer

Configure a dead-letter queue (DLQ) for the Lambda function

Answer

Increase the Lambda function timeout

Question 2

A data engineer needs to set up a data ingestion pipeline that reads from Amazon MSK (Managed Streaming for Kafka) and writes to Amazon S3 with transformations. The data is in Avro format and must be converted to Parquet. Which THREE components should be used together? (Choose THREE.)

Accepted Answer

AWS Lambda function to convert Avro to Parquet as a Firehose transformation. Option C is correct because Amazon Kinesis Data Firehose supports Amazon MSK as a source, allowing it to ingest records from Kafka topics and deliver them to Amazon S3. Option D is correct because the Amazon MSK cluster is the required source system that holds the Kafka topics being read. Option A is correct because Firehose supports AWS Lambda as a data transformation function, which can convert Avro records into Parquet before delivery to S3. Option B is not appropriate because Athena is a query service, not a streaming ingestion or format-conversion component for this pipeline. Option E is not appropriate because an AWS Glue ETL job reads from MSK in batch or micro-batch style and is not the Firehose-based ingestion path described here.

Answer

Amazon Athena to convert the data format

Answer

AWS Glue ETL job to read from MSK

Question 3

A data engineer needs to ingest streaming data from thousands of IoT devices into Amazon S3 in near real-time. The data must be processed with minimal latency and stored in a columnar format for analytics. Which service should the engineer use to ingest the data?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is designed to ingest streaming data and deliver it directly to Amazon S3 with near-real-time latency (typically 60 seconds or more, depending on buffer settings). It automatically converts data to columnar formats like Parquet or ORC, which is essential for analytics, and requires no custom code for the delivery pipeline, minimizing operational overhead.

Answer

Amazon Kinesis Data Analytics

Answer

Amazon Simple Queue Service (SQS)

Answer

Amazon Kinesis Data Streams with a Lambda consumer

Question 4

A data engineer needs to design a data ingestion pipeline that captures change data capture (CDC) events from an on-premises SQL Server database to Amazon S3 with low latency. The pipeline must handle schema changes and ensure exactly-once delivery semantics. Which combination of AWS services should the engineer use?

Accepted Answer

AWS Database Migration Service (DMS) with Amazon Kinesis Data Firehose to Amazon S3. AWS DMS can capture ongoing changes from SQL Server using its CDC capability and stream them directly into Amazon Kinesis Data Firehose, which buffers and delivers data to Amazon S3 with low latency. DMS handles schema changes by propagating them to the target, and Kinesis Data Firehose, combined with DMS's transactional integrity, supports exactly-once delivery semantics when configured with a primary key and appropriate error handling.

Answer

AWS AppFlow with SQL Server connector to Amazon S3

Answer

AWS Glue ETL job with JDBC connection to SQL Server and writing to Amazon S3

Answer

Amazon Kinesis Data Streams with AWS Lambda consumer writing to Amazon S3

Question 5

A company ingests JSON logs into Amazon S3 using Kinesis Data Firehose. The logs contain a timestamp field, but the delivery to S3 is delayed by up to 15 minutes during peak hours. The business requires near-real-time availability (under 2 minutes). Which configuration change should the data engineer make?

Accepted Answer

Decrease the buffer interval to 60 seconds. Decreasing the buffer interval to 60 seconds forces Kinesis Data Firehose to deliver data to S3 more frequently, reducing the maximum delivery latency to under 2 minutes. The default buffer interval is 300 seconds (5 minutes), and by lowering it to 60 seconds, you ensure that data is flushed to S3 within one minute of ingestion, meeting the near-real-time requirement even during peak hours when buffering might otherwise delay delivery.

Answer

Increase the number of shards in the Kinesis Data Firehose stream

Answer

Increase the buffer size to 128 MB

Answer

Enable buffering hints in the Firehose delivery stream

Question 6

A company needs to ingest data from an external API that returns CSV files daily. The files range from 100 MB to 2 GB. The data should be landed in Amazon S3 and then transformed using AWS Glue. Which ingestion method is most cost-effective and requires the least operational overhead?

Accepted Answer

Schedule an AWS Lambda function to download the CSV file and upload it to Amazon S3. Scheduling an AWS Lambda function to download the CSV file from the external API and upload it to Amazon S3 is the most cost-effective and operationally lightweight approach. Lambda can handle files up to 2 GB (with appropriate memory and timeout settings) and runs on a serverless, pay-per-execution model, eliminating the need for infrastructure management. This method directly addresses the daily, batch-oriented nature of the data ingestion without requiring additional services or complex configurations.

Answer

Set up AWS DataSync to transfer the file from the API endpoint to S3

Answer

Use Amazon Kinesis Data Firehose with a direct PUT

Answer

Deploy an AWS Direct Connect connection to the external API for faster transfer

Question 7

A data engineer notices that an AWS Glue ETL job is running slower than expected. The job reads from Amazon S3, joins two datasets, and writes the result back to S3. The job uses the default worker type (G.1X) and 10 DPUs. Which action is most likely to improve performance?

Accepted Answer

Repartition the data before the join operation. The default G.1X worker type provides 16 GB of memory and 4 vCPUs per DPU. With 10 DPUs, the job likely has sufficient compute but suffers from data skew or inefficient partitioning during the join. Repartitioning the data before the join ensures that keys are evenly distributed across partitions, reducing shuffle overhead and preventing straggler tasks, which directly improves performance.

Answer

Increase the number of DPUs to 20

Answer

Use coalesce to reduce the number of output files

Answer

Change the worker type to G.2X

Question 8

A company ingests streaming data from social media feeds into Amazon Kinesis Data Streams. The data is consumed by an AWS Lambda function that transforms and writes to Amazon S3. Recently, the Lambda function started timing out and dropping records. The data volume has tripled. Which actions should the data engineer take to resolve this? (Choose TWO.)

Accepted Answer

Increase the number of shards in the Kinesis data stream. Option A is correct because Kinesis Data Streams scales throughput by shard: each shard supports 1 MB/s ingress and 2 MB/s egress, and a Lambda consumer processes one batch per shard invocation, so tripling the data volume requires adding shards to raise parallel read capacity and prevent the consumer from falling behind. Option E is correct because Lambda allocates CPU proportionally to memory, so increasing the memory setting speeds up the transformation logic and reduces per-invocation duration, directly addressing the timeouts. Option B is not appropriate because Kinesis Data Firehose does not perform arbitrary custom transformations the way the existing Lambda code does, and it is a delivery service rather than a drop-in replacement for a Kinesis Data Streams consumer. Option C is wrong because the maximum Lambda timeout is 15 minutes, but simply extending the timeout does not fix a throughput bottleneck and risks higher latency and cost. Option D is wrong because reserved concurrency caps the function's concurrency and would throttle processing rather than increase throughput to handle the tripled volume.

Answer

Replace Lambda with Amazon Kinesis Data Firehose for the transformation

Answer

Increase the Lambda function timeout to 15 minutes

Answer

Set a reserved concurrency on the Lambda function

Question 9

Refer to the exhibit. A data engineer runs an AWS Glue job that fails with an 'Access Denied' error when writing to S3. The IAM role attached to the job has s3:PutObject permission on the output bucket. What additional configuration is most likely missing?

Accepted Answer

The S3 bucket policy does not grant access to the Glue job's IAM role. The IAM role attached to the Glue job has s3:PutObject permission, but the S3 bucket policy must explicitly grant access to that IAM role (or its principal) for the write operation to succeed. Even if the IAM role allows the action, the bucket policy acts as a separate access control layer; if it denies or does not include the role, the request fails with 'Access Denied'. This is a common cross-account or service-specific permission issue where both identity-based and resource-based policies must align.

Answer

The Glue job is not configured to write to S3 with the correct prefix

Answer

The Glue job is running in a VPC without an S3 VPC endpoint

Answer

The S3 bucket is encrypted with AWS KMS and the IAM role lacks kms:Decrypt permission

Question 10

A data engineer is building a pipeline to ingest JSON files from Amazon S3 into Amazon Redshift. The files are 100 MB each and arrive every 5 minutes. Which service is BEST suited for this ingestion?

Accepted Answer

Amazon Redshift COPY command. The Amazon Redshift COPY command is the most efficient and best-suited service for bulk loading 100 MB JSON files from S3 into Redshift at regular 5-minute intervals. It is optimized for high-throughput, parallel ingestion directly from S3, minimizing latency and resource overhead compared to other services.

Answer

AWS Glue ETL job

Answer

AWS Lambda with Redshift Data API

Answer

Amazon Kinesis Data Firehose with Redshift destination

Question 11

A company uses AWS Glue DataBrew to clean and transform data for analytics. The source data is in Parquet format in Amazon S3. The transformation includes filtering rows and adding calculated columns. What is the MOST cost-effective way to run these transformations on a schedule?

Accepted Answer

Create a Glue DataBrew recipe and schedule the job using a cron expression. AWS Glue DataBrew is purpose-built for visual data preparation, and scheduling a DataBrew recipe job with a cron expression directly meets the requirement to run filtering and column calculations on Parquet data in S3 without writing code. This is the most cost-effective approach as it avoids provisioning or managing compute resources beyond the serverless DataBrew job runs.

Answer

Use Amazon EMR with Spark

Answer

Create an AWS Lambda function triggered by S3 events

Answer

Use AWS Glue ETL with PySpark

Question 12

A company uses Amazon Kinesis Data Analytics for Apache Flink to process streaming data. The application reads from a Kinesis stream with 10 shards and writes to an S3 bucket. The application is experiencing high latency. Analysis shows that the application is not keeping up with the incoming data rate. Which action would MOST effectively reduce latency?

Accepted Answer

Increase the Parallelism of the Flink application. The high latency is caused by the Flink application not keeping up with the incoming data rate, which indicates a processing bottleneck within the application itself. Increasing the Parallelism of the Flink application (Option B) directly increases the number of parallel subtasks that can process data concurrently, improving throughput and reducing latency. This is the most effective action because it addresses the root cause—insufficient compute resources for stream processing—without changing the source or sink configuration.

Answer

Increase the number of shards in the Kinesis stream

Answer

Enable exactly-once delivery to S3

Answer

Use a larger Kinesis Data Analytics application (increase KPU)

Question 13

A company uses AWS Lake Formation to manage data lake permissions. A data engineer needs to grant an IAM role 'Read' access to a specific database and all its tables in the Data Catalog. What is the MOST efficient way to achieve this?

Accepted Answer

Grant 'Select' on the database and select 'Include' to apply to all tables. Lake Formation allows granting 'Select' permission on a database with the 'Include' option, which automatically applies the same permission to all current and future tables in that database. This is the most efficient way to grant read access to an IAM role for a specific database and all its tables in the Data Catalog, as it avoids manual per-table grants.

Answer

Grant 'Super' permission on the Data Catalog

Answer

Add the IAM role to the Lake Formation administrators group

Answer

Grant 'Describe' on the database and 'Select' on each table individually

Question 14

Which THREE factors should be considered when choosing between Amazon Kinesis Data Streams and Amazon Kinesis Data Firehose for a streaming ingestion architecture? (Choose 3.)

Accepted Answer

Ability to deliver data directly to Amazon Redshift. Option A is correct because Kinesis Data Firehose natively delivers streaming data directly to Amazon Redshift (via S3 with a COPY command), whereas Kinesis Data Streams requires custom consumers to load into Redshift. Option B is correct because Kinesis Data Streams supports retention from 24 hours up to 365 days, while Firehose does not retain data for later replay, so a retention requirement beyond 7 days forces Data Streams. Option E is correct because custom processing with AWS Lambda or the Kinesis Client Library (KCL) is a Data Streams capability, while Firehose only offers Lambda-based transformation of records in transit and no KCL consumer model. Option C is not a differentiator because both services scale ingestion automatically (Firehose fully managed, Data Streams via shard scaling/on-demand mode). Option D is not a reliable selection factor because Firehose is generally the lower-cost, pay-per-GB option, so cost alone does not favor Data Streams.

Answer

Automatic scaling of ingestion throughput

Answer

Lower cost per GB ingested

Question 15

A company uses Amazon Kinesis Data Streams to ingest clickstream data from a website. The data is then consumed by a custom application for real-time analytics. Recently, the application has been experiencing high latency. The operations team suspects the shard count is insufficient. How should the team increase the shard count of the existing stream?

Accepted Answer

Use the UpdateShardCount API to increase the shard count for the stream.. The UpdateShardCount API is the correct method to increase the shard count of an existing Kinesis Data Stream without data loss or downtime. It allows you to specify a target shard count, and Kinesis automatically splits shards to achieve that count, redistributing the hash key range across the new shards. This directly addresses the high latency caused by insufficient shard count by increasing the stream's throughput capacity.

Answer

Delete the existing stream and create a new one with a higher shard count.

Answer

Manually split a shard using the SplitShard API on each existing shard.

Answer

Modify the PutRecord calls to include a new shard key that distributes data across more shards.