DEA-C01 Data Ingestion and Transformation Practice Test 27 — 15 Questions

Question 1

A company is designing a data ingestion pipeline for real-time analytics. The source is a relational database, and the target is Amazon Redshift. The pipeline must handle schema changes in the source database automatically. Which combination of services should be used?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

AWS DMS and AWS Glue. AWS DMS (Database Migration Service) can capture ongoing changes from a relational database source using Change Data Capture (CDC) and replicate them to Amazon Redshift. AWS Glue can be used to handle schema evolution by crawling the source and updating the target schema, ensuring the pipeline adapts to schema changes automatically. Together, they provide a real-time ingestion pipeline with schema flexibility.

Answer

Amazon S3 and Amazon Athena

Answer

AWS Glue and Amazon Redshift COPY

Answer

Amazon Kinesis Data Streams and AWS Lambda

Question 2

A company has a nightly batch job that processes 100 GB of data from an Amazon S3 bucket and loads it into an Amazon Redshift table. The job currently runs on an Amazon EMR cluster. Which service would reduce operational overhead while providing similar functionality?

Accepted Answer

AWS Glue. AWS Glue is a serverless ETL service that can process 100 GB of data from S3 and load it into Redshift without managing any infrastructure. It provides built-in job scheduling, automatic retries, and a Spark-based engine that handles large-scale data transformations, directly replacing the EMR cluster's functionality while eliminating operational overhead.

Answer

AWS Database Migration Service

Answer

Amazon Redshift Spectrum

Answer

Amazon Athena

Question 3

Which TWO AWS services can be used to ingest streaming data from a mobile application into Amazon S3 for near-real-time analytics? (Choose 2.)

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is a fully managed service designed to load streaming data directly into Amazon S3, Redshift, Elasticsearch, or Splunk without requiring custom code. It can capture and transform streaming data from mobile applications in near-real-time and automatically deliver it to S3, making it ideal for near-real-time analytics pipelines.

Answer

Amazon DynamoDB Streams

Answer

AWS Glue

Answer

Amazon SQS

Question 4

Which THREE factors should be considered when selecting a data ingestion service for a high-volume, real-time streaming pipeline that requires exactly-once processing semantics? (Choose 3.)

Accepted Answer

Ability to replay records from a checkpoint. In high-volume streaming pipelines, the ability to replay records from a checkpoint is essential for achieving exactly-once processing semantics. Services like Amazon Kinesis Data Streams and Apache Kafka allow consumers to reset to a specific sequence number or offset, enabling reprocessing of unacknowledged records after a failure without data loss or duplication.

Answer

Support for schema evolution

Answer

Ability to transform data in-flight

Question 5

Which TWO AWS services can be used to transform data in an Amazon S3 data lake before loading into Amazon Redshift? (Choose 2.)

Accepted Answer

AWS Glue. AWS Glue (Option D) is correct because it provides a fully managed ETL service that can crawl, catalog, and transform data in Amazon S3 data lakes. With AWS Glue, you can create ETL jobs using a serverless Spark environment to transform raw data into optimized formats (e.g., Parquet) and load it into Amazon Redshift via JDBC connections, making it ideal for preparing data before ingestion.

Answer

AWS Lambda

Answer

Amazon Athena

Answer

Amazon EMR

Question 6

The exhibit shows an AWS CLI command and its output. A data engineer wants to copy only objects larger than 10 MB from the S3 bucket to another bucket for processing. Which approach should be used to automate this task?

Accepted Answer

Use AWS CLI with a script to filter and copy objects. The command lists objects larger than 10 MB. To automate copying, a script using AWS CLI with the --query parameter can filter and copy. Using S3 Batch Operations allows performing actions on a list of objects. The correct approach is to use AWS CLI with a script that iterates over the filtered list and uses aws s3 cp. S3 replication is for continuous sync, not one-time copy. Lambda with S3 events triggers only on new objects, not existing ones. S3 Inventory provides metadata but not direct copy.

Answer

Use S3 replication rules to replicate objects above 10 MB

Answer

Use S3 Inventory to generate a list and then copy

Answer

Use AWS Lambda with S3 event notifications

Question 7

The exhibit shows an IAM policy attached to a role used by an AWS Glue ETL job. The job reads from an S3 bucket and writes to another S3 bucket. However, the job fails with an access denied error when trying to write to the output bucket. What is the most likely cause?

Accepted Answer

The policy only allows PutObject on the my-data-lake bucket, but the job writes to a different bucket. The policy allows s3:PutObject on my-data-lake/*, but if the output bucket is different (e.g., my-output-bucket), the policy does not cover it. The error is due to missing permissions on the output bucket. The Glue service role may not have permissions to write to the output bucket. The policy does not restrict resource to only one bucket, but the ARN specifies my-data-lake. The job might be trying to write to a different bucket. There is no issue with Glue actions.

Answer

The policy is missing permissions for AWS KMS to decrypt/encrypt objects

Answer

The policy does not allow glue:StartJobRun on the specific job

Answer

The policy does not include s3:ListBucket permission

Question 8

The exhibit shows the output of describing an Amazon Kinesis Data Stream. A producer is sending records but the consumer is not receiving all records. What is the most likely cause?

Accepted Answer

The stream has only one shard, causing write throttling. The stream has only one shard, which provides a maximum throughput of 1 MB/s or 1000 records/s for writes. If the producer exceeds this, records will be throttled. The retention period is 24 hours, which is fine. The stream status is ACTIVE. There is no indication of a faulty shard. The consumer might be slow, but the question asks for cause of not receiving all records; throttling due to insufficient shards is a common issue.

Answer

The stream is in ACTIVE status, which prevents reading

Answer

The retention period is too short

Answer

The hash key range is too wide

Question 9

A data pipeline ingests CSV files from an S3 bucket into a Redshift table using the COPY command. Recently, files with inconsistent column delimiters (some use pipes, others use commas) have been arriving. The pipeline must handle both delimiters without manual intervention. What is the MOST efficient solution?

Accepted Answer

Create an AWS Lambda function triggered by S3 events that reads the first line of each file, detects the delimiter, and runs the COPY command with the appropriate DELIMITER option.. It provides an automated, event-driven solution that dynamically detects the delimiter of each incoming CSV file and executes the COPY command with the appropriate DELIMITER option. This approach eliminates manual intervention and leverages AWS Lambda's ability to process S3 events in near real-time, making it the most efficient for handling inconsistent delimiters.

Answer

Configure the COPY command with a fixed delimiter (e.g., comma) and manually convert files with pipes before ingestion.

Answer

Use AWS Glue to crawl the S3 bucket and automatically detect the schema and delimiter before writing to Redshift.

Answer

Use Amazon Athena to query the files with the OpenCSVSerDe, which automatically detects delimiters, and then write the results to Redshift.

Question 10

A company uses Amazon Kinesis Data Streams to ingest clickstream data. The data is then processed by a Kinesis Data Analytics application running SQL queries. The analytics application is falling behind and processing records with increasing latency. The stream has 4 shards, and the average record size is 5 KB. What is the MOST effective way to improve processing latency?

Accepted Answer

Increase the parallelism of the Kinesis Data Analytics application.. Increasing the parallelism of the Kinesis Data Analytics application (e.g., by increasing the number of in-application streams or ParallelismPerKPU) allows it to consume from the stream faster, reducing latency. Option A is wrong because 5 KB is well below the 1 MB/s shard limit, so increasing shards is unnecessary. Option B is wrong because enhanced fan-out is for consumers that need low latency, but does not help if the application is CPU-bound. Option D is wrong because increasing the retention period does not affect processing speed; it only keeps data longer.

Answer

Increase the number of shards in the Kinesis stream to 8.

Answer

Enable enhanced fan-out on the Kinesis stream for the analytics application.

Answer

Increase the retention period of the Kinesis stream to 7 days.

Question 11

A data engineer needs to ingest data from a relational database (MySQL) into Amazon S3 for analytics. The database is 500 GB and the job must run daily with incremental updates. Which AWS service is BEST suited for this task?

Accepted Answer

AWS Database Migration Service (DMS) with a replication task.. AWS DMS with a replication task is the best choice because it is specifically designed for continuous, incremental data replication from relational databases like MySQL to Amazon S3. DMS supports ongoing replication (change data capture) to capture incremental changes without custom scripting, and it can handle the initial 500 GB load efficiently. Other services either lack native incremental support or require additional configuration for this use case.

Answer

Amazon EMR with Apache Sqoop.

Answer

Amazon Kinesis Data Firehose with a database source.

Answer

AWS Glue ETL job with a JDBC connection.

Question 12

A company uses AWS Lambda to process messages from an Amazon SQS queue. The messages contain JSON payloads that need to be transformed and written to an Amazon DynamoDB table. Recently, the Lambda function has been timing out and messages are being sent to the dead-letter queue (DLQ). What is the BEST way to troubleshoot and resolve this issue?

Accepted Answer

Increase the Lambda function timeout and monitor DynamoDB write capacity to ensure it is not throttling.. The Lambda function is timing out, which suggests the function's execution duration is exceeding its configured timeout. Increasing the timeout gives the function more time to process messages. Additionally, if DynamoDB write capacity is insufficient, throttling can cause retries that further delay processing, so monitoring and possibly increasing write capacity units addresses the root cause of timeouts.

Answer

Use a standard SQS queue instead of a DLQ to reprocess failed messages automatically.

Answer

Switch the SQS queue to a FIFO queue to ensure exactly-once processing.

Answer

Increase the visibility timeout of the SQS queue to 30 minutes.

Question 13

A data pipeline uses Amazon Kinesis Data Firehose to ingest log data from web servers and deliver it to Amazon S3. The data is then transformed by an AWS Glue job before being loaded into Amazon Redshift. The pipeline must handle a sudden spike in log volume without data loss. Which configuration change is MOST appropriate?

Accepted Answer

Configure Kinesis Data Firehose to back up all data to S3 in case of delivery failures.. Kinesis Data Firehose can be configured to back up all data to Amazon S3 in case of delivery failures, ensuring no data loss during spikes. This feature writes incoming data to a separate S3 bucket as a safety net when the primary destination (e.g., Redshift via Glue) is unavailable or overwhelmed, directly addressing the requirement to handle sudden volume spikes without data loss.

Answer

Increase the AWS Glue job timeout and allocate more DPUs.

Answer

Increase the number of nodes in the Redshift cluster to handle higher load.

Answer

Increase the S3 bucket size limit and enable versioning.

Question 14

A company wants to ingest streaming data from IoT devices into Amazon S3 using Amazon Kinesis Data Firehose. The data must be transformed from JSON to Parquet format before landing in S3. What is the SIMPLEST way to achieve this?

Accepted Answer

Configure Kinesis Data Firehose with a built-in Parquet converter.. Amazon Kinesis Data Firehose has a built-in Parquet conversion feature that uses an AWS Glue schema to convert incoming JSON data to Parquet format. This is the simplest approach because it requires no custom code or additional services; you only need to provide a schema and enable the conversion in the Firehose delivery stream configuration. Option B (using Lambda) is more complex, as it requires writing and maintaining a custom transformation function.

Answer

Use an AWS Lambda function as a data transformation in Kinesis Data Firehose to convert JSON to Parquet.

Answer

Use Kinesis Data Firehose to deliver data directly to S3 in JSON format and run a nightly Glue job to convert to Parquet.

Answer

Use Kinesis Data Analytics to convert the data to Parquet before sending to Firehose.

Question 15

A data engineer is designing a pipeline to ingest change data capture (CDC) events from an Amazon RDS for PostgreSQL database into Amazon S3. The CDC events are captured using AWS DMS. The data must be available for querying within 5 minutes of the change. Which approach meets these requirements?

Accepted Answer

Use AWS DMS to replicate data directly to S3 in near real-time.. AWS DMS supports continuous replication (change data capture) directly to Amazon S3 as a target endpoint. DMS can write CDC events to S3 in near real-time (typically seconds to minutes), meeting the 5-minute latency requirement without intermediate services. The data is stored in comma-separated value (CSV) or Parquet format, ready for querying via Athena or Glue.

Answer

Export the database to S3 using pg_dump and then use AWS Glue to load into S3 in Parquet format.

Answer

Use AWS DMS to replicate data to Amazon Redshift, then unload to S3.

Answer

Use AWS DMS to replicate data to an SQS queue, then process with Lambda to write to S3.