DEA-C01 Data Ingestion and Transformation Practice Test 26 — 15 Questions

Question 1

A company uses Amazon Kinesis Data Streams to ingest clickstream data from a website. The data is consumed by an AWS Lambda function that writes to Amazon DynamoDB. The Lambda function is seeing high error rates due to DynamoDB write throttling. Which action should be taken to reduce throttling?

Accepted Answer

Enable auto scaling on the DynamoDB table. Enabling DynamoDB auto scaling increases write capacity automatically when needed. Using Kinesis Data Firehose would change the architecture but does not address throttling directly. Increasing Lambda memory does not help with DynamoDB throttling. Using SQS would add a queue but does not increase DynamoDB capacity.

Answer

Use Amazon Kinesis Data Firehose instead of Kinesis Data Streams

Answer

Add an Amazon SQS queue between Lambda and DynamoDB

Answer

Increase the Lambda function memory

Question 2

A data pipeline uses AWS Glue to read from an Amazon S3 bucket containing millions of small CSV files (each < 1 MB). The ETL job is slow. Which optimization would most improve performance?

Accepted Answer

Use S3 file grouping to combine small files. Using Amazon S3 file grouping or converting to columnar format like Parquet reduces the number of files and improves read performance. Increasing workers helps, but file consolidation is more impactful. Using G.1X worker type may help, but grouping files is key. Using Spark SQL directly does not address the small files problem.

Answer

Write the ETL script using PySpark instead of Scala

Answer

Increase the number of Glue workers

Answer

Use the G.1X worker type for more memory

Question 3

A company wants to transform data in Amazon S3 using SQL queries without provisioning servers. The transformations are ad-hoc and run occasionally. Which service should be used?

Accepted Answer

Amazon Athena. Amazon Athena allows querying S3 data with standard SQL without server management. AWS Glue is for scheduled ETL jobs. Amazon EMR requires cluster provisioning. Amazon Redshift Spectrum queries S3 but requires a Redshift cluster.

Answer

AWS Glue

Answer

Amazon Redshift Spectrum

Answer

Amazon EMR

Question 4

A company uses Amazon Kinesis Data Firehose to deliver streaming data to Amazon S3. The data is in JSON format and needs to be converted to Parquet. However, the conversion is failing. What is the most likely cause?

Accepted Answer

The schema is not defined in the AWS Glue Data Catalog. Kinesis Data Firehose requires a schema to convert to Parquet. This schema can be provided by a Glue Data Catalog table. If the schema is not defined, the conversion fails. Data size is not an issue for conversion. Kinesis Data Streams is not involved here. Lambda transformation can convert to Parquet but is not required.

Answer

The data transformation Lambda function is not converting to Parquet

Answer

The data size exceeds the 1 MB limit per record

Answer

The delivery stream is configured to use Kinesis Data Streams as source

Question 5

A company is designing a data ingestion pipeline for real-time analytics. The source is a relational database, and the target is Amazon Redshift. The pipeline must handle schema changes in the source database automatically. Which combination of services should be used?

Accepted Answer

AWS DMS and AWS Glue. AWS DMS can continuously replicate data from the source database to S3 in CDC mode. AWS Glue can then detect schema changes and transform the data before loading into Redshift. Kinesis Data Streams is for streaming data, not database CDC. Athena is for querying, not for handling schema evolution. Lambda alone would require custom code for schema detection.

Answer

Amazon S3 and Amazon Athena

Answer

AWS Glue and Amazon Redshift COPY

Answer

Amazon Kinesis Data Streams and AWS Lambda

Question 6

A company has a nightly batch job that processes 100 GB of data from an Amazon S3 bucket and loads it into an Amazon Redshift table. The job currently runs on an Amazon EMR cluster. Which service would reduce operational overhead while providing similar functionality?

Accepted Answer

AWS Glue. AWS Glue is a serverless ETL service that can run scheduled jobs and reduce overhead compared to managing EMR clusters. Redshift Spectrum is for querying S3, not for ETL. Athena is for ad-hoc queries. DMS is for database migration.

Answer

AWS Database Migration Service

Answer

Amazon Redshift Spectrum

Answer

Amazon Athena

Question 7

Which TWO AWS services can be used to ingest streaming data from a mobile application into Amazon S3 for near-real-time analytics? (Choose 2.)

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose can directly ingest streaming data and deliver to S3. Amazon Kinesis Data Streams can also ingest data, but requires a consumer to write to S3. AWS Glue is for batch ETL. Amazon SQS is for message queuing. Amazon DynamoDB Streams captures changes in DynamoDB, not from mobile apps.

Answer

Amazon DynamoDB Streams

Answer

AWS Glue

Answer

Amazon SQS

Question 8

Which THREE factors should be considered when selecting a data ingestion service for a high-volume, real-time streaming pipeline that requires exactly-once processing semantics? (Choose 3.)

Accepted Answer

Ability to replay records from a checkpoint. Exactly-once processing requires services that support idempotent writes and checkpoints. Kinesis Data Streams supports record replay and checkpointing. AWS Lambda can be used with Kinesis for idempotent processing. Amazon S3 provides idempotent PUT operations. Amazon DynamoDB can store checkpoints. Amazon Kinesis Data Firehose provides at-least-once delivery, not exactly-once. AWS Glue is batch-oriented.

Answer

Support for schema evolution

Answer

Ability to transform data in-flight

Question 9

Which TWO AWS services can be used to transform data in an Amazon S3 data lake before loading into Amazon Redshift? (Choose 2.)

Accepted Answer

AWS Glue. AWS Glue can run ETL jobs to transform S3 data and load into Redshift. Amazon Redshift Spectrum can query S3 data directly and load results into Redshift tables. Amazon Athena is for querying, not loading. Amazon EMR requires cluster management. AWS Lambda is for small data volumes.

Answer

AWS Lambda

Answer

Amazon Athena

Answer

Amazon EMR

Question 10

The exhibit shows an AWS CLI command and its output. A data engineer wants to copy only objects larger than 10 MB from the S3 bucket to another bucket for processing. Which approach should be used to automate this task?

Accepted Answer

Use AWS CLI with a script to filter and copy objects. The command lists objects larger than 10 MB. To automate copying, a script using AWS CLI with the --query parameter can filter and copy. Using S3 Batch Operations allows performing actions on a list of objects. The correct approach is to use AWS CLI with a script that iterates over the filtered list and uses aws s3 cp. S3 replication is for continuous sync, not one-time copy. Lambda with S3 events triggers only on new objects, not existing ones. S3 Inventory provides metadata but not direct copy.

Answer

Use S3 replication rules to replicate objects above 10 MB

Answer

Use S3 Inventory to generate a list and then copy

Answer

Use AWS Lambda with S3 event notifications

Question 11

The exhibit shows an IAM policy attached to a role used by an AWS Glue ETL job. The job reads from an S3 bucket and writes to another S3 bucket. However, the job fails with an access denied error when trying to write to the output bucket. What is the most likely cause?

Accepted Answer

The policy only allows PutObject on the my-data-lake bucket, but the job writes to a different bucket. The policy allows s3:PutObject on my-data-lake/*, but if the output bucket is different (e.g., my-output-bucket), the policy does not cover it. The error is due to missing permissions on the output bucket. The Glue service role may not have permissions to write to the output bucket. The policy does not restrict resource to only one bucket, but the ARN specifies my-data-lake. The job might be trying to write to a different bucket. There is no issue with Glue actions.

Answer

The policy is missing permissions for AWS KMS to decrypt/encrypt objects

Answer

The policy does not allow glue:StartJobRun on the specific job

Answer

The policy does not include s3:ListBucket permission

Question 12

The exhibit shows the output of describing an Amazon Kinesis Data Stream. A producer is sending records but the consumer is not receiving all records. What is the most likely cause?

Accepted Answer

The stream has only one shard, causing write throttling. The stream has only one shard, which provides a maximum throughput of 1 MB/s or 1000 records/s for writes. If the producer exceeds this, records will be throttled. The retention period is 24 hours, which is fine. The stream status is ACTIVE. There is no indication of a faulty shard. The consumer might be slow, but the question asks for cause of not receiving all records; throttling due to insufficient shards is a common issue.

Answer

The stream is in ACTIVE status, which prevents reading

Answer

The retention period is too short

Answer

The hash key range is too wide

Question 13

A data pipeline ingests CSV files from an S3 bucket into a Redshift table using the COPY command. Recently, files with inconsistent column delimiters (some use pipes, others use commas) have been arriving. The pipeline must handle both delimiters without manual intervention. What is the MOST efficient solution?

Accepted Answer

Create an AWS Lambda function triggered by S3 events that reads the first line of each file, detects the delimiter, and runs the COPY command with the appropriate DELIMITER option.. Option C is correct because using a Lambda function to inspect the first line and set the DELIMITER parameter dynamically handles multiple delimiters without manual intervention, and it's efficient as it runs per object. Option A is wrong because it requires manual reconfiguration per file. Option B is wrong because AWS Glue is overkill for simple delimiter detection. Option D is wrong because it still requires manual schema redefinition.

Answer

Configure the COPY command with a fixed delimiter (e.g., comma) and manually convert files with pipes before ingestion.

Answer

Use AWS Glue to crawl the S3 bucket and automatically detect the schema and delimiter before writing to Redshift.

Answer

Use Amazon Athena to query the files with the OpenCSVSerDe, which automatically detects delimiters, and then write the results to Redshift.

Question 14

A company uses Amazon Kinesis Data Streams to ingest clickstream data. The data is then processed by a Kinesis Data Analytics application running SQL queries. The analytics application is falling behind and processing records with increasing latency. The stream has 4 shards, and the average record size is 5 KB. What is the MOST effective way to improve processing latency?

Accepted Answer

Increase the parallelism of the Kinesis Data Analytics application.. Option D is correct because increasing the parallelism of the Kinesis Data Analytics application (e.g., by increasing the number of in-application streams or ParallelismPerKPU) allows it to consume from the stream faster, reducing latency. Option A is wrong because 5 KB is well below the 1 MB/s shard limit, so increasing shards is unnecessary. Option B is wrong because enhanced fan-out is for consumers that need low latency, but does not help if the application is CPU-bound. Option C is wrong because increasing record TTL does not affect processing speed.

Answer

Increase the number of shards in the Kinesis stream to 8.

Answer

Enable enhanced fan-out on the Kinesis stream for the analytics application.

Answer

Increase the retention period of the Kinesis stream to 7 days.

Question 15

A data engineer needs to ingest data from a relational database (MySQL) into Amazon S3 for analytics. The database is 500 GB and the job must run daily with incremental updates. Which AWS service is BEST suited for this task?

Accepted Answer

AWS Database Migration Service (DMS) with a replication task.. Option A is correct because AWS Database Migration Service (DMS) supports ongoing replication from MySQL to S3 with change data capture (CDC), enabling incremental updates. Option B is wrong because AWS Glue can do batch jobs but does not natively support CDC as seamlessly as DMS. Option C is wrong because Amazon Kinesis is for streaming data, not database snapshots. Option D is wrong because Amazon EMR is overkill for simple database ingestion.

Answer

Amazon EMR with Apache Sqoop.

Answer

Amazon Kinesis Data Firehose with a database source.

Answer

AWS Glue ETL job with a JDBC connection.