DEA-C01 Data Ingestion and Transformation Practice Test 4 — 15 Questions

Question 1

Match each AWS service to its primary purpose in data engineering.

Question 2

Match each AWS data migration tool to its primary function.

Question 3

Match each AWS database service to its primary use case.

Question 4

Match each AWS security service to its purpose in data protection.

Question 5

A company is streaming clickstream data from a website into Amazon Kinesis Data Streams. The data is then consumed by a Lambda function that transforms the records and writes them to an S3 bucket in Parquet format. Recently, the Lambda function has been timing out and the S3 bucket is not receiving all expected records. The Kinesis stream has a shard count of 10 and the Lambda function's reserved concurrency is set to the default. Which change would MOST likely resolve the issue?

Accepted Answer

Increase the batch size from the default 100 to 500 records per invocation.. The Lambda function is timing out because it cannot keep up with the incoming data rate. Increasing the batch size allows each invocation to process more records, reducing the number of invocations and the overhead. Option A is wrong because increasing concurrency would help but may exceed account limits. Option B is wrong because decreasing batch window would increase invocation frequency. Option D is wrong because S3 destination is not an output target for Kinesis streams directly.

Answer

Decrease the batch window from the default 300 seconds to 60 seconds.

Answer

Configure the Kinesis stream to directly write to S3 using a delivery stream.

Answer

Increase the Lambda function's reserved concurrency.

Question 6

A data engineer is using AWS Glue to perform ETL on data stored in an S3 bucket. The source data is in CSV format with a header row, and the target is a set of Parquet files partitioned by date. The engineer notices that the Glue job is reading all files in the source prefix, including temporary files that should be ignored. What is the MOST efficient way to exclude these temporary files?

Accepted Answer

Use an S3 prefix exclusion pattern in the Glue job's source path.. Using a S3 prefix exclusion pattern in the Glue job's S3 path is the most efficient way to exclude files. Option A is wrong because changing to Parquet does not exclude files. Option B is wrong because a custom classifier is for schema inference. Option D is wrong because Lambda would add complexity.

Answer

Change the source format from CSV to Parquet.

Answer

Set up an S3 event notification to trigger a Lambda function that moves temporary files.

Answer

Create a custom classifier in the Glue Data Catalog.

Question 7

A company uses Amazon Kinesis Data Firehose to ingest JSON logs from multiple sources into an S3 data lake. The data is then consumed by Amazon Athena for analysis. Recently, some queries have been failing with the error 'HIVE_BAD_DATA: Field xyz's type is an unsupported type'. The firehose delivery stream transforms the data using a Lambda function that converts timestamps to Unix epoch. What is the MOST likely cause of the query failure?

Accepted Answer

Some records contain timestamps that were not converted to epoch, so Athena infers the column as a string.. The error indicates that the data type detected by Athena's schema inference does not match the actual data. Since the Lambda function converts timestamps to Unix epoch (a number), Athena may infer the column as a string due to some records not being converted properly. Option A is wrong because partitions do not cause this error. Option B is wrong because S3 permissions would cause a different error. Option D is wrong because data format is likely compatible.

Answer

The data is in JSON format instead of Parquet.

Answer

The S3 partitions are not registered in the Glue Data Catalog.

Answer

The IAM role for Firehose does not have permission to write to S3.

Question 8

A data engineer is designing a data ingestion pipeline to load data from an on-premises Oracle database to Amazon S3. The pipeline should capture changes in near real-time (within minutes) and minimize impact on the source database. The source table has a 'last_modified' timestamp column. Which service combination would meet these requirements?

Accepted Answer

AWS DMS with a replication task in CDC mode, writing to S3 in Parquet format.. AWS Database Migration Service (DMS) with change data capture (CDC) can capture changes from Oracle in near real-time and write to S3. Option A is wrong because Glue does not support CDC. Option B is wrong because Data Pipeline is batch-oriented. Option D is wrong because Firehose does not connect to databases directly.

Answer

Amazon Kinesis Data Firehose with a Lambda function that queries Oracle.

Answer

AWS Data Pipeline with a periodic SQL query activity to copy full table snapshots.

Answer

AWS Glue with a JDBC connection to Oracle, running a crawler every 5 minutes.

Question 9

A company uses AWS Glue to process data from multiple S3 buckets. The Glue job runs daily and reads data from a bucket that contains millions of small files (each < 1 MB). The job has been running for hours and is often close to the 8-hour timeout limit. Which optimization would MOST reduce the job's runtime?

Accepted Answer

Pre-process the data to consolidate small files into larger files before the Glue job.. Small files cause overhead in reading and processing. Grouping them into larger files (e.g., by using S3 batch operations or a compaction step) reduces the number of files and improves performance. Option A is wrong because increasing DPUs may help but not as much as file consolidation. Option B is wrong because Spark is already used by Glue. Option D is wrong because Parquet helps but if files are small, the benefit is limited.

Answer

Convert the source data from CSV to Parquet format.

Answer

Increase the number of DPUs allocated to the Glue job.

Answer

Use a larger Spark shuffle partition size.

Question 10

A data engineer is setting up an Amazon Kinesis Data Firehose delivery stream to load data into Amazon Redshift. The data is coming from an application that produces JSON records. The engineer needs to transform the data to match the Redshift table schema. Which approach is the MOST cost-effective and requires the least operational overhead?

Accepted Answer

Configure a Lambda function in the Firehose delivery stream to transform records before delivery.. Using Firehose's built-in Lambda transformation function is the simplest and most cost-effective approach. Option A is wrong because Firehose cannot directly write to Redshift without transformation. Option B is wrong because Glue adds extra cost and complexity. Option D is wrong because KCL requires custom code and management.

Answer

Use AWS Glue as a transformation step between Firehose and Redshift, with a trigger on S3.

Answer

Use Kinesis Data Firehose with direct PUT to Redshift and rely on Redshift's COPY command to transform.

Answer

Use the Kinesis Client Library (KCL) to consume the stream, transform in an EC2 instance, and then load to Redshift.

Question 11

A company uses Amazon S3 to store raw data and AWS Glue to run ETL jobs that transform the data into analytics-ready tables. The Glue job reads from a source with a schema that changes frequently (new columns added). The engineer wants the Glue job to automatically adapt to schema changes without manual intervention. Which configuration should the engineer use?

Accepted Answer

Enable the 'Update schema' option in the Glue job's output target configuration.. Glue's 'Update schema' option in the job's output target allows the job to automatically incorporate new columns. Option A is wrong because a crawler after the job would add lag. Option B is wrong because schema-on-read is not automatic. Option D is wrong because catalog partitions are for partitioning, not schema evolution.

Answer

Schedule a Glue crawler to run after each ETL job to update the Data Catalog.

Answer

Set the job to use schema-on-read by storing data in Parquet format.

Answer

Use Glue's partition indexes to automatically detect new columns.

Question 12

A company is using Amazon Kinesis Data Streams to ingest real-time clickstream data. The data is consumed by a fleet of EC2 instances running a custom application that processes the records and writes to DynamoDB. The application is experiencing high latency and records are being processed slower than they are produced. The stream has 5 shards. Which action would MOST effectively improve processing speed?

Accepted Answer

Increase the number of shards in the Kinesis stream.. Increasing the number of shards increases the throughput of the stream, allowing more parallel consumers. Option A is wrong because increasing instance size may not help if the bottleneck is the stream. Option C is wrong because adding more consumers without more shards doesn't help (each shard supports one consumer per application). Option D is wrong because KCL handles shard distribution, but more shards are needed.

Answer

Use the Kinesis Client Library (KCL) to automatically distribute shards among instances.

Answer

Increase the EC2 instance size to provide more CPU and memory.

Answer

Add more EC2 instances consuming from the same stream without changing shard count.

Question 13

A company is building a data lake on Amazon S3 and wants to ingest data from multiple AWS services (CloudTrail, VPC Flow Logs, and ALB logs). The data should be stored in a central S3 bucket with a common partitioning scheme. Which service can be used to collect and centralize this data with minimal configuration?

Accepted Answer

Configure each source service to deliver logs directly to the central S3 bucket.. Amazon S3 can be configured as a destination for CloudTrail, VPC Flow Logs, and ALB logs directly, but centralizing requires custom logic. AWS Glue can crawl and catalog, but not ingest. AWS Data Pipeline can copy data but requires setup. However, the best answer is to use S3 replication or a simple Lambda function. But among the options, the most suitable is to use AWS Glue with a custom script? Actually, the question expects S3 cross-region replication? Let's see. The correct answer is to use S3 as a central bucket and configure each service to deliver logs to that bucket. But the options may include 'Use Amazon S3 event notifications to trigger a Lambda function that copies logs to a central bucket.' That is plausible. However, the simplest is to configure each service to deliver to the same bucket. But the question likely expects using S3 replication? I'll go with: Configure each service to deliver logs to a common S3 bucket prefix.

Answer

Use AWS Data Pipeline to copy logs from each source S3 bucket to the central bucket.

Answer

Use AWS Glue to crawl the logs from each source and write to a central S3 bucket.

Answer

Set up Amazon Kinesis Data Firehose to ingest logs from each service and write to S3.

Question 14

A company is running a critical application that generates millions of small JSON files every hour in an S3 bucket. A data engineer needs to process these files in near real-time using AWS Glue. The engineer wants to minimize the latency between file arrival and Glue job start. Which TWO actions should the engineer take?

Accepted Answer

Pre-process the files to consolidate them into larger files before the Glue job runs.. Using S3 event notifications to trigger a Lambda that starts the Glue job reduces latency compared to scheduled jobs. Also, grouping small files into larger ones before Glue processing reduces the overhead. Option A is wrong because increasing DPUs does not reduce start latency. Option C is wrong because batch window increases latency.

Answer

Increase the Glue job's batch window to 600 seconds.

Answer

Increase the number of DPUs for the Glue job to accelerate processing.

Question 15

A data engineer needs to ingest data from an Amazon RDS MySQL database into a data lake on Amazon S3. The engineer wants to perform an initial full load and then capture incremental changes. Which TWO AWS services can be combined to achieve this?

Accepted Answer

Amazon S3. AWS DMS can do full load and CDC. S3 can be the target. Option B is wrong because Glue does not support CDC from MySQL directly. Option C is wrong because Firehose doesn't connect to MySQL. Option D is wrong because Data Pipeline does not support CDC.

Answer

Amazon Kinesis Data Firehose

Answer

AWS Glue

Answer

AWS Data Pipeline