MLS-C01 Data Engineering Practice Test 22 — 15 Questions

Question 1

A data engineer needs to transform large CSV files stored in Amazon S3 into Parquet format before loading into Amazon Redshift. The transformation logic is complex and requires custom Python code. Which AWS service should be used to perform this transformation with minimal operational overhead?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

AWS Glue. AWS Glue is the correct answer because it is a fully managed, serverless ETL service that can handle large CSV files, convert them to Parquet, and load into Amazon Redshift with minimal operational overhead. AWS Glue provides a built-in Spark environment and supports custom Python code via Spark jobs. Option B (AWS Lambda) has a 15-minute timeout and is not designed for large-scale data transformations. Option C (Amazon EMR) requires managing clusters, increasing operational overhead. Option D (AWS Data Pipeline) is a legacy service with less flexibility and is not optimized for complex transformations like CSV to Parquet.

Answer

AWS Lambda

Answer

Amazon EMR

Answer

AWS Data Pipeline

Question 2

A company is streaming clickstream data from a website to Amazon Kinesis Data Streams. The data is consumed by a Lambda function that enriches each record with geolocation information before writing to an S3 bucket. Recently, the Lambda function has been failing with throttling errors. What is the MOST likely cause?

Accepted Answer

The Lambda function's concurrent execution limit has been reached. The most likely cause is that the Lambda function's concurrent execution limit has been reached. Kinesis Data Streams invokes Lambda functions per shard, and with high throughput or many shards, concurrent invocations can exceed the default Lambda concurrency limit (1000 per region). This results in throttling errors. Option A is incorrect because the Lambda payload limit for asynchronous invocation (used by Kinesis) is 256 KB, not 6 MB. Option C is incorrect; setting reserved concurrency too high would not cause throttling—it could actually help avoid throttling. Option D is incorrect because the default shard limit is 500, but shard limits are a Kinesis concern, not directly causing Lambda throttling.

Answer

The Lambda function's payload size exceeds the 6 MB limit

Answer

The Lambda function's reserved concurrency is set too high

Answer

The Kinesis stream has exceeded the default shard limit of 500

Question 3

A data scientist wants to run a one-time SQL query on a large dataset stored in Amazon S3 (CSV format, 2 TB) using Amazon Athena. The query involves joining this dataset with a smaller table stored in Amazon RDS. What is the MOST cost-effective and performant approach?

Accepted Answer

Export the RDS table to S3 in Parquet format, then use Athena to join the two S3 datasets. Exporting the RDS table to S3 as Parquet and running the join in Athena avoids data transfer costs and leverages Athena's fast query engine. Option B (federated query) adds complexity and may be slower. Option C (Redshift Spectrum) requires a Redshift cluster. Option D (Glue ETL) is overkill for a one-time query.

Answer

Use Amazon Redshift Spectrum to query both S3 and RDS

Answer

Use Athena Federated Query to query RDS directly

Answer

Use AWS Glue ETL to join the data and write results back to S3, then query with Athena

Question 4

A company is using Amazon Kinesis Data Firehose to load streaming data into an S3 bucket. The data schema evolves over time, with new columns added. The data must be queryable using Amazon Athena. What is the BEST way to handle schema changes?

Accepted Answer

Use AWS Glue Crawlers to automatically detect schema changes and update the table metadata. AWS Glue Crawlers can automatically detect schema changes in the data stored in S3 and update the AWS Glue Data Catalog metadata used by Athena. This allows Athena to query the evolving schema without manual intervention. Option A (manual update) is not the best because it requires manual effort and is error-prone. Option B (converting to JSON) is not necessary; Athena can handle various formats including Parquet, ORC, etc., and schema evolution is better handled by Glue Crawlers. Option D (recreating the table daily) is disruptive and not the best practice.

Answer

Manually update the Athena table definition each time a new column is added

Answer

Configure Firehose to convert the data to Apache JSON format

Answer

Recreate the Athena table daily to pick up new columns

Question 5

A company runs a daily batch ETL job using AWS Glue that reads from Amazon RDS (MySQL), transforms the data, and writes to Amazon Redshift. The job takes 6 hours and processes 500 GB of data. Management wants to reduce the runtime. Which action would be MOST effective?

Accepted Answer

Increase the number of DPUs allocated to the Glue job. Increasing the number of DPUs (Data Processing Units) allocated to the AWS Glue job directly increases the parallelism of the Spark-based ETL job, allowing it to process the 500 GB of data faster. Since the bottleneck is the Glue job's compute capacity, adding more DPUs reduces runtime without changing the source or target infrastructure.

Answer

Increase the node size of the Redshift cluster

Answer

Use the Redshift COPY command to load data directly from RDS

Answer

Use Amazon RDS with Provisioned IOPS SSD storage

Question 6

A company is using Amazon Kinesis Data Analytics for Apache Flink to process real-time sensor data. The application reads from a Kinesis data stream, performs windowed aggregations, and writes results to an S3 bucket. Recently, the application has been experiencing high latency and checkpoint failures. What is the MOST likely cause?

Accepted Answer

The S3 destination bucket is located in a different AWS Region than the Kinesis application. The S3 destination bucket is located in a different AWS Region than the Kinesis application. Cross-region data transfer introduces network latency and increases the likelihood of checkpoint failures because Apache Flink checkpoints require writes to complete within a timeout. Option A (insufficient shards) would cause throttling (ProvisionedThroughputExceededException) but not directly checkpoint failures. Option C (record size > 1 MB) is impossible because Kinesis Data Streams enforces a 1 MB maximum record size. Option D (low parallelism) could cause backpressure but not typically checkpoint failures unless resources are severely constrained. Therefore, the most likely cause is the cross-region S3 bucket.

Answer

The number of shards in the Kinesis stream is insufficient for the data volume

Answer

The record size in the Kinesis stream exceeds the 1 MB limit

Answer

The parallelism of the Flink application is set too low for the number of shards

Question 7

A data engineer needs to ingest data from an on-premises Apache Kafka cluster into Amazon S3 with minimal latency (under 5 minutes) for real-time analytics. The data volume is approximately 10 MB per second. Which solution is MOST cost-effective and meets the latency requirement?

Accepted Answer

Use Amazon MSK to mirror the on-premises Kafka cluster, then use Kinesis Firehose to write to S3. Amazon MSK (Managed Streaming for Apache Kafka) can mirror the on-premises Kafka topics to the cloud with near-real-time replication (typically under 1 minute), and then Kinesis Firehose can be configured with a 60-second buffer interval to deliver data to Amazon S3, meeting the under-5-minute latency requirement. This solution is cost-effective because MSK eliminates the need to manage Kafka infrastructure, and Firehose provides serverless, pay-per-use data delivery without provisioning servers.

Option B (S3 Transfer Acceleration) is designed for accelerating file uploads over long distances, not for continuous streaming of data from Kafka; it would require additional application changes and is not suitable for real-time streaming.

Option C (Kinesis Data Streams with Direct Connect) would require re-architecting the data pipeline to send data directly to Kinesis, adding complexity and cost (Direct Connect bandwidth). Kinesis Data Streams also requires managing shards and does not natively integrate with Kafka replication.

Option D (VPN + Lambda) introduces additional latency due to VPN overhead and Lambda cold starts, and Lambda is not optimized for high-throughput streaming of 10 MB/s; it would likely cause throttling or increased costs. Therefore, option A is the most cost-effective and meets the latency requirement.

Answer

Use Amazon S3 Transfer Acceleration for direct uploads from on-premises

Answer

Use Amazon Kinesis Data Streams with a Direct Connect connection from on-premises

Answer

Set up a VPN connection and use AWS Lambda to consume from Kafka and write to S3

Question 8

A company uses Amazon Redshift for data warehousing. The data engineering team needs to load data from multiple S3 buckets into Redshift daily. Each bucket contains files in different formats (CSV, JSON, Parquet). Which AWS service is BEST suited to automate this ingestion process?

Accepted Answer

AWS Glue. AWS Glue is a fully managed ETL service that can crawl S3 buckets to discover schema, handle various formats (CSV, JSON, Parquet), and load data into Redshift. It automates the ingestion process without the need for manual infrastructure management. Amazon EMR with Spark requires more setup and management, AWS Data Pipeline is less flexible and older, and AWS Database Migration Service is designed for migrating entire databases, not for loading from S3.

Answer

Amazon EMR with Apache Spark

Answer

AWS Data Pipeline

Answer

AWS Database Migration Service (DMS)

Question 9

A company is using Amazon Kinesis Data Streams with 10 shards to ingest clickstream data. Each record is approximately 50 KB. The data is consumed by a Lambda function that writes to DynamoDB. The Lambda function is experiencing throttling errors. Which TWO actions should the data engineer take to resolve the issue? (Choose TWO.)

Accepted Answer

Request a limit increase for the Lambda function's concurrent execution limit. The Lambda function is experiencing throttling errors because it is being invoked too frequently. To resolve this, the data engineer should increase the Lambda function's concurrent execution limit (option C) to allow more simultaneous executions, and increase the batch size in the Lambda event source mapping (option E) to process more records per invocation, reducing the number of invocations. Option A (increase record size) is irrelevant as it would increase data volume. Option B (switch to Kinesis Data Firehose) changes the architecture and does not directly address Lambda throttling. Option D (increase the number of shards) would increase throughput but also potentially increase concurrency without solving the throttling issue. Therefore, the correct answers are C and E.

Answer

Increase the record size to 1 MB to reduce the number of records

Answer

Switch to Kinesis Data Firehose instead of Data Streams

Answer

Increase the number of shards in the Kinesis stream

Question 10

A data engineer is designing an ETL pipeline using AWS Glue to process data from Amazon S3 and load it into Amazon Redshift. The pipeline must handle incremental data loads and ensure data consistency. Which THREE features should the engineer use to achieve this? (Choose THREE.)

Accepted Answer

Pushdown predicates to filter partitions in S3. (pushdown predicates) filters S3 partitions, reducing data scanned and enabling efficient incremental loads. Option D (Redshift transactional tables with automatic commit) ensures data consistency during writes. Option E (Glue job bookmarks) tracks processed data, supporting incremental processing. Option B (Glue data preview) is used for development and does not contribute to incremental loading or consistency. Option C (Glue partition filters) is less efficient than pushdown predicates for filtering partitions.

Answer

Glue data preview to validate transformation logic

Answer

Glue partition filters to limit data scanned

Question 11

A company stores IoT sensor data in Amazon S3 and uses Amazon Athena for ad-hoc queries. The data is partitioned by date, but queries are still slow and expensive. Which TWO actions can improve query performance and reduce cost? (Choose TWO.)

Accepted Answer

Use S3 lifecycle policies to compact small files into larger ones. Compacts small files into larger ones, reducing the number of objects and minimizing metadata overhead, which improves query performance. Option B converts data from CSV to Parquet, a columnar format that reduces the amount of data scanned by Athena, lowering cost and speeding up queries. Option C (disabling encryption) does not affect performance and is not recommended. Option D (using Glue) is a different service and not a direct improvement for Athena queries. Option E (increasing partitions to hour-level) can create many small files, degrading performance.

Answer

Disable server-side encryption on the S3 bucket

Answer

Use AWS Glue instead of Athena for querying

Answer

Increase the number of partitions to hour-level granularity

Question 12

A data engineer is troubleshooting an AWS Glue job that reads from and writes to the S3 bucket 'data-lake-bucket'. The job fails when trying to write to the 'sensitive/' prefix. The IAM policy attached to the Glue job's IAM role is shown in the exhibit. What is the MOST likely reason for the failure?

Accepted Answer

The IAM role has an explicit deny for s3:PutObject on the 'sensitive/' prefix. Even though the first statement allows s3:PutObject on the entire bucket, the second statement explicitly denies s3:PutObject on the 'sensitive/' prefix. Explicit deny overrides any allow. Option A is wrong because the policy allows GetObject. Option C is wrong because the policy covers the bucket. Option D is wrong because there is a deny statement.

Answer

The IAM role does not have permission to read objects from the bucket

Answer

The IAM policy does not specify the bucket resource correctly

Answer

The IAM policy lacks a required condition for encryption

Question 13

A data engineer runs the AWS CLI command shown in the exhibit to find large log files in S3. The command returns an empty list, but the engineer knows there are files larger than 1 MB in that prefix. What is the MOST likely issue?

Accepted Answer

The command does not paginate through all objects; only the first 1000 are returned. The `list-objects` command returns up to 1000 objects per call. If there are more than 1000 objects under the prefix, the command only examines the first 1000 objects. Since the engineer knows there are files larger than 1 MB, those files likely appear after the first 1000 objects. To find them, pagination is required (e.g., using `--page-size` or `--max-items` and `--starting-token`). Option A is incorrect because the JMESPath query is syntactically valid. Option C is incorrect because the engineer confirmed objects exist under the prefix. Option D is incorrect because `Size` is in bytes, so `1000000` correctly represents 1 MB.

Answer

The JMESPath query syntax is incorrect

Answer

The prefix is incorrect; there are no objects under that prefix

Answer

The Size value is in kilobytes, not bytes

Question 14

A Lambda function is triggered by S3 events. The event payload shown in the exhibit is received by the Lambda function. The function is supposed to process the CSV file and load it into DynamoDB. However, the function fails because it cannot read the file. What is the MOST likely cause?

Accepted Answer

The Lambda function's IAM role does not have s3:GetObject permission. The Lambda function cannot read the file from S3 because its IAM role does not have the s3:GetObject permission. Option A is wrong because the failure is not due to DynamoDB write permissions; the function fails before writing. Option C is wrong because the S3 bucket exists, as indicated by the event triggering. Option D is wrong because the event notification is correctly configured to trigger the Lambda function, as evidenced by the function receiving the event.

Answer

The Lambda function lacks DynamoDB write permissions

Answer

The S3 bucket does not exist

Answer

The S3 event notification is misconfigured

Question 15

A company is building a data pipeline to process streaming data from IoT devices. The data must be ingested with low latency, transformed in real-time using custom logic, and stored in Amazon S3 partitioned by device ID and timestamp. Which combination of AWS services should the company use to meet these requirements?

Accepted Answer

Amazon Kinesis Data Streams with AWS Lambda and Amazon S3. Amazon Kinesis Data Streams provides low-latency ingestion of streaming data, AWS Lambda can apply custom transformation logic in real-time, and the transformed data can be stored in Amazon S3 with partitioning by device ID and timestamp using AWS Lambda to write to S3 with appropriate prefix. Option A is incorrect because Kinesis Data Firehose does not support custom transformation without invoking a Lambda function and cannot partition on write at the level of granularity required (device ID and timestamp). Option B is incorrect because Amazon MSK adds operational overhead and is more complex than needed; although an S3 sink connector can write to S3, it does not easily support custom transformation and partitioning by device ID and timestamp without additional configuration. Option C is incorrect because DynamoDB Streams is designed for change data capture from DynamoDB tables and is not suitable for direct ingestion of high-volume IoT streaming data.

Answer

Amazon Kinesis Data Firehose with direct S3 delivery

Answer

Amazon Managed Streaming for Apache Kafka (MSK) with Amazon S3 sink connector

Answer

Amazon DynamoDB Streams with AWS Lambda and Amazon S3