MLS-C01 Data Engineering Practice Test 12 — 15 Questions

Question 1

A data scientist wants to run a one-time SQL query on a large dataset stored in Amazon S3 (CSV format, 2 TB) using Amazon Athena. The query involves joining this dataset with a smaller table stored in Amazon RDS. What is the MOST cost-effective and performant approach?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Export the RDS table to S3 in Parquet format, then use Athena to join the two S3 datasets. Exporting the small RDS table to S3 in Parquet format and then joining it with the 2 TB S3 dataset in Athena is the most cost-effective and performant approach. Athena charges per TB scanned, and Parquet is columnar and compressed, drastically reducing bytes scanned. Joining two S3 datasets in Athena avoids data movement and leverages Athena's distributed SQL engine.

Answer

Use Amazon Redshift Spectrum to query both S3 and RDS

Answer

Use Athena Federated Query to query RDS directly

Answer

Use AWS Glue ETL to join the data and write results back to S3, then query with Athena

Question 2

A company is using Amazon Kinesis Data Analytics for Apache Flink to process real-time sensor data. The application reads from a Kinesis data stream, performs windowed aggregations, and writes results to an S3 bucket. Recently, the application has been experiencing high latency and checkpoint failures. What is the MOST likely cause?

Accepted Answer

The S3 destination bucket is located in a different AWS Region than the Kinesis application. Cross-Region writes from a Flink application to an S3 bucket in a different Region introduce significant network latency and can cause checkpoint timeouts, since checkpoints must complete within the configured timeout and cross-Region S3 writes are slower and less reliable. Co-locating the S3 bucket with the Flink application resolves the latency and checkpoint failures.

Answer

The number of shards in the Kinesis stream is insufficient for the data volume

Answer

The record size in the Kinesis stream exceeds the 1 MB limit

Answer

The parallelism of the Flink application is set too low for the number of shards

Question 3

A data engineer needs to ingest data from an on-premises Apache Kafka cluster into Amazon S3 with minimal latency (under 5 minutes) for real-time analytics. The data volume is approximately 10 MB per second. Which solution is MOST cost-effective and meets the latency requirement?

Accepted Answer

Use Amazon MSK to mirror the on-premises Kafka cluster, then use Kinesis Firehose to write to S3. Amazon MSK can mirror an on-premises Kafka cluster using MirrorMaker 2.0 or MSK Replicator, providing a managed Kafka endpoint in AWS. Kinesis Data Firehose then delivers the data to S3 with a configurable buffer interval (as low as 60 seconds), easily meeting the sub-5-minute latency requirement. This combination is cost-effective because MSK handles the streaming replication and Firehose provides a fully managed, serverless delivery to S3 without needing to manage consumers or Lambda functions.

Answer

Use Amazon S3 Transfer Acceleration for direct uploads from on-premises

Answer

Use Amazon Kinesis Data Streams with a Direct Connect connection from on-premises

Answer

Set up a VPN connection and use AWS Lambda to consume from Kafka and write to S3

Question 4

A data engineer is designing a data lake on Amazon S3. The data comes from various sources and must be stored in a way that supports both batch and real-time analytics. The engineer needs to partition the data to optimize query performance in Amazon Athena. Which partitioning strategy is MOST appropriate?

Accepted Answer

Partition by year, then month, then day, then hour. Partitioning by year, month, day, and hour is most appropriate because it aligns with common query patterns for time-series data, allowing Athena to prune partitions and scan less data. This hierarchical partitioning optimizes performance for both batch and real-time analytics.

Answer

Partition by a hash of the record ID to distribute data evenly

Answer

Do not partition; use a single prefix for all data

Answer

Partition by source type and then by date

Question 5

A company is using Amazon Kinesis Data Streams with 10 shards to ingest clickstream data. Each record is approximately 50 KB. The data is consumed by a Lambda function that writes to DynamoDB. The Lambda function is experiencing throttling errors. Which TWO actions should the data engineer take to resolve the issue? (Choose TWO.)

Accepted Answer

Request a limit increase for the Lambda function's concurrent execution limit. Option C is correct because Lambda throttling errors typically stem from hitting the account's concurrent execution limit; requesting a limit increase allows more simultaneous Lambda invocations to process the Kinesis stream without being throttled. Option E is correct because increasing the batch size in the Lambda event source mapping lets each invocation process more records per poll, reducing the total number of concurrent invocations needed and thus alleviating throttling. Option A is incorrect because increasing record size to 1 MB would not reduce throttling and could worsen it by consuming more Kinesis throughput per record. Option B is incorrect because switching to Kinesis Data Firehose does not address Lambda concurrency throttling and changes the architecture unnecessarily. Option D is incorrect because adding shards increases parallelism and can actually increase the number of concurrent Lambda invocations, potentially worsening throttling rather than resolving it.

Answer

Increase the record size to 1 MB to reduce the number of records

Answer

Switch to Kinesis Data Firehose instead of Data Streams

Answer

Increase the number of shards in the Kinesis stream

Question 6

A data engineer is designing an ETL pipeline using AWS Glue to process data from Amazon S3 and load it into Amazon Redshift. The pipeline must handle incremental data loads and ensure data consistency. Which THREE features should the engineer use to achieve this? (Choose THREE.)

Accepted Answer

Pushdown predicates to filter partitions in S3. Option E (Glue job bookmarks) is correct because job bookmarks persist state across job runs so that AWS Glue only reprocesses new or changed data in S3, which is exactly how incremental loads are implemented. Option A (pushdown predicates) is correct because they let Glue push filtering conditions down to the data source so only relevant S3 partitions are read, reducing scan volume and supporting efficient incremental processing. Option D (Redshift transactional tables with automatic commit) is correct because Redshift's transactional ACID semantics, combined with committing each load, ensure that data is written atomically and consistently, which is required for the consistency guarantee. Option B (Glue data preview) is not correct because it is only a development aid for inspecting sample data, not a mechanism for incremental loading or consistency. Option C (Glue partition filters) is not correct because, while partition pruning reduces scanned data, it is a general optimization already covered by pushdown predicates and does not itself provide incremental state tracking or transactional consistency.

Answer

Glue data preview to validate transformation logic

Answer

Glue partition filters to limit data scanned

Question 7

A company stores IoT sensor data in Amazon S3 and uses Amazon Athena for ad-hoc queries. The data is partitioned by date, but queries are still slow and expensive. Which TWO actions can improve query performance and reduce cost? (Choose TWO.)

Accepted Answer

Use S3 lifecycle policies to compact small files into larger ones. Option B is correct because converting the data from CSV to Parquet, a columnar format, allows Athena to read only the columns referenced in each query and applies efficient compression and predicate pushdown, which dramatically reduces the amount of data scanned and therefore lowers cost and improves performance. Option A is correct because S3 lifecycle policies can be used to transition or expire objects, and in this context they support compaction of many small files into fewer larger ones; fewer, larger objects reduce per-file overhead (S3 GET requests, listing, and partition metadata processing) so Athena scans more efficiently. Option C is wrong because disabling server-side encryption does not affect query performance or cost and would weaken security. Option D is wrong because AWS Glue is primarily an ETL/catalog service, not a replacement query engine for ad-hoc Athena SQL, and switching would not inherently reduce scan cost. Option E is wrong because adding hour-level partitions increases partition metadata overhead and does not reduce the bytes scanned unless queries actually filter on the hour, so it can even hurt performance and cost.

Answer

Disable server-side encryption on the S3 bucket

Answer

Use AWS Glue instead of Athena for querying

Answer

Increase the number of partitions to hour-level granularity

Question 8

A data engineer is troubleshooting an AWS Glue job that reads from and writes to the S3 bucket 'data-lake-bucket'. The job fails when trying to write to the 'sensitive/' prefix. The IAM policy attached to the Glue job's IAM role is shown in the exhibit. What is the MOST likely reason for the failure?

Accepted Answer

The IAM role has an explicit deny for s3:PutObject on the 'sensitive/' prefix. The most likely reason the Glue job fails to write to the sensitive/ prefix is that the IAM role's policy contains an explicit deny for s3:PutObject on that prefix. In AWS IAM, an explicit deny always overrides any allow, so even if the role has broad s3 permissions, the deny on sensitive/* blocks the write. This matches the exhibit showing a Deny statement scoped to the sensitive prefix.

Answer

The IAM role does not have permission to read objects from the bucket

Answer

The IAM policy does not specify the bucket resource correctly

Answer

The IAM policy lacks a required condition for encryption

Question 9

A data engineer runs the AWS CLI command shown in the exhibit to find large log files in S3. The command returns an empty list, but the engineer knows there are files larger than 1 MB in that prefix. What is the MOST likely issue?

Accepted Answer

The command does not paginate through all objects; only the first 1000 are returned. The `list-objects` command returns up to 1000 objects per call. If there are more than 1000 objects under the prefix, the command only examines the first 1000 objects. Since the engineer knows there are files larger than 1 MB, those files likely appear after the first 1000 objects. To find them, pagination is required (e.g., using `--page-size` or `--max-items` and `--starting-token`). Option A is incorrect because the JMESPath query is syntactically valid. Option C is incorrect because the engineer confirmed objects exist under the prefix. Option D is incorrect because `Size` is in bytes, so `1000000` correctly represents 1 MB.

Answer

The JMESPath query syntax is incorrect

Answer

The prefix is incorrect; there are no objects under that prefix

Answer

The Size value is in kilobytes, not bytes

Question 10

A company is building a data pipeline to process streaming data from IoT devices. The data must be ingested with low latency, transformed in real-time using custom logic, and stored in Amazon S3 partitioned by device ID and timestamp. Which combination of AWS services should the company use to meet these requirements?

Accepted Answer

Amazon Kinesis Data Streams with AWS Lambda and Amazon S3. Amazon Kinesis Data Streams ingests streaming data with low latency and supports custom real-time transformation via AWS Lambda (using Kinesis as an event source). Lambda can then write the transformed records to Amazon S3 with partitioning by device ID and timestamp using the S3 object key structure. This combination meets all three requirements: low-latency ingestion, custom real-time transformation, and partitioned S3 storage.

Answer

Amazon Kinesis Data Firehose with direct S3 delivery

Answer

Amazon Managed Streaming for Apache Kafka (MSK) with Amazon S3 sink connector

Answer

Amazon DynamoDB Streams with AWS Lambda and Amazon S3

Question 11

A data scientist is using Amazon SageMaker to train a model. The training data is stored in Amazon S3 and is approximately 500 GB. The data scientist notices that the training job is taking a long time to start because the data is being copied to the training instance's storage. The data scientist wants to reduce the startup time for subsequent training jobs. Which action should the data scientist take?

Accepted Answer

Use Pipe input mode instead of File input mode for the training job. Pipe input mode streams data directly from Amazon S3 to the training instance as the training job runs, eliminating the initial download time required by File input mode. This significantly reduces startup time for large datasets like 500 GB. It is the recommended approach for large datasets to reduce training start latency.

Answer

Use an EBS-optimized instance type

Answer

Use Amazon FSx for Lustre as a high-performance file system mounted to the training instance

Answer

Increase the size of the training instance's Amazon EBS storage volume

Question 12

A company stores sensitive customer data in Amazon S3. The company must ensure that data is encrypted at rest. The company also needs to manage the encryption keys using an AWS service that allows automatic rotation of keys. Which solution meets these requirements?

Accepted Answer

Use Server-Side Encryption with AWS KMS-Managed Keys (SSE-KMS) and enable automatic key rotation. SSE-KMS encrypts objects at rest using keys managed in AWS KMS, and KMS supports automatic annual rotation of customer master keys (CMKs), satisfying both the encryption-at-rest and automatic key rotation requirements. Enabling automatic rotation on the KMS key means AWS generates new backing key material each year while retaining old versions for decryption. This is the only option that combines AWS-managed key storage with built-in rotation.

Answer

Use client-side encryption with AWS CloudHSM

Answer

Use Server-Side Encryption with Customer-Provided Keys (SSE-C)

Answer

Use Server-Side Encryption with Amazon S3-Managed Keys (SSE-S3)

Question 13

A company is building a data pipeline that ingests data from on-premises databases into Amazon S3 using AWS Database Migration Service (AWS DMS). The company wants to capture continuous changes from the source database and replicate them to S3 in near-real time. Which AWS DMS configuration should the company use?

Accepted Answer

Create a CDC-only task with S3 as the target endpoint. A CDC-only task with S3 as the target endpoint is correct because the requirement is to capture continuous changes in near-real time. AWS DMS Change Data Capture (CDC) reads ongoing changes from the source database's transaction logs (e.g., Oracle redo logs, SQL Server transaction log, MySQL binlog) and streams them to the target without re-copying existing data. Since the pipeline is already ingesting data and only needs ongoing replication, CDC-only avoids redundant full-load overhead.

Answer

Create a full-load task to copy the existing data

Answer

Create a full-load plus CDC task with S3 target

Answer

Create a validation task to compare source and target

Question 14

A company is building a data pipeline that uses Amazon Kinesis Data Streams to ingest real-time events. The pipeline then uses AWS Lambda to process the events and store results in Amazon DynamoDB. The company wants to ensure that the Lambda function can process all events without data loss and without duplicating processing. Which TWO configuration steps should the company take?

Accepted Answer

Increase the data retention period of the Kinesis stream to 7 days to allow reprocessing. Option A is correct because increasing the Kinesis Data Streams retention period to 7 days (the maximum, up from the default 24 hours) keeps records available in the stream so that if the Lambda consumer falls behind or fails, events can still be reprocessed from their original position rather than being lost after the default retention window expires. Option E is correct because disabling Lambda's automatic retries on failure prevents the same batch of Kinesis records from being invoked and processed again after an error, which is the primary mechanism that causes duplicate processing in Kinesis-to-Lambda integrations. Option B is incorrect because a small batch window only reduces latency; it does not guarantee lossless or exactly-once processing and can even increase the number of partial batches. Option C is incorrect because the iterator age metric is a monitoring/diagnostic tool, not a configuration step that ensures no data loss or duplication. Option D is incorrect because using a single shard limits throughput and does not by itself prevent data loss or duplicate processing; ordering is already guaranteed per shard, and parallel processing across shards is not the cause of duplication.

Answer

Set the Lambda function's batch window to a small value (e.g., 1 second) to reduce processing latency

Answer

Enable the 'iterator age' metric in Amazon CloudWatch to monitor consumer lag

Answer

Use a single shard for the Kinesis stream to ensure order and avoid parallel processing issues

Question 15

A company is using AWS Glue to catalog data stored in Amazon S3. The data is partitioned by year, month, day, and hour. The company runs hourly ETL jobs that add new partitions. The Glue crawler is scheduled to run every hour to update the Data Catalog. However, the crawler is taking longer than expected and is not completing before the next crawler run starts. Which action could the company take to resolve this issue?

Accepted Answer

Use multiple crawlers, each configured to crawl a different path (e.g., one for year=2023, one for year=2024). Splitting the crawl across multiple crawlers, each targeting a different partition path (e.g., year=2023 vs year=2024), parallelizes the work and reduces the time any single crawler needs to complete. This directly addresses the overlapping-run problem by shortening each crawler's runtime so it finishes before the next scheduled run.

Answer

Increase the throughput of the crawler by configuring the 'Schema updates' option

Answer

Enable partition indexing on the table to speed up the crawler

Answer

Decrease the crawler schedule frequency to every 2 hours to avoid overlapping runs

Answer

Increase the number of crawler instances by configuring the 'Crawler queue' to process multiple partitions in parallel