MLS-C01 Data Engineering Practice Test 24 — 15 Questions

Question 1

A company uses Amazon Kinesis Data Firehose to deliver streaming data to Amazon S3. The data contains personally identifiable information (PII) that must be redacted before storage. Which AWS service can be integrated with Kinesis Data Firehose to transform the data in real time?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

AWS Lambda. AWS Lambda can be integrated as a data transformation function within a Kinesis Data Firehose delivery stream. When Firehose receives incoming records, it can invoke a Lambda function synchronously to process each batch of data, allowing you to redact PII fields in real time before the data is written to Amazon S3.

Answer

Amazon Athena

Answer

Amazon Kinesis Data Analytics

Answer

Amazon EMR

Question 2

A data engineer needs to transfer 50 TB of historical data from an on-premises Hadoop cluster to Amazon S3. The company has a 100 Mbps internet connection and a tight deadline of two weeks. Which AWS service should the engineer use to transfer the data most efficiently?

Accepted Answer

AWS Snowball Edge. B is correct because transferring 50 TB over a 100 Mbps connection would take approximately 48 days (50 TB * 8 / 100 Mbps / 86400 seconds/day), far exceeding the two-week deadline. AWS Snowball Edge is a physical data transport device that can securely transfer petabytes of data offline, bypassing network bandwidth constraints entirely. For large datasets and tight deadlines, Snowball Edge is the most efficient AWS service.

Answer

AWS Storage Gateway (Volume Gateway)

Answer

Amazon S3 Transfer Acceleration

Answer

AWS DataSync over the internet

Question 3

A company uses Amazon Redshift for its data warehouse. The data engineering team needs to load 10 TB of data from Amazon S3 into Redshift every night. The team wants to minimize the load time and use the fewest number of COPY commands. The data is in CSV format and is partitioned by date in S3. Which approach should the team take?

Accepted Answer

Use a manifest file with a single COPY command.. Using a manifest file with a single COPY command is the most efficient approach because it allows Redshift to load data from multiple S3 objects (partitioned by date) in parallel, automatically splitting the workload across cluster nodes. This minimizes load time by leveraging Redshift's parallel processing without requiring multiple COPY commands or manual concatenation, and it avoids the overhead of additional services like AWS Glue for a straightforward bulk load.

Answer

Use multiple COPY commands, one per partition.

Answer

Concatenate all data into a single large file before loading.

Answer

Use AWS Glue to transform the data and then load into Redshift.

Question 4

A data engineer needs to schedule an AWS Glue ETL job to run every hour. The job reads from an Amazon DynamoDB table and writes to Amazon S3. Which AWS service should the engineer use to trigger the Glue job on schedule?

Accepted Answer

Amazon EventBridge (CloudWatch Events). Amazon EventBridge (formerly CloudWatch Events) can trigger AWS Glue jobs on a schedule using cron or rate expressions. Option A is incorrect because Amazon Kinesis Data Streams is for real-time streaming data, not scheduling. Option B is incorrect because AWS Step Functions is for orchestrating workflows, but scheduling is typically done via EventBridge. Option D is incorrect because AWS Lambda is a compute service, not a scheduler, though it can be used in conjunction with EventBridge to trigger Glue, but the direct scheduler is EventBridge.

Answer

Amazon Kinesis Data Streams

Answer

AWS Step Functions

Answer

AWS Lambda

Question 5

A data engineering team is designing a data pipeline that processes streaming data from Amazon Kinesis Data Streams using AWS Lambda. The team notices that some records are being processed multiple times (duplicates). Which TWO steps should the team take to ensure exactly-once processing?

Accepted Answer

Design the Lambda function to be idempotent.. Option A is correct because Lambda's Kinesis event source mapping is at-least-once by design, so the function must be idempotent to make repeated deliveries of the same record produce the same result without side effects. Option B is correct because tracking a unique record identifier (e.g., Kinesis sequence number or a producer-supplied ID) in an external store like DynamoDB lets the function detect and skip records it has already processed, which is the standard pattern for achieving exactly-once semantics on top of at-least-once delivery. Option C is not correct because increasing batch size only changes invocation frequency and batching, not the at-least-once delivery guarantee, so duplicates can still occur. Option D is not correct because KPL provides reliable, ordered delivery with aggregation and retries, but it does not guarantee exactly-once delivery to consumers. Option E is not correct because disabling retries would cause data loss on transient failures and does not prevent duplicates from Kinesis checkpointing behavior.

Answer

Increase the batch size to reduce the number of invocations.

Answer

Use Kinesis Producer Library (KPL) to guarantee exactly-once delivery.

Answer

Disable retries on the Lambda function.

Question 6

A data engineer is building a data pipeline using AWS Glue. The pipeline reads data from Amazon S3, transforms it, and writes it back to S3 in a different format. The engineer needs to handle schema evolution (new columns added over time). Which TWO features of AWS Glue can help manage schema evolution?

Accepted Answer

AWS Glue Data Catalog. Option A, the AWS Glue Data Catalog, is correct because it serves as the central metadata repository that stores table schemas, partition information, and schema versions, allowing the pipeline to detect and reconcile newly added columns as the underlying S3 data evolves. Option B, AWS Glue DynamicFrame, is correct because it is a schema-flexible abstraction that resolves schema inconsistencies at runtime, supports the ResolveChoice and relationalize transformations, and can automatically merge or reconcile new columns without requiring a rigid, pre-defined schema. Option C, AWS Lake Formation, is not correct because it is a permission and governance service for managing access to data lakes, not a mechanism for handling schema evolution. Option D, Amazon Athena, is not correct because it is an interactive query service that reads from the Data Catalog but does not itself manage or evolve schemas for Glue ETL jobs. Option E, Amazon S3 object tags, is not correct because tags are key-value metadata for classification, lifecycle, and access control, and they do not describe or evolve the columnar schema of the data.

Answer

AWS Lake Formation

Answer

Amazon Athena

Answer

Amazon S3 object tags

Question 7

A company is using Amazon Kinesis Data Streams to ingest real-time clickstream data. The data is consumed by a Kinesis Data Analytics application that runs SQL queries. The application has been failing intermittently with 'ProvisionedThroughputExceededException' errors. Which action should be taken to resolve this issue?

Accepted Answer

Increase the number of shards in the Kinesis data stream.. The 'ProvisionedThroughputExceededException' error indicates that the Kinesis Data Stream's read or write throughput limits have been exceeded. Increasing the number of shards in the stream directly increases the total provisioned throughput, allowing the Kinesis Data Analytics application to consume data without throttling.

Answer

Disable error logging in the Kinesis Data Analytics application.

Answer

Increase the record size in the Kinesis data stream.

Answer

Switch from Kinesis Data Analytics to Kinesis Data Firehose.

Question 8

A data engineering team is designing a data pipeline to process large CSV files (10-50 GB each) stored in Amazon S3. The pipeline must transform the data using AWS Glue and load it into Amazon Redshift for analytics. The team wants to minimize costs while ensuring the pipeline can handle peak loads. Which approach is the most cost-effective?

Accepted Answer

Use AWS Glue with Spark and dynamic frames, scaling the number of workers based on file size.. AWS Glue with Spark and dynamic frames is the most cost-effective approach because it is serverless, automatically scales workers based on file size, and is optimized for ETL on large CSV files (10-50 GB) in S3. Dynamic frames provide built-in transformations and schema inference, reducing development effort, while the ability to adjust the number of workers allows handling peak loads without over-provisioning. This minimizes idle compute costs compared to always-on clusters like EMR.

Answer

Use AWS Lambda to process each file and load into Redshift.

Answer

Use Amazon EMR with Hive to transform the data and load into Redshift.

Answer

Use an AWS Glue Python shell job with a single r5.xlarge worker.

Question 9

A data engineer needs to set up a data pipeline that ingests data from an Amazon RDS MySQL database into Amazon S3. The pipeline should run daily and capture incremental changes (inserts, updates, deletes) from the source database. Which AWS service should be used as the data ingestion tool?

Accepted Answer

AWS Database Migration Service (DMS) with continuous change data capture (CDC).. AWS DMS with continuous CDC is the correct choice because it is specifically designed to capture incremental changes (inserts, updates, deletes) from a relational database like Amazon RDS MySQL and replicate them to Amazon S3. DMS uses the MySQL binary log (binlog) to track row-level changes in near real-time, making it ideal for daily incremental pipelines. Other services either lack native CDC support or are not optimized for database-to-object-store incremental ingestion.

Answer

Amazon Kinesis Data Streams with a Lambda function.

Answer

AWS Data Pipeline with a SQL activity.

Answer

AWS Glue with a scheduled crawler.

Question 10

A company is building a data lake on Amazon S3 and wants to use AWS Glue to catalog the data. The data includes CSV, Parquet, and JSON files. The team wants to ensure that the Glue crawler can infer the schema correctly and update the Data Catalog when new partitions are added. Which crawler configuration should be used?

Accepted Answer

Create a single crawler that includes all file extensions and set the 'Update all new and existing partitions' option.. A single AWS Glue crawler can handle multiple file formats (CSV, Parquet, JSON) in a data lake on Amazon S3, and enabling 'Update all new and existing partitions' ensures the Data Catalog is refreshed with both new partitions and any schema changes in existing partitions. This configuration maintains a consistent and up-to-date catalog without manual intervention, which is essential for downstream analytics and machine learning workloads.

Answer

Create separate crawlers for each file format and schedule them at different times.

Answer

Use a crawler that only catalogs Parquet files because they are more efficient.

Answer

Use a crawler with 'Update all new and existing partitions' disabled to avoid schema conflicts.

Question 11

Which THREE factors should be considered when choosing between Amazon Kinesis Data Streams and Amazon Kinesis Data Firehose for a real-time data ingestion pipeline? (Choose three.)

Accepted Answer

The need for custom real-time processing logic using consumer applications.. Option B is correct because Kinesis Data Streams is designed for custom consumer applications (using KCL, Kinesis Client Library, or AWS Lambda) that implement bespoke real-time processing logic, whereas Firehose only delivers to destinations with optional Lambda-based transformation. Option C is correct because latency is a key differentiator: Data Streams offers sub-second to low-second latency for consumers, while Firehose buffers data and delivers in micro-batches, typically resulting in 60-second-or-more latency (or as low as ~5 seconds with near-real-time settings). Option D is correct because Data Streams requires you to provision and manage shards (or use on-demand mode) and handle scaling/resizing, while Firehose is fully managed and auto-scales without shard management. Option A is not correct because built-in data transformation and analytics is not a primary differentiator in this comparison — Firehose offers Lambda-based transformation and Data Streams integrates with Kinesis Data Analytics, so both can support transformation/analytics to varying degrees. Option E is not correct because neither service natively guarantees exactly-once delivery semantics end-to-end; Data Streams provides at-least-once delivery with record-level ordering per shard, and Firehose provides at-least-once delivery to destinations, so exactly-once is not a valid selection factor between them.

Answer

The need for built-in data transformation and analytics.

Answer

The requirement for exactly-once delivery semantics.

Question 12

A company is streaming real-time sensor data from IoT devices to Amazon Kinesis Data Streams. The data is then consumed by an AWS Lambda function that enriches the records with metadata from an Amazon DynamoDB table and writes the results to an Amazon S3 bucket. Recently, the Lambda function has been failing with 'ProvisionedThroughputExceededException' errors from DynamoDB. The data volume is variable, with occasional bursts. Which solution should a data engineer implement to resolve this issue without losing data?

Accepted Answer

Enable DynamoDB auto scaling for the table to automatically adjust read capacity based on demand.. DynamoDB auto scaling dynamically adjusts the table's provisioned read capacity based on actual traffic patterns, handling bursty sensor data without manual intervention. This prevents ProvisionedThroughputExceededExceptions while ensuring no data loss, as the Lambda function can retry failed operations. Auto scaling is the most cost-effective and operationally efficient solution for variable workloads.

Answer

Increase the DynamoDB table's provisioned read capacity units to a high static value.

Answer

Use an Amazon SQS queue to buffer the Lambda requests before querying DynamoDB.

Answer

Configure an Amazon SNS topic to throttle the data stream before it reaches Lambda.

Question 13

An e-commerce company uses Amazon Redshift for analytics. The data engineering team needs to load daily sales data from an S3 bucket that receives new files every hour. The data must be loaded into Redshift with minimal impact on query performance during the day, and they need to handle late-arriving data (files that appear after the daily load). Which approach should they use?

Accepted Answer

Use a staging table to load data incrementally with a MERGE operation, and schedule a late-arriving data job to merge files that arrive after the daily load.. It uses a staging table to incrementally load data with a MERGE operation, which minimizes impact on query performance by avoiding full table overwrites. The separate late-arriving data job handles files that appear after the daily load, ensuring completeness without blocking ongoing queries. This approach aligns with Redshift's best practices for incremental loads and late-arriving data handling.

Answer

Use AWS Glue ETL to copy the data from S3 to Redshift, overwriting the existing data each day.

Answer

Stream the data from S3 using Amazon Kinesis Firehose to load into Redshift continuously.

Answer

Use Amazon Redshift Spectrum to query data directly from S3 and create external tables.

Question 14

A company is using Amazon Redshift for data warehousing. The data engineering team observes that query performance degrades over time due to data skew. Which three strategies should the team implement to improve performance?

Accepted Answer

Choose appropriate distribution keys based on join and group-by columns.. Option A is correct because choosing a distribution key on columns frequently used in joins or GROUP BY clauses co-locates matching rows on the same slice, eliminating data movement (redistribution or broadcast) across nodes and directly mitigating skew-related slowdowns. Option C is correct because VACUUM reclaims space from deleted rows and re-sorts data to restore sort-key ordering, while ANALYZE refreshes table statistics so the query planner generates efficient plans; both are essential maintenance tasks as data changes over time. Option D is correct because defining appropriate sort keys (compound or interleaved) enables zone maps to skip irrelevant blocks during scans, reducing I/O and improving query performance on large tables. Option B is not correct because adding nodes increases capacity and parallelism but does not fix the underlying skew problem, and a poorly chosen distribution key can actually worsen skew across the additional slices. Option E is not correct because Amazon Redshift does not use traditional indexes; it relies on sort keys, distribution keys, and zone maps, so dropping indexes is not a valid Redshift tuning strategy.

Answer

Increase the number of nodes in the Redshift cluster.

Answer

Drop unused indexes on large tables.

Question 15

Refer to the exhibit. A data engineer runs an Athena query and gets a failure. What is the most likely cause?

Accepted Answer

The IAM role used does not have permissions to write to S3.. The most likely cause for an Athena query failure is that the IAM role used does not have the necessary permissions to write query results to the specified S3 bucket. While syntax errors or encryption settings can cause issues, permissions are a frequent and common cause of failure in practice.

Answer

The query result location uses a bucket with default encryption enabled.

Answer

The SQL query syntax is incorrect.

Answer

The output S3 bucket specified in the query result configuration already exists.