MLS-C01 Data Engineering Practice Test 16 — 15 Questions

Question 1

A data engineer needs to collect and analyze log data from multiple EC2 instances in real-time. The solution should be serverless and scalable. Which TWO AWS services should be used?

Accepted Answer

Amazon Kinesis Data Firehose. Amazon Kinesis Data Firehose is the correct choice because it is a fully managed, serverless service that can capture, transform, and load streaming log data from EC2 instances into destinations like Amazon S3 or Amazon OpenSearch Service in near real-time, with no infrastructure to manage. It automatically scales to handle high-throughput data streams, making it ideal for real-time log analytics.

Answer

Amazon EMR

Answer

Amazon Athena

Answer

Amazon S3

Question 2

A company is using AWS Glue ETL jobs to transform data. The jobs are failing due to insufficient memory. The data processing involves complex joins and aggregations. Which THREE actions can improve job performance and reduce memory usage?

Accepted Answer

Filter and project data early in the transformation to reduce data volume. Filtering and projecting data early in the transformation reduces the volume of data that must be processed in subsequent operations like joins and aggregations. By using pushdown predicates and selecting only necessary columns, you minimize the data shuffled across the cluster, which directly reduces memory pressure and improves job performance in AWS Glue ETL.

Answer

Decrease the number of DPUs allocated to the job

Answer

Use a single node cluster to avoid shuffle overhead

Question 3

A data engineering team needs to ingest streaming data from thousands of IoT devices into Amazon S3 for near-real-time analytics. The solution must handle data that arrives in bursts and must be able to reprocess failed records automatically. Which combination of AWS services should the team use?

Accepted Answer

Amazon Kinesis Data Streams with AWS Lambda. Amazon Kinesis Data Streams is designed for real-time ingestion of large volumes of streaming data, such as from thousands of IoT devices, and can handle bursty traffic by scaling shards. AWS Lambda can be used as a consumer to process records in near-real-time, and Kinesis Data Streams supports automatic retries and checkpointing, enabling reprocessing of failed records without data loss.

Answer

AWS Glue with Amazon S3

Answer

Amazon SQS with AWS Lambda

Answer

Amazon DynamoDB Streams with AWS Lambda

Question 4

A data engineer is designing a data pipeline that transforms raw JSON files (each 50-200 KB) in Amazon S3 into Parquet format using AWS Glue. The pipeline must minimize data processing costs and handle a high volume of small files (millions per day). The engineer configures a Glue ETL job with Spark, but the job is slow and expensive due to overhead of reading many small files. Which optimization should the engineer implement to reduce cost and improve performance?

Accepted Answer

Use S3 object grouping or batch operations to combine small files before Glue processing.. The primary performance bottleneck with many small files in S3 is the overhead of listing, opening, and reading each file individually in Spark. By grouping or batching small files into larger objects (e.g., using S3 Batch Operations or a pre-processing step), you reduce the number of input splits and task launches, which dramatically lowers the cost and runtime of the Glue ETL job. This directly addresses the root cause of the inefficiency rather than merely scaling resources.

Answer

Increase the worker type to G.2X for more memory per worker.

Answer

Increase the number of DPUs allocated to the Glue job.

Answer

Change the output format from Parquet to CSV to reduce compression overhead.

Question 5

A company uses Amazon Kinesis Data Firehose to deliver streaming data to Amazon S3. The data contains personally identifiable information (PII) that must be redacted before storage. Which AWS service can be integrated with Kinesis Data Firehose to transform the data in real time?

Accepted Answer

AWS Lambda. AWS Lambda can be integrated as a data transformation function within a Kinesis Data Firehose delivery stream. When Firehose receives incoming records, it can invoke a Lambda function synchronously to process each batch of data, allowing you to redact PII fields in real time before the data is written to Amazon S3.

Answer

Amazon Athena

Answer

Amazon Kinesis Data Analytics

Answer

Amazon EMR

Question 6

A data engineering team needs to build a data lake on Amazon S3 that will be queried by Amazon Athena and Amazon Redshift Spectrum. The data will be ingested from multiple sources in various formats (CSV, JSON, Parquet). Which partitioning strategy will provide the best query performance for date-range queries?

Accepted Answer

Partition by year, month, and day in a hierarchical structure.. Partitioning by year, month, and day in a hierarchical structure minimizes the amount of data scanned by Amazon Athena and Redshift Spectrum for date-range queries. Athena and Redshift Spectrum both charge per byte scanned, so reducing the scan size directly improves performance and reduces cost. A hierarchical partition structure (e.g., s3://bucket/year=2023/month=11/day=01/) allows the query engine to prune partitions at each level, efficiently skipping irrelevant directories for queries like WHERE date BETWEEN '2023-11-01' AND '2023-11-30'.

Answer

Partition by date with one partition per day in a flat structure.

Answer

Do not partition; let Athena scan the entire dataset.

Answer

Partition by source system first, then by date.

Question 7

A company has an AWS Glue ETL job that reads data from an Amazon RDS for MySQL table and writes to Amazon S3 in Parquet format. The job runs daily and processes 500 GB of data. Recently, the job has been failing with memory errors during the write phase. The data schema is wide (200 columns). Which change should a data engineer make to the Glue job to resolve the memory issue?

Accepted Answer

Configure the write operation with 'groupSize' to limit records per file.. The memory error occurs because the wide schema (200 columns) and large data volume (500 GB) cause the Spark executors to run out of memory when writing Parquet files, as each executor attempts to buffer entire partitions. Configuring 'groupSize' limits the number of records written per file, reducing the per-executor memory footprint and preventing out-of-memory errors during the write phase.

Answer

Increase the number of DPUs for the Glue job.

Answer

Change the output format from Parquet to CSV.

Answer

Use the JDBC connection with fetchSize parameter.

Question 8

A data engineer needs to transfer 50 TB of historical data from an on-premises Hadoop cluster to Amazon S3. The company has a 100 Mbps internet connection and a tight deadline of two weeks. Which AWS service should the engineer use to transfer the data most efficiently?

Accepted Answer

AWS Snowball Edge. B is correct because transferring 50 TB over a 100 Mbps connection would take approximately 48 days (50 TB * 8 / 100 Mbps / 86400 seconds/day), far exceeding the two-week deadline. AWS Snowball Edge is a physical data transport device that can securely transfer petabytes of data offline, bypassing network bandwidth constraints entirely. For large datasets and tight deadlines, Snowball Edge is the most efficient AWS service.

Answer

AWS Storage Gateway (Volume Gateway)

Answer

Amazon S3 Transfer Acceleration

Answer

AWS DataSync over the internet

Question 9

A data engineering team is building a real-time fraud detection system. Transactions are ingested via Amazon Kinesis Data Streams, and a machine learning model (deployed on Amazon SageMaker) scores each transaction. The team needs to store the raw transactions and the model's predictions in Amazon S3 for later analysis. Which architecture should the team use?

Accepted Answer

Use Amazon Kinesis Data Analytics for Apache Flink to enrich records with SageMaker predictions, then output to Firehose for S3.. It uses Amazon Kinesis Data Analytics for Apache Flink to perform real-time enrichment by invoking the SageMaker endpoint for each transaction, then streams the enriched records to Kinesis Data Firehose for reliable, batched delivery to S3. This architecture handles the asynchronous nature of model inference without blocking the ingestion stream, and Firehose provides automatic retry and compression for S3 storage.

Answer

Use AWS Lambda to read from Kinesis, invoke SageMaker, and write directly to S3.

Answer

Use Amazon Kinesis Data Firehose with a transformation Lambda to call SageMaker.

Answer

Use AWS Lambda to invoke the SageMaker endpoint for each record, then write to S3 via Firehose.

Question 10

A company uses Amazon Redshift for its data warehouse. The data engineering team needs to load 10 TB of data from Amazon S3 into Redshift every night. The team wants to minimize the load time and use the fewest number of COPY commands. The data is in CSV format and is partitioned by date in S3. Which approach should the team take?

Accepted Answer

Use a manifest file with a single COPY command.. Using a manifest file with a single COPY command is the most efficient approach because it allows Redshift to load data from multiple S3 objects (partitioned by date) in parallel, automatically splitting the workload across cluster nodes. This minimizes load time by leveraging Redshift's parallel processing without requiring multiple COPY commands or manual concatenation, and it avoids the overhead of additional services like AWS Glue for a straightforward bulk load.

Answer

Use multiple COPY commands, one per partition.

Answer

Concatenate all data into a single large file before loading.

Answer

Use AWS Glue to transform the data and then load into Redshift.

Question 11

A data engineer needs to schedule an AWS Glue ETL job to run every hour. The job reads from an Amazon DynamoDB table and writes to Amazon S3. Which AWS service should the engineer use to trigger the Glue job on schedule?

Accepted Answer

Amazon EventBridge (CloudWatch Events). Amazon EventBridge (formerly CloudWatch Events) can trigger AWS Glue jobs on a schedule using cron or rate expressions. Option A is incorrect because Amazon Kinesis Data Streams is for real-time streaming data, not scheduling. Option B is incorrect because AWS Step Functions is for orchestrating workflows, but scheduling is typically done via EventBridge. Option D is incorrect because AWS Lambda is a compute service, not a scheduler, though it can be used in conjunction with EventBridge to trigger Glue, but the direct scheduler is EventBridge.

Answer

Amazon Kinesis Data Streams

Answer

AWS Step Functions

Answer

AWS Lambda

Question 12

A data engineering team is designing a data pipeline that processes streaming data from Amazon Kinesis Data Streams using AWS Lambda. The team notices that some records are being processed multiple times (duplicates). Which TWO steps should the team take to ensure exactly-once processing?

Accepted Answer

Design the Lambda function to be idempotent.. Options A and B are correct. Making the Lambda function idempotent ensures that processing the same record multiple times does not cause duplicates downstream. Using a unique identifier per record and storing processed IDs in an external store like DynamoDB allows deduplication by checking if a record has already been processed. Option C is incorrect because increasing batch size does not prevent duplicates and may increase the chance of processing failures. Option D is incorrect because KPL provides exactly-once delivery to Kinesis Data Streams, not from the stream to Lambda, so deduplication is still needed. Option E is incorrect because disabling retries can lead to data loss without guaranteeing exactly-once processing.

Answer

Increase the batch size to reduce the number of invocations.

Answer

Use Kinesis Producer Library (KPL) to guarantee exactly-once delivery.

Answer

Disable retries on the Lambda function.

Question 13

A company uses Amazon Athena to query a data lake in Amazon S3. The data is partitioned by year, month, day, and hour. The team notices that queries are slow and expensive. The team wants to improve performance and reduce costs. Which THREE actions should the team take?

Accepted Answer

Ensure queries filter on partition columns (year, month, day, hour).. Athena charges based on the amount of data scanned per query. By filtering on partition columns (year, month, day, hour), Athena uses partition pruning to skip reading irrelevant S3 prefixes, drastically reducing the data scanned and thus lowering both cost and query latency.

Answer

Increase the number of partitions by adding a partition for minute.

Answer

Use CSV format with GZIP compression.

Question 14

A data engineer is building a data pipeline using AWS Glue. The pipeline reads data from Amazon S3, transforms it, and writes it back to S3 in a different format. The engineer needs to handle schema evolution (new columns added over time). Which TWO features of AWS Glue can help manage schema evolution?

Accepted Answer

AWS Glue Data Catalog. AWS Glue Data Catalog is correct because it stores schema metadata and can be updated automatically or manually to reflect new columns added to source data, enabling schema evolution tracking. AWS Glue DynamicFrame is correct because it provides a flexible, schema-on-read structure that can accommodate varying schemas across records, allowing transformations to handle new columns without breaking the pipeline.

Answer

AWS Lake Formation

Answer

Amazon Athena

Answer

Amazon S3 object tags

Question 15

A data engineer uses the IAM policy above for an AWS Lambda function that processes data in S3 and triggers an AWS Glue job. The Lambda function is unable to start the Glue job. What is the most likely cause?

Accepted Answer

The policy does not include iam:PassRole permission.. The Lambda function needs to pass an IAM role to AWS Glue when starting a job, which requires the `iam:PassRole` permission. Without this permission, the `StartJobRun` API call fails even if the Lambda has permissions to invoke Glue. The policy shown lacks this critical permission, making option C the correct answer.

Answer

The Glue job name in the resource ARN is misspelled.

Answer

The policy does not allow s3:PutObject on the bucket.

Answer

The policy does not include s3:GetObject on the bucket.