MLS-C01 Data Engineering Practice Test 24 — 15 Questions

Question 1

A company uses Amazon EMR to run Spark jobs on a cluster with 10 core nodes of type r5.xlarge. The jobs are I/O intensive and read large amounts of data from S3. The team notices high network throughput but low CPU utilization. Which configuration change would improve job performance at the same cost?

Accepted Answer

Replace the core nodes with r5d.xlarge instances that have local SSDs.. Option B is correct because r5d instances include local NVMe SSDs, which can be used for caching intermediate data, reducing network I/O and improving performance for I/O intensive jobs. Option A is wrong because increasing core nodes increases cost. Option C is wrong because using spot instances reduces cost but not performance. Option D is wrong because moving to m5 instances (general purpose) may not improve I/O.

Answer

Change the instance type to m5.xlarge (general purpose) to balance resources.

Answer

Increase the number of core nodes to 20.

Answer

Use spot instances for the core nodes to save cost and reinvest in more nodes.

Question 2

A company wants to build a data lake on Amazon S3. The data lake will store raw data in its original format and also transformed data in Parquet. The data is generated by various sources and must be cataloged for discovery. Which service should the company use to automatically discover, catalog, and make the data searchable?

Accepted Answer

AWS Glue Data Catalog. Option D is correct. AWS Glue Data Catalog is a central metadata repository that can automatically crawl S3 data sources to populate the catalog. Option A is wrong because Athena is a query engine, not a catalog. Option B is wrong because EMR is a processing framework. Option C is wrong because S3 is storage.

Answer

Amazon S3

Answer

Amazon Athena

Answer

Amazon EMR

Question 3

Refer to the exhibit. A data engineer has attached this IAM policy to an IAM role used by an AWS Glue ETL job. The job reads from an S3 bucket (data-bucket) that is encrypted with SSE-KMS using the key arn:aws:kms:us-east-1:123456789012:key/abc123, transforms the data, and writes the result to a different S3 bucket (output-bucket) encrypted with a different KMS key (arn:aws:kms:us-east-1:123456789012:key/xyz789). When the job runs, it fails with an access denied error. What is the cause?

Accepted Answer

The policy does not grant kms:Encrypt permission for the output bucket's KMS key.. Option C is correct because the policy grants kms:Decrypt and kms:GenerateDataKey for the input bucket's key, but does not grant the equivalent permissions for the output bucket's key (xyz789). The job needs to encrypt the output, so it needs kms:Encrypt (or GenerateDataKey) for the output key. Option A is wrong because the policy includes s3:PutObject. Option B is wrong because the Glue catalog permissions are sufficient. Option D is wrong because the policy includes s3:GetObject.

Answer

The policy does not include s3:GetObject permission for the output bucket.

Answer

The policy does not include glue:CreateTable permission.

Answer

The policy does not include s3:PutObject permission for the output bucket.

Question 4

A company uses Amazon Kinesis Data Firehose to deliver data to Amazon S3. The data must be transformed before delivery using AWS Lambda. The Lambda function adds a timestamp field. The Firehose stream receives up to 10,000 records per second. The transformation currently takes 500 ms per record. What should the team do to ensure the transformation can keep up with the incoming data without data loss?

Accepted Answer

Increase the Lambda concurrency limit for the function to handle parallel invocations.. Option B is correct because increasing the Lambda concurrency limit ensures that multiple Lambda invocations can run in parallel to handle the high throughput. Option A is wrong because increasing the buffer size would cause delays and potential data loss if the buffer fills up. Option C is wrong because increasing the number of shards applies to Kinesis Data Streams, not Firehose. Option D is wrong because Lambda functions in a VPC may have reduced network performance and are not needed for adding a timestamp.

Answer

Increase the number of shards in the Kinesis stream.

Answer

Place the Lambda function in a VPC to improve network performance.

Answer

Increase the S3 buffer size and buffer interval in the Firehose delivery stream.

Question 5

A company runs a critical data pipeline using Apache Spark on Amazon EMR. The pipeline reads data from Amazon S3, performs complex transformations, and writes results back to S3. The job runs every hour and must complete within 30 minutes. Recently, the job has been taking longer and occasionally failing due to executor losses. The team suspects memory pressure. Which action should the team take to improve stability and performance without increasing cost?

Accepted Answer

Enable Spark dynamic allocation to adjust executors based on workload.. Option C is correct because enabling dynamic allocation allows Spark to release idle executors and request more when needed, reducing memory pressure. Option A is wrong because increasing the number of core nodes increases cost. Option B is wrong because increasing executor memory per node may cause YARN containers to fail if the instance memory is exceeded. Option D is wrong because reducing shuffle partitions may reduce parallelism and increase memory per task.

Answer

Increase the spark.executor.memory setting to allocate more memory per executor.

Answer

Increase the number of core nodes in the EMR cluster.

Answer

Decrease the number of shuffle partitions (spark.sql.shuffle.partitions) to reduce overhead.

Question 6

A company runs a real-time analytics platform that ingests IoT sensor data from millions of devices. The data is sent to Amazon Kinesis Data Streams with 16 shards. A custom Java application using the Kinesis Client Library (KCL) processes the data and writes aggregated results to Amazon DynamoDB. The application runs on a fleet of EC2 instances in an Auto Scaling group. Recently, the team noticed that some records are being processed multiple times, resulting in duplicate entries in DynamoDB. The application uses the DynamoDB PutItem API to write records. The team needs to eliminate duplicates without significantly increasing latency. Which solution should the team implement?

Accepted Answer

Use DynamoDB TransactWriteItems with a condition check that the record's Kinesis sequence number does not already exist in the table.. Option B is correct because using a DynamoDB transaction with a condition check on the Kinesis sequence number ensures that each record is written only once. Option A is wrong because idempotent writes would require a unique identifier; using PutItem with a condition expression on a unique attribute (like the sequence number) is effectively the same as option B but transactions provide atomicity. Option C is wrong because idempotent writes in DynamoDB are not natively supported; you must use conditional writes. Option D is wrong because adding a FIFO queue adds latency and complexity without guaranteeing exactly-once processing in the consumer.

Answer

Enable DynamoDB auto scaling to increase write capacity and reduce throttling, which causes retries and duplicates.

Answer

Place an Amazon SQS FIFO queue between the KCL application and DynamoDB to deduplicate messages.

Answer

Modify the application to use DynamoDB BatchWriteItem instead of PutItem to reduce the number of write requests.

Question 7

A company uses AWS Glue to run ETL jobs that process data from Amazon RDS for PostgreSQL and load it into Amazon Redshift. The Glue job runs nightly and takes 6 hours to complete. The Redshift cluster is a single dc2.large node. The team needs to reduce the load time to under 3 hours. The data volume is 200 GB per night. The team is considering using Amazon Redshift Spectrum to query data directly from S3 instead of loading it. However, the data transformation logic is complex and requires multiple joins and aggregations that are currently performed in Glue. Which approach should the team recommend to meet the time requirement?

Accepted Answer

Increase the Redshift cluster to a multi-node cluster with dc2.8xlarge nodes to improve COPY and query performance.. Option C is correct because increasing the node size or number of nodes provides more compute resources for the COPY command and any subsequent processing. Using a larger node type like dc2.8xlarge or adding nodes increases parallelism. Option A is wrong because Redshift Spectrum does not replace the transformation logic; the complex transformations would still need to be run, possibly in Glue or Redshift. Option B is wrong because staging data in S3 does not reduce the transformation time. Option D is wrong because using a single node with elastic resize is not possible for dc2; also, splitting the load does not reduce total time if the bottleneck is compute.

Answer

Use Redshift Spectrum to create external tables and run the transformations directly in Redshift, bypassing the Glue job.

Answer

Split the Glue job into multiple parallel jobs that each load a portion of the data into separate Redshift tables, then use UNION ALL views.

Answer

Stage the data in S3 in Parquet format and use a COPY command with the PARQUET option to load data faster.

Question 8

A company uses Amazon S3 to store log files from various applications. The logs are in JSON format and are appended to existing files every few minutes. A data analyst wants to run SQL queries on the logs using Amazon Athena. However, queries return incomplete results because Athena does not support modifying data. The team needs to enable querying of the latest log data with minimal changes to the existing ingestion process. Which solution should the team implement?

Accepted Answer

Create an Athena table using the Hive JSON SerDe that reads the logs directly from the existing S3 bucket.. Option D is correct because Athena supports reading JSON data with the Hive JSON SerDe. By creating a table with the appropriate SerDe, the analyst can query the JSON logs directly. Option A is wrong because converting to Parquet would require changing the ingestion process. Option B is wrong because Glue ETL to load into Redshift is overkill and adds latency. Option C is wrong because Kinesis Data Firehose would require changing the ingestion pipeline.

Answer

Convert the logs to Parquet format using a scheduled AWS Glue job and store them in a separate S3 bucket.

Answer

Stream the logs to Amazon Kinesis Data Firehose, which writes the data to S3 in Parquet format.

Answer

Use AWS Glue to load the JSON logs into Amazon Redshift and query using Redshift.

Question 9

A data engineering team is building a real-time clickstream analytics pipeline on AWS. They need to ingest millions of events per second from mobile apps and websites, process them with low latency, and store the results in Amazon S3 for downstream analysis. Which combination of AWS services should the team use to minimize operational overhead while meeting these requirements?

Accepted Answer

Use Amazon Kinesis Data Streams for ingestion, Amazon Kinesis Data Analytics for real-time processing, and Amazon Kinesis Data Firehose to deliver results to Amazon S3.. Option C is correct because Amazon Kinesis Data Streams can handle high-throughput ingestion, Kinesis Data Analytics processes streaming data with low latency, and Kinesis Data Firehose delivers processed data to S3 with minimal overhead. Option A is wrong because AWS Glue is a batch ETL service, not suitable for real-time processing. Option B is wrong because Amazon EMR is a managed Hadoop cluster that requires more operational overhead. Option D is wrong because Amazon MQ is a message broker for standard messaging protocols, not optimized for real-time analytics.

Answer

Use Amazon MQ to ingest streaming data, AWS Lambda to process each message, and save output to Amazon S3.

Answer

Use Amazon Kinesis Data Streams to ingest data, Amazon EMR to process with Spark Streaming, and save output to Amazon S3.

Answer

Use AWS Glue to ingest data into Amazon RDS, then use AWS Glue ETL jobs to transform and load into Amazon S3.

Question 10

A data engineer is designing a data lake on Amazon S3. The data comes from various sources, including IoT devices, web logs, and transactional databases. The engineer needs to organize the data in a way that supports efficient querying using Amazon Athena and allows for easy management of access permissions. Which S3 bucket structure is the most appropriate?

Accepted Answer

Use a prefix structure like s3://bucket/source/year/month/day/.. Option B is correct because partitioning by source, year, month, day allows Athena to prune partitions, reducing scan costs and improving performance. Option A is wrong because storing all data in a flat structure forces full scans. Option C is wrong because prefix-based access controls can be applied at the source level within the partitioned structure. Option D is wrong because using date as the first partition level is less intuitive for managing permissions by source.

Answer

Store all data in a single prefix without any partitioning.

Answer

Store all data in separate S3 buckets for each source and date.

Answer

Use a prefix structure like s3://bucket/date/source/.

Question 11

A company uses AWS Glue to run ETL jobs that transform data from Amazon RDS for MySQL to Amazon S3. The current job runs daily and takes 3 hours to process 100 GB of data. The company expects data volume to grow 10x in the next year. They need to reduce job runtime and cost. Which approach should they take?

Accepted Answer

Use parallel reads with pushdown predicates in the Glue job's source connection, and write the output in columnar format (Parquet) partitioned by date.. Option D is correct because parallel reads from RDS with pushdown predicates reduce the load on the source and speed up extraction; using columnar formats like Parquet reduces storage and scanning costs in Athena. Option A is wrong because increasing DPUs without changing the extraction method may not help if the bottleneck is the source database. Option B is wrong because S3 Select is for server-side filtering, not for Glue jobs. Option C is wrong because Redshift Spectrum is for querying data in S3, not for transforming it.

Answer

Use S3 Select with Glue to filter data before transformation.

Answer

Increase the number of Glue DPUs to 100 and enable job bookmarking.

Answer

Use Amazon Redshift Spectrum to perform transformations in place on S3.

Question 12

A data engineer is tasked with building a pipeline to process streaming data from IoT devices. The devices send data in JSON format every second. The pipeline must aggregate data in 5-minute windows and store the results in Amazon S3. The engineer needs to handle late-arriving data (up to 1 hour) and ensure exactly-once semantics. Which combination of AWS services should they use?

Accepted Answer

Amazon Kinesis Data Streams for ingestion, Amazon Kinesis Data Analytics for windowed aggregations, and Amazon Kinesis Data Firehose to write to Amazon S3.. Option A is correct because Kinesis Data Analytics supports windowed aggregations, can handle late data via watermarking, and provides exactly-once processing when used with Kinesis Data Streams. Option B is wrong because Kinesis Data Firehose does not allow custom windowed aggregations. Option C is wrong because Glue Streaming is a batch-oriented service. Option D is wrong because Lambda does not have built-in support for exactly-once semantics for streaming applications.

Answer

Amazon Kinesis Data Streams for ingestion, AWS Glue Streaming ETL for aggregation, and Amazon S3 for storage.

Answer

Amazon SQS for ingestion, AWS Lambda for aggregation, and Amazon S3 for storage.

Answer

Amazon Kinesis Data Streams for ingestion, Amazon Kinesis Data Firehose for transformation, and Amazon S3 for storage.

Question 13

A research institution is building a data lake to store genomics data. Each experiment generates multiple files totaling about 500 GB. The data is stored in Amazon S3 and needs to be processed by multiple machine learning (ML) training jobs running on Amazon SageMaker. The data has a high churn rate; after 30 days, most data becomes irrelevant and should be moved to Amazon S3 Glacier Deep Archive. The institution wants to minimize storage costs while maintaining data durability. Which S3 storage class should they use for the first 30 days?

Accepted Answer

Use S3 Intelligent-Tiering for all data, and set a lifecycle policy to transition to S3 Glacier Deep Archive after 30 days.. Option C is correct because S3 Intelligent-Tiering automatically moves data between access tiers based on usage, which is ideal for data with unknown or changing access patterns. Option A is wrong because S3 Standard is more expensive for data that may not be accessed frequently after the initial processing. Option B is wrong because S3 One Zone-IA is not durable across AZs. Option D is wrong because S3 Glacier Instant Retrieval is for long-lived, rarely accessed data requiring millisecond access; not cost-effective for the first 30 days.

Answer

Use S3 One Zone-IA for all data, and set a lifecycle policy to transition to S3 Glacier Deep Archive after 30 days.

Answer

Use S3 Standard for all data, and set a lifecycle policy to transition to S3 Glacier Deep Archive after 30 days.

Answer

Use S3 Glacier Instant Retrieval for all data, and set a lifecycle policy to transition to S3 Glacier Deep Archive after 30 days.

Question 14

A company uses AWS Glue crawlers to populate the AWS Glue Data Catalog from Amazon S3. The data is partitioned by year/month/day/hour. The crawler runs every hour and adds new partitions. However, the data engineer notices that the crawler is taking longer to run as the number of partitions grows, and sometimes it misses new partitions. What is the most cost-effective and reliable way to address this?

Accepted Answer

Enable the crawler's partition index feature.. Option B is correct because enabling the crawler's partition index feature allows Glue to quickly find new partitions without re-scanning the entire table. Option A is wrong because increasing the crawler's frequency will not help if the crawler is already missing partitions due to scanning overhead. Option C is wrong because manually adding partitions is error-prone and does not scale. Option D is wrong because Athena MSCK REPAIR TABLE is a manual command and requires running it after data arrives, not automated.

Answer

Manually add new partitions using ALTER TABLE ADD PARTITION in Athena.

Answer

Use the Athena MSCK REPAIR TABLE command after the crawler runs.

Answer

Increase the crawler's schedule to run every 30 minutes.

Question 15

A data engineer needs to transfer 50 TB of historical data from an on-premises Hadoop cluster to Amazon S3. The company has a 100 Mbps internet connection and the data must be transferred within 5 days. Which AWS service is best suited for this task?

Accepted Answer

AWS Snowball Edge. Option C is correct because AWS Snowball Edge is a physical device that can transfer large amounts of data over a network faster than a 100 Mbps connection. Option A is wrong because AWS DataSync is designed for network transfers, but 100 Mbps would take much longer than 5 days. Option B is wrong because AWS Storage Gateway is for ongoing hybrid cloud storage, not large-scale data migration. Option D is wrong because S3 Transfer Acceleration speeds up internet transfers but still limited by the 100 Mbps connection.

Answer

AWS DataSync

Answer

Amazon S3 Transfer Acceleration

Answer

AWS Storage Gateway