MLS-C01 Data Engineering Practice Test 17 — 15 Questions

Question 1

Refer to the exhibit. A company is using the Kinesis stream 'my-stream' with one shard. The producer is sending 1000 records per second, each 1 KB. The consumer is reading from the stream using the Kinesis Client Library (KCL). The consumer is able to process 500 records per second per shard. What is the most likely cause of the consumer falling behind?

Accepted Answer

The stream has only one shard, which limits the read throughput to 1 MB/s.. The stream has only 1 shard, which provides a read throughput of 2 MB/s (or 1000 records per second of 1 KB each). The consumer can process 500 records per second, which is half the incoming rate, so it will fall behind. Increasing the number of shards would increase read throughput. Option A (not enough shards) is correct. Option B (retention period) does not affect ingestion. Option C (encryption) may add overhead but is not the main issue. Option D (consumer code) could be optimized, but the root cause is shard count.

Answer

The retention period is set to 24 hours, which is too short.

Answer

The stream uses KMS encryption, which adds latency.

Answer

The consumer application is not using enhanced fan-out.

Question 2

Refer to the exhibit. A data engineer is troubleshooting an AWS Glue job that fails with an 'AccessDenied' error when trying to write to the S3 bucket 'my-data-lake'. The IAM policy attached to the Glue service role is shown. What is the missing permission?

Accepted Answer

s3:ListBucket. The policy allows s3:GetObject and s3:PutObject on the bucket's objects, but it does not allow s3:ListBucket on the bucket itself. Many Glue operations require ListBucket to discover objects. Option D (s3:ListBucket) is missing. Option A (s3:DeleteObject) is not needed. Option B (s3:GetBucketLocation) is not required. Option C (s3:PutObjectAcl) is not needed.

Answer

s3:PutObjectAcl

Answer

s3:GetBucketLocation

Answer

s3:DeleteObject

Question 3

Refer to the exhibit. A team deploys this CloudFormation stack. The Kinesis stream is created, but the Firehose delivery stream fails to create with a 'Resource handler returned message: Unable to assume role' error. What is the most likely cause?

Accepted Answer

The IAM role 'firehose-role' does not exist in the AWS account.. The Firehose role 'firehose-role' is specified with a static ARN that includes an account ID. If the stack is deployed in a different account, or if the role does not exist, the assumption fails. Option A (role does not exist in the account) is the most likely. Option B (stream encryption) is not related. Option C (retention period) is fine. Option D (shard count) is fine.

Answer

The shard count of 2 is too low for Firehose to read from.

Answer

The Kinesis stream is encrypted with KMS, but Firehose does not have permission to decrypt.

Answer

The retention period of 168 hours is too long for Firehose.

Question 4

A company is streaming real-time sensor data from IoT devices to Amazon Kinesis Data Streams. The data is then consumed by an AWS Lambda function that enriches the records with metadata from an Amazon DynamoDB table and writes the results to an Amazon S3 bucket. Recently, the Lambda function has been failing with 'ProvisionedThroughputExceededException' errors from DynamoDB. The data volume is variable, with occasional bursts. Which solution should a data engineer implement to resolve this issue without losing data?

Accepted Answer

Enable DynamoDB auto scaling for the table to automatically adjust read capacity based on demand.. Option B is correct because enabling DynamoDB auto scaling dynamically adjusts read/write capacity based on traffic patterns, handling bursts without manual intervention. Option A (increasing read capacity units) is costly and may not handle all peaks. Option C (SQS) introduces latency and does not address the DynamoDB throttling directly. Option D (SNS) is for push notifications, not for resolving throughput issues.

Answer

Increase the DynamoDB table's provisioned read capacity units to a high static value.

Answer

Use an Amazon SQS queue to buffer the Lambda requests before querying DynamoDB.

Answer

Configure an Amazon SNS topic to throttle the data stream before it reaches Lambda.

Question 5

An e-commerce company uses Amazon Redshift for analytics. The data engineering team needs to load daily sales data from an S3 bucket that receives new files every hour. The data must be loaded into Redshift with minimal impact on query performance during the day, and they need to handle late-arriving data (files that appear after the daily load). Which approach should they use?

Accepted Answer

Use a staging table to load data incrementally with a MERGE operation, and schedule a late-arriving data job to merge files that arrive after the daily load.. Option A is correct because staging tables allow incremental loads with upsert logic using a staging table, and a late-arriving data process can merge the additional records later without blocking queries. Option B (COPY with auto staging) Redshift Spectrum queries data in S3 without loading, which may be slower for frequent queries. Option C (Kinesis Firehose) is real-time streaming, suitable for near-real-time but not for batched daily loads with late data handling. Option D (Glue ETL with overwrite) overwrites data, losing late-arriving data.

Answer

Use AWS Glue ETL to copy the data from S3 to Redshift, overwriting the existing data each day.

Answer

Stream the data from S3 using Amazon Kinesis Firehose to load into Redshift continuously.

Answer

Use Amazon Redshift Spectrum to query data directly from S3 and create external tables.

Question 6

A data scientist needs to train a machine learning model using a large dataset (500 GB) stored in an S3 bucket. The training will be performed on a SageMaker notebook instance. The data scientist wants to minimize data transfer costs and reduce training time. Which data ingestion approach should the data engineer recommend?

Accepted Answer

Use the SageMaker SDK to directly read the data from S3 during training without copying it to the notebook.. Option C is correct because using S3 as a data source for SageMaker directly avoids copying data into the notebook instance, reducing transfer costs and time. Option A (copying to notebook EBS) incurs transfer costs and uses local storage. Option B (EFS) adds cost and complexity. Option D (RDS) is for structured data and incurs additional costs.

Answer

Copy the dataset to the notebook instance's attached EBS volume before training.

Answer

Load the dataset into an Amazon RDS database and query it from the notebook.

Answer

Mount the S3 bucket to the notebook instance using Amazon Elastic File System (EFS).

Question 7

A company is using AWS Glue to run ETL jobs that transform data from multiple sources into a data lake on S3. The jobs are scheduled to run hourly. Recently, the jobs have been failing intermittently with 'MemoryError' exceptions. The data volume has grown over time. The data engineer needs to resolve this issue cost-effectively. Which action should be taken?

Accepted Answer

Increase the number of DPUs allocated to the Glue job and use a larger worker type.. Option B is correct because Glue jobs can be configured to use more DPUs (Data Processing Units) to increase memory, and using worker type G.2X or G.4X provides more memory per worker. Option A (increasing S3 timeout) does not address memory. Option C (Spark partitioning) may help but is more complex and may not be sufficient if memory is insufficient. Option D (changing to Python shell) reduces memory and will likely fail.

Answer

Increase the S3 timeout settings in the Glue job configuration.

Answer

Switch the Glue job type from Spark to Python shell to reduce memory overhead.

Answer

Repartition the data using Spark's repartition method before processing.

Question 8

A data engineering team needs to set up a data pipeline that ingests streaming data from an Apache Kafka cluster running on Amazon EKS into an S3 data lake. The data must be stored in Parquet format, partitioned by date and event type. The team wants a fully managed solution with minimal operational overhead. Which solution should they choose?

Accepted Answer

Use Amazon MSK (Managed Streaming for Apache Kafka) and configure an MSK Connect S3 sink connector.. Option D is correct because Amazon MSK (Managed Streaming for Kafka) is a fully managed Kafka service, and MSK Connect with an S3 sink connector can deliver data directly to S3 in Parquet format. Option A (Kinesis Data Analytics) is for real-time analytics, not for data lake ingestion. Option B (Kinesis Data Firehose) works with Kinesis streams, not Kafka directly. Option C (Glue ETL) is batch-oriented and adds latency.

Answer

Set up a Kinesis Data Firehose delivery stream that reads from Kafka and writes to S3.

Answer

Use AWS Glue ETL jobs to pull data from Kafka cluster periodically.

Answer

Create a Kinesis Data Analytics application to read from Kafka and write to S3.

Question 9

A data scientist is training a deep learning model using a large dataset stored in S3. The training job runs on a SageMaker training instance with a GPU. The data engineer notices that the GPU utilization is low, and the training is I/O bound. The data is read directly from S3 using the SageMaker SDK. Which change should the data engineer recommend to improve GPU utilization?

Accepted Answer

Use SageMaker Pipe mode to stream data directly from S3 to the training container.. Option B is correct because enabling SageMaker Pipe mode streams data directly from S3 to the training container without writing to disk, reducing I/O bottlenecks and improving GPU utilization. Option A (increasing batch size) may cause memory issues. Option C (using EFS) adds network latency. Option D (using EBS) still involves disk I/O.

Answer

Increase the batch size in the training script to process more data per step.

Answer

Mount the S3 bucket to the training instance using Amazon Elastic File System (EFS).

Answer

Copy the entire dataset to an Amazon EBS volume attached to the training instance.

Question 10

A company uses Amazon DynamoDB as the primary data store for a real-time recommendation engine. The data engineering team needs to export a daily snapshot of the DynamoDB table to S3 for offline analytics. The table is large (10 TB) and has a high read/write throughput. Which method will export the data with the least impact on the production workload?

Accepted Answer

Use the DynamoDB export to S3 feature available in the AWS Console or CLI.. Option D is correct because DynamoDB's export to S3 feature uses the table's backup capability, which reads from the storage layer without consuming read capacity units, thus having zero impact on production. Option A (scan) consumes RCUs. Option B (Data Pipeline) also uses scan. Option C (Glue ETL) also consumes RCUs.

Answer

Use AWS Data Pipeline to export the DynamoDB table to S3.

Answer

Use DynamoDB Scan API with parallel scans to export data to S3.

Answer

Use AWS Glue ETL job with a DynamoDB connection to export data.

Question 11

A data engineer is building a data pipeline that uses AWS Lambda to process records from an SQS queue and write results to an S3 bucket. The Lambda function processes each record individually and writes a separate file to S3. The team notices high latency and wants to reduce the number of S3 PUT requests to improve performance and reduce cost. Which approach should the data engineer take?

Accepted Answer

Aggregate multiple records into a single file in a DynamoDB table, then periodically write the aggregated data to S3.. Option A is correct because buffering records in a DynamoDB table and then using a batch write to S3 reduces the number of PUT requests. Option B (increasing Lambda memory) does not reduce S3 PUT count. Option C (S3 batch operations) is for existing objects, not for incoming data. Option D (multipart upload) is for large objects, not for many small objects.

Answer

Use S3 multipart upload for each record to improve throughput.

Answer

Increase the Lambda function's memory allocation to improve processing speed.

Answer

Use S3 Batch Operations to process the records in batches.

Question 12

A company has a large number of small CSV files (hundreds of thousands) in an S3 bucket. A data engineer needs to run a SQL query on this data using Amazon Athena. The queries are currently slow and expensive. Which two actions will improve query performance and reduce cost?

Accepted Answer

Partition the data by a commonly filtered column (e.g., date).. Option A and D are correct. Partitioning the data reduces the amount of data scanned by Athena, improving performance and reducing cost. Converting to columnar format (Parquet) further reduces scanned data and improves compression. Option B (compression) helps but is less impactful than partitioning and columnar format. Option C (more partitions) could help but too many small partitions may hurt performance. Option E (increasing S3 request rate) is not a direct action for Athena.

Answer

Increase the S3 request rate per prefix to improve read throughput.

Answer

Compress the CSV files using gzip.

Answer

Increase the number of partitions by splitting files into smaller ones.

Question 13

A data engineer needs to design a data ingestion pipeline that ingests data from a MySQL database hosted on-premises into Amazon S3 for analytics. The pipeline must capture change data (CDC) and run continuously with low latency. Which two services should the data engineer use?

Accepted Answer

AWS Database Migration Service (DMS) with ongoing replication.. Option A and D are correct. AWS DMS can capture ongoing changes from MySQL using CDC and replicate them to S3. Option B (Kinesis Data Streams) can receive CDC data from DMS but is not directly needed if DMS writes to S3. Option C (Glue ETL) is batch-oriented. Option E (AppFlow) is for SaaS applications, not for on-premises databases.

Answer

Amazon AppFlow.

Answer

AWS Glue ETL jobs scheduled at regular intervals.

Answer

Amazon Kinesis Data Streams.

Question 14

A company is using Amazon Redshift for data warehousing. The data engineering team observes that query performance degrades over time due to data skew. Which three strategies should the team implement to improve performance?

Accepted Answer

Choose appropriate distribution keys based on join and group-by columns.. Option A, B, and E are correct. Choosing appropriate distribution keys reduces data movement. Regular VACUUM and ANALYZE reclaims space and updates statistics. Using sort keys improves query performance by reducing scans. Option C (increasing node count) may help but is costly and not a targeted fix for skew. Option D (dropping indexes) is not applicable to Redshift (no indexes).

Answer

Increase the number of nodes in the Redshift cluster.

Answer

Drop unused indexes on large tables.

Question 15

Refer to the exhibit. An IAM policy is attached to a data engineering team's role. The team needs to upload data to the 'confidential' prefix in the 'my-data-lake' bucket. However, they are receiving 'AccessDenied' errors. What is the likely cause?

Accepted Answer

The Deny statement with the condition explicitly denies access to the 'confidential' prefix for accounts other than 123456789012.. Option A is correct. The Deny statement denies all s3 actions on the confidential prefix for any principal account that is not 123456789012. If the team's role is from a different account (e.g., 111111111111), the Deny applies. Option B is incorrect because the Deny has a condition, but it still denies for other accounts. Option C is incorrect because the Allow statement does not include the confidential prefix explicitly; however, the Deny overrides the Allow. Option D is incorrect because the condition does not require the team to use a specific source IP.

Answer

The condition in the Deny statement requires the team to use a specific source IP address.

Answer

The Allow statement only grants GetObject and PutObject, but the team needs ListBucket.

Answer

The Allow statement's resource does not include the 'confidential' prefix.