MLS-C01 Data Engineering Practice Test 17 — 15 Questions

Question 1

A machine learning team stores training data in Amazon S3 in Parquet format. They use Amazon SageMaker to train a model and need to access the data efficiently. The data is partitioned by date and stored in a bucket with versioning enabled. The team wants to ensure that the training job reads only the latest version of each object and minimizes data transfer costs. Which solution should they implement?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use the SageMaker training job's input channel with the S3 prefix and set the 'InputMode' to 'FastFile' to stream data directly from S3.. The requirement is to read the latest version of objects and minimize data transfer costs. FastFile mode streams data from S3 during training, so it always accesses the current object versions and avoids downloading the entire dataset upfront, reducing storage and transfer overhead. Other options either do not address versioning or add unnecessary cost and complexity.

Answer

Enable S3 Transfer Acceleration on the bucket and use the accelerated endpoint in the training job's input configuration.

Answer

Use Amazon S3 Select to retrieve only the required columns from each Parquet object.

Answer

Configure the SageMaker training job to use the S3 bucket's latest object versions by specifying the version ID in the manifest file.

Question 2

A company uses AWS Glue to process data from an Amazon S3 data lake. The Glue ETL job reads JSON files, transforms them, and writes Parquet files to another S3 location. The job runs daily and processes data partitioned by year/month/day. Recently, the job started failing with an error indicating that the number of partitions exceeds the allowed limit. The data engineer needs to resolve this issue with minimal changes to the existing job. What should the data engineer do?

Accepted Answer

Request a quota increase for the maximum number of partitions per table in AWS Glue.. The error indicates that the partition count exceeds the AWS Glue service quota. The most direct and minimal-change solution is to request a quota increase for the maximum number of partitions per table. Other options either do not affect the partition limit or add unnecessary complexity. Increasing DPUs or using indexes does not resolve the quota issue, and bookmarks do not reduce the partition count.

Answer

Enable AWS Glue job bookmarks to track processed data and avoid reprocessing old partitions.

Answer

Use AWS Glue partition indexes to optimize partition pruning and reduce the number of partitions scanned.

Answer

Increase the number of DPUs for the Glue job to handle more partitions.

Question 3

A data engineer is building a data pipeline that ingests data from an Amazon Kinesis Data Stream into an Amazon S3 bucket. The data must be stored in Parquet format and partitioned by year, month, day, and hour for efficient querying with Amazon Athena. The engineer needs to choose the most operationally efficient solution to transform and write the data. (Choose two.)

Accepted Answer

Use AWS Glue streaming ETL with the 'UpdateCatalog' option enabled to automatically partition the output in the AWS Glue Data Catalog.. The most operationally efficient solution is to use AWS Glue streaming ETL to read from Kinesis, transform, and write Parquet with partitioning, and to enable the 'UpdateCatalog' option to automatically manage partitions in the Data Catalog. This minimizes custom code and infrastructure management. Other options either require multiple services or custom code, increasing operational overhead and complexity.

Answer

Use an AWS Lambda function to read from Kinesis Data Stream, transform the data, and write to S3 in Parquet format with partitioning.

Answer

Use Amazon Kinesis Data Analytics to run SQL on the stream and output to S3 in Parquet format with partitioning.

Answer

Use Amazon Kinesis Data Firehose to deliver the stream to S3, then run an AWS Glue ETL job to convert the data to Parquet and repartition.

Question 4

A data science team uses Amazon SageMaker to train models on a large dataset stored in S3. The dataset is 500 GB in CSV format and is updated daily. The team wants to optimize data loading for training jobs to reduce I/O wait time. Which data ingestion strategy is MOST effective?

Accepted Answer

Use SageMaker Pipe input mode to stream data directly from S3.. SageMaker Pipe input mode streams data directly from S3 to the training algorithm without writing to the instance's EBS volume, eliminating disk I/O bottlenecks. This is especially effective for large datasets (500 GB) that are updated daily, as it reduces startup time and avoids the need to download the entire dataset before training begins.

Answer

Use SageMaker File input mode and increase the EBS volume size to 1 TB.

Answer

Convert the CSV files to Parquet format and use File input mode.

Answer

Load the data into an Amazon EFS file system and mount it to the training instance.

Question 5

A data engineer needs to transform large CSV files stored in S3 into Parquet format and load them into a data warehouse for analysis. The transformation must be cost-effective and serverless. Which AWS service should be used?

Accepted Answer

AWS Glue. AWS Glue is the correct choice because it provides a fully managed, serverless ETL service that can automatically convert CSV files from S3 into Parquet format using its built-in Spark engine. It is cost-effective as you only pay for the resources consumed during the job execution, and it integrates directly with data warehouses like Amazon Redshift for loading transformed data.

Answer

Amazon Athena

Answer

Amazon EMR with Spark

Answer

AWS Data Pipeline

Question 6

A company uses Amazon Kinesis Data Firehose to deliver streaming data to an S3 bucket. The data is JSON and must be partitioned by year, month, and day. The delivery stream is configured with a buffer interval of 60 seconds and buffer size of 5 MB. The data producer sends about 1 MB per second. The data is arriving in S3 but the partitions are not being created as expected. What is the MOST likely reason?

Accepted Answer

The delivery stream does not have dynamic partitioning enabled with the appropriate custom prefix.. Kinesis Data Firehose requires dynamic partitioning to be explicitly enabled and configured with a custom prefix (e.g., 'year=!{timestamp:yyyy}/month=!{timestamp:MM}/day=!{timestamp:dd}/') to automatically partition data by year, month, and day. Without this setting, Firehose writes all data to a single S3 prefix, ignoring the desired partition structure.

Answer

The data is encrypted with AWS KMS and Firehose cannot write to encrypted buckets.

Answer

The buffer interval is too short for the data volume, causing incomplete records.

Answer

The S3 bucket has versioning enabled, which prevents partitioning.

Question 7

A data scientist uses Amazon SageMaker to train a model. The training dataset is 10 GB and stored in S3. The training job uses a ml.m5.large instance. The data must be available on the local file system during training. Which input mode should be used?

Accepted Answer

File input mode. File input mode is correct because it downloads the entire training dataset from S3 to the local file system of the ml.m5.large instance before training begins, ensuring the data is available locally as required. This mode is suitable for datasets up to 10 GB, as the instance's local storage (typically 8 GB for ml.m5.large) may be insufficient, but SageMaker uses the instance's Amazon EBS volume (up to 512 GB) for file input mode, making it viable.

Answer

Local input mode

Answer

Batch input mode

Answer

Pipe input mode

Question 8

A company uses Amazon Redshift as a data warehouse. They need to load 50 TB of clickstream data from S3 into Redshift daily. The data arrives in 5-minute intervals as gzipped CSV files. The target table has a sort key and a distribution key. The load must complete within 2 hours. Which approach is MOST efficient?

Accepted Answer

Use the COPY command with a manifest file and gzip compression.. The COPY command is the most efficient way to load large volumes of data into Amazon Redshift because it uses the cluster's massively parallel processing (MPP) architecture to read data directly from S3 in parallel across all nodes. With a manifest file, you can specify multiple gzipped CSV files, and the gzip compression reduces network I/O and storage overhead. This approach can easily load 50 TB within 2 hours, especially when the target table has a sort key and distribution key, as COPY automatically leverages these for optimal data distribution and sorting during the load.

Answer

Use AWS Glue to transform the data and write to Redshift using JDBC.

Answer

Use a staging table and then merge using a stored procedure.

Answer

Use a series of INSERT statements from a Lambda function.

Question 9

A machine learning engineer needs to process a large dataset that does not fit on a single Amazon SageMaker notebook instance's EBS volume. The data is stored in S3. What is the MOST efficient way to access the data from the notebook?

Accepted Answer

Read the data directly from S3 using the boto3 library.. Reading data directly from S3 using the boto3 library is the most efficient approach for a dataset that exceeds the notebook instance's EBS volume capacity. Boto3 allows you to stream data in chunks or use S3 Select for server-side filtering, avoiding the need to download the entire dataset to local storage. This method leverages S3's high-throughput API and eliminates the bottleneck of writing to a local EBS volume, which is limited in size and I/O performance.

Answer

Increase the EBS volume size to 5 TB.

Answer

Mount the S3 bucket as a file system using s3fs.

Answer

Use SageMaker File input mode in the notebook.

Question 10

An organization stores sensitive customer data in S3. A data pipeline uses AWS Glue to transform the data and load it into Amazon Redshift. The security team requires that data be encrypted at rest in S3 and in transit between S3 and Glue, and between Glue and Redshift. Which configuration meets these requirements?

Accepted Answer

Enable S3 default encryption with SSE-KMS, use a VPC endpoint for S3, and configure Glue to use SSL for Redshift connection.. It ensures encryption at rest in S3 via SSE-KMS, encrypts data in transit between S3 and Glue by using a VPC endpoint (which enforces HTTPS/TLS), and encrypts data in transit between Glue and Redshift by configuring SSL for the Redshift connection. SSE-KMS provides envelope encryption with a customer-managed key, while the VPC endpoint and SSL satisfy the in-transit encryption requirements.

Answer

Use S3 client-side encryption, and use VPC Peering between Glue and Redshift.

Answer

Use S3 default encryption with SSE-KMS, and use Network Load Balancer for Redshift.

Answer

Enable S3 server-side encryption with SSE-S3, and use SSL for both Glue connections.

Question 11

A company uses Amazon SageMaker to train and deploy machine learning models. The training data is stored in Amazon S3 (Parquet format, 10 TB). The data scientists have been running training jobs using the File mode input, but the jobs are taking too long due to data download time. They want to reduce the training start-up time and overall training time. Which solution is MOST cost-effective and efficient?

Accepted Answer

Configure the SageMaker training job to use Pipe mode, which streams data directly from S3 without downloading to the instance's local storage.. Pipe mode in SageMaker streams training data directly from Amazon S3 to the training algorithm without first downloading it to the instance's local storage. This eliminates the data download step, significantly reducing startup time and overall training time for large datasets like 10 TB. It is the most cost-effective because it avoids the need for larger instances or additional data transfer acceleration services.

Answer

Use S3 Transfer Acceleration to speed up the data transfer from S3 to the training instance.

Answer

Use larger EC2 instances with more vCPUs and memory to speed up the training process.

Answer

Enable Elastic Fabric Adapter (EFA) on the training instances to improve network throughput.

Question 12

A data engineering team is designing a data lake on AWS for machine learning workloads. The data includes structured, semi-structured, and unstructured data. The team needs to ensure that the data is cataloged, easily discoverable, and can be queried by Amazon Athena and Amazon EMR. The team also wants to enforce fine-grained access control at the column and row level for sensitive data. Which combination of AWS services should the team use? (Select TWO.)

Accepted Answer

AWS Lake Formation. AWS Lake Formation (A) is correct because it is the service that provides centralized fine-grained access control for data lakes, allowing column-level and row-level security policies to be enforced across analytics services such as Amazon Athena and Amazon EMR. AWS Glue Data Catalog (C) is correct because it serves as the persistent metadata store that catalogs structured, semi-structured, and unstructured data, making the data discoverable and queryable by Athena and EMR. Together, Lake Formation governs permissions while the Glue Data Catalog provides the metadata backbone for the data lake. IAM (B) handles authentication and broad AWS resource permissions but does not by itself deliver column- and row-level data lake access control. Amazon RDS for PostgreSQL (D) is a relational database engine, not a data lake catalog or governance service. Amazon DynamoDB (E) is a NoSQL key-value store and does not provide data lake cataloging or fine-grained analytics access control.

Answer

AWS Identity and Access Management (IAM)

Answer

Amazon RDS for PostgreSQL

Answer

Amazon DynamoDB

Question 13

A data scientist needs to transform raw JSON data from an S3 bucket into Parquet format using AWS Glue. The job must be cost-effective and run only when new data arrives. Which solution should be used?

Accepted Answer

Create a Glue ETL job triggered by an S3 event notification via Lambda.. It uses an S3 event notification to invoke a Lambda function, which then triggers an AWS Glue ETL job only when new data arrives. This event-driven architecture ensures cost-effectiveness by avoiding continuous or scheduled runs, and it directly transforms raw JSON into Parquet format as required.

Answer

Create a Glue crawler that runs continuously.

Answer

Schedule a Glue ETL job to run every hour.

Answer

Use Glue DataBrew to transform data and schedule it daily.

Question 14

A team is building a data pipeline to process terabytes of log data daily using Amazon EMR. The data arrives in 5-minute windows and must be available for querying within 30 minutes. The data is originally in gzip-compressed CSV files. Which approach will minimize processing time and cost?

Accepted Answer

Use Amazon EMR with Spark to convert data to Parquet and store in S3, using spot instances for task nodes.. Converting gzip-compressed CSV to Parquet reduces storage size and improves query performance due to columnar storage and predicate pushdown. Using spot instances for task nodes significantly lowers compute cost, while the 30-minute SLA is achievable with Spark on EMR processing 5-minute windows of data.

Answer

Use Amazon EMR with Spark to convert data to Parquet and use on-demand instances.

Answer

Use AWS Glue to convert data to gzip-compressed CSV and query with Athena.

Answer

Use Amazon EMR with Hive to transform data to compressed CSV and store in S3.

Question 15

A data engineer is designing a streaming pipeline using Amazon Kinesis Data Analytics for Apache Flink. The pipeline reads from a Kinesis data stream and writes to a S3 bucket. The job must recover quickly from failures without reprocessing large amounts of data. Which TWO configurations should be used? (Choose TWO)

Accepted Answer

Enable checkpointing with a state backend like RocksDB.. Option A is correct because enabling checkpointing with a durable state backend such as RocksDB (via the EmbeddedRocksDBStateBackend or RocksDBStateBackend) allows Apache Flink to periodically snapshot operator state, so after a failure the job resumes from the last completed checkpoint instead of replaying the entire stream, which directly satisfies the requirement to recover quickly without reprocessing large amounts of data. Option C is correct because configuring the S3 sink with exactly-once delivery semantics (for example, using the Flink FileSink with RollingPolicy and OnCheckpointRollingPolicy, or the StreamingFileSink with exactly-once mode) ties file commit to completed checkpoints, ensuring that on recovery only data after the last checkpoint is reprocessed and no duplicates or partial files are written to S3. Option B is not appropriate because an in-memory state backend keeps state only in the TaskManager heap and does not persist snapshots durably, so it cannot support fast recovery from failures. Option D is incorrect because setting parallelism to the maximum number of shards is a throughput/scaling consideration and does not by itself provide fast failure recovery or avoid reprocessing. Option E is incorrect because extending Kinesis stream retention to 365 days only lengthens how long records remain readable; it does not improve checkpoint-based recovery or reduce reprocessing after a job failure.

Answer

Use in-memory state backend for low latency.

Answer

Set the parallelism to the maximum number of shards.

Answer

Increase the retention period of the Kinesis stream to 365 days.