MLS-C01 Data Engineering Practice Test 15 — 15 Questions

Question 1

A machine learning engineer needs to train a model on a dataset stored in Amazon S3 in CSV format. The dataset is 200 GB and is updated daily with new records. The engineer wants to minimize training time and cost for each training run. The training job will run on multiple instances and must read the data efficiently. What should the engineer do to prepare the data for training?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Convert the CSV data to Apache Parquet format and store it in Amazon S3, then use SageMaker Pipe mode with the Parquet files.. Converting CSV to Parquet reduces storage size and enables columnar reads, which speeds up training. SageMaker Pipe mode streams data from S3 to the training instances without downloading the full dataset, reducing startup time and EBS costs. This is optimal for large, tabular datasets and multi-instance training, as it minimizes I/O and cost.

Answer

Partition the CSV data by date and use SageMaker File mode with multiple instances to read partitions in parallel.

Answer

Use Amazon EMR to convert the data to TFRecord format and store it in Amazon S3, then use SageMaker Pipe mode with the TFRecord files.

Answer

Keep the data in CSV format and use SageMaker File mode to download the entire dataset to each training instance.

Question 2

A data engineer is designing a pipeline that ingests JSON records from Amazon Kinesis Data Streams and writes them to Amazon S3 in Parquet format. The records contain nested fields and the schema evolves frequently. The engineer needs the pipeline to automatically discover the schema and handle the nested data without manual intervention. The solution must minimize operational overhead. Which AWS service should the engineer use?

Accepted Answer

AWS Glue with a crawler and an ETL job using the AWS Glue Data Catalog. AWS Glue crawlers automatically discover schemas, including nested JSON structures, and keep the Data Catalog updated as schemas evolve. An AWS Glue ETL job can then read from Kinesis Data Streams, perform transformations, and write Parquet to S3. This serverless approach eliminates cluster management, directly addressing the need to minimize operational overhead while handling nested data and schema changes.

Answer

AWS Lambda with a Python script that parses JSON and uses the pandas library to write Parquet to S3

Answer

Amazon EMR with a Spark Structured Streaming application that reads from Kinesis and writes Parquet to S3

Answer

Amazon Kinesis Data Firehose with a record format conversion to Parquet using an AWS Glue table

Question 3

A machine learning team stores training data in Amazon S3 and uses AWS Glue Data Catalog tables to query it from Amazon Athena. A data engineer must ensure that a new column added to the source Parquet files is immediately queryable in Athena without rewriting existing data, and that queries scan only the columns referenced. Which combination of actions should the engineer take to meet these requirements with the LEAST operational overhead?

Accepted Answer

Run an AWS Glue crawler to update the table schema in the Data Catalog, then query the table in Athena using columnar projection so only referenced columns are read.. A Glue crawler refreshes the Data Catalog table with the newly added column, which Athena then exposes immediately without rewriting data. Because Parquet is columnar, Athena reads only referenced columns, satisfying the scan-efficiency requirement. Rewriting the dataset, converting to CSV, or hand-editing metadata either adds unnecessary cost or fails to deliver column projection.

Answer

Convert the Parquet files to CSV so Athena can infer the new column at query time without updating the Data Catalog.

Answer

Manually edit the table definition in the AWS Glue Data Catalog to add the column, then run MSCK REPAIR TABLE in Athena to load the new partition metadata.

Answer

Create a new AWS Glue ETL job that reads the Parquet files, adds the new column, and writes a fully rewritten dataset to a new S3 prefix, then point Athena at the new location.

Question 4

A financial company ingests sensitive transaction data into an Amazon Kinesis Data Stream. The data must be encrypted at rest and in transit. The company also needs to rotate the encryption keys annually without disrupting the stream. The security team requires that the keys be managed by AWS and that access to the keys be audited. Which solution meets these requirements?

Accepted Answer

Use server-side encryption with AWS KMS customer managed keys (CMKs) for the Kinesis stream, and enable automatic key rotation.. Using AWS KMS customer managed keys with server-side encryption for Kinesis Data Streams enables encryption at rest, and Kinesis encrypts data in transit by default. Customer managed keys support automatic annual rotation and provide audit trails through CloudTrail. This solution satisfies all requirements: encryption, key management by AWS, annual rotation, and auditing.

Answer

Use server-side encryption with AWS KMS customer managed keys, but disable key rotation because Kinesis does not support automatic rotation.

Answer

Use server-side encryption with AWS KMS AWS managed keys for the Kinesis stream, and manually rotate the keys by creating a new stream.

Answer

Encrypt data at rest using client-side encryption with a custom key stored in AWS Secrets Manager, and rotate the key annually using a Lambda function.

Question 5

A data engineer is using Amazon Kinesis Data Firehose to deliver application logs to Amazon S3. The logs are JSON and must be converted to Apache Parquet before storage to reduce query cost. The engineer configures the delivery stream with an AWS Lambda function for record transformation. After testing, the delivered S3 objects still contain raw JSON. What is the most likely cause?

Accepted Answer

The Lambda function returns the transformed records without setting the recordId to match the incoming record.. For Firehose Lambda transformation, the function must return a response containing recordId, result, and data for each record. Firehose uses recordId to match transformed output to the original. Without a matching recordId, the transformation is considered unsuccessful and Firehose delivers the original data. Ensuring the recordId is copied from the input resolves the issue.

Answer

The Lambda function must be invoked with an event source mapping from the Firehose delivery stream.

Answer

The Lambda function's memory is too low to process the JSON records into Parquet format.

Answer

The Firehose delivery stream IAM role lacks s3:PutObject permission on the destination bucket.

Question 6

An ML engineer needs to run a SageMaker training job on a 400 GB dataset stored as a single gzip-compressed CSV file in Amazon S3. The training script expects the data to be available as a local directory of files, and the job repeatedly fails with an out-of-disk-space error on the training instance after several minutes. The engineer wants to preserve the SageMaker File mode workflow with minimal changes to the training script. Which approach will resolve the failure while keeping File mode?

Accepted Answer

Re-upload the dataset as multiple smaller shards in S3 and point the training job's input channel to the S3 prefix so SageMaker downloads the shards into the local directory.. File mode stages the full contents of each input channel onto the training instance before the script runs, so a single 400 GB compressed object can exhaust local storage once decompressed. Sharding the dataset into many smaller objects under one S3 prefix keeps File mode intact, distributes the storage footprint, and matches the script's expectation of a local directory of files.

Answer

Switch the input channel to Pipe mode so the data is streamed from S3 without being fully copied to the instance's local disk.

Answer

Enable Amazon S3 Transfer Acceleration on the bucket and set the input channel's S3DataDistributionType to FullyReplicated.

Answer

Increase the training instance's attached Amazon EBS volume size using the VolumeSizeInGB parameter in the algorithm specification.

Question 7

A data engineer is building a pipeline that ingests data from an on-premises Oracle database into Amazon S3 for machine learning training. The database is large (10 TB) and the engineer needs to perform a one-time full load followed by incremental loads. The pipeline must handle schema changes and minimize downtime. The engineer wants to use AWS Database Migration Service (AWS DMS) with an Amazon S3 target. Which AWS DMS configuration should the engineer use to meet these requirements?

Accepted Answer

Use AWS DMS with change data capture (CDC) enabled, and configure the S3 target with the Parquet output format and partition by date.. Enabling CDC in AWS DMS allows the initial full load and subsequent incremental changes to be replicated continuously, minimizing downtime. Using Parquet format with date partitioning on the S3 target optimizes storage and query performance for machine learning. This configuration also supports schema evolution if DMS is configured to replicate DDL changes.

Answer

Use AWS DMS with full load only, and use AWS Schema Conversion Tool to handle schema changes.

Answer

Use AWS DMS with full load only, and schedule periodic full loads using AWS Glue jobs to capture changes.

Answer

Use AWS DMS with CDC enabled, and configure the S3 target with the CSV output format and no partitioning.

Question 8

A data engineer is using AWS Glue to catalog data stored in Amazon S3. The data is in JSON format and is partitioned by year, month, and day. The engineer notices that queries in Amazon Athena are slow and expensive because they scan all partitions. The engineer wants to improve query performance and reduce cost. What should the engineer do?

Accepted Answer

Enable partition projection in the AWS Glue Data Catalog table properties.. Partition projection in AWS Glue Data Catalog allows Athena to compute partition locations on the fly using a specified pattern, eliminating the need to store and retrieve all partition metadata. This reduces query planning time and cost because Athena can prune partitions more efficiently. It is particularly useful for highly partitioned datasets with a regular structure, such as date-based partitions.

Answer

Increase the number of DPUs for the AWS Glue ETL job that populates the catalog.

Answer

Run the MSCK REPAIR TABLE command in Athena to update the partition metadata.

Answer

Convert the JSON data to Parquet format and update the table definition in AWS Glue.

Question 9

A data engineer is designing an ingestion pipeline that must land JSON events from an Application Load Balancer access log stream into Amazon S3 within 60 seconds of arrival, then convert them to Parquet for Athena. The pipeline must handle occasional bursts of up to 50 MB/s and must not lose records if a downstream consumer fails. Which architecture meets these requirements?

Accepted Answer

Send events to Amazon Kinesis Data Streams with enhanced fan-out, use AWS Lambda to convert each record to Parquet, and write to S3 with the Lambda function configured with a dead-letter queue for failed invocations.. Kinesis Data Streams retains records durably and supports replay, so a failed consumer can resume without data loss, while enhanced fan-out provides dedicated read throughput for bursts. Lambda handles the conversion to Parquet, and a dead-letter queue preserves failed invocations for reprocessing, satisfying the no-loss requirement within the latency window.

Answer

Write events directly to Amazon S3 using the S3 PutObject API from the load balancer, then use an AWS Glue crawler on a schedule to convert the objects to Parquet.

Answer

Send events to Amazon Kinesis Data Streams with a retention period of 24 hours, run an Amazon Kinesis Data Analytics application to convert records, and have a consumer write Parquet to S3 with checkpointing.

Answer

Use Amazon Kinesis Data Firehose with an AWS Lambda function for record transformation, buffering to 60 seconds or 5 MB before delivering to S3, and enable S3 versioning on the destination bucket.

Question 10

A media analytics company ingests thousands of JSON event files per hour into an Amazon S3 bucket named raw-events. An AWS Glue crawler runs hourly against that bucket. Analysts report that Athena queries fail with 'HIVE_BAD_DATA' errors and that columns intermittently show as string in the Glue Data Catalog even though numeric values are present. A data engineer must make the catalog stable and queries reliable with the least operational overhead. (Choose two.)

Accepted Answer

Convert the incoming JSON to a partitioned Parquet dataset with an AWS Glue ETL job and point the crawler at the Parquet prefix.. The instability comes from schema-on-read inference over inconsistent JSON. Materializing a typed Parquet dataset gives the catalog a deterministic schema, while a manually defined table with crawler schema-change protection locks the column types in place. Together they eliminate the intermittent string inference and the SerDe conversion failures without adding heavy operational work.

Answer

Increase the crawler's DPU capacity and shorten its schedule to every 15 minutes so it samples more files.

Answer

Set the Athena workgroup query result location to an S3 bucket in the same Region as the raw-events bucket.

Answer

Enable S3 Object Lock in governance mode on the raw-events bucket so files cannot be modified between crawler runs.

Question 11

A company stores sensitive customer data in Amazon S3 and uses AWS Glue DataBrew to profile and transform the data. The security team requires that all data be encrypted at rest with AWS KMS keys that the company manages, and that DataBrew jobs log all column-level transformations for audit. Which two actions should the data engineer take to meet these requirements? (Choose two.)

Accepted Answer

Configure the DataBrew job to use a customer managed KMS key for the S3 output location and ensure the DataBrew role has kms:GenerateDataKey and kms:Decrypt permissions.. Meeting the requirements needs two distinct controls. First, encrypt DataBrew output with a customer managed KMS key by specifying the key and granting the job role kms:GenerateDataKey and kms:Decrypt. Second, enable DataBrew job logging to CloudWatch Logs with a log level that records transformation details. Together these provide company-managed encryption at rest and an auditable record of column-level changes.

Answer

Use AWS Glue crawlers to catalog the S3 data and enable encryption in the Data Catalog settings.

Answer

Attach an S3 bucket policy that denies unencrypted PutObject requests and enable default encryption with SSE-S3.

Answer

Configure DataBrew to use AWS CloudTrail data events for S3 and enable CloudTrail Insights.

Question 12

A machine learning engineer needs to store training data in Amazon S3. The data will be accessed frequently by SageMaker training jobs and must be highly durable and available. The engineer wants to minimize storage costs while meeting these requirements. Which S3 storage class should the engineer choose?

Accepted Answer

S3 Standard. S3 Standard provides the high durability, availability, and performance required for frequently accessed training data. While other classes may have lower storage costs, they are designed for infrequent access or have trade-offs in durability, availability, or retrieval costs. S3 Standard is the most cost-effective for this use case when considering overall access patterns.

Answer

S3 Standard-Infrequent Access (S3 Standard-IA)

Answer

S3 One Zone-IA

Answer

S3 Intelligent-Tiering

Question 13

A machine learning engineer is building a training dataset from multiple sources. The data is stored in Amazon S3 in various formats: CSV, JSON, and Parquet. The engineer needs to create a unified dataset for training a model using Amazon SageMaker. The engineer wants to minimize data preparation time and ensure the training job can read the data efficiently. Which two actions should the engineer take? (Choose two.)

Accepted Answer

Convert all data to a single format, such as Parquet, using an AWS Glue ETL job, and store the output in Amazon S3.. Using AWS Glue to crawl the data creates a unified schema in the Data Catalog, which helps understand and access the data. Then, an AWS Glue ETL job can convert all data to Parquet, a columnar format that is efficient for training. This combination minimizes data preparation time and ensures SageMaker training jobs can read the data efficiently, leveraging Glue's serverless ETL capabilities.

Answer

Mount the S3 bucket as a file system on the SageMaker training instances using Amazon FSx for Lustre.

Answer

Use SageMaker Ground Truth to label the data and create a unified dataset.

Answer

Use Amazon Athena to query the data directly from S3 and write the results to a new S3 bucket in CSV format.

Question 14

A machine learning engineer is preparing a training dataset stored in Amazon S3 in CSV format. The dataset is 50 GB and contains many missing values. The engineer needs to transform the data using AWS Glue and store it in Parquet format for efficient querying by Amazon Athena. The transformation must handle the missing values by replacing them with the median of each numeric column. Which approach should the engineer take to meet these requirements?

Accepted Answer

Create an AWS Glue ETL job that uses the FillMissingValues transform with the median option for each numeric column.. The FillMissingValues transform in AWS Glue provides a straightforward, built-in method to impute missing values using median for numeric columns. It handles the computation across the dataset in a distributed manner and integrates seamlessly into an ETL job. Other options either do not support median imputation, require a different service, or involve unnecessary custom code. Therefore, using FillMissingValues is the most efficient and appropriate solution.

Answer

Use AWS Glue DataBrew to create a recipe that replaces missing values with the median, then run the recipe as a job to output Parquet.

Answer

Create an AWS Glue ETL job that uses the DynamicFrame class with the ResolveChoice transform to replace missing values with the median.

Answer

Write a custom PySpark script in an AWS Glue ETL job that loads the data, computes the median for each numeric column, and uses the fillna method to replace missing values.

Question 15

A machine learning engineer is using Amazon SageMaker Ground Truth to label a dataset of images. The labeling job is configured with a private workforce and uses a custom labeling task. After the job completes, the engineer notices that the output manifest file contains labels but the consolidated annotations are missing. The engineer needs to obtain the final labeled dataset in a format suitable for training. What should the engineer do to resolve this issue?

Accepted Answer

Use the SageMaker Ground Truth console or API to run the consolidation task on the output manifest, which generates the consolidated annotations file.. SageMaker Ground Truth labeling jobs typically produce an output manifest and a consolidated annotations file. If the consolidated annotations are missing, it is often because the consolidation step failed or was not triggered. The correct action is to manually run the consolidation task using the Ground Truth console or API, which processes the output manifest and generates the consolidated file. This file is essential for training because it combines all worker responses into a single dataset.

Answer

Run the AWS Glue job that was automatically created by the labeling job to consolidate the annotations.

Answer

Use the AWS CLI command aws sagemaker create-labeling-job with the --auto-labeling flag to automatically generate consolidated annotations.

Answer

Re-run the labeling job with the same input manifest and ensure that the output manifest is stored in the same S3 bucket.