MLA-C01 Data Preparation for Machine Learning Practice Question

A machine learning engineer needs to split a dataset into training, validation, and test sets for a SageMaker training job. The dataset is stored in Amazon S3 as a single CSV file. The engineer wants to ensure that the splits are reproducible and that the test set is never used during training or hyperparameter tuning. Which approach should the engineer use?

⚠ Common exam trap

The trap here is assuming that SageMaker training jobs automatically split data or that using all splits in training is acceptable.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Manually split the CSV file using a Python script with a fixed random seed, write the three splits to separate S3 prefixes, and use the training and validation prefixes in the training job, keeping the test prefix for final evaluation.

To ensure reproducibility and prevent data leakage, the engineer should split the data with a fixed random seed and store each split in separate S3 locations. The training job should only consume the training and validation sets, while the test set is reserved for final model evaluation. This is a standard best practice for ML workflows.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✓
Manually split the CSV file using a Python script with a fixed random seed, write the three splits to separate S3 prefixes, and use the training and validation prefixes in the training job, keeping the test prefix for final evaluation.
Why this is correct
Using a Python script with a fixed random seed ensures reproducibility. Writing the splits to separate S3 prefixes allows the training job to access only the training and validation data, while the test set remains untouched for final evaluation. This meets all requirements.
✗
Use the SageMaker training job's built-in data splitting feature by specifying a validation split percentage in the hyperparameters.
Why it's wrong here
SageMaker training jobs do not have a built-in data splitting feature that automatically creates train/validation/test splits from a single file. The validation split percentage is not a standard hyperparameter. This approach would require custom code and does not guarantee a separate test set.
✗
Use SageMaker Data Wrangler to split the data into three parts and export them to S3, then use all three parts in the training job with different channels.
Why it's wrong here
While Data Wrangler can split data, using all three parts in the training job would expose the test set during training, violating the requirement. The test set must be held out entirely until final evaluation. This approach does not prevent leakage.
✗
Use Amazon Athena to run a query that randomly assigns rows to three splits, and store the results in S3. Then use all three splits in the training job.
Why it's wrong here
Athena can split data, but using all three splits in the training job again exposes the test set. Additionally, Athena's random assignment may not be reproducible without a fixed seed. This approach fails to keep the test set separate and reproducible.

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

MLA-C01 Data Preparation for Machine Learning Practice Question

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question