MLA-C01 Data Preparation for Machine Learning Practice Question
A financial services company is developing a fraud detection model using Amazon SageMaker. They have a dataset with 10 million transactions, each with 300 features. The dataset is highly imbalanced (0.1% fraud). They have performed feature engineering and now need to split the data for training, validation, and test sets. The data is stored in CSV files in Amazon S3. They plan to use SageMaker's built-in XGBoost algorithm. To ensure proper evaluation and avoid data leakage, which data splitting strategy should they use?
⚠ Common exam trap
Test-takers frequently choose random splitting (Option A) out of habit, forgetting that imbalanced datasets require stratified sampling to avoid evaluation sets with zero positive cases, which would render metrics like precision and recall undefined.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Perform a stratified split on the target variable to ensure each set has the same fraud ratio.
A stratified split preserves the original 0.1% fraud ratio across training, validation, and test sets, which is critical for imbalanced datasets. This ensures each subset is representative of the population, allowing SageMaker's XGBoost to be evaluated fairly without data leakage. Random splits (Option A) could accidentally create a validation or test set with zero fraud cases, making evaluation meaningless.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Randomly shuffle the entire dataset and then split into 80% training, 10% validation, 10% test.
Why it's wrong here
Shuffling before splitting lets near-duplicate transactions from the same account or card appear in both training and test partitions, leaking information and inflating fraud-detection scores. Random splitting suits independent, identically distributed records, not temporally or entity-correlated financial transactions.
- ✗
Use k-fold cross-validation on the entire dataset and average the results.
Why it's wrong here
K-fold cross-validation reuses every record for both training and validation across folds, so the held-out test set is never genuinely unseen, and it gives no final unbiased estimate. It suits small datasets needing variance reduction, not a 10-million-row fraud problem requiring a separate test partition.
- ✓
Perform a stratified split on the target variable to ensure each set has the same fraud ratio.
Why this is correct
A stratified split preserves the 0.1% fraud ratio across training, validation and test sets, preventing the minority class from being absent or severely under-represented in any split. This satisfies the stem's requirement for proper evaluation of the imbalanced target, since random splitting could yield validation sets with too few fraud cases to assess model performance reliably.
- ✗
Apply SMOTE to balance the dataset first, then split randomly into training, validation, and test sets.
Why it's wrong here
SMOTE generates synthetic minority samples before splitting, so interpolated points derived from training records land in validation and test sets, leaking label information and distorting precision-recall. SMOTE belongs inside the training pipeline only, after the split, never applied to the full dataset.
Quick reference
AWS S3 Storage Class Comparison
| Storage Class | Min Duration | Retrieval | Use Case |
|---|---|---|---|
| S3 Standard | None | Immediate | Frequently accessed data |
| S3 Standard-IA | 30 days | Immediate | Infrequent access, rapid retrieval |
| S3 One Zone-IA | 30 days | Immediate | Non-critical infrequent data |
| S3 Intelligent-Tiering | None | Immediate–hours | Unknown or changing access patterns |
| S3 Glacier Instant | 90 days | Milliseconds | Archive with instant retrieval |
| S3 Glacier Flexible | 90 days | Minutes–hours | Archive, flexible retrieval |
| S3 Glacier Deep Archive | 180 days | Hours | Long-term compliance archive |
Go deeper
Related to this question
About these practice questions
Courseiva writes every MLA-C01 question from scratch — 665 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLA-C01 exam.