Addressing Class Imbalance in SageMaker Autopilot
An ML team is using SageMaker Autopilot to automatically build a binary classification model. The dataset has 500,000 rows and 200 columns, with a severe class imbalance (1% positive). Which configuration should the team set to address the imbalance?
Quick Answer
The correct answer is to specify the objective metric as 'F1' or 'AUC' when configuring the SageMaker Autopilot job. This works because these metrics are threshold-independent and focus on the trade-off between precision and recall or true positive and false positive rates, making them inherently robust to severe class imbalance where accuracy would be misleading. On the AWS Certified Machine Learning Specialty MLS-C01 exam, this question tests your understanding that Autopilot does not support manual data-level techniques like undersampling or SMOTE, nor does it expose a direct class-weight parameter; instead, it relies on the objective metric to guide model optimization. A common trap is assuming you must preprocess the data manually, but Autopilot’s built-in algorithms can handle imbalance when given the right objective. Memory tip: For imbalanced binary classification, remember “F1 or AUC, not accuracy” to avoid the 99% accuracy trap.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Specify the 'objective' as 'F1' or 'AUC' to optimize for imbalanced data.
SageMaker Autopilot allows specifying the objective metric for optimization. For imbalanced datasets, metrics like F1 score or AUC are more appropriate than accuracy because they account for precision and recall or the trade-off between true positive and false positive rates. By setting the objective to 'F1' or 'AUC', Autopilot will optimize the model for these metrics, which better handle class imbalance. Option B (MulticlassClassification) is for multi-class problems, not binary imbalance; Option C (SMOTE) is not supported by Autopilot; Option D (stratified splitting) helps ensure representative validation splits but does not directly address the imbalance in model optimization.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Specify the 'objective' as 'F1' or 'AUC' to optimize for imbalanced data.
Why this is correct
F1 and AUC are better metrics for imbalanced classification.
- ✗
Set the 'problem_type' to 'MulticlassClassification' to handle imbalance.
Why it's wrong here
The problem type is binary; multiclass is incorrect.
- ✗
Use the 'AutoML' job with 'EnsembleMode' and 'SMOTE' sampling.
Why it's wrong here
SMOTE is not a built-in option in Autopilot.
- ✗
Configure the data split to use stratified sampling based on the target.
Why it's wrong here
Stratification helps validation but does not address imbalance in training.
Go deeper
Related to this question
About these practice questions
One of 1,672 original MLS-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on MLS-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A data scientist is using SageMaker Autopilot to automatically build a classification model. The dataset is highly imbalanced (1% positive class). Which configuration should the scientist set to handle the class imbalance?
hard- A.Set the problem_type to 'BinaryClassification' and enable 'balance_class_weights'.
- ✓ B.Use the 'AutoMLJobObjective' with 'F1' metric.
- C.Set the 'sample_weight' attribute in the input data.
- D.Manually downsample the majority class before training.
Why B: SageMaker Autopilot does not support direct class weighting or sample weights for imbalanced datasets. By setting the objective metric to 'F1', Autopilot will optimize the model for the harmonic mean of precision and recall, which is more robust to class imbalance than accuracy. This encourages the model to pay attention to the minority (positive) class during training and hyperparameter tuning.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.