MLA-C01 Practice Question: Refer to the exhibit

Exhibit

{
    "HyperParameterTuningJobConfig": {
        "Strategy": "Bayesian",
        "HyperParameterTuningJobObjective": {
            "Type": "Maximize",
            "MetricName": "validation:accuracy"
        },
        "ResourceLimits": {
            "MaxNumberOfTrainingJobs": 20,
            "MaxParallelTrainingJobs": 5
        },
        "TrainingJobDefinition": {
            "StaticHyperParameters": {
                "epochs": "50"
            },
            "AlgorithmSpecification": {
                "TrainingImage": "some-image",
                "TrainingInputMode": "File"
            },
            "InputDataConfig": [
                {
                    "ChannelName": "train",
                    "DataSource": { "S3DataSource": { "S3DataType": "S3Prefix", "S3Uri": "s3://bucket/train.csv" } }
                }
            ],
            "OutputDataConfig": { "S3OutputPath": "s3://bucket/output" },
            "ResourceConfig": { "InstanceType": "ml.m5.large", "InstanceCount": 1 },
            "StoppingCondition": { "MaxRuntimeInSeconds": 3600 }
        }
    }
}

Refer to the exhibit. A data scientist configured an automatic model tuning job for a classification model. The tuning job completed after 20 training jobs, but the best validation accuracy was only 0.65. What is the most effective way to potentially improve the result?

⚠ Common exam trap

AWS often tests the misconception that increasing parallelism (MaxParallelTrainingJobs) improves model quality, when in fact it only speeds up execution without increasing the total number of trials, which is the key lever for better hyperparameter optimization.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Increase MaxNumberOfTrainingJobs to 100

Increasing MaxNumberOfTrainingJobs to 100 allows the automatic model tuning job to explore a larger hyperparameter space, giving the Bayesian optimization strategy (the default) more trials to converge on a better configuration. With only 20 training jobs, the tuner may not have had enough iterations to balance exploration and exploitation, especially for a complex classification model. More jobs increase the likelihood of finding a hyperparameter combination that yields higher validation accuracy.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✓
Increase MaxNumberOfTrainingJobs to 100
Why this is correct
The tuning job explored only 20 configurations, likely too few to locate a strong region of the hyperparameter space. Raising MaxNumberOfTrainingJobs to 100 lets the tuner evaluate more candidates, improving the chance of higher validation accuracy.
✗
Change the strategy to Random
Why it's wrong here
Random search samples hyperparameters independently rather than exhaustively, so it explores the space differently but does not guarantee finding better values than the Bayesian strategy already used. It suits very large search spaces where exhaustive tuning is impractical, not a 20-job run that stalled at 0.65 accuracy.
✗
Change the objective metric to training:accuracy
Why it's wrong here
Training accuracy measures fit on data the model has already seen, so it rises even when generalisation worsens; tuning toward it encourages overfitting rather than lifting validation accuracy. It is useful only for diagnosing underfitting, where both training and validation scores are low.
✗
Increase MaxParallelTrainingJobs to 10
Why it's wrong here
MaxParallelTrainingJobs only controls how many jobs run concurrently; it changes wall-clock duration, not the hyperparameter search space or objective. Raising parallelism is correct when a tuning job is too slow, but it cannot lift a 0.65 validation accuracy ceiling.

Go deeper

MLA-C01 Practice Question: Refer to the exhibit

Exhibit

Why each option matters

Option-by-option breakdown

Related to this question