MLA-C01 Practice Test 30 — 15 Questions

Question 1

A company uses SageMaker Neo to compile a trained model for deployment on edge devices. What is the primary benefit of using Neo?

Accepted Answer

It reduces model size and improves inference speed on target hardware. SageMaker Neo compiles trained models into optimized executables for specific target hardware (CPU, GPU, or edge accelerators), producing smaller artifacts and faster inference by leveraging hardware-specific instruction sets. It is designed for edge and constrained deployments where latency, memory, and compute are limited. Neo does not handle monitoring, retraining, or endpoint provisioning.

Answer

It monitors model drift in production

Answer

It automatically retrains the model on new data

Answer

It provides a serverless inference endpoint

Question 2

A company wants to reduce costs for a SageMaker real-time endpoint that has variable traffic. Which feature allows the endpoint to automatically adjust instance count based on demand?

Accepted Answer

Auto Scaling for SageMaker endpoints. Auto Scaling for SageMaker endpoints is the native capability that dynamically adjusts the number of instances behind a real-time endpoint based on CloudWatch metrics such as InvocationsPerInstance or ModelLatency. It uses Application Auto Scaling policies (target tracking or step scaling) to add instances during traffic spikes and remove them during lulls, directly reducing cost for variable workloads. Savings Plans and Inference Recommender do not perform runtime scaling.

Answer

SageMaker Savings Plans

Answer

SageMaker Inference Recommender

Answer

SageMaker Model Monitor

Question 3

An organization wants to ensure that only approved model versions can be deployed to production. They use the SageMaker Model Registry to track model versions. How can they enforce that only approved models are deployed?

Accepted Answer

Use IAM policies to restrict deployment to only Approved model versions. AWS IAM policies can be used to conditionally restrict SageMaker API actions (e.g., CreateEndpointConfig, CreateModel) based on the model version's approval status. By evaluating the `sagemaker:ModelPackageApprovalStatus` condition key in an IAM policy, you can enforce that only model versions with an `Approved` status can be deployed, providing a native, automated, and auditable enforcement mechanism without manual intervention or external dependencies.

Answer

Manually review each model before deployment

Answer

Use SageMaker Model Monitor to check model quality after deployment

Answer

Store model metadata in a DynamoDB table and check it before deployment

Question 4

A company is building a fraud detection model on an imbalanced dataset (99% legitimate, 1% fraudulent). To improve recall on the minority class, they want to resample data. Which combination of techniques should they use?

Accepted Answer

SMOTE on training set only. SMOTE (Synthetic Minority Over-sampling Technique) generates synthetic minority-class samples by interpolating between existing minority instances. Applying SMOTE only to the training set prevents synthetic samples from leaking into the test set, which would inflate evaluation metrics and produce an overly optimistic model. This is the correct resampling approach for improving recall on an imbalanced fraud dataset.

Answer

SMOTE on entire dataset before train/test split

Answer

Random oversampling of minority class before train/test split

Answer

Random undersampling of majority class

Question 5

A data scientist is training a deep learning model on Amazon SageMaker and notices that the training loss decreases but the validation loss starts increasing after a certain number of epochs. The model is likely overfitting. Which SageMaker feature can they use to detect and diagnose this issue during training?

Accepted Answer

SageMaker Debugger. SageMaker Debugger is the correct choice because it provides real-time monitoring of training metrics, including loss values, and can automatically detect anomalies such as overfitting (where training loss decreases but validation loss increases). It allows you to set rules (e.g., `OverfitRule`) that trigger alerts or stop training when overfitting is detected, enabling proactive diagnosis during the training job.

Answer

SageMaker Model Monitor

Answer

SageMaker Automatic Model Tuning

Answer

SageMaker Experiments

Question 6

A financial services company uses SageMaker to train a fraud detection model. They have imbalanced data with 1% fraud. They trained a Gradient Boosting model using SMOTE for oversampling and achieved 99% accuracy on the test set, but the fraud recall is only 10%. The data scientist is concerned about the model's performance. Which change is most likely to improve fraud recall without sacrificing too much precision?

Accepted Answer

Increase the weight of the fraud class in the loss function.. The model achieves 99% accuracy but only 10% fraud recall because the class imbalance causes the loss function to be dominated by the majority (non-fraud) class. Increasing the weight of the fraud class in the loss function (e.g., via scale_pos_weight in XGBoost or class_weight in scikit-learn) directly penalizes misclassification of fraud cases more heavily, forcing the model to prioritize recall on the minority class. This is a targeted intervention at the learning objective itself, unlike changing the evaluation metric, which only changes how you measure performance without altering what the model learns.

Answer

Use a different evaluation metric like F1-score during training.

Answer

Reduce the SMOTE sampling ratio to create more synthetic samples.

Answer

Use a random undersampling of the majority class.

Question 7

A machine learning engineer is evaluating a binary classification model that predicts customer churn. The model achieves 95% accuracy, but the engineer suspects class imbalance is causing a misleading metric. Which THREE evaluation steps should the engineer perform to properly assess the model? (Choose THREE.)

Accepted Answer

Calculate precision, recall, and F1-score. Option B is correct because precision, recall, and F1-score are threshold-based classification metrics that reveal how well the model identifies the minority churn class, unlike accuracy which can be inflated by class imbalance. Option D is correct because the ROC curve plots True Positive Rate against False Positive Rate across thresholds and AUC summarizes discriminative ability independently of a single threshold, making it robust for imbalanced binary classification. Option E is correct because the confusion matrix breaks predictions into TP, TN, FP, and FN counts, exposing exactly how many churners are missed or falsely flagged and serving as the basis for precision, recall, and F1. Options A and C are incorrect because RMSE and MAE are regression error metrics measuring continuous prediction deviations, not appropriate for evaluating a binary classification model's class-imbalanced performance.

Answer

Calculate RMSE

Answer

Compute Mean Absolute Error (MAE)

Question 8

Refer to the exhibit. A data scientist ran a training job using a custom algorithm container. The job failed with the error shown. What is the most likely cause?

Accepted Answer

The algorithm script references an undefined variable or metric named 'loss'. The error shown in the exhibit indicates that the training job failed because the algorithm script references an undefined variable or metric named 'loss'. In SageMaker custom algorithm containers, the training script must define and log metrics explicitly; if the script tries to use a metric like 'loss' without defining it or if it's misspelled, the job fails with a NameError or similar. The other options relate to infrastructure or configuration issues that would produce different error messages.

Answer

The S3 output path is incorrect

Answer

The training image is not accessible

Answer

The instance type is insufficient

Question 9

Refer to the exhibit. A SageMaker training job logs show training AUC increasing but validation AUC plateauing at 0.880. What is the most likely issue?

Accepted Answer

Overfitting. The training AUC increasing while validation AUC plateaus at 0.880 is a classic sign of overfitting. The model is learning noise and patterns specific to the training data that do not generalize to unseen validation data, causing the validation metric to stall despite continued improvement on the training set.

Answer

Learning rate too high

Answer

Underfitting

Answer

Insufficient training data

Question 10

A machine learning engineer needs to select features for a regression model. The dataset contains 50 numeric features, and the target variable is continuous. The engineer wants to reduce dimensionality by selecting features that have the strongest linear relationship with the target. Which feature selection method is MOST appropriate?

Accepted Answer

Correlation analysis. Correlation analysis (Option B) is the most appropriate method because it measures the linear relationship between each feature and the continuous target variable. Features with the strongest absolute correlation coefficients are selected, directly aligning with the goal of reducing dimensionality by choosing features with the strongest linear relationship.

Answer

Lasso regularization

Answer

Mutual information

Answer

Recursive feature elimination (RFE)

Question 11

A trained model needs to be deployed for real-time inference with low latency. Which AWS service is best suited for this?

Accepted Answer

SageMaker endpoints. SageMaker endpoints are designed for real-time inference by provisioning persistent, auto-scaled HTTPS endpoints that return predictions with millisecond latency. They support automatic scaling, A/B testing, and can be deployed behind a VPC for low-latency access, making them the ideal choice for serving a trained model in production.

Answer

SageMaker Batch Transform

Answer

SageMaker Hyperparameter Tuning

Answer

AWS Lambda with model packaged

Question 12

A machine learning engineer is deploying a model using SageMaker and needs to ensure that the endpoint can automatically scale based on traffic patterns. Which TWO actions should the engineer take? (Choose two.)

Accepted Answer

Define a scaling policy using Application Auto Scaling for the SageMaker endpoint variant.. Option A is correct because SageMaker endpoint variants are scaled through Application Auto Scaling, which is the AWS service that registers the SageMaker variant as a scalable target and applies a scaling policy (target-tracking or step scaling) to adjust the desired instance count. Option B is correct because Application Auto Scaling policies are driven by Amazon CloudWatch alarms, and the InvocationsPerInstance metric is the standard SageMaker metric used to scale on traffic per instance. Option C is incorrect because SageMaker Model Monitor detects data drift and quality issues, not traffic-based scaling. Option D is incorrect because multi-model endpoints consolidate multiple models on shared infrastructure to reduce hosting cost, not to autoscale on traffic patterns. Option E is incorrect because batch transform is an offline, batch inference mechanism and does not serve a real-time, autoscaling endpoint.

Answer

Enable SageMaker Model Monitor to detect data drift.

Answer

Configure a multi-model endpoint to serve multiple models.

Answer

Use SageMaker batch transform to handle variable traffic.

Question 13

A machine learning engineer is training a deep learning model on SageMaker and notices that the training loss decreases rapidly in the first few epochs but then plateaus. The validation loss starts increasing after 10 epochs. Which action should the engineer take to improve generalization?

Accepted Answer

Use early stopping with validation loss monitoring. Early stopping is the correct action because the validation loss increasing after 10 epochs while training loss continues to decrease is a classic sign of overfitting. By monitoring validation loss and halting training when it stops improving (e.g., using a patience parameter), the engineer prevents the model from memorizing noise in the training data, thereby improving generalization. SageMaker's built-in training job features or the `EarlyStopping` callback in frameworks like TensorFlow or PyTorch can implement this directly.

Answer

Add more layers to the model

Answer

Increase the learning rate

Answer

Decrease the batch size

Question 14

A machine learning engineer notices that the latency of a SageMaker endpoint has increased over time. They need to identify which component (model inference vs. pre/post-processing) contributes most to the latency. Which CloudWatch metrics should they examine?

Accepted Answer

ModelLatency and OverheadLatency. SageMaker endpoints emit CloudWatch metrics that break down total latency into model inference time (ModelLatency) and the time spent in pre/post-processing (OverheadLatency). By comparing these two metrics, the engineer can pinpoint whether the bottleneck is in the inference code or in the custom preprocessing/postprocessing logic. Option D directly provides both metrics needed for this root-cause analysis.

Answer

Latency and ModelLatency

Answer

Invocations and 4XXError

Answer

5XXError and MemoryUtilization

Question 15

A team is using SageMaker Pipelines to train a model. The pipeline has multiple steps: data processing, training, evaluation, and registration. They use a Condition step to evaluate the model's accuracy and if it exceeds a threshold, register the model. They run the pipeline and the training step succeeds, but the pipeline fails at the Condition step with an error: 'Unable to evaluate condition: the property 'Accuracy' does not exist.' The evaluation step output is a JSON file with key 'accuracy'. What is the most likely cause?

Accepted Answer

The Condition step is referencing the wrong property name.. The Condition step in SageMaker Pipelines evaluates a property from the output of a previous step. The error 'Unable to evaluate condition: the property 'Accuracy' does not exist' indicates that the Condition step is looking for a property named 'Accuracy' (capital A), but the evaluation step outputs a JSON file with the key 'accuracy' (lowercase a). This mismatch in property name casing causes the condition to fail, even though the evaluation step produced the correct output.

Answer

The evaluation step did not produce the output correctly.

Answer

The training step output is being used instead of the evaluation step output.

Answer

The pipeline definition has a syntax error.