MLA-C01 ML Model Development Practice Test 2 — 15 Questions

Question 1

A data scientist is training a regression model in Amazon SageMaker. The dataset contains missing values in several features. The scientist wants to handle missing values as part of the training pipeline to ensure consistency between training and inference. Which approach should the scientist use?

Accepted Answer

Use a scikit-learn container in SageMaker to create a preprocessing step that imputes missing values and include it in the inference pipeline.. Option D is correct because it uses a scikit-learn container within SageMaker to create a preprocessing step that imputes missing values, then includes that step in the inference pipeline. This ensures the same imputation logic (e.g., mean, median, or custom strategy) is applied consistently during both training and inference, preventing data drift and maintaining reproducibility. SageMaker Pipelines or the built-in scikit-learn container allow the preprocessing to be serialized as part of the model artifact, so inference requests automatically undergo the same transformation.

Answer

Impute missing values in a separate Jupyter notebook and save the cleaned data.

Answer

Use SageMaker Autopilot to automatically handle missing values.

Answer

Drop all rows with missing values before training.

Question 2

A machine learning team is using Amazon SageMaker to train a model. They notice that the training job is taking longer than expected and the logs show repeated warnings about 'loss not decreasing'. Which SageMaker feature should they use to diagnose and visualize the training process?

Accepted Answer

Amazon SageMaker Debugger. Amazon SageMaker Debugger is the correct choice because it provides real-time monitoring and visualization of training metrics, including loss values, gradients, and weights. The repeated 'loss not decreasing' warnings indicate a training issue (e.g., vanishing gradients or learning rate problems), and Debugger can capture these tensors and emit alerts or trigger actions (like stopping the job) via built-in or custom rules. It also integrates with SageMaker Studio for interactive visualization of the training progress.

Answer

Amazon SageMaker Clarify

Answer

Amazon SageMaker Experiments

Answer

Amazon SageMaker Model Monitor

Question 3

A data scientist is building a text classification model using Amazon SageMaker. The dataset is stored as a CSV file in Amazon S3. The scientist wants to use the SageMaker built-in BlazingText algorithm. Which of the following steps are required to prepare the data for training? (Choose TWO.)

Accepted Answer

Convert the CSV file to the format of a single file with one instance per line.. Option C is correct because BlazingText expects input data in a single file where each line represents one training instance. This is a specific requirement of the algorithm's input format, not a general SageMaker practice. The CSV file must be converted to this line-per-instance format for BlazingText to process it correctly.

Answer

Convert the text to one-hot encoded vectors.

Answer

Tokenize and remove stop words from the text.

Answer

Upload the data to an Amazon SageMaker notebook instance.

Question 4

An MLOps team is designing a CI/CD pipeline for deploying machine learning models to production on Amazon SageMaker. They want to ensure that the deployment process is automated and that models are automatically rolled back if performance degrades. Which of the following AWS services or features should they use to achieve this? (Choose THREE.)

Accepted Answer

Amazon SageMaker Model Registry. Amazon SageMaker Model Registry is correct because it provides a centralized catalog for managing, versioning, and approving ML models. It enables automated deployment by triggering CI/CD pipelines when a model version is approved, and supports automatic rollback by allowing you to revert to a previous approved version if performance degrades, as detected by monitoring metrics.

Answer

Amazon SageMaker Ground Truth

Answer

AWS CloudTrail

Question 5

A financial services company is deploying a real-time fraud detection model using Amazon SageMaker. The model is a gradient boosting model (XGBoost) trained on historical transaction data. The inference endpoint uses an ml.m5.2xlarge instance with a single variant. Recently, the company has experienced a 3x increase in transaction volume during peak hours, causing inference latency to exceed the 200ms SLA. The data science team has already optimized the model by reducing the number of trees and feature set, but the latency remains high during spikes. The team considers using SageMaker's built-in scaling policies. They currently have a single endpoint with one production variant. The team wants to maintain low latency without over-provisioning resources. They have ruled out model changes. Which approach should the team take?

Accepted Answer

Configure an Application Auto Scaling target tracking scaling policy for the variant based on the 'SageMakerVariantInvocationsPerInstance' metric, with a target value that keeps the inference latency within the SLA.. Option A is correct because SageMaker's built-in target tracking scaling policy using the 'SageMakerVariantInvocationsPerInstance' metric allows the endpoint to automatically adjust the instance count based on real-time invocation load. By setting a target value that correlates with the 200ms SLA, the policy dynamically scales out during traffic spikes and scales in during lulls, preventing over-provisioning while maintaining low latency. This approach directly addresses the 3x peak-hour volume increase without requiring manual intervention or model changes.

Answer

Deploy the model on multiple endpoints behind an Application Load Balancer.

Answer

Use scheduled scaling to increase the instance count during known peak hours.

Answer

Manually increase the instance count during peak hours.

Question 6

A data science team at a financial services company is deploying a real-time fraud detection model using Amazon SageMaker. The model is a gradient boosting classifier trained on historical transaction data. The model is deployed to a SageMaker endpoint with an ML.M5.LARGE instance for real-time inference. After deployment, the team observes that the endpoint's latency spikes to over 2 seconds during peak hours (10:00-12:00 and 14:00-16:00), causing timeouts for client applications. The average latency during off-peak hours is 200 ms. The team has enabled auto-scaling with a target average CPU utilization of 70%, but the endpoint still experiences high latency during peak hours. The instance count never scales beyond 2 instances during peaks. The model size is 500 MB, and each request includes 200 features. The team needs to reduce latency to under 500 ms at the 99th percentile during peak hours without increasing costs beyond the current budget. Which course of action should the team take?

Accepted Answer

Switch the endpoint instance type to a GPU instance such as ml.g4dn.xlarge to accelerate inference.. Option C is correct because GPU instances like ml.g4dn.xlarge are optimized for compute-intensive workloads such as gradient boosting inference, which involves numerous matrix operations. By offloading the computation to the GPU, the model can process each request faster, reducing latency from over 2 seconds to under 500 ms at the 99th percentile without increasing the instance count or budget. This directly addresses the root cause—CPU-bound inference during peak hours—while keeping costs stable.

Answer

Configure SageMaker batch transform for the real-time endpoint to process requests asynchronously.

Answer

Increase the auto-scaling maximum instance count to 10 and set target CPU utilization to 50%.

Answer

Enable data compression on the endpoint to reduce payload size and network latency.

Question 7

A machine learning engineer is developing a text classification model using Amazon SageMaker. The dataset consists of 1 million customer reviews, with labels indicating sentiment (positive, negative, neutral). The engineer uses a pre-trained BERT model from the Hugging Face Model Hub and fine-tunes it on the dataset using SageMaker's Hugging Face estimator with a ml.p3.2xlarge instance. After 2 hours of training, the training job fails with a 'ResourceExhaustedError: CUDA out of memory' error. The error occurs during the forward pass of the first epoch. The engineer confirms that the batch size is set to 32, the maximum sequence length is 512 tokens, and the dataset is stored in a S3 bucket in the same AWS region. The engineer needs to complete fine-tuning without increasing instance costs. Which course of action should the engineer take?

Accepted Answer

Reduce the batch size to 8 and enable gradient accumulation with 4 steps to maintain effective batch size.. Option A is correct because reducing the batch size to 8 directly lowers GPU memory usage per forward pass, and enabling gradient accumulation with 4 steps allows the model to simulate the original effective batch size of 32 (8 × 4 = 32) without increasing memory footprint. This approach resolves the CUDA out-of-memory error while keeping the same instance type (ml.p3.2xlarge) and without incurring additional costs.

Answer

Enable SageMaker Managed Spot Training to reduce costs and use the savings to upgrade to a ml.p3.8xlarge instance.

Answer

Switch to a CPU-based instance like ml.c5.2xlarge to avoid GPU memory constraints.

Answer

Reduce the maximum sequence length to 128 tokens to lower memory consumption.

Question 8

A data scientist is training a binary classification model using a dataset that has a severe class imbalance (90% negative, 10% positive). Which technique should be used to address the imbalance during model training?

Accepted Answer

Apply random oversampling of the minority class. Random oversampling of the minority class (Option C) directly addresses class imbalance by duplicating or synthesizing examples from the positive class, which balances the training distribution and prevents the model from becoming biased toward the majority class. This technique is specifically designed to mitigate the skewed gradient updates that occur when the minority class is underrepresented, leading to better recall and precision for the positive class in binary classification tasks.

Answer

Use a larger batch size

Answer

Use L2 regularization

Answer

Increase the learning rate

Question 9

A machine learning engineer is training a model using SageMaker's built-in XGBoost algorithm. The training job fails with an error indicating insufficient memory. Which parameter should be adjusted to reduce memory usage?

Accepted Answer

max_depth. Option A is correct because decreasing max_depth reduces the size of each tree, which lowers memory consumption during training. Option B (subsample) reduces the number of rows per iteration but may not directly address tree size. Option C (colsample_bytree) reduces the number of features per tree but is less effective than max_depth. Option D (num_round) increases the number of trees but does not directly reduce per-tree memory.

Answer

subsample

Answer

num_round

Answer

colsample_bytree

Question 10

A team is tuning hyperparameters for a neural network using SageMaker's HyperparameterTuningJob with Bayesian optimization. After several trials, the objective metric has not improved significantly. Which action is most likely to help continue making progress?

Accepted Answer

Switch to random search strategy. Option D is correct because switching to random search introduces exploration and can help escape local optima that Bayesian optimization might be stuck exploiting. Option A (switch to Bayesian) is already in use. Option B (warm start) uses previous results but does not change the search strategy. Option C (expand ranges) might help if the optimum lies outside current ranges, but stagnation often requires more exploration.

Answer

Expand the hyperparameter ranges

Answer

Use a warm start with previous tuning results

Answer

Switch to Bayesian search

Question 11

A data scientist is training a large model on SageMaker and wants to reduce training time by using multiple GPUs. The model is small enough to fit on a single GPU but training is slow. Which SageMaker feature should be used?

Accepted Answer

Data parallelism using SageMaker's Distributed Data Parallel. Option D is correct because SageMaker's Distributed Data Parallel (DDP) replicates the model across multiple GPUs and splits mini-batches, speeding up training for models that fit on a single GPU. Option A (Model Parallel) is for models too large for one GPU. Option B (larger instance) may provide more vCPUs but not necessarily more GPUs. Option C (Elastic Inference) accelerates inference, not training.

Answer

Use a larger instance with more vCPUs

Answer

Model parallelism using SageMaker's Model Parallel

Answer

Use Elastic Inference

Question 12

A machine learning engineer is training a deep learning model using TensorFlow in SageMaker. The training runs on an ml.p3.16xlarge instance (8 GPUs). The engineer notices that GPU utilization is low (~30%) and time per epoch is high. The model uses a custom training loop. Which configuration change is most likely to improve GPU utilization?

Accepted Answer

Increase the batch size to match GPU memory. Option C is correct because increasing the batch size increases the computational work per GPU, keeping them busier and improving utilization. Option A (mixed precision) can improve throughput but not necessarily utilization if batch size remains small. Option B (SageMaker Managed Warm Pools) is for inference. Option D (reducing data loading workers) could worsen data starvation, decreasing utilization.

Answer

Reduce the number of data loading workers

Answer

Use mixed precision training

Answer

Enable SageMaker Managed Warm Pools

Question 13

A data scientist is performing feature engineering on a dataset containing a categorical feature with high cardinality (over 1000 unique values). Which encoding method is most appropriate to use as input for a tree-based model?

Accepted Answer

Label encoding. Option A is correct because label encoding assigns integer labels to categories, and tree-based models can effectively split on these ordinal-like values without creating a large number of features. Option B (one-hot encoding) would produce too many features. Option C (target encoding) risks data leakage. Option D (binary encoding) creates fewer features than one-hot but still many and may not be as interpretable for trees.

Answer

One-hot encoding

Answer

Target encoding

Answer

Binary encoding

Question 14

A team is evaluating classification models for a medical diagnosis application. The cost of a false negative is much higher than the cost of a false positive. Which metric should be optimized during model selection?

Accepted Answer

Recall. Option B is correct because recall (true positive rate) focuses on minimizing false negatives, which is the priority when a missed diagnosis is costly. Option A (precision) minimizes false positives. Option C (accuracy) treats all errors equally. Option D (F1 score) balances precision and recall but does not emphasize recall over precision.

Answer

Accuracy

Answer

F1 score

Answer

Precision

Question 15

A machine learning engineer is using SageMaker to train a model with the built-in LightGBM algorithm. The engineer wants to use early stopping to prevent overfitting. The training job is configured with a validation dataset. Which hyperparameter should be set to enable early stopping?

Accepted Answer

early_stopping_rounds. Option B is correct because the SageMaker LightGBM implementation uses the early_stopping_rounds hyperparameter to specify the number of consecutive rounds without improvement before stopping. Option A (num_iterations) sets the maximum number of rounds. Option C (num_boost_round) is an alias for the number of boosting rounds. Option D (early_stopping) is not a valid hyperparameter name.

Answer

num_iterations

Answer

early_stopping

Answer

num_boost_round