MLS-C01 Modeling Practice Test 28 — 15 Questions

Question 1

A company is deploying a real-time inference endpoint with SageMaker. The model is a large neural network that requires GPU acceleration. Which TWO configurations must be set?

Accepted Answer

Instance type with GPU. Deploying a real-time inference endpoint with a large neural network that requires GPU acceleration necessitates selecting an instance type with a GPU, such as the ml.p3 or ml.g4dn series, to provide the parallel processing power needed for low-latency inference. Without a GPU instance, the model would fall back to CPU, leading to unacceptable inference times for large neural networks.

Answer

Batch transform job

Answer

Production variant

Answer

Training container image

Question 2

A data scientist is building a binary classifier and wants to evaluate model performance. Which THREE metrics are most commonly used?

Accepted Answer

Precision. Precision is a core metric for binary classifiers, measuring the proportion of true positive predictions among all positive predictions. It is especially important when the cost of false positives is high, such as in spam detection or fraud alert systems.

Answer

Mean Absolute Error

Answer

RMSE

Question 3

Refer to the exhibit. A data scientist is trying to create a SageMaker training job but receives an access denied error. The IAM policy shown is attached to their role. What is the most likely reason for the error?

Accepted Answer

The policy only allows CreateTrainingJob when the training job status is 'Failed', which is never true initially. The IAM policy uses a `Condition` block with `sagemaker:TrainingJobStatus` set to `Failed`. When a `CreateTrainingJob` API call is made, the training job status is not yet set (it is `Creating` or `InProgress`), so the condition evaluates to false, and the request is denied. The policy only grants permission when the status equals `Failed`, which never occurs at creation time.

Answer

The Action is not allowed because 'CreateTrainingJob' is misspelled

Answer

There is an explicit deny in another policy

Answer

The Resource is set to '*' which does not include the specific training job ARN

Question 4

Refer to the exhibit. A data scientist runs the AWS CLI command to create a SageMaker training job. The job fails immediately with 'ValidationException: Invalid instance type'. What is the most likely issue?

Accepted Answer

The training image URI is for a different AWS region. The error 'ValidationException: Invalid instance type' occurs because the training image URI specified in the command points to an Amazon ECR repository in a different AWS region than where the SageMaker training job is being created. SageMaker validates that the image URI is accessible from the current region; if the URI references a region that does not contain the XGBoost image or the instance type is not supported in that region's ECR, the validation fails. The instance type itself (ml.m5.large) is valid for XGBoost, but the mismatch between the image's region and the job's region triggers the exception.

Answer

The IAM role ARN is invalid

Answer

The S3 bucket 'my-bucket' does not exist or the role lacks permissions

Answer

The instance type ml.m5.large does not support the XGBoost image

Question 5

Refer to the exhibit. What is the recall of the model?

Accepted Answer

0.80. Recall is calculated as True Positives divided by the sum of True Positives and False Negatives. From the confusion matrix, True Positives = 80 and False Negatives = 20, so recall = 80 / (80 + 20) = 0.80. Option B is correct.

Answer

0.85

Answer

0.89

Answer

0.90

Question 6

A data scientist is training a classification model on an imbalanced dataset where the positive class represents only 5% of the data. Which technique would BEST address the class imbalance without discarding data?

Accepted Answer

Use SMOTE to generate synthetic samples for the minority class. SMOTE (Synthetic Minority Oversampling Technique) is the best choice because it generates synthetic samples for the minority class by interpolating between existing minority instances, effectively balancing the dataset without discarding any data. This avoids the information loss of undersampling and the overfitting risk of simple random oversampling, making it ideal for a 5% positive class scenario.

Answer

Randomly undersample the majority class

Answer

Adjust the decision threshold to 0.95

Answer

Randomly oversample the minority class with replacement

Question 7

A data scientist is training a linear regression model and observes that the training loss is low but validation loss is high. Which step should the data scientist take to address this issue?

Accepted Answer

Apply L2 regularization to the model. The model is overfitting (low training loss, high validation loss). L2 regularization adds a penalty on the magnitude of coefficients, which discourages complexity and reduces overfitting. Increasing training epochs (B) would likely worsen overfitting by allowing the model to memorize more. Reducing the training dataset size (C) would provide less data, making overfitting worse. Adding more features (D) increases model complexity and typically exacerbates overfitting.

Answer

Increase the number of training epochs

Answer

Reduce the size of the training dataset

Answer

Add more features to the model

Question 8

A company uses Amazon SageMaker to train a deep learning model on a GPU instance. The training job is taking too long. Which action would MOST likely reduce training time?

Accepted Answer

Use a larger GPU instance type, such as p3.16xlarge. Using a larger GPU instance like p3.16xlarge provides significantly more GPU memory, CUDA cores, and memory bandwidth, which allows for larger batch sizes and more efficient parallel processing of matrix operations. This directly reduces training time for deep learning models by enabling faster forward and backward passes through the network, especially when the model is large enough to fully utilize the additional GPU resources.

Answer

Reduce the mini-batch size

Answer

Use distributed data parallelism across multiple smaller instances

Answer

Reduce the number of epochs

Question 9

A data scientist is building a binary classifier for loan default prediction. The cost of a false negative (missing a default) is 10 times higher than the cost of a false positive. Which evaluation metric is MOST appropriate?

Accepted Answer

F-beta score with beta=2. The F-beta score with beta=2 is the most appropriate metric because it weights recall (sensitivity) higher than precision, which is critical when false negatives are 10 times more costly than false positives. Beta=2 means recall is considered 2^2 = 4 times more important than precision, directly aligning with the asymmetric cost structure. This allows the model to be tuned to minimize missed defaults, even at the expense of more false alarms.

Answer

Precision

Answer

Accuracy

Answer

Area under the ROC curve

Question 10

A data scientist is training a random forest model. During hyperparameter tuning, which parameter is MOST effective at reducing overfitting?

Accepted Answer

Decrease the maximum depth of each tree. Decreasing the maximum depth of each tree limits the complexity of individual trees, preventing them from memorizing noise and outliers in the training data. This directly reduces overfitting by enforcing simpler decision boundaries, which is a core regularization technique for ensemble methods like Random Forest.

Answer

Increase the number of trees

Answer

Increase the number of features considered per split

Answer

Increase the maximum depth of each tree

Question 11

A company is using Amazon SageMaker to train a model. The training job is using a large dataset stored in S3. The data scientist notices that the training job is spending a significant amount of time reading data from S3. Which approach would BEST reduce data loading time?

Accepted Answer

Use the Pipe mode input for the training data. Pipe mode streams data directly from S3 into the training algorithm without first downloading it to the training instance's local storage. This eliminates the I/O bottleneck of writing large datasets to disk, significantly reducing data loading time compared to File mode, which downloads the entire dataset before training begins.

Answer

Use the File mode input with a larger instance

Answer

Use a larger training instance with more CPU

Answer

Increase the batch size to reduce the number of batches

Question 12

A data scientist is using Amazon SageMaker Autopilot to automatically build a model. The dataset contains a mix of numerical and categorical features. After the experiment completes, Autopilot provides several candidate pipelines. Which pipeline is MOST likely to be ranked highest by Autopilot?

Accepted Answer

The pipeline with the lowest validation loss. Amazon SageMaker Autopilot ranks candidate pipelines by their objective metric on the validation dataset, which is typically the validation loss (e.g., cross-entropy for classification or mean squared error for regression). The pipeline with the lowest validation loss generalizes best to unseen data, making it the highest-ranked candidate. Autopilot uses hold-out validation or cross-validation to compute this metric, ensuring the ranking reflects out-of-sample performance rather than overfitting to the training set.

Answer

The pipeline with the simplest model (e.g., linear classifier)

Answer

The pipeline with the fastest training time

Answer

The pipeline with the lowest training loss

Question 13

A data scientist is training a neural network for image classification. The training loss is decreasing steadily, but the validation loss starts increasing after a few epochs. What is the MOST likely cause?

Accepted Answer

The model is overfitting to the training data. The validation loss increasing while the training loss continues to decrease is the classic signature of overfitting. The model is memorizing the training data (including noise) rather than learning generalizable patterns, causing it to perform poorly on unseen validation data.

Answer

The learning rate is too high

Answer

The gradients are vanishing

Answer

The model is underfitting

Question 14

Which TWO approaches are valid for handling missing categorical values in a dataset before training a machine learning model?

Accepted Answer

Impute missing values with the mode of the column. The mode (most frequent value) is the only valid measure of central tendency for categorical data, as it identifies the most common category. Imputing with the mode preserves the distribution of categories and is a standard technique for handling missing categorical values in preprocessing pipelines like scikit-learn's SimpleImputer with strategy='most_frequent'.

Answer

Remove all rows with missing values

Answer

Impute missing values with the median of the column

Answer

Impute missing values with the mean of the column

Question 15

Which THREE factors should be considered when choosing between a parametric and a non-parametric machine learning model?

Accepted Answer

Non-parametric models are generally more flexible. Non-parametric models, such as k-nearest neighbors or decision trees, do not assume a fixed functional form for the data, allowing them to capture complex, non-linear relationships. This flexibility makes them well-suited for datasets where the underlying distribution is unknown or highly irregular, but it also increases the risk of overfitting if not properly regularized.

Answer

Parametric models have lower bias than non-parametric models

Answer

Non-parametric models are less prone to overfitting