MLS-C01 Mock Exam 94 — 25 Questions

Question 1

A data scientist is training a linear regression model to predict house prices. The dataset includes features such as square footage, number of bedrooms, and location. After training, the model achieves an R² of 0.85 on the training set but only 0.60 on the test set. Which of the following is the MOST likely cause of this discrepancy?

Accepted Answer

The model is overfitting the training data. A high R² on the training set (0.85) paired with a significantly lower R² on the test set (0.60) is a classic symptom of overfitting. The model has learned noise and specific patterns in the training data that do not generalize to unseen data, causing poor test performance. Regularization techniques like Lasso or Ridge, or reducing model complexity, would typically address this issue.

Answer

There is multicollinearity among the features

Answer

The model is underfitting the training data

Answer

There is data leakage between the training and test sets

Question 2

A media company uses Amazon SageMaker to train a deep learning model for video classification. The training job uses a single ml.p3.2xlarge instance and processes 50 GB of labeled video data stored in Amazon S3. The training completes successfully in 12 hours. However, the data scientists report that the model’s accuracy is lower than expected. They suspect the training data contains labeling errors. To improve model accuracy without incurring significant additional cost, they want to identify and remove mislabeled training examples before retraining. They have a small budget of $50 and need to complete the analysis within 2 hours. Which approach should the data scientists take?

Accepted Answer

Train a small, fast model on a random sample of the data (e.g., 1 GB) using a cheaper instance like ml.m5.xlarge, then use the model's prediction confidence to flag low-confidence examples as potential mislabels for manual review.. Option C is correct because training a small, fast model on a 1 GB random sample using a cheaper instance (ml.m5.xlarge) allows the team to quickly identify low-confidence predictions, which are strong indicators of mislabeled examples. This approach fits within the $50 budget and 2-hour time constraint, as it avoids processing the full 50 GB dataset and leverages a lightweight model for rapid iteration. By flagging only suspicious samples for manual review, the team can efficiently clean the training data without incurring the cost of re-labeling the entire dataset.

Answer

Use SageMaker Ground Truth to create a new labeling job for the entire dataset, then compare the new labels with the original labels to identify discrepancies.

Answer

Use SageMaker Clarify to generate a bias report for the training data and remove instances that contribute to bias.

Answer

Manually review all 50 GB of video data to correct labels.

Question 3

A financial services company is building a fraud detection model using a large dataset of credit card transactions. The dataset contains 10 million rows with 50 features, including transaction amount, merchant category, time of day, and customer historical features. The label is binary: fraudulent (1% of data) or legitimate. The company wants to deploy a real-time inference endpoint using Amazon SageMaker that can score transactions with sub-100ms latency. The current model is a gradient boosting model (XGBoost) trained on a sample of 1 million rows due to memory constraints. The model achieves 0.95 AUC on a held-out test set but the fraud recall (sensitivity) is only 0.4, which is unacceptable because the cost of missing a fraud is high. The data science team has access to a larger compute instance (ml.m5.24xlarge) for training. Which course of action is most likely to improve fraud recall while maintaining latency requirements?

Accepted Answer

Train the XGBoost model on the full 10 million rows using an ml.p3.2xlarge instance with GPU support, and apply SMOTE oversampling to the minority class before training.. Option A is correct because training on the full 10 million rows with a GPU-accelerated instance (ml.p3.2xlarge) allows the XGBoost model to learn from the complete data distribution, addressing the bias introduced by the 1 million row sample. Applying SMOTE oversampling to the minority class (fraud) directly tackles the class imbalance (1% fraud), which is the root cause of the low recall (0.4). SMOTE generates synthetic fraudulent examples, improving the model's ability to detect fraud without significantly increasing inference latency, as the model architecture and deployment remain unchanged.

Answer

Engineer additional features from transaction time and merchant category, then retrain the XGBoost model on the same 1 million row sample.

Answer

Downsample the majority class to 1% of the original size to create a balanced dataset of 200,000 rows, then retrain the XGBoost model on this balanced sample.

Answer

Replace XGBoost with a logistic regression model trained on the full dataset, as linear models are faster to train and may generalize better on large data.

Question 4

A machine learning engineer is working on a regression problem to predict house prices. The dataset contains 500,000 rows and 20 features, including 'sqft_living', 'bedrooms', 'bathrooms', 'floors', 'waterfront', 'view', 'condition', 'grade', 'yr_built', 'zipcode', and 'lat'. After performing exploratory data analysis, the engineer notices that the 'sqft_living' feature has a right-skewed distribution with a long tail. The 'zipcode' feature is categorical with 70 unique values. The 'lat' feature is continuous. The engineer wants to prepare the data for a linear regression model. Which action should the engineer take to improve model performance?

Accepted Answer

Apply a log transformation to the 'sqft_living' feature.. Linear regression assumes that features are approximately normally distributed, and a right-skewed distribution like 'sqft_living' can violate this assumption, leading to poor model performance. Applying a log transformation compresses the long tail, making the distribution more symmetric and helping the model learn a linear relationship between the feature and the target. This is a standard preprocessing step for skewed features in regression tasks.

Answer

Remove the 'sqft_living' feature because it violates the normality assumption.

Answer

One-hot encode the 'zipcode' feature to capture location effects.

Answer

Apply standard scaling (z-score) to the 'sqft_living' feature.

Question 5

A financial services company uses Amazon SageMaker to train a fraud detection model. The training data is stored in an S3 bucket encrypted with AWS KMS. The SageMaker training job is configured to use a custom Docker container that reads data from S3 and writes model artifacts back to S3. The training job fails with the error: 'Unable to write model artifact to s3://my-bucket/output/model.tar.gz. Access Denied.' The IAM role used by the training job has the following permissions: s3:GetObject and s3:PutObject on the bucket, and kms:Decrypt on the KMS key. The training job is not using a VPC. What is the MOST likely cause of the failure?

Accepted Answer

The IAM role does not have kms:GenerateDataKey permission on the KMS key. Option B is correct because SageMaker training jobs need kms:GenerateDataKey permission to write encrypted objects. Option A is wrong because the role already has s3:PutObject. Option C is wrong because the job is not in a VPC, so no VPC endpoint issues. Option D is wrong because S3 Batch Operations is irrelevant.

Answer

The S3 bucket is in a different region than the training job

Answer

The S3 bucket requires S3 Batch Operations for writing artifacts

Answer

The IAM role does not have s3:PutObject permission on the output bucket

Question 6

A company is using Amazon SageMaker Ground Truth to build a training dataset for an image classification model. The company has a large number of unlabeled images stored in Amazon S3. The data science team wants to use a private workforce consisting of internal employees to label the images. The team creates a labeling job with a private workforce. After starting the job, the team notices that the labeling tasks are not being assigned to any workers. The workers have been added to the private workforce and have received their login credentials. What is the MOST likely cause of this issue?

Accepted Answer

The workers have not been added to the work team that is assigned to the labeling job.. For a private workforce, workers must be added to a work team that is associated with the labeling job. Option C is correct. Option A (IAM permissions) would prevent login, not task assignment. Option B (S3 permissions) would affect data access, not task visibility. Option D (task type) does not affect assignment.

Answer

The labeling job is configured for a different task type than image classification.

Answer

The S3 bucket containing the images has incorrect permissions, preventing workers from viewing the images.

Answer

The workers do not have the required IAM permissions to access the labeling portal.

Question 7

A retail company uses Amazon SageMaker to train a model for product demand forecasting. The dataset contains daily sales data for 10,000 products over 3 years. The data includes features like price, promotions, holidays, and seasonality. The data scientist uses a linear regression model and gets an RMSE of 50 units. However, the business requires more accurate forecasts, especially for products with high variability. The scientist notices that the residuals show a pattern: the model underestimates demand during promotional periods. Which approach should the scientist take to improve the model?

Accepted Answer

Add interaction features between promotion and other variables.. Option D is correct because adding interaction terms between promotion and other features can capture the promotional effect. Option A is wrong because more data may not fix the bias. Option B is wrong because moving to deep learning may be overkill. Option C is wrong because excluding promotions removes important information.

Answer

Collect more historical data for training.

Answer

Use a deep learning model like LSTM.

Answer

Remove promotion features to simplify the model.

Question 8

A data scientist is working on a binary classification problem to predict loan default. The dataset has 200,000 samples and 50 features. The target variable is imbalanced: 5% default, 95% non-default. The scientist trains a logistic regression model and achieves 95% accuracy, but the recall for the default class is only 20%. The business requires that at least 70% of actual defaults be identified (recall >= 0.7). Which approach should the scientist take to improve recall without significantly sacrificing precision?

Accepted Answer

Use oversampling techniques like SMOTE to create synthetic samples of the minority class. SMOTE (Synthetic Minority Oversampling Technique) creates synthetic samples for the minority class by interpolating between existing minority instances, which increases the representation of the default class in the training data. This directly addresses the low recall (20%) by providing the logistic regression model with more balanced class distributions, enabling it to learn decision boundaries that capture more true positives without discarding majority class information. Unlike simple oversampling, SMOTE reduces overfitting risk by generating novel samples rather than duplicating existing ones, which helps maintain precision while improving recall.

Answer

Use random undersampling of the majority class to balance the dataset

Answer

Change the decision threshold to 0.3

Answer

Increase the regularization strength (C) in logistic regression

Question 9

A data scientist is using Amazon SageMaker to train a natural language processing model using a custom Docker container. The training script reads data from an S3 bucket and writes checkpoints to an S3 bucket. The training job is failing with the error 'Unable to write to checkpoint path: s3://my-bucket/checkpoints/'. The IAM role associated with the training job has the following policy: {'Effect': 'Allow', 'Action': 's3:PutObject', 'Resource': 'arn:aws:s3:::my-bucket/checkpoints/*'}. The bucket 'my-bucket' exists and the prefix 'checkpoints/' is empty. What is the most likely cause of the failure?

Accepted Answer

The IAM role is missing the s3:ListBucket permission. The error 'Unable to write to checkpoint path' occurs because the SageMaker training job's IAM role lacks the `s3:ListBucket` permission. Even though the role has `s3:PutObject` on the checkpoint prefix, SageMaker's S3 client first performs a `ListObjects` (or `HeadObject`) call to verify the bucket exists and to check the prefix state before writing. Without `s3:ListBucket` on the bucket itself, the API call fails, causing the write operation to abort.

Answer

The IAM role does not have s3:PutObject permission

Answer

The S3 bucket does not exist

Answer

The checkpoint prefix already contains objects

Question 10

A company runs a real-time recommendation system on SageMaker with a model that uses a deep neural network. The endpoint uses a single ml.p3.2xlarge instance. Recently, the number of users has grown, and the endpoint's latency has increased from 50ms to 200ms, exceeding the SLA of 100ms. The model inference code is optimized and cannot be improved further. The company wants to reduce latency while minimizing cost. The data scientist has the following options: A. Switch to a larger instance type with more GPU memory, such as ml.p3.8xlarge. B. Use SageMaker's Elastic Inference to attach an EI accelerator to the existing instance. C. Deploy the model on multiple smaller instances (e.g., ml.p3.2xlarge) behind a load balancer and distribute traffic. D. Convert the model to use TensorFlow Lite and deploy on a CPU-based instance. Which option is the MOST cost-effective and meets the latency requirement?

Accepted Answer

Use SageMaker's Elastic Inference. Option B is the most cost-effective because Elastic Inference provides dedicated GPU acceleration at a fraction of the cost of a full GPU instance. Option A increases cost significantly. Option C may reduce latency but increases cost and complexity. Option D may not maintain accuracy and may not meet latency requirements on CPU.

Answer

Convert to TensorFlow Lite on CPU

Answer

Switch to a larger instance type, e.g., ml.p3.8xlarge

Answer

Deploy on multiple smaller instances behind a load balancer

Question 11

A company is using Amazon SageMaker to deploy a machine learning model that predicts equipment failure. The model is a binary classifier that outputs a probability. The company wants to set a threshold such that the model correctly identifies 95% of actual failures (recall >= 0.95). The model's precision at the current threshold of 0.5 is 0.7. The data scientist evaluates the model on a test set and obtains the following confusion matrix at threshold 0.5: TP=95, FN=5, FP=40, TN=860. The total actual positives are 100. Which threshold adjustment should the data scientist make to achieve the recall goal?

Accepted Answer

Keep the threshold at 0.5. Option B is correct. Lowering the threshold increases recall by classifying more instances as positive, which reduces false negatives. Currently recall is 95/100 = 0.95, so recall is already 95%. Actually, recall is already 95% at threshold 0.5. So the requirement is already met. But the question might imply that recall needs to be at least 95%, which it is. However, the stem says 'the company wants to set a threshold such that the model correctly identifies 95% of actual failures (recall >= 0.95)'. At threshold 0.5, recall is 95/100 = 0.95, so it meets the requirement. So no adjustment is needed. But the options include 'Keep the threshold at 0.5' as option D. So D is correct. Let me check: If recall is already 0.95, then no change needed. So answer D.

Answer

Decrease the threshold to 0.1

Answer

Increase the threshold to 0.7

Answer

Decrease the threshold to 0.3

Question 12

A company uses Amazon SageMaker to train a linear regression model on a dataset with 10 million rows and 50 features. The training job takes 8 hours to complete. A data scientist wants to reduce the training time to under 2 hours without changing the dataset size or the model algorithm. The SageMaker instance type currently used is ml.m5.2xlarge. Which action should the data scientist take to achieve the desired training time?

Accepted Answer

Use SageMaker's distributed training with 4 ml.m5.2xlarge instances.. Option A is correct because using a distributed training approach with multiple ml.m5.2xlarge instances will parallelize the computation, reducing wall-clock time. Option B (increasing to ml.m5.4xlarge) provides more compute but not enough to reduce time from 8 to 2 hours (only 2x improvement). Option C (changing to ml.p3.2xlarge with GPU) is not optimal for linear regression, which is CPU-bound. Option D (reducing features) changes the dataset and is not allowed.

Answer

Change the instance type to ml.p3.2xlarge (GPU instance).

Answer

Change the instance type to ml.m5.4xlarge (double the vCPUs and memory).

Answer

Reduce the number of features from 50 to 25.

Question 13

A company uses Amazon SageMaker to host a model for real-time inference. The model is a large ensemble of 10 deep learning models, each 500 MB. The total model size is 5 GB, which exceeds the 5 GB limit for SageMaker real-time endpoints. The data scientist wants to reduce the model size without significantly impacting accuracy. The ensemble uses averaging of predictions from all models. The scientist has access to a validation set with 10,000 samples. Which technique should the scientist use to reduce the model size?

Accepted Answer

Use model distillation to train a smaller model that approximates the ensemble. Option A is correct. Model distillation trains a smaller student model to mimic the ensemble, reducing size while preserving accuracy. Option B is wrong because price-aware instance selection does not reduce model size. Option C is wrong because SageMaker Neo is for optimization, not size reduction below 5 GB. Option D is wrong because pruning alone may not reduce size enough.

Answer

Use a more expensive instance type to host the model

Answer

Use SageMaker Neo to compile and optimize the model

Answer

Apply weight pruning to each model in the ensemble

Question 14

A data scientist is training a binary classifier using logistic regression. The dataset has 100 features and 1 million samples. After training, the model achieves AUC of 0.85 on the test set. The business wants to understand which features contribute most to predictions. Which technique should the data scientist use?

Accepted Answer

Use the coefficients of the logistic regression model as feature importance. Coefficients of logistic regression are natural measures of feature importance.

Answer

Use t-SNE to visualize feature importance

Answer

Use a random forest model and its feature importance attribute

Answer

Use Principal Component Analysis (PCA) to find important components

Question 15

A company is using SageMaker to train a text classification model using a built-in BlazingText algorithm. The dataset has 500,000 documents, each labeled with one of 10 categories. The training time is taking longer than expected. The data scientist wants to speed up training without increasing cost. The training job is using a single ml.m4.xlarge instance. The code uses default hyperparameters. Which change is MOST likely to reduce training time? A. Use a larger instance type, such as ml.m4.4xlarge. B. Increase the learning rate significantly. C. Use SageMaker Managed Spot Training. D. Use the 'mode' hyperparameter set to 'batch_skipgram' instead of 'supervised'. The company has a fixed budget and wants to minimize cost while reducing training time. Which option should the data scientist choose?

Accepted Answer

Use SageMaker Managed Spot Training. Option C is the best because spot instances can reduce cost and training time is not affected; they can use the same instance type at lower cost, allowing them to use more instances if needed. Option A increases cost. Option B may cause the model not to converge. Option D changes the problem to unsupervised, not appropriate.

Answer

Increase the learning rate significantly

Answer

Use the 'mode' hyperparameter set to 'batch_skipgram' instead of 'supervised'

Answer

Use a larger instance type, such as ml.m4.4xlarge

Question 16

A financial services company is developing a fraud detection model using gradient boosting. The dataset contains 10 million transactions with 0.1% fraudulent. The model is trained on a SageMaker ml.m5.2xlarge instance and takes 8 hours. The team needs to reduce training time without sacrificing model performance. They have permission to use up to 4 instances. What should they do?

Accepted Answer

Switch to a built-in XGBoost with GPU support and use a p3.2xlarge instance. GPU instances like p3.2xlarge accelerate XGBoost training substantially.

Answer

Use SageMaker hyperparameter tuning to find faster hyperparameters

Answer

Use SageMaker's distributed training with data parallelism across 4 ml.m5.2xlarge instances

Answer

Use SageMaker managed spot training with checkpointing

Question 17

A bank is building a credit risk model using a large dataset with 500 features and 2 million samples. The dataset contains many categorical features with high cardinality (e.g., zip code, occupation). The model must be deployed on SageMaker and provide real-time predictions with low latency. They also need to explain individual predictions for regulatory compliance. Which approach is most appropriate?

Accepted Answer

Use a gradient boosting model with ordinal encoding for categorical features and use SageMaker's built-in XGBoost with SHAP. XGBoost with ordinal encoding and SHAP balances performance, latency, and explainability.

Answer

Use a linear model with target encoding for categorical features and deploy with SageMaker's built-in linear learner algorithm

Answer

Use a deep neural network with embedding layers for categorical features and use SageMaker's built-in Debugger for explanations

Answer

Use XGBoost with one-hot encoding for categorical features and deploy with SageMaker's built-in SHAP explainer

Question 18

A company is using SageMaker built-in object detection algorithm to detect defects in manufacturing images. The model is trained on 10,000 labeled images and achieves 95% accuracy. However, in production, the model misclassifies many defective items as non-defective (false negatives). The business requires recall > 90% for the defect class. Which action should they take?

Accepted Answer

Adjust the decision threshold of the model to increase recall at the expense of precision. Threshold tuning directly optimizes recall for a given class.

Answer

Use a different algorithm such as semantic segmentation

Answer

Use SageMaker's Automatic Model Tuning to find better hyperparameters

Answer

Retrain the model with more images of non-defective items

Question 19

A research lab is training a large language model (LLM) on SageMaker using PyTorch. The model has 1 billion parameters and does not fit on a single GPU. They have access to a cluster of 16 p4d.24xlarge instances (each with 8 A100 GPUs). They need to train the model with minimal changes to the training script. Which SageMaker feature should they use?

Accepted Answer

SageMaker's model parallelism with automatic partitioning. SageMaker's model parallelism is designed for large models that don't fit on a single device.

Answer

SageMaker's distributed data parallelism with Horovod

Answer

Use SageMaker's built-in BlazingText algorithm

Answer

SageMaker's managed spot training with checkpointing

Question 20

A startup is building a recommendation system for an e-commerce platform using collaborative filtering. They have a dataset of user-item interactions (ratings) with 1 million users and 100,000 items. The data is sparse (99% missing ratings). They need to train a model on Amazon SageMaker that can handle large-scale sparse data efficiently. Which approach should they use?

Accepted Answer

Use the built-in Factorization Machines algorithm in SageMaker. SageMaker's Factorization Machines handle sparse data efficiently and are designed for recommendation tasks.

Answer

Use PCA to reduce dimensionality and then apply k-nearest neighbors

Answer

Use the built-in XGBoost algorithm with one-hot encoding for user and item IDs

Answer

Implement a neural network with dense layers using the built-in MXNet framework