MLS-C01 Mock Exam 92 — 25 Questions

Question 1

A company is building a binary classification model to predict customer churn. The dataset has 10,000 samples with 500 churners (positive class). The data scientist trains a logistic regression model and obtains an accuracy of 95%. However, the model predicts all customers as non-churn. Which metric should the data scientist use to evaluate the model's performance?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

F1-score. The F1-score is the harmonic mean of precision and recall, making it robust to class imbalance. Since the model predicts all customers as non-churn (accuracy 95% due to 9500 non-churners), precision for the positive class is undefined (0 true positives) and recall is 0, so the F1-score correctly reveals the model's failure to identify any churners.

Answer

AUC-ROC

Answer

Accuracy

Answer

Confusion matrix

Question 2

A data scientist is building a recommendation system for an e-commerce platform using Amazon SageMaker. The system needs to provide personalized product recommendations based on user purchase history and product metadata. The dataset contains 10 million users and 1 million products. Which algorithm should the data scientist use as the core of the recommendation engine?

Accepted Answer

Factorization Machines. Factorization Machines (FM) are specifically designed for high-dimensional sparse data like user-item interactions, making them ideal for recommendation systems with 10 million users and 1 million products. FM can capture pairwise feature interactions (e.g., user-product affinities) efficiently using factorized parameters, which scales well to large datasets and supports personalized recommendations from purchase history and metadata.

Answer

Linear Learner

Answer

XGBoost

Answer

K-Means

Question 3

A data scientist is training a deep learning model on a large dataset using Amazon SageMaker. The training job is taking too long. Which action would MOST likely reduce training time without sacrificing model accuracy?

Accepted Answer

Enable data parallelism across multiple GPUs. Enabling data parallelism across multiple GPUs distributes the training workload across several devices, allowing larger batch sizes and faster gradient computation per epoch. Amazon SageMaker's distributed training libraries (e.g., SageMaker Data Parallelism) use all-reduce algorithms to synchronize gradients efficiently, which reduces wall-clock training time without altering the model architecture or loss function, thus preserving accuracy.

Answer

Use a smaller instance type for training

Answer

Implement early stopping with a low patience value

Answer

Reduce the number of epochs by half

Question 4

A machine learning engineer is trying to deploy a model using a SageMaker endpoint but receives an access denied error. The IAM policy attached to the role is shown in the exhibit. What is the MOST likely cause of the error?

Accepted Answer

The policy does not include iam:PassRole.. The error occurs because the IAM role used by SageMaker does not have the iam:PassRole permission, which is required to allow SageMaker to assume the role and access the necessary resources (e.g., S3 buckets, EC2 instances) during endpoint deployment. Without this permission, SageMaker cannot pass the role to the service, resulting in an access denied error even if other SageMaker actions are allowed.

Answer

The policy does not include sagemaker:CreateEndpoint.

Answer

The policy does not specify resource ARNs.

Answer

The policy does not include sagemaker:InvokeEndpoint.

Question 5

A data engineer is performing exploratory data analysis on a large dataset stored in Amazon S3 (10 TB in CSV format). The dataset has 2000 columns and 50 million rows. The engineer needs to compute summary statistics (mean, median, standard deviation) for each numeric column and identify missing values. Which approach is MOST cost-effective and time-efficient?

Accepted Answer

Convert the data to Apache Parquet format, then use Amazon Athena to run SQL queries for statistics.. Using Amazon Athena with columnar formats like Parquet after converting from CSV reduces query costs and improves performance. Option A (Redshift Spectrum) requires setting up a Redshift cluster, which is overkill. Option B (SageMaker Data Wrangler) may struggle with 2000 columns. Option D (AWS Glue ETL) is more expensive and slower for simple statistics.

Answer

Use Amazon Redshift Spectrum to query the data directly from S3.

Answer

Load the data into Amazon SageMaker Data Wrangler and compute statistics interactively.

Answer

Use AWS Glue ETL to compute statistics and write results to S3.

Question 6

A data scientist is training a random forest model for a binary classification task. The dataset has 100,000 samples and 500 features. The model is overfitting. Which TWO actions are MOST likely to reduce overfitting?

Accepted Answer

Reduce the maximum depth of each tree. Reducing the maximum depth of each tree limits the complexity of individual trees, preventing them from memorizing noise and specific patterns in the training data. This is a standard regularization technique for random forests that directly combats overfitting by controlling the variance of the model.

Answer

Increase the number of trees in the forest

Answer

Increase the number of features considered at each split

Answer

Use all features for each tree

Question 7

A data scientist is training a neural network for image classification. The training loss is not decreasing significantly, and the validation loss is high. Which TWO actions should the scientist take to address potential vanishing gradients?

Accepted Answer

Use ReLU activation functions in hidden layers. ReLU activation functions help mitigate vanishing gradients because they output a constant gradient of 1 for positive inputs, preventing the gradient from shrinking as it propagates backward through many layers. This avoids the exponential decay of gradients that occurs with saturating activations like sigmoid or tanh, enabling effective training of deep networks.

Answer

Increase the learning rate

Answer

Switch activation functions from ReLU to sigmoid

Answer

Remove dropout layers

Question 8

A data scientist is using Amazon SageMaker to train a neural network. The training job fails with the error 'ResourceLimitExceeded: The account-level service limit for ml.p3.8xlarge for training job usage is 0.' What is the most likely cause and solution?

Accepted Answer

The account has not requested a limit increase for ml.p3.8xlarge; submit a limit increase request via AWS Support.. The error message explicitly states that the account-level service limit for ml.p3.8xlarge for training job usage is 0, which means the account has not been granted any capacity for that instance type. AWS enforces service quotas (limits) per account per region, and for GPU-intensive instances like ml.p3.8xlarge, the default limit is often 0 unless a limit increase request has been submitted and approved. Therefore, the correct solution is to request a limit increase via AWS Support.

Answer

The training job is using spot instances; switch to on-demand instances.

Answer

The instance type is not available in the current region; switch to a different region.

Answer

The instance type is too large; use a smaller instance type like ml.m5.large.

Question 9

A data scientist is using Amazon SageMaker Debugger to monitor training. Which THREE types of issues can Debugger monitor?

Accepted Answer

Poor weight initialization. Amazon SageMaker Debugger can monitor training for poor weight initialization by analyzing tensors and gradients during the training process. It uses built-in rules to detect if weights are initialized with values that are too large or too small, which can lead to slow convergence or failure to learn. This is a core capability of Debugger's real-time monitoring of model parameters.

Answer

Hardware failures

Answer

Data drift

Question 10

A data scientist is tuning a gradient boosting model using Amazon SageMaker Automatic Model Tuning. The objective metric is AUC. The training job converges quickly but the final model has low AUC on the validation set. Which hyperparameter should the data scientist adjust to improve validation AUC?

Accepted Answer

Decrease the learning rate and increase the number of rounds. Decreasing the learning rate and increasing the number of rounds is the correct approach because a low learning rate forces the model to take smaller steps toward the optimum, reducing overfitting and allowing more trees to contribute to the ensemble. This combination often improves generalization and validation AUC when the training job converges too quickly, indicating that the model is overfitting or underfitting due to aggressive learning.

Answer

Increase the subsample ratio of training data

Answer

Increase the learning rate

Answer

Increase the maximum depth of trees

Question 11

A company is using Amazon SageMaker to deploy a model for real-time inference. The model receives requests that are small but arrive in bursts. The data scientist wants to minimize latency and cost. Which deployment option is MOST appropriate?

Accepted Answer

Use Amazon SageMaker Serverless Inference. Amazon SageMaker Serverless Inference is the most appropriate option because it automatically scales compute resources based on request volume, charges only for the compute time used during inference (per-millisecond billing), and has no idle costs. This matches the bursty, small-request pattern perfectly, minimizing both latency and cost without requiring manual instance management.

Answer

Use a real-time endpoint with a single instance

Answer

Use a multi-model endpoint with auto-scaling

Answer

Use a batch transform job triggered by a schedule

Question 12

A data scientist is deploying a SageMaker model using CloudFormation. The stack creation fails with the above error. What is the MOST likely cause?

Accepted Answer

The Docker image has not been pushed to the ECR repository. The error indicates that SageMaker cannot find the Docker image specified in the `PrimaryContainer` of the model definition. CloudFormation creates the SageMaker model by referencing an ECR image URI; if that image has not been pushed to the specified ECR repository, the model creation fails immediately. This is the most common cause when the stack creation fails with an error about a missing or inaccessible image.

Answer

The IAM role does not have permissions to access ECR

Answer

The model name is incorrect

Answer

The instance type specified in the endpoint configuration is not available

Question 13

A company is using Amazon SageMaker to train a XGBoost model on a large dataset. The training job is taking a long time. The data scientist wants to reduce training time without sacrificing model accuracy. The dataset is 100 GB in CSV format stored in S3. What is the most effective approach?

Accepted Answer

Use Pipe mode to stream data from S3 instead of downloading it first.. SageMaker's Pipe mode streams data directly from S3 to the training algorithm without writing it to disk, eliminating the I/O bottleneck of downloading the full 100 GB dataset. This reduces training time significantly by overlapping data loading with computation, while preserving model accuracy since the entire dataset is still used.

Answer

Reduce the number of instances to avoid communication overhead.

Answer

Use random sampling to reduce the dataset size to 10 GB.

Answer

Use SageMaker Managed Spot Training to reduce cost, but training time may increase due to interruptions.

Question 14

A data scientist is training a neural network for image classification. The training loss is decreasing steadily, but the validation loss starts increasing after a few epochs. What is the MOST likely cause?

Accepted Answer

The model is overfitting to the training data. The validation loss increasing while the training loss continues to decrease is the classic signature of overfitting. The model is memorizing the training data (including noise) rather than learning generalizable patterns, causing it to perform poorly on unseen validation data.

Answer

The learning rate is too high

Answer

The gradients are vanishing

Answer

The model is underfitting

Question 15

A company is building a data pipeline using AWS Glue to transform data from Amazon RDS to Amazon S3. The pipeline runs daily and processes about 500 GB of data. The team notices that the job is taking longer than expected. Which change would MOST improve the job performance?

Accepted Answer

Increase the number of DPUs for the Glue job. Increasing the number of DPUs (Data Processing Units) for the AWS Glue job directly allocates more distributed computing resources, allowing the job to process the 500 GB dataset in parallel across multiple workers. This is the most effective way to reduce runtime for a large-scale ETL job, as Glue's Spark-based execution scales horizontally with DPU count.

Answer

Disable job bookmarking

Answer

Upgrade the RDS instance to a larger class

Answer

Use smaller file sizes in S3 output

Question 16

A data scientist is performing EDA on a dataset with 100 features. They want to identify which features are most predictive of the target using a model-agnostic method. Which technique should they use?

Accepted Answer

Permutation feature importance. Permutation feature importance is the correct model-agnostic method because it measures the increase in prediction error after permuting a feature's values, breaking the relationship with the target, and works with any model. Pearson correlation (A) is bivariate and only captures linear relationships. L1 regularization (B) is model-specific to linear models and embeds feature selection within the model. SHAP values (C) are model-specific as they rely on game theory and require model outputs for calculation.

Answer

Pearson correlation matrix

Answer

L1 regularization

Answer

SHAP values

Question 17

A data scientist is exploring a dataset with a binary target variable. Which TWO metrics are appropriate for evaluating the balance of the target classes? (Choose two.)

Accepted Answer

Count plot of the target variable. Options A and D are correct. A count plot directly visualizes the frequency of each class in the target variable, making it easy to assess balance. Similarly, value_counts() returns the exact count of each class, providing a numeric measure of balance. Option B (histogram) is designed for continuous variables; while it can depict the counts of two categories, it is not the standard or most appropriate method for evaluating binary class balance. Option C (scatter plot) is used to explore relationships between two numeric features and does not show class distribution. Option E (correlation matrix) measures linear relationships between numeric features and is irrelevant for assessing target class balance.

Answer

Histogram of a feature

Answer

Scatter plot of two features colored by target

Answer

Correlation matrix of all features

Question 18

A data scientist is using Amazon SageMaker's built-in BlazingText algorithm for word2vec embeddings. The dataset is a corpus of 10 million documents. After training, the data scientist observes that the learned embeddings do not capture semantic similarity well (e.g., 'king' and 'queen' are not close). Which hyperparameter adjustment is most likely to improve the quality of embeddings?

Accepted Answer

Increase the vector dimensionality. Increasing the vector dimensionality allows the model to capture more nuanced semantic relationships and co-occurrence patterns in the data. With 10 million documents, the default dimensionality (typically 100 or 300) may be insufficient to encode the rich contextual information, so raising it (e.g., to 300 or 500) gives the model more capacity to learn high-quality embeddings where words like 'king' and 'queen' become closer in vector space.

Answer

Decrease the window size

Answer

Decrease the number of negative samples

Answer

Increase the learning rate

Question 19

A data engineering team uses AWS Glue to run ETL jobs. They notice that jobs are taking longer to complete as data volume grows. They want to optimize performance without increasing cost significantly. Which THREE strategies should they consider?

Accepted Answer

Partition the input data in S3. Partitioning input data in S3 (B) allows AWS Glue to use partition pruning, reading only the relevant subsets of data instead of scanning the entire dataset. This reduces I/O and processing time, directly addressing the performance degradation caused by growing data volumes.

Answer

Remove partitioning from the output

Answer

Use Amazon EMR instead of Glue

Question 20

An IAM policy is attached to a SageMaker notebook instance. The data scientist wants to use the notebook to train a model using data from S3 bucket 'my-bucket'. However, the training job fails with an access denied error. What is the MOST likely cause?

Accepted Answer

The notebook instance role does not have iam:PassRole permission to pass the SageMaker execution role. The most likely cause is that the notebook instance role lacks the iam:PassRole permission, which is required to pass the SageMaker execution role to the training job. When a SageMaker notebook instance creates a training job, it must pass an execution role that the training job will assume to access resources like S3. Without iam:PassRole on the notebook's role, the API call fails with an access denied error, even if all other permissions are correctly configured.

Answer

The sagemaker:CreateTrainingJob permission is not allowed on the specific resource

Answer

The S3 bucket resource ARN is incorrectly formatted

Answer

The s3:GetObject permission is missing for the bucket