MLS-C01 Practice Questions 2 — 50 Questions

Question 1

A data scientist is tuning a gradient boosting model using Amazon SageMaker's Automatic Model Tuning (hyperparameter optimization). The objective metric is validation:auc. After 50 training jobs, the best model still has a validation AUC of only 0.65. The scientist suspects overfitting because the training AUC is 0.99. Which hyperparameter configuration is MOST likely to reduce overfitting?

Accepted Answer

Increase lambda from 1 to 10. Increasing lambda (L2 regularization) from 1 to 10 adds a stronger penalty on the magnitude of leaf weights in the gradient boosting model. This directly reduces overfitting by discouraging the model from fitting noise in the training data, which is consistent with the observed gap between training AUC (0.99) and validation AUC (0.65). In XGBoost, lambda controls the L2 regularization term on weights, and a higher value forces the model to be simpler and more generalizable.

Answer

Increase num_round from 100 to 500

Answer

Increase max_depth from 6 to 12

Answer

Increase subsample from 0.5 to 1.0

Question 2

A company is building a recommendation system for an e-commerce platform. They have user-item interaction data and want to use matrix factorization. However, the dataset is sparse (99% missing interactions). Which approach should the data scientist take to train the model effectively?

Accepted Answer

Use alternating least squares (ALS) with implicit feedback and assign lower confidence to unobserved interactions. Option B is correct because Alternating Least Squares (ALS) with implicit feedback is specifically designed to handle sparse implicit feedback datasets by assigning lower confidence to unobserved interactions (e.g., confidence = 1 + alpha * r_ui, where r_ui is 0 for unobserved). This avoids the pitfalls of treating missing values as zeros (which distorts the factorization) and scales well to 99% sparsity by leveraging weighted regularization.

Answer

Impute missing values with zeros and use singular value decomposition (SVD)

Answer

Remove all users and items with fewer than 10 interactions to reduce sparsity

Answer

Use item-based collaborative filtering with cosine similarity

Question 3

A data scientist is training a binary classification model on a dataset with 10,000 features. The model overfits severely. Which technique is MOST appropriate to reduce overfitting?

Accepted Answer

Apply L1 regularization (Lasso). L1 regularization (Lasso) can shrink some feature coefficients to zero, effectively performing feature selection and reducing overfitting. Early stopping is more for iterative algorithms. PCA reduces dimensionality but may lose interpretability. Increasing max depth would worsen overfitting.

Answer

Use early stopping during training

Answer

Use PCA to reduce dimensionality

Answer

Increase the max depth of the model

Question 4

A data scientist is training a model using Amazon SageMaker and wants to automatically stop training when the model stops improving. Which feature should be used?

Accepted Answer

Enable early stopping in the training job configuration.. Option D is correct because SageMaker's built-in early stopping feature automatically halts a training job when the model's objective metric (e.g., loss or accuracy) ceases to improve over a specified number of steps or epochs. This is configured directly in the training job's `StoppingCondition` parameter, which monitors the metric defined in the `MetricDefinitions` and stops training if no improvement is detected, saving compute time and avoiding overfitting.

Answer

Use SageMaker Debugger to monitor the loss metric.

Answer

Configure a CloudWatch alarm on the training job's CPU utilization.

Answer

Use SageMaker Hyperparameter Tuning with random search.

Question 5

A company is using Amazon SageMaker to train a model with a custom algorithm. The training script reads data from an S3 bucket using boto3. The training job fails with an 'AccessDenied' error when trying to access the S3 bucket. The IAM role attached to the SageMaker notebook instance has full S3 access. What is the most likely cause?

Accepted Answer

The SageMaker execution role used for the training job does not have S3 access permissions.. The IAM role attached to the SageMaker notebook instance is used for interactive development, but training jobs run under a separate SageMaker execution role. Even if the notebook role has full S3 access, the training job's execution role must also have explicit S3 permissions. The 'AccessDenied' error indicates that the execution role lacks the necessary s3:GetObject or s3:ListBucket actions for the S3 bucket.

Answer

The S3 bucket has a bucket policy that denies access from the SageMaker service.

Answer

The training script is using an incorrect S3 bucket name.

Answer

The SageMaker training job is not configured to use the S3 VPC endpoint.

Question 6

A company is using Amazon SageMaker to train a deep learning model. The training job is failing with an error 'CUDA out of memory'. The training instance is an ml.p3.2xlarge with 16 GB GPU memory. The model architecture and batch size are appropriate for this instance size. What is the most likely cause of this error?

Accepted Answer

Enable automatic mixed precision (AMP) training to reduce memory usage.. Option C is correct because enabling automatic mixed precision (AMP) training reduces GPU memory usage by storing tensors in half-precision (FP16) where possible, while keeping critical operations in full precision (FP32). This directly addresses the 'CUDA out of memory' error on an ml.p3.2xlarge instance (16 GB GPU memory) without changing the model architecture or batch size, which are already appropriate.

Answer

Reduce the number of epochs.

Answer

Increase the number of GPUs by using a distributed training instance type.

Answer

Use a smaller instance type to force lower memory usage.

Question 7

A company is using Amazon SageMaker to train a deep learning model on a large dataset stored in S3. The training job is failing with an OutOfMemory error. The data scientist wants to minimize cost while resolving the issue. Which action should the data scientist take?

Accepted Answer

Change the input mode from 'File' to 'Pipe'.. The OutOfMemory error occurs because the 'File' input mode downloads the entire training dataset to the instance's local storage before training begins, consuming significant memory. Switching to 'Pipe' mode streams data directly from S3 to the training algorithm, reducing memory footprint and avoiding the need for larger instances. This minimizes cost by using the existing instance type while resolving the memory issue.

Answer

Increase the instance type to one with more memory.

Answer

Use the 'auto' setting for the input mode.

Answer

Reduce the batch size hyperparameter.

Question 8

A company is using Amazon SageMaker to train a XGBoost model on a large dataset. The training job is taking a long time. The data scientist wants to reduce training time without sacrificing model accuracy. The dataset is 100 GB in CSV format stored in S3. What is the most effective approach?

Accepted Answer

Use Pipe mode to stream data from S3 instead of downloading it first.. Option B is correct because SageMaker's Pipe mode streams data directly from S3 to the training algorithm without writing it to disk, eliminating the I/O bottleneck of downloading the full 100 GB dataset. This reduces training time significantly by overlapping data loading with computation, while preserving model accuracy since the entire dataset is still used.

Answer

Reduce the number of instances to avoid communication overhead.

Answer

Use random sampling to reduce the dataset size to 10 GB.

Answer

Use SageMaker Managed Spot Training to reduce cost, but training time may increase due to interruptions.

Question 9

A data scientist is deploying a model using Amazon SageMaker. The model endpoint needs to handle real-time inference requests with low latency. The model is a large ensemble of 10 deep learning models, each approximately 500 MB. What is the most cost-effective deployment strategy that meets the low-latency requirement?

Accepted Answer

Use a SageMaker multi-model endpoint to host all models on one or more instances.. A SageMaker multi-model endpoint (MME) allows hosting multiple models on a single or few instances, dynamically loading them from Amazon S3 into memory as needed. This is the most cost-effective option for a large ensemble of 500 MB models because it avoids the expense of separate endpoints or multiple instances per model, while still supporting low-latency real-time inference by keeping frequently used models cached.

Answer

Deploy each model to a separate endpoint and use a load balancer.

Answer

Use a single endpoint with multiple instances behind it.

Answer

Use a SageMaker batch transform job to process inference requests in batches.

Question 10

An IAM policy is attached to a SageMaker execution role. A data scientist tries to create a training job using a custom algorithm stored in an ECR repository. The training job fails with an 'AccessDenied' error when pulling the Docker image from ECR. What is the missing permission?

Accepted Answer

ecr:GetDownloadUrlForLayer and ecr:BatchGetImage on the ECR repository. When SageMaker pulls a custom Docker image from ECR during training job creation, the execution role needs permissions to download the image layers. The required actions are ecr:GetDownloadUrlForLayer (to generate pre-signed URLs for each layer) and ecr:BatchGetImage (to retrieve image metadata and layer manifests). Without these, the 'AccessDenied' error occurs because SageMaker cannot authenticate or fetch the container image from the ECR repository.

Answer

ecr:PutImage on the ECR repository

Answer

s3:GetObject on the ECR repository

Answer

sagemaker:CreateTrainingJob on the ECR resource

Question 11

A machine learning engineer is deploying a model using AWS Lambda for real-time inference. The model is a scikit-learn RandomForestClassifier with 100 trees, serialized as a pickle file of 150 MB. The Lambda function has 3 GB memory allocated. However, the inference requests are timing out after 30 seconds. What is the most likely cause?

Accepted Answer

The model is loaded from S3 on every invocation, causing high latency.. Option C is correct because the default behavior of loading a model from S3 on every Lambda invocation introduces significant latency. Each invocation must download the 150 MB pickle file from S3 over the network, deserialize it, and then run inference, which easily exceeds the 30-second timeout. The model should be loaded once outside the handler (in global scope) and reused across invocations to avoid this overhead.

Answer

scikit-learn is not compatible with AWS Lambda.

Answer

The Lambda function does not have enough memory to load the model.

Answer

The Lambda function timeout is set too low; increase it to 5 minutes.

Question 12

A data scientist is using Amazon SageMaker for hyperparameter tuning. The tuning job uses a Bayesian optimization strategy. After 10 training jobs, the objective metric (validation accuracy) has plateaued at 0.85. The data scientist wants to explore more diverse hyperparameter combinations. What should the data scientist do?

Accepted Answer

Increase the exploration weight in the tuning job configuration.. In Bayesian optimization, the exploration weight controls the trade-off between exploring new hyperparameter regions and exploiting known good regions. Increasing this weight encourages the acquisition function to sample more diverse combinations, which can help escape a plateau. Option C is correct because it directly addresses the need for greater diversity in the search space.

Answer

Decrease the exploration weight in the tuning job configuration.

Answer

Switch to random search strategy.

Answer

Increase the number of parallel training jobs.

Question 13

A company is using Amazon SageMaker to deploy a model for real-time inference. The model endpoint is behind an Application Load Balancer (ALB) for A/B testing. The data scientist notices that the endpoint is returning HTTP 503 errors intermittently. The CloudWatch metrics show that the endpoint's Invocations metric is within limits, but the ModelLatency metric has high variance. What is the most likely cause?

Accepted Answer

The endpoint instance type does not have enough memory for the model.. Option C is correct because high variance in ModelLatency combined with intermittent 503 errors strongly indicates that the model container is running out of memory under load. When memory is insufficient, the inference process may be killed by the kernel (OOM killer) or the container may be throttled, causing sporadic failures that manifest as 503s even though the Invocations metric (request count) appears within limits. The latency spikes occur because the container struggles to allocate memory for each request, leading to timeouts or crashes.

Answer

The model container is using a custom inference code that has a bug.

Answer

The ALB health check is misconfigured and marking instances unhealthy.

Answer

The endpoint is configured with too few instances; increase the instance count.

Question 14

A DevOps engineer created a SageMaker notebook instance using the Terraform configuration shown. The notebook instance is in a VPC with a public subnet. However, the notebook instance cannot access the internet. What is the most likely cause?

Accepted Answer

The subnet does not have a route to an internet gateway.. Option C is correct because a SageMaker notebook instance in a VPC with a public subnet requires a route to an internet gateway (IGW) in the subnet's route table to access the internet. Without that route, traffic from the notebook cannot reach the internet, even if `direct_internet_access` is enabled. The Terraform configuration likely omitted the route to the IGW, causing the connectivity failure.

Answer

The role_arn is incorrect or missing permissions.

Answer

The instance type ml.t2.medium does not support internet access.

Answer

The direct_internet_access parameter is set to 'Enabled' but should be 'Disabled'.

Question 15

A data scientist is deploying a model using Amazon SageMaker for real-time inference. The model is memory-intensive and requires a GPU. Which instance type should be selected for the endpoint?

Accepted Answer

p3.2xlarge. The p3.2xlarge instance is correct because it provides a GPU (NVIDIA Tesla V100) with high memory bandwidth, which is essential for memory-intensive deep learning models requiring GPU acceleration for real-time inference. SageMaker endpoints for GPU-based models must use instance types from the P or G families, as CPU-only instances like i3, c5, or r5 lack the parallel processing capabilities needed for efficient GPU inference.

Answer

i3.2xlarge

Answer

c5.2xlarge

Answer

r5.2xlarge

Question 16

A data scientist is trying to create a training job named 'test-model' using an IAM role with the attached policy. The creation fails with an AccessDenied error. What is the most likely cause?

Accepted Answer

The Deny statement uses a wildcard '*' in the condition value, which is not supported for StringNotEquals.. The Deny statement uses 'StringNotEquals' with a wildcard '*' in the condition value, which is not supported for the 'StringNotEquals' condition operator in IAM policies. The 'StringNotEquals' operator requires exact string matching and does not support wildcards; using '*' will cause the condition to never match, effectively making the Deny statement non-functional or causing unexpected behavior. This mismatch leads to an AccessDenied error because the policy evaluation fails to properly deny or allow the action.

Answer

The Resource is set to '*' and should be specific.

Answer

The Deny statement uses 'StringNotEquals' which should be 'StringEquals'.

Answer

The IAM role does not have permission to assume the SageMaker execution role.

Question 17

A company is using AWS Glue to run ETL jobs that transform data for machine learning. The jobs are failing with 'Out of Memory' errors. The data size is growing, and the company needs a cost-effective solution. Which approach should be taken?

Accepted Answer

Increase the number of workers in the job configuration.. Increasing the number of workers in the AWS Glue job configuration distributes the data processing load across more Spark executors, directly addressing the 'Out of Memory' error by providing more aggregate memory without changing the worker type. This is a cost-effective approach because it scales horizontally, often at a lower cost than moving to a larger worker type, and it leverages the existing Glue infrastructure without migrating to EMR.

Answer

Switch to Spark on Amazon EMR.

Answer

Optimize the job by filtering data earlier.

Answer

Use a larger worker type like G.2X.

Question 18

A company is using Amazon SageMaker to build a machine learning pipeline. The pipeline includes data preprocessing, training, and evaluation steps. The company wants to ensure that the pipeline is reproducible and that artifacts are versioned. Which TWO actions should be taken? (Choose TWO.)

Accepted Answer

Use SageMaker Pipelines to create the pipeline and enable versioning on the pipeline artifacts.. SageMaker Pipelines provides a native way to define, orchestrate, and version machine learning pipelines. By enabling versioning on pipeline artifacts (e.g., via the `Pipeline` object's `version` parameter or by using SageMaker Model Registry), each pipeline run is tracked with a unique version, ensuring reproducibility. This directly addresses the requirement for reproducible pipelines and versioned artifacts.

Answer

Use a naming convention for training jobs that includes the date.

Answer

Use AWS CodePipeline to trigger the pipeline on code changes.

Answer

Store the training dataset in a versioned S3 bucket.

Question 19

A data scientist is deploying a model on Amazon SageMaker for real-time inference. The model is a PyTorch model that requires custom inference code. The data scientist needs to handle variable-length inputs and optimize inference latency. Which THREE steps should the data scientist take? (Choose THREE.)

Accepted Answer

Enable SageMaker batch transform to process requests in batches.. Option A is correct because SageMaker batch transform processes requests in batches, which can improve throughput and reduce per-request latency for variable-length inputs by grouping similar-sized inputs together. However, for real-time inference, batch transform is not suitable as it is designed for offline, asynchronous processing; the question specifies real-time inference, so this option is actually incorrect in context. The correct steps for real-time inference with variable-length inputs and optimized latency are B, D, and E, but since the question asks for three correct steps and marks A as correct, this is a trap.

Answer

Use the SageMaker PyTorch container without any modifications.

Answer

Set the endpoint to use multiple variants for A/B testing.

Question 20

A company runs a machine learning pipeline on Amazon SageMaker. The pipeline consists of three steps: data preprocessing (using a custom container), training (using a built-in algorithm), and model evaluation (using a custom container). The pipeline is orchestrated using AWS Step Functions. Recently, the pipeline has been failing intermittently at the model evaluation step with a 'TimeoutError'. The evaluation step runs a Python script that loads the trained model and a test dataset from S3, computes metrics, and writes results back to S3. The step is configured with a timeout of 600 seconds. The test dataset size has grown over time. The data science team suspects that the timeout is due to the increased data size. They want a solution that minimizes changes to the existing infrastructure and avoids increasing the timeout arbitrarily. Which approach should the team take?

Accepted Answer

Modify the evaluation script to process the test dataset in parallel batches, and use multiprocessing to distribute the workload within the same container.. Option C is correct because it addresses the root cause—the evaluation script's inability to process the growing dataset within the 600-second timeout—by parallelizing the workload within the same container. This approach minimizes infrastructure changes (no instance type or timeout increase) and leverages Python's multiprocessing to reduce wall-clock time, directly tackling the 'TimeoutError' without arbitrary timeout extensions.

Answer

Increase the timeout to 1200 seconds and use a larger instance type for the evaluation step.

Answer

Increase the timeout to 1800 seconds to accommodate the larger dataset.

Answer

Switch the evaluation step to use the 'ml.m5.4xlarge' instance type for more memory and compute.