MLS-C01 Machine Learning Implementation and Operations Practice Test 16 — 15 Questions

Question 1

A data scientist is training a neural network on a GPU instance in Amazon SageMaker. The training job fails with an 'OutOfMemoryError'. Which action should the data scientist take to resolve this issue?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Reduce the batch size in the training script.. An OutOfMemoryError during GPU training indicates that the GPU's memory is exhausted. Reducing the batch size directly decreases the memory footprint per training step, as fewer samples and their corresponding activations are stored simultaneously. This is the most immediate and effective fix without changing the instance type or training architecture.

Answer

Enable automatic hyperparameter tuning.

Answer

Switch to distributed training across multiple instances.

Answer

Use a smaller instance type with less GPU memory.

Question 2

A machine learning engineer is deploying a model to an Amazon SageMaker endpoint. The model is a PyTorch model that requires a custom inference script. The engineer notices that the endpoint is returning 500 errors after deployment. Which step should the engineer take to debug the issue?

Accepted Answer

View the CloudWatch Logs for the endpoint.. When a SageMaker endpoint returns 500 errors, it typically indicates an application-level failure in the inference container, such as an unhandled exception in the custom inference script. CloudWatch Logs capture the stdout and stderr of the container, including Python tracebacks, which directly reveal the root cause. Option D is correct because viewing these logs is the first and most effective step to diagnose the specific error.

Answer

Redeploy the endpoint with a different instance type.

Answer

Check the CloudWatch metrics for the endpoint.

Answer

Modify the inference script and update the endpoint.

Question 3

A data scientist is using AWS Glue to prepare training data. The job reads from an S3 bucket, performs transformations, and writes to another S3 bucket. The job is failing due to insufficient memory. Which solution should the data scientist use to fix this?

Accepted Answer

Increase the number of DPU (Data Processing Units) for the job.. The job is failing due to insufficient memory, which is a resource constraint. Increasing the number of DPU (Data Processing Units) allocates more memory and compute capacity to the AWS Glue job, directly addressing the out-of-memory error. This is the standard approach to scale Glue ETL jobs when they hit memory limits.

Answer

Use AWS Glue's job bookmark feature.

Answer

Use Amazon Athena instead of AWS Glue.

Answer

Use a columnar file format like Parquet.

Question 4

A company is building a fraud detection model. The dataset is highly imbalanced (99% legitimate, 1% fraud). The data scientist trains a model using Amazon SageMaker's built-in XGBoost algorithm. The model achieves 99% accuracy but only catches 10% of fraud cases. Which technique should the data scientist apply to improve recall for the minority class?

Accepted Answer

Set the scale_pos_weight hyperparameter in XGBoost.. Setting the scale_pos_weight hyperparameter in XGBoost adjusts the weight of the positive (minority) class during training, effectively penalizing misclassifications of fraud cases more heavily. This directly addresses the class imbalance by forcing the model to focus on the minority class, which improves recall without altering the dataset distribution. The current 99% accuracy with only 10% fraud recall indicates the model is biased toward the majority class, and scale_pos_weight is the most direct and efficient fix within XGBoost.

Answer

Use random under-sampling of the majority class.

Answer

Use mean squared error as the objective function.

Answer

Use SMOTE to oversample the minority class.

Question 5

A data scientist needs to version control datasets used for machine learning experiments. Which AWS service should the data scientist use?

Accepted Answer

Amazon S3 with versioning enabled. Amazon S3 with versioning enabled is the correct choice because it provides a simple, scalable, and cost-effective way to version control datasets. S3 versioning preserves every object version, allowing you to retrieve, restore, or compare previous dataset states, which is essential for reproducibility in ML experiments. This directly meets the requirement for dataset version control without additional overhead.

Answer

AWS Lake Formation

Answer

Amazon SageMaker Feature Store

Answer

Amazon SageMaker Model Registry

Question 6

A company is using Amazon SageMaker to train a model on a large dataset stored in S3. The training job is taking a long time due to slow data loading. Which action can the data scientist take to reduce data loading time?

Accepted Answer

Use Pipe mode to stream data from S3.. Pipe mode streams data directly from S3 into the training algorithm without first downloading it to the local storage, eliminating the bottleneck of disk I/O and reducing data loading time. This is especially effective for large datasets where the time to copy data to EBS (File mode) dominates the training job duration.

Answer

Use File mode and copy data to Amazon EBS.

Answer

Use a larger instance type with more memory.

Answer

Enable data augmentation during training.

Question 7

A data scientist is using Amazon SageMaker to train a model. The training data is stored in an S3 bucket encrypted with AWS KMS. Which TWO actions are necessary to allow SageMaker to access the data?

Accepted Answer

Ensure the SageMaker execution role has s3:GetObject permission.. A is correct because the SageMaker execution role must have the s3:GetObject permission to read objects from the S3 bucket. Without this IAM permission, SageMaker cannot retrieve the training data, even if the bucket is otherwise accessible.

Answer

Enable S3 Transfer Acceleration.

Answer

Set up a VPC endpoint for S3.

Answer

Add a bucket policy allowing SageMaker access.

Question 8

A company is deploying a machine learning model using Amazon SageMaker. The model needs to be updated frequently. Which THREE practices should the company implement for model versioning and deployment?

Accepted Answer

Use AWS CodePipeline to automate the training and deployment pipeline.. Option A is correct because AWS CodePipeline can orchestrate the CI/CD workflow that automates retraining and redeployment of frequently updated SageMaker models, ensuring repeatable and auditable releases. Option B is correct because the SageMaker Model Registry catalogs model versions with metadata and approval status, providing the versioning backbone needed to track and promote models. Option E is correct because canary deployments on SageMaker endpoints let you shift a small percentage of traffic to a new model version, validate it in production, and roll back quickly if issues arise. Option C is not appropriate because manually updating endpoint configurations is error-prone, unscalable, and defeats automation for frequent updates. Option D is not appropriate because storing datasets in a single S3 bucket without versioning removes traceability between training data and model versions, undermining reproducibility.

Answer

Manually update the endpoint configuration each time.

Answer

Store all training datasets in a single S3 bucket without versioning.

Question 9

A data scientist is reviewing the training logs from a SageMaker training job. The model's loss decreases steadily and accuracy increases. However, when the model is evaluated on a holdout test set, the accuracy is only 0.65. Which issue does this behavior suggest?

Accepted Answer

The model is overfitting to the training data.. The model shows strong performance on the training set (loss decreases, accuracy increases) but poor generalization to the holdout test set (accuracy 0.65). This classic symptom indicates overfitting, where the model has memorized the training data rather than learning generalizable patterns. In SageMaker, this can occur when the model is too complex relative to the amount of training data, or when regularization techniques like dropout or L2 weight decay are insufficient.

Answer

The learning rate is too high.

Answer

The model is underfitting the training data.

Answer

The training data is corrupted.

Question 10

A data scientist is using Amazon SageMaker to train a custom image classification model using a PyTorch script. The training job runs successfully but the model accuracy is lower than expected. The scientist wants to debug the training process by inspecting gradients and layer outputs. Which SageMaker feature should be used to capture this internal state during training?

Accepted Answer

Use SageMaker Debugger to capture tensors and gradients.. SageMaker Debugger is specifically designed to capture internal model state such as tensors, gradients, and weights during training. It allows you to set rules to monitor for issues like vanishing gradients or overfitting, and to save these tensors for later analysis. This directly addresses the need to inspect gradients and layer outputs to diagnose low accuracy.

Answer

Use SageMaker Experiments to track hyperparameters and metrics.

Answer

Use SageMaker Profiler to profile system bottlenecks.

Answer

Use SageMaker Model Monitor to detect data drift.

Question 11

A company uses Amazon SageMaker to train a model using a custom Docker container. The training job fails with an error: "Unable to write to /opt/ml/output/data". The data scientist checks the container and finds that the /opt/ml directory is not writable. What is the MOST likely cause?

Accepted Answer

The container runs as a non-root user that lacks write permissions to /opt/ml.. The error 'Unable to write to /opt/ml/output/data' indicates a permission issue. By default, SageMaker training containers run as a non-root user (uid 1000) for security reasons. If the Docker image is built with /opt/ml owned by root and without world-writable permissions, the non-root user cannot write to that directory, causing the failure.

Answer

The Docker image is built from a base image that does not have the required libraries.

Answer

The SageMaker training job is configured with insufficient memory.

Answer

The training script is not copying the model to /opt/ml/model.

Question 12

An ML team wants to perform batch inference on a large dataset stored in Amazon S3 using a pre-trained model. The team needs to process the data in parallel across multiple instances to reduce processing time. Which approach should they use?

Accepted Answer

Use SageMaker Batch Transform with multiple instances.. SageMaker Batch Transform is designed specifically for batch inference on large datasets stored in Amazon S3. It automatically distributes the data across multiple instances, processes them in parallel, and writes the results back to S3, making it the optimal choice for reducing processing time.

Answer

Use SageMaker Processing to run a custom inference script.

Answer

Use SageMaker Training to run inference as a training job.

Answer

Use SageMaker Ground Truth to process the data.

Question 13

Refer to the exhibit. An ML engineer attaches this IAM policy to a user. The user wants to invoke the SageMaker endpoint my-endpoint from an EC2 instance with public IP 52.1.1.1. What will happen?

Accepted Answer

The invocation is allowed because the source IP is not in the denied ranges.. The IAM policy explicitly allows the `sagemaker:InvokeEndpoint` action, and the `Deny` statement only denies requests from IP addresses in the ranges 10.0.0.0/8 or 192.168.0.0/16. Since the EC2 instance has a public IP of 52.1.1.1, which is not within those denied ranges, the invocation is allowed. The policy does not require the user to be in a VPC or to have endpoint creation permissions for invoking an existing endpoint.

Answer

The invocation fails because the user does not have permission to create an endpoint.

Answer

The invocation is denied because the Deny statement applies to all resources.

Answer

The invocation is denied because the user is not in a VPC.

Question 14

A financial services company is deploying a machine learning model for credit risk assessment. The model must have an inference latency under 200ms and must be able to handle up to 1000 transactions per second (TPS). The company wants to minimize costs. The model is a gradient boosting model implemented in XGBoost. Which SageMaker deployment option should the team choose?

Accepted Answer

Deploy the model on a SageMaker real-time endpoint with multiple instances behind a load balancer.. A SageMaker real-time endpoint with multiple instances behind a load balancer provides the sub-200ms inference latency and 1000 TPS throughput required for synchronous, low-latency predictions. XGBoost models are well-suited for real-time endpoints, and horizontal scaling with multiple instances ensures high availability and cost efficiency by matching capacity to demand.

Answer

Use SageMaker Batch Transform to process transactions in batches.

Answer

Use SageMaker asynchronous inference for queued requests.

Answer

Use SageMaker Serverless Inference for automatic scaling.

Question 15

A company wants to use Amazon SageMaker to host a model that was trained using a custom algorithm. The model artifact is stored in Amazon S3. The company wants to ensure that the endpoint can automatically scale based on the number of incoming requests. Which configuration should the company use?

Accepted Answer

Create a SageMaker real-time endpoint and configure automatic scaling using a target tracking policy.. A SageMaker real-time endpoint with automatic scaling using a target tracking policy allows the endpoint to dynamically adjust the number of instances based on the incoming request load. This configuration is ideal for hosting a custom algorithm model artifact stored in S3, as it provides low-latency inference and can scale out or in based on a target metric like average CPU utilization or request count per instance.

Answer

Create a SageMaker multi-model endpoint with automatic scaling.

Answer

Use SageMaker Serverless Inference which scales automatically.

Answer

Use SageMaker Batch Transform with a scheduled job.