MLS-C01 Machine Learning Implementation and Operations Practice Test 26 — 15 Questions

Question 1

A machine learning engineer is designing an automated ML pipeline for training and deploying models. The pipeline must include data validation, model training, hyperparameter tuning, and model deployment. The engineer wants to use AWS services that integrate well and provide version control. Which THREE services should be combined to achieve this? (Choose THREE.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

AWS Step Functions. Amazon SageMaker (E) is the core service for building, training, tuning, and deploying ML models, providing built-in hyperparameter tuning jobs, model registry versioning, and endpoints for deployment. AWS Step Functions (B) orchestrates the multi-step ML workflow (data validation, training, tuning, deployment) as a state machine, coordinating SageMaker jobs and handling retries and branching logic. AWS CodePipeline (C) provides CI/CD automation and version control integration, triggering and managing the pipeline stages from source commits through deployment. AWS Glue (A) is a serverless ETL service for data cataloging and transformation, not a full ML training/tuning/deployment platform, and Amazon EMR (D) is a big-data processing framework for Hadoop/Spark clusters, which is unnecessary here since SageMaker handles the ML lifecycle.

Answer

AWS Glue

Answer

Amazon EMR

Question 2

A company is using Amazon SageMaker to host a real-time inference endpoint for a natural language processing model. The endpoint is configured with an ml.m5.large instance. After deployment, the company observes that the inference latency is higher than expected, and the endpoint is experiencing CPU utilization near 100% during peak hours. The model is a PyTorch model that uses a transformer architecture. The company wants to reduce latency without increasing cost significantly. Which approach should the company take?

Accepted Answer

Change the endpoint instance type to ml.g4dn.xlarge to use GPU acceleration.. The endpoint is CPU-bound on an ml.m5.large (CPU-only) instance running a transformer model, causing high latency and 100% CPU. Switching to an ml.g4dn.xlarge instance provides GPU acceleration, which is well-suited for transformer inference and can dramatically reduce latency. The cost increase is moderate compared to scaling out many CPU instances, and it addresses the root cause (CPU saturation).

Answer

Configure the endpoint with Auto Scaling to add more instances during peak hours.

Answer

Switch to batch transform for inference.

Answer

Attach an Elastic Inference accelerator to the existing instance.

Question 3

A company is deploying a machine learning model using AWS Lambda for real-time inference. The model is a large ensemble model that takes approximately 500 MB of memory. The Lambda function is configured with 1024 MB of memory and a timeout of 15 seconds. The company observes that the function frequently times out during inference. The company wants to keep using Lambda for its serverless benefits. Which solution should the company implement to reduce inference time?

Accepted Answer

Increase the Lambda function memory to 3008 MB to provide more CPU resources.. Lambda has a maximum memory of 10,240 MB and a maximum timeout of 15 minutes. Increasing memory to 3008 MB gives more CPU power and reduces inference time. Option A is correct. Option B (SageMaker) moves away from serverless, which the company wants to keep. Option C (Step Functions) adds orchestration overhead and does not directly reduce inference time. Option D (ElastiCache) adds latency and cost and does not address the timeout issue.

Answer

Deploy the model on Amazon SageMaker hosting instead of Lambda.

Answer

Use AWS Step Functions to invoke the Lambda function asynchronously.

Answer

Use Amazon ElastiCache to cache model predictions and reduce computation.

Question 4

A company is using Amazon SageMaker Ground Truth to build a training dataset for an image classification model. The company has a large number of unlabeled images stored in Amazon S3. The data science team wants to use a private workforce consisting of internal employees to label the images. The team creates a labeling job with a private workforce. After starting the job, the team notices that the labeling tasks are not being assigned to any workers. The workers have been added to the private workforce and have received their login credentials. What is the MOST likely cause of this issue?

Accepted Answer

The workers have not been added to the work team that is assigned to the labeling job.. In Amazon SageMaker Ground Truth, a private workforce consists of work teams, and workers must be assigned to a work team that is associated with the labeling job. If workers are added to the private workforce but not to the specific work team used by the labeling job, they will not receive tasks. The most likely cause is that the workers are not in the assigned work team.

Answer

The labeling job is configured for a different task type than image classification.

Answer

The S3 bucket containing the images has incorrect permissions, preventing workers from viewing the images.

Answer

The workers do not have the required IAM permissions to access the labeling portal.

Question 5

A company is using Amazon SageMaker to train a deep learning model. The training job uses a script that reads data from Amazon S3 using the SageMaker SDK's `s3_input` method. The training job runs on a single ml.p3.2xlarge instance. The data scientist notices that the GPU utilization is very low during training, often below 20%. The training dataset is large, approximately 50 GB, stored as TFRecord files in S3. What is the MOST likely cause of low GPU utilization?

Accepted Answer

The data loading pipeline is not optimized, causing the GPU to wait for data.. Low GPU utilization (below 20%) with a large dataset (50 GB TFRecord) on a single GPU instance almost always indicates the input pipeline cannot feed data fast enough, so the GPU idles waiting for batches. This is a classic data-loading bottleneck: insufficient prefetching, too few parallel readers, small batch sizes, or reading directly from S3 without the Fast File Mode / Pipe Mode optimizations. Optimizing the tf.data pipeline (interleave, prefetch, parallel map, larger batch) is the fix.

Answer

The training script is using a CPU-only version of TensorFlow.

Answer

The batch size is too large, causing the GPU to run out of memory.

Answer

The instance type does not have enough GPU memory for the model.

Question 6

A company uses Amazon SageMaker to deploy a model for real-time inference. The model is a linear regression model that was trained using the SageMaker built-in Linear Learner algorithm. The endpoint is configured with an ml.m5.large instance. After deployment, the company notices that the endpoint returns incorrect predictions. The training data was normalized, but the inference requests send raw feature values without normalization. What should the company do to fix the issue?

Accepted Answer

Create a SageMaker inference pipeline that includes a preprocessing step to normalize the input data before passing it to the model.. The model was trained on normalized data, so it expects input features with the same normalization statistics (mean and standard deviation) applied during training. Raw feature values at inference time cause a mismatch, leading to incorrect predictions. A SageMaker inference pipeline allows you to include a preprocessing step that normalizes the input data before it reaches the model, ensuring consistency.

Answer

Retrain the model using raw data without normalization.

Answer

Change the endpoint instance type to a GPU instance to handle the raw data.

Answer

Use a batch transform job to preprocess the data before sending it to the endpoint.

Question 7

A company is using Amazon SageMaker to train a model using a custom Docker container. The training script writes model artifacts to the `/opt/ml/model` directory. The training job completes successfully, but the model artifacts are not uploaded to the S3 output path specified in the training job. The company has verified that the SageMaker execution role has the necessary S3 permissions. The Docker container is built using a base image that is not one of the official SageMaker Docker images. What is the MOST likely reason for the failure to upload model artifacts?

Accepted Answer

The custom container does not include the SageMaker training toolkit, which handles artifact uploads.. The SageMaker training toolkit is required in custom containers to handle the automatic upload of model artifacts to S3. Without it, even if the script writes to /opt/ml/model and has correct permissions, SageMaker cannot perform the upload. Option B is correct because the container lacks the toolkit. Option A is incorrect because the entry point is unrelated to the upload failure. Option C is incorrect as the output path formatting would cause a different error. Option D is incorrect because permissions were already verified.

Answer

The training script's entry point is not correctly specified in the container.

Answer

The output path in the training job configuration is incorrectly formatted.

Answer

The SageMaker execution role does not have s3:PutObject permission on the output bucket.

Question 8

A machine learning engineer is deploying a real-time inference service on Amazon SageMaker. The model must serve predictions with minimal latency, and the engineer needs to monitor the endpoint for data quality drift and model quality drift. The engineer also wants to receive automated alerts when drift exceeds a threshold. Which two actions should the engineer take to meet these requirements? (Choose two.)

Accepted Answer

Enable SageMaker Model Monitor on the endpoint with a data quality baseline and a monitoring schedule.. To monitor an endpoint for data and model quality drift, SageMaker Model Monitor is the appropriate service. It requires a baseline and a monitoring schedule to continuously evaluate data captured from the endpoint. The metrics emitted to CloudWatch can be used to create alarms that trigger SNS notifications, providing automated alerts. Together, these two actions satisfy the monitoring and alerting requirements.

Answer

Enable SageMaker Clarify to detect bias in the endpoint's predictions and send alerts via CloudWatch Events.

Answer

Configure the endpoint to use a multi-model server to reduce latency by loading only the required model.

Answer

Use SageMaker Debugger to capture tensors during inference and set built-in rules for drift detection.

Question 9

A company is running a hyperparameter tuning job on Amazon SageMaker to optimize a neural network. The tuning job is using Bayesian optimization and has completed 20 training jobs. The best model so far has an accuracy of 92%. The company wants to reduce the cost of the tuning job without significantly impacting the final model performance. Which action should a machine learning engineer take?

Accepted Answer

Enable early stopping in the hyperparameter tuning job to stop poorly performing training jobs early.. Enabling early stopping in a SageMaker hyperparameter tuning job allows the tuner to automatically stop training jobs that are not improving on the objective metric. This reduces the number of epochs and compute resources spent on unpromising trials, lowering cost. The best-performing jobs continue, so the final model quality is maintained. This is the recommended approach to optimize cost without sacrificing performance.

Answer

Change the objective metric to a less resource-intensive metric like training loss instead of validation accuracy.

Answer

Reduce the maximum number of training jobs for the tuning job to 10.

Answer

Use a smaller instance type for the training jobs to reduce cost per job.

Question 10

A machine learning engineer is deploying a real-time inference endpoint on Amazon SageMaker. The model artifacts are stored in an Amazon S3 bucket that is encrypted with AWS Key Management Service (KMS) using a customer managed key. The SageMaker execution role has permissions to access the S3 bucket but does not have any KMS permissions. When the engineer attempts to create the endpoint, the deployment fails with an error indicating that the model data cannot be downloaded. Which action should the engineer take to resolve this issue?

Accepted Answer

Add a policy to the SageMaker execution role granting kms:Decrypt and kms:GenerateDataKey permissions for the customer managed key.. The failure occurs because the SageMaker execution role lacks permissions to decrypt the model artifacts encrypted with a customer managed KMS key. Although the role has S3 access, KMS decryption is a separate authorization step. Adding kms:Decrypt and kms:GenerateDataKey to the execution role allows SageMaker to access the encrypted objects and deploy the endpoint successfully.

Answer

Configure the S3 bucket to use server-side encryption with Amazon S3 managed keys (SSE-S3) instead of the customer managed key.

Answer

Add a bucket policy to the S3 bucket that allows the SageMaker service principal to perform s3:GetObject on the model artifacts.

Answer

Modify the SageMaker notebook instance's IAM role to include kms:Decrypt permissions for the customer managed key.

Question 11

A machine learning engineer is deploying a model to an Amazon SageMaker endpoint that will receive a sudden spike in traffic. The model is hosted on a single ml.m5.large instance and the engineer wants to ensure the endpoint can automatically scale to handle the increased load without manual intervention. The engineer also wants to minimize latency. Which solution should the engineer implement?

Accepted Answer

Configure an Application Auto Scaling policy for the SageMaker endpoint based on the SageMakerVariantInvocationsPerInstance metric.. Application Auto Scaling for SageMaker endpoints automatically adjusts the number of instances based on a specified metric. Using the SageMakerVariantInvocationsPerInstance metric allows scaling in response to traffic spikes, ensuring sufficient capacity and minimizing latency. This is the recommended and most straightforward solution for automatic scaling of SageMaker endpoints.

Answer

Enable SageMaker Inference Recommender to automatically select the optimal instance type and scaling policy.

Answer

Use an AWS Lambda function to monitor endpoint latency and manually adjust the instance count via the SageMaker API.

Answer

Deploy the model to multiple endpoints and use Amazon Route 53 weighted routing to distribute traffic.

Question 12

A data scientist is training a deep learning model using Amazon SageMaker. The training job uses a large dataset stored in Amazon S3 and runs on multiple GPU instances. The data scientist notices that the training job is slow because the GPUs are underutilized, and the training data loading is a bottleneck. The dataset consists of many small files. Which solution will improve GPU utilization and reduce training time?

Accepted Answer

Use Pipe mode with the RecordIO format and augment the data pipeline with a high-throughput data loader that prefetches batches.. Pipe mode with RecordIO streams data directly from S3, avoiding download delays and reducing per-file overhead. RecordIO concatenates many examples into fewer files, which is ideal for many small files. A prefetching data loader keeps the GPU fed by preparing the next batch while the current one is processed, maximizing utilization and reducing overall training time.

Answer

Use Pipe mode with the TensorFlowRecord format to stream data directly from S3 to the training instances.

Answer

Use File mode with the dataset downloaded to the training instances' Amazon EBS volumes, and increase the volume size.

Answer

Use File mode with a distributed file system like Amazon FSx for Lustre mounted to the training instances.

Question 13

An ML engineer trains a model on Amazon SageMaker using a dataset stored in Amazon S3. The training job completes successfully, but the engineer notices that the model artifacts are not saved to the specified S3 output path. The training job's IAM role has full access to the input S3 bucket but only read access to the output bucket. What is the most likely cause of the missing model artifacts?

Accepted Answer

The training job's IAM role lacks s3:PutObject permission for the output S3 bucket.. SageMaker training jobs need both read access to input data and write access to the output S3 bucket. The role's read-only permission on the output bucket prevents the job from uploading model artifacts. Granting s3:PutObject on the output prefix ensures artifacts are stored. The other options either misstate the permissions or incorrectly attribute the cause to missing configuration or local filesystem issues.

Answer

The training job's IAM role lacks s3:GetObject permission for the input S3 bucket.

Answer

The training container does not have write access to the local filesystem to save the model.

Answer

The SageMaker training job did not specify an output S3 path in the estimator configuration.

Question 14

A machine learning engineer is deploying a real-time fraud detection model on Amazon SageMaker. The model is a PyTorch neural network that requires GPU acceleration. The engineer needs to ensure that the endpoint automatically scales the number of instances based on the number of invocations per instance. The engineer has already created the SageMaker model and endpoint configuration. Which scaling policy should the engineer configure to meet these requirements?

Accepted Answer

Target tracking scaling policy with a predefined metric of SageMakerVariantInvocationsPerInstance.. The correct answer is the target tracking scaling policy using the predefined SageMakerVariantInvocationsPerInstance metric because it directly scales based on the number of invocations per instance, which is exactly what the engineer needs. This policy is simple to configure and automatically adjusts capacity to maintain the target value, ensuring optimal performance and cost efficiency for the real-time fraud detection endpoint.

Answer

Step scaling policy with a CloudWatch alarm on the CPUUtilization metric of the endpoint instances.

Answer

Target tracking scaling policy with a custom metric published to Amazon CloudWatch that measures the number of invocations per instance.

Answer

Scheduled scaling policy that increases the instance count during peak business hours.

Question 15

A data scientist has trained a linear regression model using Amazon SageMaker and wants to deploy it for real-time predictions. The model expects a CSV input with a single record per request. The data scientist needs to create an endpoint that can be invoked with low latency. Which SageMaker deployment option should they use?

Accepted Answer

Create a real-time endpoint using the SageMaker hosting services.. Real-time endpoints provide persistent, low-latency inference for individual requests, making them ideal for the described use case. Batch transform is for offline processing, Serverless Inference may have cold starts, and Lambda is not a SageMaker deployment option. Therefore, creating a real-time endpoint is the correct choice.

Answer

Use SageMaker Serverless Inference to deploy the model.

Answer

Create a batch transform job to process the data.

Answer

Deploy the model to an AWS Lambda function using the SageMaker SDK.