MLS-C01 Machine Learning Implementation and Operations Practice Test 28 — 15 Questions

Question 1

A data scientist is using Amazon SageMaker to train a model. The training job must write model artifacts to an Amazon S3 bucket. The data scientist wants to ensure that the training job can access the S3 bucket securely without embedding long-term AWS credentials in the training container. Which mechanism should the data scientist use to grant the training job access to the S3 bucket?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Attach an IAM role to the SageMaker training job with a policy that allows access to the S3 bucket.. SageMaker training jobs assume an IAM role specified when the job is created. This role provides temporary AWS credentials to the training container, enabling secure access to AWS services like S3. Attaching an IAM role with a least-privilege policy is the standard, secure method. Storing credentials in a Dockerfile or Secrets Manager is insecure or overly complex. Public S3 buckets are unacceptable. Therefore, using an IAM role is the correct approach.

Answer

Configure the S3 bucket to allow public read and write access so the training job can access it without credentials.

Answer

Use AWS Secrets Manager to store AWS credentials and retrieve them in the training script at runtime.

Answer

Store AWS access keys as environment variables in the training container's Dockerfile.

Question 2

A company has deployed a real-time inference endpoint on Amazon SageMaker using a custom container. The endpoint receives a steady stream of requests but occasionally experiences spikes in traffic. The company wants to automatically scale the endpoint based on the number of invocations per instance to handle the load efficiently while minimizing costs. Which scaling policy should the company configure?

Accepted Answer

Target tracking scaling policy with a predefined metric of SageMakerVariantInvocationsPerInstance. The SageMakerVariantInvocationsPerInstance metric measures the average number of invocations per instance, directly indicating the load. A target tracking policy using this metric automatically adjusts instance count to maintain a specified target, handling traffic spikes efficiently. Other options either rely on less direct metrics or are not responsive to dynamic changes, making them less suitable for this use case.

Answer

Target tracking scaling policy with a custom metric of MemoryUtilization

Answer

Scheduled scaling policy to increase instances during peak hours

Answer

Step scaling policy based on the CPUUtilization metric

Question 3

A company has a production SageMaker real-time endpoint serving a fraud detection model. The model was trained on transaction data from several months ago, and the fraud patterns have shifted. The team wants to detect when the live request data diverges from the training data distribution so they can trigger retraining, without building custom monitoring infrastructure. Which solution should they implement?

Accepted Answer

Enable SageMaker Model Monitor with a data quality baseline created from the training dataset, and configure a monitoring schedule with CloudWatch alerts.. SageMaker Model Monitor is the managed capability for this need: it generates a baseline of statistics and constraints from the training data, then a monitoring schedule analyzes captured endpoint data and reports violations to CloudWatch. Because the analysis and alerting are built in, the team gets distribution drift detection and an alarm path to retraining without writing custom monitoring code.

Answer

Attach a custom CloudWatch Logs subscription filter to the endpoint's container logs and write a Lambda function that computes KL divergence for each feature.

Answer

Create an A/B test between the current model and a retrained model, and use the traffic split results to infer data drift.

Answer

Enable SageMaker Debugger on the endpoint and configure rules to detect distribution changes in incoming requests.

Question 4

A machine learning engineer has trained a model using Amazon SageMaker and wants to deploy it to a real-time endpoint. The model artifacts are stored in Amazon S3. The engineer needs to create a SageMaker model, configure an endpoint configuration, and then create the endpoint. Which sequence of API calls should the engineer use to deploy the model?

Accepted Answer

CreateModel, CreateEndpointConfig, CreateEndpoint. Deploying a model to a SageMaker endpoint involves three steps: creating a model that points to the model artifacts and inference code, creating an endpoint configuration that specifies the instance type and count, and creating the endpoint using that configuration. This sequence ensures that the necessary resources are available before the endpoint is launched, allowing for a successful deployment.

Answer

CreateModel, CreateEndpoint, CreateEndpointConfig

Answer

CreateEndpoint, CreateModel, CreateEndpointConfig

Answer

CreateEndpointConfig, CreateModel, CreateEndpoint

Question 5

A machine learning engineer has deployed a real-time SageMaker endpoint backed by three production variants. The engineer wants to shift 10% of live inference traffic from the current model to a newly trained model that is already registered and hosted on the same endpoint, while keeping the remaining 90% on the current model. The change must take effect immediately without redeploying the endpoint. Which action should the engineer take?

Accepted Answer

Call UpdateEndpointWeightsAndCapacities with the new variant's DesiredWeight set to 0.1 and the current variant's DesiredWeight set to 0.9.. A multi-variant endpoint already hosts both models, so traffic can be rebalanced in place. UpdateEndpointWeightsAndCapacities changes each variant's DesiredWeight immediately, which is exactly the mechanism for gradual canary rollouts and A/B splits. Because the weights are fractions summing to 1, setting the new variant to 0.1 and the incumbent to 0.9 delivers the required 10/90 live split without recreating the endpoint or interrupting existing callers.

Answer

Enable automatic scaling on the new variant and rely on Application Auto Scaling to distribute incoming traffic between variants.

Answer

Call CreateEndpointConfig with new ProductionVariants and then CreateEndpoint, letting SageMaker replace the existing endpoint.

Answer

Modify the endpoint's data capture configuration so requests are mirrored to both variants and compare results offline.

Question 6

A company is deploying a real-time inference endpoint on Amazon SageMaker. They need to ensure the endpoint can automatically scale based on incoming traffic and that the scaling policy targets a specific utilization metric. Which two actions should they take? (Choose two.)

Accepted Answer

Configure an Application Auto Scaling policy for the SageMaker endpoint variant.. To automatically scale a SageMaker endpoint, you must configure Application Auto Scaling and define a scaling policy. Target tracking with the SageMakerVariantInvocationsPerInstance metric is a common and effective policy that adjusts instance count to maintain a target invocation rate. Together, these actions provide dynamic scaling without manual intervention, meeting the company's need.

Answer

Enable Elastic Inference on the endpoint to handle traffic spikes.

Answer

Use the SageMaker UpdateEndpoint API to manually adjust the instance count.

Answer

Create a CloudWatch alarm that triggers an AWS Lambda function to call UpdateEndpoint.

Question 7

A company wants to deploy a trained SageMaker model to production with the ability to shift a small percentage of live traffic to a new model version for testing. The company needs to monitor the new model's performance without affecting the majority of users. Which deployment strategy should be used?

Accepted Answer

SageMaker endpoint with production variants and traffic distribution. Using production variants on a SageMaker endpoint allows you to deploy multiple models and assign traffic weights. By setting a low weight for the new variant, you can test its performance on a small slice of live traffic while the existing variant serves the majority, enabling safe experimentation.

Answer

SageMaker Serverless Inference

Answer

SageMaker Batch Transform with a random split

Answer

SageMaker multi-model endpoint

Question 8

A data scientist is using Amazon SageMaker to train a model. The training job must write model artifacts to an Amazon S3 bucket. The data scientist wants to ensure that the training job can access the S3 bucket securely without embedding long-term AWS credentials in the training container. Which mechanism should the data scientist use to grant the training job access to the S3 bucket?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Change the hyperparameter tuning strategy to Random.. The tuning job is likely using the default Bayesian strategy, which can exploit promising hyperparameters too early and miss better configurations. Switching to a Random strategy ensures a more thorough exploration of the hyperparameter space by sampling values randomly, which can prevent premature convergence and lead to better overall performance. Other options do not directly address the exploration issue.

Answer

Use a smaller learning rate for the model.

Answer

Decrease the value of the MaxNumberOfTrainingJobs parameter.

Answer

Increase the value of the MaxNumberOfTrainingJobs parameter.

Question 9

A machine learning engineer needs to store and version large datasets used for training models in Amazon SageMaker. The datasets are updated frequently, and the engineer wants to track changes and ensure reproducibility of training jobs. Which AWS service should they use?

Accepted Answer

Amazon S3 with versioning enabled. Amazon S3 is the primary storage for SageMaker training data, and enabling versioning allows you to keep multiple variants of a dataset. This makes it possible to reference a specific version in a training job, ensuring reproducibility. Other services either do not store the data or lack the versioning capability needed for large-scale ML datasets.

Answer

Amazon RDS with automated backups

Answer

Amazon EFS with AWS Backup

Answer

AWS Glue Data Catalog

Question 10

A machine learning engineer has a SageMaker training job that reads training data from an Amazon S3 bucket in the same AWS account. The job is running in a VPC and needs to download the data quickly. The engineer has configured the training job to use a private subnet with a NAT gateway. Data transfer is slow. Which action will improve the data transfer speed?

Accepted Answer

Configure the training job to use a VPC endpoint for Amazon S3.. The training job is in a private subnet with a NAT gateway, so S3 traffic goes through the NAT gateway to the internet, causing slow transfer. A VPC endpoint for S3 enables private connectivity within the AWS network, improving speed and reducing cost. Other options do not address the network path bottleneck or are not suitable for SageMaker training data access.

Answer

Enable S3 Transfer Acceleration on the training data bucket.

Answer

Move the training data to an Amazon EFS file system and mount it to the training instances.

Answer

Increase the instance count of the training job.

Question 11

A company is deploying a machine learning model to a SageMaker endpoint. The model requires a GPU for inference and must handle occasional bursts of traffic. The company wants to minimize cost while ensuring that the endpoint scales automatically. Which scaling policy should the company use?

Accepted Answer

Target tracking scaling policy based on the SageMakerVariantInvocationsPerInstance metric.. Target tracking scaling with SageMakerVariantInvocationsPerInstance automatically adjusts the number of instances to maintain a target invocation rate per instance. This handles bursts by scaling out quickly and scales in during low traffic, minimizing cost. Other policies are either less responsive, based on less relevant metrics, or do not provide automatic scaling based on load.

Answer

Use a larger instance type with more GPU memory and disable autoscaling.

Answer

Scheduled scaling policy based on expected traffic patterns.

Answer

Step scaling policy based on the CPUUtilization metric.

Question 12

A company uses Amazon SageMaker to train a recommendation model. The training dataset grows daily and is stored in Amazon S3. The data science team wants to avoid re-downloading the entire dataset for each training job, so they configure SageMaker Pipe mode with an augmented manifest file. Which statement accurately describes how Pipe mode streams data to the training algorithm?

Accepted Answer

Pipe mode streams data directly from Amazon S3 to the training container without downloading the full dataset to the training instance's Amazon EBS volume.. Pipe mode streams data directly from Amazon S3 to the training container, bypassing the need to download the full dataset to the training instance's EBS volume. This reduces startup time and disk space requirements, making it suitable for large or frequently updated datasets. The other options confuse Pipe mode with File mode or misstate its memory and compatibility characteristics.

Answer

Pipe mode streams data from Amazon S3 but requires the training algorithm to read the entire dataset into memory before processing any records.

Answer

Pipe mode downloads the entire dataset to the training instance's Amazon EBS volume before training starts, then streams from local disk.

Answer

Pipe mode streams data from Amazon S3 only when the training job uses the SageMaker built-in algorithms; custom containers must use File mode.

Question 13

A company has a real-time inference endpoint on Amazon SageMaker that uses a custom container. The endpoint is experiencing high latency during peak traffic. The company wants to reduce latency by enabling GPU acceleration and ensuring that the model server can handle multiple concurrent requests efficiently. The custom container currently runs a Flask application that processes one request at a time. Which solution will meet these requirements with the LEAST operational overhead?

Accepted Answer

Migrate the custom container to use NVIDIA Triton Inference Server with dynamic batching and deploy on a GPU instance.. NVIDIA Triton Inference Server is designed for high-performance inference, supporting dynamic batching and concurrent model execution on GPUs. It maximizes GPU utilization and reduces latency by batching requests, which is ideal for the scenario. Other options either require custom code changes, do not leverage GPU efficiently, or scale horizontally without reducing per-request latency.

Answer

Reconfigure the Flask application to use Gunicorn with multiple worker processes and deploy on a GPU instance.

Answer

Use SageMaker's multi-model server to host multiple models on a single endpoint and enable GPU acceleration.

Answer

Increase the number of instances behind the endpoint and enable automatic scaling based on CPU utilization.

Question 14

A machine learning engineer is deploying a real-time inference endpoint using Amazon SageMaker. The model is a TensorFlow model that requires GPU acceleration for low-latency predictions. The engineer must ensure that the endpoint automatically scales based on the number of invocations per instance, and that it scales in a cost-effective manner. The endpoint will receive traffic that varies throughout the day. Which solution should the engineer implement?

Accepted Answer

Configure an Application Auto Scaling policy with a target tracking scaling policy based on the SageMakerVariantInvocationsPerInstance metric.. For dynamic scaling of SageMaker endpoints based on invocation load, Application Auto Scaling with a target tracking policy using the SageMakerVariantInvocationsPerInstance metric is the correct choice. It automatically adjusts instance count to maintain a target invocation rate per instance, ensuring low latency and cost efficiency as traffic varies. Other options either lack GPU support, require manual intervention, or do not scale based on the appropriate metric.

Answer

Use AWS Lambda with provisioned concurrency to serve the TensorFlow model on GPU-enabled instances.

Answer

Deploy the model to an Amazon ECS cluster with GPU instances and configure Service Auto Scaling based on CPU utilization.

Answer

Manually adjust the number of endpoint instances using a scheduled scaling policy based on historical traffic patterns.

Question 15

A machine learning engineer is using SageMaker Pipelines to orchestrate a training workflow. The pipeline includes a processing step that outputs a dataset to S3, followed by a training step that consumes that dataset. The engineer wants to ensure that the training step uses the output from the processing step. How should the engineer configure the training step?

Accepted Answer

Use the ProcessingOutput object from the processing step as the input to the training step.. SageMaker Pipelines allows steps to pass data by referencing the output of a previous step. The ProcessingOutput object contains the S3 URI where the processing step saves its output. By using it as input to the training step, the pipeline automatically resolves the correct location and establishes a dependency. This is the intended and most reliable method.

Answer

Add a condition step between the processing and training steps to check if the processing output exists.

Answer

Configure the training step to use the same instance type as the processing step.

Answer

Manually specify the S3 bucket and prefix that the processing step will use, and hardcode it in the training step's input.