MLA-C01 ML Solution Monitoring, Maintenance and Security Practice Test 7 — 15 Questions

Question 1

Refer to the exhibit. A data scientist reviews the CloudWatch Logs from an Amazon SageMaker real-time endpoint. What is the MOST likely root cause of the NaN output?

Accepted Answer

The input data contains out-of-range values not seen during training, causing the model to output NaN.. Option B is correct. The unusual input value (-9999.0) suggests data drift or out-of-range input that could cause the model to produce NaN. Option A is wrong because there is no memory error. Option C is wrong because no latency issue is indicated. Option D is wrong because the log shows the error during inference, not during model loading.

Answer

The model weights became corrupted due to a disk write error.

Answer

The endpoint is overloaded and returning a default NaN response.

Answer

The model artifact failed to load correctly, resulting in NaN weights.

Question 2

A company uses an Amazon SageMaker endpoint with auto-scaling. They notice that during traffic bursts, new instances take several minutes to become healthy, causing 503 errors. What is the BEST way to reduce the time to serve requests during scaling events?

Accepted Answer

Use a larger instance type so that fewer instances are needed, and the scaling threshold is triggered less often.. Option D is correct because using a larger instance type with more compute resources can handle more requests per instance, reducing the need to scale as aggressively. Option A is wrong because proactive scaling with a schedule can help but doesn't reduce the time to become healthy. Option B is wrong because decreasing cooldown period could cause thrashing. Option C is wrong because increasing maximum instances doesn't speed up each instance's startup.

Answer

Set up a scheduled scaling policy to pre-warm instances before known traffic bursts.

Answer

Decrease the cooldown period for the scaling policy to add instances faster.

Answer

Increase the maximum number of instances to allow more capacity.

Question 3

A team is deploying a model that requires GPU acceleration for inference. They are using an Amazon SageMaker real-time endpoint. The model is a large language model (LLM) that does not fit on a single GPU. Which configuration should they use to minimize latency while fitting the model?

Accepted Answer

Use SageMaker's model parallelism library to shard the model across multiple GPUs in a single instance.. Option C is correct because SageMaker supports model parallelism, allowing a model to be sharded across multiple GPUs in the same instance. Option A is wrong because SageMaker does not support multi-endpoint model parallelism. Option B is wrong because data parallelism is for training, not inference. Option D is wrong because SageMaker Neo is for optimization, not model parallelism across GPUs.

Answer

Use data parallelism with Horovod to distribute inference across GPUs.

Answer

Optimize the model with SageMaker Neo to reduce its size.

Answer

Deploy the model across multiple endpoints and use a load balancer.

Question 4

A company uses Amazon SageMaker Pipelines for automated retraining. The pipeline includes a processing step that runs a Python script. The script uses the boto3 library to call an AWS service, but the calls are being throttled. What is the MOST effective way to address this within the pipeline?

Accepted Answer

Modify the Python script to include retry with exponential backoff when receiving throttling exceptions.. Option A is correct because implementing retry logic with exponential backoff in the script is the standard approach to handle throttling. Option B is wrong because Service Quotas increase is a longer-term solution but not always available. Option C is wrong because using a larger instance does not reduce throttling. Option D is wrong because there is no built-in pipeline step for backoff; it must be in code.

Answer

Increase the instance count for the processing step to distribute the API calls.

Answer

Request a service quota increase for the throttling limit.

Answer

Add a wait step in the pipeline before the processing step.

Question 5

A healthcare company uses Amazon SageMaker to deploy a real-time inference endpoint for a diagnostic model. The endpoint is configured with a single ml.p3.2xlarge instance. The model processes patient data and returns a risk score. Recently, the endpoint has been experiencing intermittent 504 errors along with increased latency. The team uses Amazon CloudWatch to monitor the endpoint's InvocationsPerInstance and ModelLatency metrics. They observe that InvocationsPerInstance is well below the throttling threshold, but ModelLatency shows periodic spikes lasting 5-10 seconds. The endpoint's CPU utilization remains below 60%, but memory utilization occasionally spikes to 90% during those spikes. The team has checked the inference code and found no obvious memory leaks or performance bottlenecks in the custom logic. The model itself is a deep neural network hosted using Apache MXNet. The team suspects that the issue might be related to resource contention or an external dependency. What should the team do FIRST to diagnose and resolve the issue?

Accepted Answer

Enable SageMaker Debugger rules and profiling to monitor memory and CPU utilization at a fine-grained level during inference.. Option B is correct because the symptoms point to a possible memory contention issue, and enabling detailed profiling for memory and CPU can identify the root cause. Option A is wrong because increasing instance size might mask the problem without identifying it. Option C is wrong because request batching can increase memory usage and may worsen the issue. Option D is wrong because Model Monitor is for data drift, not performance diagnostics.

Answer

Implement request batching to increase throughput and reduce the number of inference requests.

Answer

Increase the instance type to a more memory-intensive instance like ml.p3.8xlarge to handle memory spikes.

Answer

Set up SageMaker Model Monitor to track data drift and model quality metrics.

Question 6

A company uses Amazon SageMaker to train and deploy a machine learning model. After deployment, they notice that the model's accuracy drops significantly over time due to changes in the underlying data distribution. Which monitoring solution should they implement to detect this issue automatically?

Accepted Answer

Set up Amazon SageMaker Model Monitor with data quality monitoring.. Option D is correct because Amazon SageMaker Model Monitor can monitor data quality and model quality drift. Option A (CloudWatch Logs) is for logs, not drift detection. Option B (CloudTrail) tracks API calls. Option C (AWS Config) tracks resource configuration.

Answer

Configure AWS Config rules to check the model accuracy metric.

Answer

Use AWS CloudTrail to monitor changes to the model's S3 bucket.

Answer

Enable Amazon CloudWatch Logs on the endpoint and set alarms on inference latency.

Question 7

A team is deploying a real-time inference endpoint in SageMaker. The model requires access to an S3 bucket containing customer data, which is encrypted with SSE-KMS. The team needs to ensure that the endpoint can decrypt the data. Which IAM role configuration is necessary?

Accepted Answer

Add kms:Decrypt permission to the SageMaker execution role for the specific KMS key.. Option A is correct because the SageMaker execution role must have permission to use the KMS key to decrypt the S3 objects. Option B is wrong because the endpoint role needs the decrypt permission, not grant to S3. Option C is insufficient because the role must have kms:Decrypt. Option D is incorrect because SageMaker does not assume a role from S3.

Answer

Add kms:GenerateDataKey permission to the SageMaker execution role.

Answer

Attach a policy to the S3 bucket granting s3:GetObject to the KMS key.

Answer

Configure the endpoint to assume the S3 bucket's IAM role.

Question 8

A data scientist trained a model using SageMaker and wants to automate the retraining process when new data becomes available. Which AWS service is best suited to trigger a SageMaker training job based on an S3 event?

Accepted Answer

Amazon EventBridge with a rule matching S3 object creation.. Option B is correct because Amazon EventBridge can listen to S3 events (e.g., PutObject) and trigger a Lambda function to start a training job. Option A (SQS) is a queue, not a trigger for direct training jobs. Option C (SWF) is a workflow service not typically used for this simple pattern. Option D (Step Functions) can orchestrate but is not directly triggered by S3 events without EventBridge.

Answer

AWS Step Functions with a scheduled trigger.

Answer

Amazon Simple Workflow Service (SWF) decider.

Answer

Amazon Simple Queue Service (SQS) with a polling script.

Question 9

After deploying a model to a SageMaker endpoint, the operations team notices high inference latency. They suspect it is due to insufficient instance capacity. Which first step should they take to diagnose the issue?

Accepted Answer

Review Amazon CloudWatch metrics for the endpoint, such as CPUUtilization and Invocations.. Option C is correct because CloudWatch metrics like Invocations, ModelLatency, and CPUUtilization can help identify if the endpoint is overloaded. Option A (retrain) doesn't address capacity. Option B (CloudTrail) does not provide performance metrics. Option D (SageMaker Debugger) is for training debugging, not inference.

Answer

Check AWS CloudTrail logs for API errors.

Answer

Use Amazon SageMaker Debugger to analyze inference performance.

Answer

Retrain the model with more training data.

Question 10

A company is using a SageMaker notebook instance to develop models. The security team requires that all data in the notebook be encrypted at rest and in transit, and that internet access be restricted. Which configuration meets these requirements?

Accepted Answer

Use a VPC-only notebook instance with a customer-managed KMS key and disable direct internet access.. Option A is correct because a VPC-only notebook with KMS encryption ensures data at rest is encrypted and no internet access. HTTPS is used for in-transit. Option B allows internet access via NAT. Option C does not encrypt at rest. Option D ignores VPC restrictions.

Answer

Use a notebook with internet access enabled but attach a security group that blocks all outbound traffic.

Answer

Use a notebook with a public subnet and a network ACL that denies all inbound traffic.

Answer

Use a VPC-only notebook with default AWS managed key for EBS encryption.

Question 11

A team uses SageMaker Pipelines to automate model retraining. After a successful pipeline run, they want to register the new model version in the SageMaker Model Registry so that it can be reviewed for approval. Which step type should they add to the pipeline?

Accepted Answer

RegisterModelStep. Option B is correct because the RegisterModel step in SageMaker Pipelines registers a model in the Model Registry. Option A (Training) only trains. Option C (Transform) is for batch inference. Option D (Condition) is for branching.

Answer

ConditionStep

Answer

TransformStep

Answer

TrainingStep

Question 12

A financial services company is deploying a model for loan approval. They must ensure that the model's predictions do not show bias against protected groups. They plan to monitor for bias drift after deployment. Which SageMaker feature should they use?

Accepted Answer

SageMaker Clarify with bias drift detection.. Option C is correct because SageMaker Clarify can detect bias and drift in predictions. Option A (Model Monitor) focuses on data quality, not bias. Option B (Debugger) is for debugging training. Option D (Ground Truth) is for labeling.

Answer

SageMaker Model Monitor with data quality monitoring.

Answer

SageMaker Debugger to capture tensors.

Answer

SageMaker Ground Truth for fairness labels.

Question 13

A company is using SageMaker endpoints for inference. To reduce costs, they want to use Automatic Scaling. However, they observe that scaling up takes several minutes, causing latency spikes during traffic bursts. What should they do to mitigate this?

Accepted Answer

Configure the endpoint with a target tracking scaling policy and pre-warm additional instances during expected traffic surges.. Option A is correct because pre-warming the endpoint reduces cold start time. Option B (larger instances) might help but not cost-effective. Option C (optimizing model) reduces computational load but not scaling delay. Option D (scale down slowly) addresses scale-down, not up.

Answer

Optimize the model to reduce inference time.

Answer

Use larger instance types to handle more requests per instance.

Answer

Set the endpoint to scale down slowly to maintain capacity.

Question 14

A company has deployed a model to a SageMaker endpoint. The security team wants to ensure that all traffic between the endpoint and the client application is encrypted and that the endpoint is not accessible from the internet. Which TWO actions should the company take? (Choose TWO.)

Accepted Answer

Configure the SageMaker endpoint to be VPC-only by setting the endpoint's VPC configuration.. Option A and D are correct. A VPC endpoint (PrivateLink) enables private connectivity from clients in the same VPC, and using HTTPS ensures encryption. Option B (public endpoint) is wrong. Option C (AWS KMS) doesn't directly encrypt network traffic. Option E (API Gateway) is unnecessary if clients are in VPC.

Answer

Place the endpoint behind an API Gateway and call it from the client.

Answer

Create the endpoint with a public endpoint and allow only the client's IP address via security group.

Answer

Use AWS KMS to encrypt data in transit between the client and the endpoint.

Question 15

A data science team uses SageMaker Studio to collaborate. They need to restrict access to certain SageMaker Studio applications (e.g., only JupyterLab, no RStudio). Which THREE steps should they take? (Choose THREE.)

Accepted Answer

Attach an IAM policy to users that denies sagemaker:CreateApp for specific app types.. Options A, B, and E are correct. A domain-level policy can restrict apps, a lifecycle configuration can enforce settings at launch, and IAM policies can limit specific applications. Option C (S3 bucket policy) doesn't control Studio apps. Option D (CloudTrail) is for auditing, not restriction.

Answer

Use an S3 bucket policy to prevent access to RStudio application artifacts.

Answer

Enable CloudTrail to log user activity and monitor for prohibited app usage.

MLA-C01 ML Solution Monitoring, Maintenance and Security Practice Test 7 — 15 Questions

Exhibit

MLA-C01 ML Solution Monitoring, Maintenance and Security Practice Test 7 — 15 Questions

Exhibit