Courseiva

CCNA ML Solution Monitoring, Maintenance, and Security Questions

34 of 109 questions · Page 2/2 · ML Solution Monitoring, Maintenance, and Security · Answers revealed

Practice these questionsDomain overviewAll questions
76
MCQhard

A financial services company must deploy a SageMaker endpoint that processes sensitive customer data. They require that all traffic between the endpoint and the model containers be encrypted, and that the endpoint cannot be accessed from outside a specific VPC. Which combination of settings should they use?

A.Use a private VPC and enable data encryption at rest using KMS
B.Enable inter-container traffic encryption and configure the endpoint with VPC-only mode
C.Enable network isolation mode and inter-container traffic encryption
D.Deploy the endpoint in a private subnet and use a VPC endpoint for SageMaker API
AnswerB

VPC-only mode makes the endpoint only accessible from the VPC, and inter-container traffic encryption encrypts data between containers.

Why this answer

Inter-container traffic encryption ensures that data between the SageMaker endpoint and the model containers is encrypted in transit, typically using TLS. Configuring the endpoint with VPC-only mode restricts all inference traffic to the specified VPC, preventing any access from outside that VPC. This combination directly addresses the requirements for encrypted inter-container traffic and VPC-restricted access.

Exam trap

The trap here is confusing network isolation mode with inter-container traffic encryption and VPC-only mode, as candidates often assume network isolation alone secures all traffic and access, but it does not encrypt inter-container communication or restrict inbound endpoint access to a VPC.

How to eliminate wrong answers

Option A is wrong because enabling data encryption at rest using KMS only protects stored data, not traffic between the endpoint and model containers, and using a private VPC alone does not enforce VPC-only mode for endpoint access. Option C is wrong because network isolation mode prevents the model container from accessing the internet but does not encrypt inter-container traffic nor restrict endpoint access to a specific VPC. Option D is wrong because deploying the endpoint in a private subnet and using a VPC endpoint for the SageMaker API controls API calls but does not encrypt inter-container traffic or enforce VPC-only mode for inference requests.

Practice this question →
77
MCQmedium

A machine learning engineer observes that model performance on a SageMaker endpoint has degraded over the past week. Ground truth labels are available with a 2-day delay. The engineer wants to automatically trigger a retraining pipeline when prediction quality drops below an acceptable threshold. Which approach is most appropriate?

A.Use SageMaker Model Monitor - Model Quality Monitor with ground truth, create a CloudWatch alarm on the metric, and trigger an AWS Lambda function to start retraining
B.Manually evaluate the model weekly and retrain as needed
C.Use SageMaker Model Monitor - Data Quality Monitor to detect drift, then trigger retraining
D.Use SageMaker Clarify to monitor bias drift and trigger retraining
AnswerA

Model Quality Monitor evaluates predictions against ground truth; CloudWatch alarm on quality metric triggers retraining.

Why this answer

SageMaker Model Monitor's Model Quality Monitor is specifically designed to compare model predictions against ground truth labels (available with a 2-day delay) and track metrics like accuracy, precision, recall, or F1 score. You can configure a CloudWatch alarm on a metric such as 'accuracy' dropping below a threshold, which triggers an AWS Lambda function to start the retraining pipeline. This automates the detection of prediction quality degradation and the retraining response without manual intervention.

Exam trap

The trap here is that candidates confuse Data Quality Monitor (which monitors input data drift) with Model Quality Monitor (which monitors prediction accuracy against ground truth), leading them to choose Option C incorrectly.

How to eliminate wrong answers

Option B is wrong because manually evaluating the model weekly is not automated and does not meet the requirement to automatically trigger retraining when prediction quality drops; it introduces latency and human error. Option C is wrong because Data Quality Monitor detects drift in input data distribution (e.g., feature skew), not in prediction quality against ground truth labels, so it cannot directly measure model performance degradation. Option D is wrong because SageMaker Clarify is used for bias detection and explainability, not for monitoring prediction quality or triggering retraining based on performance metrics.

Practice this question →
78
MCQeasy

A team wants to monitor the number of requests and latency of their SageMaker endpoint using a unified dashboard. Which AWS service should they use to create a custom dashboard with these metrics?

A.Amazon CloudWatch Dashboards
B.AWS CloudTrail
C.AWS Config
D.SageMaker Studio
AnswerA

CloudWatch Dashboards can display real-time and historical metrics from SageMaker endpoints in a customizable layout.

Why this answer

Amazon CloudWatch Dashboards allow you to create custom views of metrics from any source, including SageMaker endpoint metrics like Invocations and Latency. SageMaker itself does not provide a dashboard for these metrics.

Practice this question →
79
Multi-Selectmedium

A machine learning team needs to monitor a deployed model for both data drift and concept drift. Which TWO approaches should they implement? (Select TWO.)

Select 2 answers
A.Set up SageMaker Model Monitor for data quality monitoring
B.Use SageMaker Clarify for bias monitoring
C.Configure CloudWatch Logs Insights to query inference logs
D.Set up SageMaker Model Monitor for model quality monitoring
E.Enable SageMaker Debugger during inference
AnswersA, D

Data quality monitoring detects drift in input features.

Why this answer

SageMaker Model Monitor can be configured for data quality (data drift) and model quality (concept drift) monitoring. Data drift monitors input distribution changes, while model quality monitors prediction accuracy against ground truth.

Practice this question →
80
MCQmedium

A data scientist deploys a model and wants to monitor the endpoint's invocation latency. They notice that the CloudWatch metric 'ModelLatency' is high, but 'OverheadLatency' is low. Which statement correctly interprets these metrics?

A.The SageMaker overhead is causing the delay; check endpoint configuration
B.The model inference time is the bottleneck; consider optimizing the model or using a faster instance type
C.The endpoint is overloaded; increase the number of instances
D.The network latency is high; move the endpoint closer to clients
AnswerB

High ModelLatency indicates inference time is the issue.

Why this answer

The 'ModelLatency' metric measures the time taken by the SageMaker model container to process a single request, including inference and any preprocessing/postprocessing within the container. 'OverheadLatency' measures the time spent on SageMaker infrastructure (e.g., network I/O, request queuing, and response handling). When ModelLatency is high and OverheadLatency is low, the bottleneck is clearly the model inference time itself, not the infrastructure overhead. Therefore, optimizing the model (e.g., quantization, pruning) or upgrading to a faster instance type (e.g., GPU vs.

CPU) is the correct remediation.

Exam trap

The trap here is that candidates confuse 'ModelLatency' with overall endpoint latency and assume any high latency is due to infrastructure or scaling issues, when in fact the metric explicitly isolates the model's own inference time from overhead.

How to eliminate wrong answers

Option A is wrong because high ModelLatency with low OverheadLatency indicates the delay is inside the model container, not in SageMaker's infrastructure overhead; checking endpoint configuration would not address the model's own inference time. Option C is wrong because endpoint overload typically manifests as increased OverheadLatency (due to request queuing) or increased Invocations and 5xx errors, not as isolated high ModelLatency with low OverheadLatency. Option D is wrong because network latency is captured within OverheadLatency, not ModelLatency; moving the endpoint closer to clients would reduce OverheadLatency but would not affect the model's inference computation time.

Practice this question →
81
MCQmedium

A company deploys a real-time inference endpoint and wants to be alerted if the number of 4XX errors exceeds 10 per minute over a 5-minute period. Which steps should they take?

A.Create a CloudWatch alarm on the 4XXError metric with a threshold of 10 and an evaluation period of 5 minutes, and configure SNS notification
B.Create a CloudWatch alarm on the Invocations metric and set a threshold
C.Enable endpoint auto-scaling with a target tracking policy
D.Use SageMaker Model Monitor to capture invocations and trigger an SNS topic
AnswerA

Correct metric, threshold, and action.

Why this answer

A CloudWatch alarm on the `4XXError` metric with a threshold of 10 and an evaluation period of 5 minutes directly monitors the rate of HTTP 4XX errors from the SageMaker real-time inference endpoint. When the alarm state transitions to ALARM (i.e., the average 4XX errors per minute exceeds 10 over the 5-minute window), it triggers an SNS notification to alert the team. This is the standard approach for real-time metric-based alerting in AWS.

Exam trap

The trap here is that candidates confuse metric-based alerting (CloudWatch alarms on `4XXError`) with monitoring services (Model Monitor) or scaling mechanisms (auto-scaling), leading them to pick options that address different operational concerns.

How to eliminate wrong answers

Option B is wrong because the `Invocations` metric counts total requests, not 4XX errors, so it cannot detect error rate thresholds. Option C is wrong because endpoint auto-scaling with a target tracking policy adjusts capacity based on a target metric (e.g., Invocations per instance), not on error counts, and it does not generate alerts. Option D is wrong because SageMaker Model Monitor is designed for data quality, bias, and drift detection on captured payloads, not for real-time HTTP error rate monitoring; it cannot directly trigger alerts on 4XX error counts per minute.

Practice this question →
82
MCQmedium

A machine learning engineer notices that the latency of a SageMaker endpoint has increased over time. They need to identify which component (model inference vs. pre/post-processing) contributes most to the latency. Which CloudWatch metrics should they examine?

A.Latency and ModelLatency
B.Invocations and 4XXError
C.5XXError and MemoryUtilization
D.ModelLatency and OverheadLatency
AnswerD

ModelLatency shows inference time inside the container; OverheadLatency shows SageMaker overhead. Comparing them pinpoints the latency source.

Why this answer

SageMaker endpoints emit CloudWatch metrics that break down total latency into model inference time (ModelLatency) and the time spent in pre/post-processing (OverheadLatency). By comparing these two metrics, the engineer can pinpoint whether the bottleneck is in the inference code or in the custom preprocessing/postprocessing logic. Option D directly provides both metrics needed for this root-cause analysis.

Exam trap

The trap here is that candidates confuse the total Latency metric with a breakdown metric, assuming it alone can identify the bottleneck, when in fact only the pair of ModelLatency and OverheadLatency provides the necessary decomposition.

How to eliminate wrong answers

Option A is wrong because Latency is the total end-to-end response time, and ModelLatency alone only covers inference; together they do not isolate the pre/post-processing component. Option B is wrong because Invocations and 4XXError track request count and client-side errors, not latency breakdown. Option C is wrong because 5XXError indicates server-side failures and MemoryUtilization shows resource pressure, but neither metric decomposes latency into inference vs. overhead.

Practice this question →
83
MCQmedium

A company uses SageMaker endpoints for real-time inference. They want to automatically scale the number of instances based on the number of outstanding requests. Which auto-scaling policy type should they choose?

A.Scheduled scaling
B.Step scaling
C.Target tracking scaling
D.Simple scaling
AnswerC

Target tracking automatically adjusts capacity to keep the specified metric at the target value.

Why this answer

Target tracking scaling adjusts the instance count to maintain a target metric value (e.g., average invocation count per instance). Step scaling uses predefined scaling adjustments based on alarm breaches but does not directly track a target. Simple scaling is not recommended for production.

Scheduled scaling is for predictable patterns, not dynamic.

Practice this question →
84
MCQeasy

A machine learning team wants to monitor bias in a deployed model's predictions on an ongoing basis. Which AWS service should they use to schedule bias monitoring jobs and generate reports?

A.Amazon QuickSight with Athona queries
B.AWS CloudTrail for prediction logging
C.SageMaker Model Monitor with data quality monitoring
D.SageMaker Clarify with bias drift monitoring
AnswerD

Clarify offers bias monitoring after deployment, detecting shifts in fairness metrics.

Why this answer

SageMaker Clarify is the correct choice because it provides built-in bias drift monitoring capabilities that can be scheduled to run on a recurring basis. It evaluates predictions against pre-training and post-training bias metrics (e.g., DPL, DI, CDDL) and generates detailed reports, making it the only service designed specifically for ongoing bias monitoring in deployed models.

Exam trap

The trap here is that candidates confuse SageMaker Model Monitor (which handles data and model quality drift) with SageMaker Clarify (which handles bias and explainability drift), leading them to select Option C even though it does not support bias-specific monitoring.

How to eliminate wrong answers

Option A is wrong because Amazon QuickSight with Athena queries is a business intelligence and visualization service, not a bias monitoring tool; it lacks the ability to schedule bias detection jobs or compute bias metrics. Option B is wrong because AWS CloudTrail logs API calls for auditing and governance, not model predictions or bias metrics; it cannot schedule bias monitoring or generate bias reports. Option C is wrong because SageMaker Model Monitor with data quality monitoring focuses on detecting data drift (e.g., feature distribution changes) and model quality degradation (e.g., accuracy), not bias drift; it does not compute fairness metrics like disparate impact or equal opportunity.

Practice this question →
85
MCQmedium

A company uses SageMaker Inference Recommender to select the optimal endpoint configuration. After running the recommender, they receive a recommendation for a specific instance type and initial instance count. What should they do next to optimize costs over time?

A.Use the recommended configuration without changes, as it is already optimal
B.Purchase a Savings Plan for the recommended instance type to reduce hourly cost
C.Set up auto-scaling with a target tracking policy based on the recommended metric
D.Manually adjust the instance count daily based on observed traffic
AnswerC

Auto-scaling adjusts capacity to demand, minimizing cost while meeting performance.

Why this answer

SageMaker Inference Recommender provides a baseline configuration. To optimize costs, they should apply auto-scaling with a target tracking policy based on the recommended metric, such as invocation count or latency.

Practice this question →
86
MCQeasy

A company uses SageMaker Studio for collaborative ML development. The security team requires that all SageMaker Studio notebooks run within a VPC and cannot access the public internet. Which configuration should the administrator set?

A.Enable VPC-only mode for the SageMaker Studio domain
B.Use SageMaker notebook instances instead of Studio
C.Apply an SCP that denies internet access for all IAM users
D.Set the SageMaker Studio domain to use a public subnet with a NAT Gateway
AnswerA

This restricts all traffic to the VPC, no internet access.

Why this answer

Enabling VPC-only mode for the SageMaker Studio domain ensures that all Studio notebooks and apps are launched within the specified VPC and cannot access the public internet. This mode enforces that all network traffic, including internet-bound traffic, is routed through the VPC, and it blocks direct internet access by default, meeting the security team's requirement.

Exam trap

The trap here is that candidates may confuse VPC-only mode with simply using a private subnet, but VPC-only mode is a specific SageMaker Studio domain setting that explicitly blocks all internet access, whereas a private subnet alone could still allow outbound traffic via a NAT Gateway or VPC endpoint.

How to eliminate wrong answers

Option B is wrong because using SageMaker notebook instances instead of Studio does not inherently enforce VPC-only internet restrictions; notebook instances can still be configured with public internet access unless explicitly blocked via VPC settings. Option C is wrong because an SCP that denies internet access for all IAM users is an organization-level policy that does not directly control the network configuration of SageMaker Studio notebooks; it would affect user permissions but not the VPC routing or internet access of the Studio environment. Option D is wrong because setting the SageMaker Studio domain to use a public subnet with a NAT Gateway would actually provide outbound internet access to the notebooks, which violates the requirement that notebooks cannot access the public internet.

Practice this question →
87
MCQhard

A team is using SageMaker Clarify to detect bias drift in a deployed model's predictions. They run weekly bias monitoring jobs. The team wants to be notified when the bias metric for a sensitive feature exceeds a threshold. What is the most efficient method to achieve this?

A.Configure the Clarify monitoring job to send results to an SNS topic directly
B.After each Clarify job, run a custom Lambda that parses the report and publishes a custom CloudWatch metric; create an alarm on that metric
C.Manually review the bias report in SageMaker Studio each week
D.Use SageMaker Model Monitor - Bias Drift Monitor which automatically creates CloudWatch metrics
AnswerB

This approach translates bias metrics into CloudWatch metrics for alarming.

Why this answer

SageMaker Clarify bias monitoring jobs output a JSON report to S3 but do not natively publish CloudWatch metrics. By using a custom Lambda to parse the report and publish a custom CloudWatch metric, you can then create a CloudWatch alarm that triggers notifications when the bias metric exceeds a threshold. This is the most efficient automated method because it leverages CloudWatch's native alarm and notification capabilities without manual intervention.

Exam trap

The trap here is that candidates may confuse SageMaker Clarify's bias monitoring with SageMaker Model Monitor's built-in bias drift capabilities, assuming that Clarify automatically creates CloudWatch metrics or integrates with SNS, when in fact it only outputs to S3 and requires a custom pipeline for metric extraction and alerting.

How to eliminate wrong answers

Option A is wrong because SageMaker Clarify monitoring jobs cannot directly send results to an SNS topic; they output reports to S3, and SNS integration would require an intermediary like Lambda. Option C is wrong because manually reviewing the bias report in SageMaker Studio each week is not efficient and does not provide automated notifications. Option D is wrong because SageMaker Model Monitor's Bias Drift Monitor is a separate feature for monitoring bias drift over time, but it does not automatically create CloudWatch metrics for Clarify's bias metrics; it uses its own built-in metrics and alerts, not the custom threshold-based approach described.

Practice this question →
88
MCQeasy

An organization wants to schedule a retraining pipeline to run every Sunday night. Which AWS service should they use to trigger the pipeline on a schedule?

A.AWS Lambda
B.AWS Step Functions
C.Amazon SQS
D.Amazon EventBridge
AnswerD

EventBridge provides scheduled events using cron or rate expressions.

Why this answer

Amazon EventBridge is the correct choice because it provides a scheduled event source using cron or rate expressions to trigger target services at specified times, such as every Sunday night. It can directly invoke an AWS Step Functions state machine or a Lambda function to start the retraining pipeline, making it the native scheduling service for event-driven workflows in AWS.

Exam trap

The trap here is that candidates often confuse AWS Lambda's ability to be triggered by a schedule with Lambda itself being a scheduling service, but Lambda is only the compute target, not the scheduler — EventBridge is the service that provides the scheduled trigger.

How to eliminate wrong answers

Option A is wrong because AWS Lambda is a compute service that runs code in response to triggers, but it does not natively provide scheduling capabilities; while you can use Lambda with EventBridge, Lambda alone cannot generate scheduled events. Option B is wrong because AWS Step Functions is a workflow orchestration service that coordinates multiple AWS services, but it does not have built-in scheduling; it requires an external trigger like EventBridge to start execution on a schedule. Option C is wrong because Amazon SQS is a message queue service for decoupling application components, not a scheduling service; it cannot trigger pipelines on a schedule and relies on consumers to poll or receive messages.

Practice this question →
89
MCQhard

A data science team uses SageMaker Studio with a VPC-only mode. They need to access a private S3 bucket in the same VPC to read training data. The SageMaker Studio domain is configured with VPC-only mode. Which configuration ensures the Studio notebook can access the S3 bucket without traversing the public internet?

A.Set the S3 bucket as public and restrict access by source IP
B.Configure a NAT Gateway in the public subnet and route Studio traffic through it
C.Use a SageMaker execution role with a policy that allows s3:GetObject from any network
D.Create an S3 Gateway Endpoint in the VPC and attach a bucket policy that allows access from the VPC
AnswerD

An S3 Gateway Endpoint allows private access to S3 without internet. The bucket policy must restrict access to the VPC endpoint.

Why this answer

An S3 Gateway Endpoint provides a private, VPC-only route to S3 without traversing the public internet. By attaching a bucket policy that restricts access to the VPC endpoint, the SageMaker Studio domain (configured with VPC-only mode) can securely read training data from the private S3 bucket using AWS PrivateLink.

Exam trap

The trap here is that candidates often confuse S3 Gateway Endpoints (which are free and route traffic privately within AWS) with VPC Interface Endpoints (which use PrivateLink and incur costs), or mistakenly think IAM policies alone control network path, ignoring the need for a VPC endpoint to avoid public internet traversal.

How to eliminate wrong answers

Option A is wrong because setting the S3 bucket as public exposes it to the internet, violating the VPC-only requirement and creating a security risk; source IP restrictions are not reliable in a VPC-only context. Option B is wrong because a NAT Gateway routes traffic to the public internet, which contradicts the VPC-only mode requirement and would cause the Studio notebook to traverse the public internet to reach S3. Option C is wrong because allowing s3:GetObject from any network in the execution role does not prevent the traffic from going over the public internet; the VPC-only mode requires a private network path, not just an IAM policy.

Practice this question →
90
Multi-Selecthard

A machine learning engineer is setting up model quality monitoring for a binary classification model. They have ground truth labels available in Amazon S3. Which TWO steps are required to configure model quality monitoring? (Choose two.)

Select 2 answers
A.Create a CloudWatch Alarm for accuracy degradation
B.Use Amazon SageMaker Clarify for bias detection
C.Create a schedule for the monitoring job to run at regular intervals
D.Create a baseline for model quality using training data and ground truth labels
E.Enable data capture on the endpoint to collect predictions
AnswersC, D

A schedule defines how often the monitoring job runs to compare predictions against ground truth.

Why this answer

A baseline must be computed from training data and ground truth, and a schedule for monitoring jobs must be defined. The monitoring job then compares production predictions against ground truth.

Practice this question →
91
MCQmedium

A team is monitoring a SageMaker endpoint and notices that the average latency (ModelLatency) is increasing over time, but the number of invocations is steady. They suspect that the model's inference code is becoming slower due to memory leaks. Which metric should they also examine to confirm this hypothesis?

A.Invocations metric
B.OverheadLatency metric
C.5XXError metric
D.MemoryUtilization metric (if custom)
AnswerD

Increasing memory usage suggests a leak, which can cause latency growth.

Why this answer

SageMaker publishes memory utilization metrics if the container emits them via CloudWatch. A memory leak would show increasing memory usage over time, correlating with increasing latency.

Practice this question →
92
Multi-Selecteasy

A company wants to implement cost monitoring and optimization for SageMaker endpoints. Which TWO actions should they take? (Select TWO)

Select 2 answers
A.Manually scale instances based on historical patterns
B.Enable detailed billing reports to track endpoint costs
C.Use SageMaker Savings Plans to get discounted rates in exchange for a commitment
D.Use SageMaker Model Monitor to reduce endpoint costs
E.Right-size endpoints using SageMaker Inference Recommender
AnswersC, E

Savings Plans provide lower costs for consistent usage.

Why this answer

SageMaker Savings Plans offer discounted compute rates (up to 64% off on-demand) in exchange for a 1- or 3-year commitment to a consistent amount of compute usage (measured in dollars per hour), directly reducing endpoint costs. This is a cost optimization mechanism, not a monitoring one, and aligns with the goal of reducing spend on SageMaker endpoints.

Exam trap

The trap here is that candidates confuse cost monitoring (Option B) with cost optimization, or assume that Model Monitor (Option D) directly reduces costs when it only monitors quality, not spend.

Practice this question →
93
MCQeasy

An ML engineer wants to be notified when the average inference latency of a SageMaker endpoint exceeds 500 ms for 2 consecutive evaluation periods. Which AWS service combination should they use?

A.CloudWatch Alarm + SNS
B.SageMaker Model Monitor + SNS
C.EventBridge + Lambda
D.SageMaker Clarify + SNS
AnswerA

CloudWatch Alarm triggers on the Latency metric and publishes to SNS for notification.

Why this answer

CloudWatch collects the 'Latency' metric from SageMaker endpoints. A CloudWatch Alarm can be set on this metric with a threshold of 500 ms for 2 consecutive periods. SNS then sends notification (email, SMS, etc.).

Lambda is for automated remediation, not required for notification only.

Practice this question →
94
Multi-Selectmedium

A company uses SageMaker Model Monitor for feature attribution drift monitoring with SHAP. Which THREE prerequisites must be in place before starting the monitoring schedule? (Select THREE)

Select 3 answers
A.A SageMaker Clarify processing job that computes SHAP values on the captured data
B.Ground truth labels for the inference data
C.Baseline constraints file for data quality
D.Real-time endpoint with data capture enabled
E.A baseline SHAP explainability file from training data
AnswersA, D, E

Clarify runs the SHAP analysis on the current data to compare with baseline.

Why this answer

SageMaker Model Monitor requires a Clarify processing job to compute SHAP values on the captured data as part of the feature attribution drift monitoring setup. This job generates the necessary SHAP explainability values that are compared against the baseline to detect drift.

Exam trap

The trap here is that candidates often confuse the prerequisites for feature attribution drift monitoring with those for data quality monitoring, mistakenly selecting the baseline constraints file (Option C) instead of the baseline SHAP explainability file (Option E).

Practice this question →
95
MCQmedium

A data science team uses Amazon SageMaker Model Monitor to detect data drift in production. They notice that the schema of incoming data (number of features) has changed compared to the training baseline. Which type of monitor is BEST suited to detect this issue?

A.Bias drift monitor
B.Feature attribution drift monitor
C.Data quality monitor
D.Model quality monitor
AnswerC

Data quality monitor checks for schema violations and statistical drift in input features.

Why this answer

The Data quality monitor in SageMaker Model Monitor is specifically designed to detect violations in the input data schema, such as changes in the number of features, feature types, or missing values, by comparing incoming data against a baseline computed from the training dataset. Since the issue is a structural change in the schema (number of features), the Data quality monitor is the correct choice.

Exam trap

The trap here is that candidates confuse 'data drift' (distribution shift) with 'schema change' and incorrectly choose Feature attribution drift monitor, thinking it covers all input changes, but it only tracks importance shifts, not structural feature count violations.

How to eliminate wrong answers

Option A is wrong because Bias drift monitor focuses on detecting bias in model predictions (e.g., demographic parity) and does not monitor input schema changes. Option B is wrong because Feature attribution drift monitor (SHAP-based) tracks changes in feature importance over time, not the number or presence of features. Option D is wrong because Model quality monitor evaluates degradation in prediction accuracy (e.g., AUC, F1) against ground truth, not input data structure.

Practice this question →
96
Multi-Selectmedium

A company is deploying a SageMaker real-time endpoint and needs to monitor inference latency. Which THREE metrics are available from SageMaker for this purpose? (Choose THREE.)

Select 3 answers
A.OverheadLatency
B.Invocations
C.ModelLatency
D.MemoryUtilization
E.Latency
AnswersA, C, E

Time spent on SageMaker infrastructure overhead.

Why this answer

OverheadLatency is a SageMaker metric that measures the time taken by the SageMaker infrastructure to process the request before and after invoking the model, including network overhead and framework overhead. It is one of the three metrics specifically designed to monitor inference latency for real-time endpoints.

Exam trap

The trap here is that candidates often confuse Invocations (a request count metric) or MemoryUtilization (a resource utilization metric) with latency metrics, but SageMaker specifically provides three distinct latency-focused metrics: Latency, ModelLatency, and OverheadLatency.

Practice this question →
97
MCQmedium

A company uses SageMaker Model Monitor to detect bias drift in their real-time inference endpoint. They have collected ground truth labels and want to monitor for bias across different demographic groups. Which type of monitoring should they configure?

A.SageMaker Model Monitor – Feature Attribution Drift Monitoring
B.SageMaker Clarify – Bias Drift Monitoring
C.SageMaker Model Monitor – Model Quality Monitoring
D.SageMaker Model Monitor – Data Quality Monitoring
AnswerB

Clarify's bias drift monitoring uses ground truth labels to compute bias metrics for demographic groups.

Why this answer

SageMaker Clarify offers post-deployment bias monitoring that uses ground truth labels to compute bias metrics (e.g., difference in positive outcome rates) over time.

Practice this question →
98
MCQhard

A financial services company is deploying a fraud detection model on SageMaker. To comply with regulations, they must ensure that the model's predictions are not biased against protected groups. They plan to monitor bias drift post-deployment using SageMaker Clarify. Which data inputs are required to configure Clarify's bias drift monitoring?

A.Only the inference data with predictions
B.Only the ground truth labels for recent predictions
C.Only the training data with feature attributions
D.Baseline training data with ground truth labels and inference data with predictions
AnswerD

Clarify bias monitoring requires a baseline dataset (training data with labels) and current inference data (with predictions and ground truth when available) to compute bias metrics over time.

Why this answer

SageMaker Clarify's bias drift monitoring requires a baseline—specifically, the training data with ground truth labels—to establish the original bias metrics, and the inference data with predictions to compute post-deployment bias metrics. By comparing these two datasets, Clarify detects statistically significant shifts in bias over time, which is essential for regulatory compliance in fraud detection models.

Exam trap

The trap here is that candidates often assume only inference data is needed for monitoring, overlooking the critical requirement of a baseline training dataset with ground truth labels to measure drift against.

How to eliminate wrong answers

Option A is wrong because inference data with predictions alone lacks a baseline for comparison, making it impossible to measure drift from the original model behavior. Option B is wrong because ground truth labels for recent predictions, without a baseline training dataset, cannot establish the initial bias metrics needed for drift detection. Option C is wrong because training data with feature attributions, while useful for explainability, does not include the inference data with predictions required to compute post-deployment bias metrics.

Practice this question →
99
MCQeasy

A company wants to reduce costs for a SageMaker real-time endpoint that receives predictable traffic patterns: high during business hours and low at night. The model is a small PyTorch model. Which cost-saving strategy is most suitable?

A.Use a single large instance to handle peak load
B.Use a multi-model endpoint with multiple models
C.Configure auto-scaling with a scheduled scaling policy to add instances during business hours and reduce at night
D.Switch to batch transform jobs and run nightly
AnswerC

Matches capacity to predictable demand, minimizing cost.

Why this answer

Auto-scaling with a schedule can adjust instance count based on time, matching capacity to demand. This is more efficient than manual scaling or using a larger instance.

Practice this question →
100
MCQmedium

A team has deployed a real-time inference endpoint and wants to automatically scale based on CPU utilization. Which scaling policy type should they use with Application Auto Scaling for SageMaker endpoints?

A.Target tracking scaling
B.Step scaling
C.Predictive scaling
D.Simple scaling
AnswerA

Target tracking scaling automatically maintains a target metric value, such as average CPU utilization.

Why this answer

Target tracking scaling adjusts the number of instances based on a target metric value (e.g., CPU utilization at 50%). Step scaling uses step adjustments, and simple scaling is deprecated. Predictive scaling is not supported for SageMaker endpoints.

Practice this question →
101
Multi-Selectmedium

A company uses SageMaker Model Monitor to detect data drift. They want to receive alerts when drift is detected and automatically trigger a retraining pipeline. Which TWO steps should they implement? (Select TWO.)

Select 2 answers
A.Configure Model Monitor to directly invoke a SageMaker Pipeline when drift is detected
B.Configure a SageMaker Processing job to run periodically and check drift
C.Set up an SNS subscription that triggers a Lambda function to start the SageMaker Pipeline
D.Create a CloudWatch Alarm on the data quality violation metric that publishes to an SNS topic
E.Create an EventBridge rule that triggers on Model Monitor drift events to start the pipeline
AnswersC, D

Lambda function subscribed to SNS can start the pipeline programmatically.

Why this answer

Amazon SNS can be used to publish a notification when Model Monitor detects data drift, and a Lambda function subscribed to that SNS topic can invoke the SageMaker Pipeline to trigger retraining. This decouples the monitoring from the pipeline execution, allowing for flexible, event-driven automation. Option D is correct because Model Monitor emits CloudWatch metrics for data quality violations, and you can create a CloudWatch Alarm on those metrics to publish to an SNS topic, which can then trigger a retraining pipeline via Lambda or other integrations.

Exam trap

The trap here is that candidates may think Model Monitor can directly trigger pipelines or emit EventBridge events, but in reality it relies on CloudWatch metrics and SNS for downstream automation.

Practice this question →
102
MCQhard

A financial services company deploys a fraud detection model with a SageMaker endpoint. They need to ensure that all data sent to the endpoint is encrypted in transit and at rest, and that the endpoint cannot be accessed from the public internet. Which combination of settings should they use?

A.Use endpoint data encryption with an AWS managed key and enable public endpoint access
B.Enable inter-container traffic encryption and disable VPC-only mode
C.Enable VPC-only mode, inter-container traffic encryption, and use a KMS key for endpoint encryption
D.Deploy the endpoint in a private subnet without SageMaker VPC-only mode
AnswerC

VPC-only isolates the endpoint; inter-container encryption secures traffic; KMS encrypts data at rest.

Why this answer

VPC-only mode ensures the endpoint is private. Inter-container traffic encryption is needed for data in transit between containers in multi-model endpoints. KMS encryption secures data at rest on the instance storage.

Practice this question →
103
MCQhard

A financial services company needs to deploy a SageMaker endpoint that only accepts inference requests from within a specific VPC and denies all public traffic. The endpoint must also encrypt data in transit between containers. How should the endpoint be configured?

A.Deploy the endpoint in a public subnet and restrict security group ingress to the VPC CIDR
B.Enable VPC-only mode for the endpoint and disable public access
C.Use a privateLink endpoint and enable data encryption at rest
D.Configure the endpoint with network isolation mode and enable inter-container traffic encryption
AnswerD

Network isolation blocks public internet access; inter-container traffic encryption secures data in transit between containers.

Why this answer

Enabling network isolation mode ensures the SageMaker endpoint is deployed within a VPC and cannot be accessed from the public internet, satisfying the requirement to deny all public traffic. Additionally, enabling inter-container traffic encryption (using TLS) encrypts data in transit between the containers hosting the model, meeting the encryption requirement. This configuration is specific to SageMaker endpoints and directly addresses both constraints.

Exam trap

The trap here is that candidates confuse 'network isolation mode' with simply deploying in a VPC, or they mistakenly think that a PrivateLink endpoint or security group rules alone can block all public traffic, when in fact network isolation is the only way to ensure the endpoint has no public endpoint URL.

How to eliminate wrong answers

Option A is wrong because deploying the endpoint in a public subnet does not prevent public traffic; security group ingress rules alone cannot block all public access since the endpoint would still have a public endpoint URL. Option B is wrong because 'VPC-only mode' is not a valid SageMaker endpoint configuration; SageMaker endpoints are either publicly accessible or deployed in a VPC with network isolation, but there is no toggle for 'VPC-only mode' that disables public access. Option C is wrong because using a PrivateLink endpoint (AWS PrivateLink) is for accessing the endpoint from other VPCs or on-premises networks, not for restricting public traffic, and enabling data encryption at rest does not address encryption of data in transit between containers.

Practice this question →
104
Multi-Selectmedium

A machine learning team needs to automatically retrain a model when concept drift is detected in the deployed endpoint's predictions. Which TWO steps should they take? (Choose TWO.)

Select 2 answers
A.Schedule retraining with Amazon EventBridge on a fixed schedule
B.Create a CloudWatch alarm on a model quality metric (e.g., accuracy) and trigger a Lambda function to start a retraining job
C.Set up SageMaker Model Monitor - Model Quality Monitor to compute prediction quality metrics against ground truth
D.Configure SageMaker Model Monitor - Data Quality Monitor to detect input drift
E.Use SageMaker Clarify to monitor bias drift
AnswersB, C

Alarm triggers retraining pipeline when quality drops.

Why this answer

Model Quality Monitor compares predictions with ground truth to detect concept drift. When an alarm triggers, a Lambda function can start a retraining pipeline. Data Quality Monitor is for data drift, not concept drift.

Practice this question →
105
MCQeasy

A company wants to reduce costs for a production SageMaker endpoint that has predictable traffic patterns. They have purchased a Savings Plan. What additional step can they take to further optimize costs while maintaining performance?

A.Use SageMaker Inference Recommender to right-size the endpoint
B.Reduce the number of instances to one, regardless of load
C.Switch from real-time to batch inference
D.Disable auto-scaling
AnswerA

Inference Recommender tests different instance types and configurations to find the most cost-effective option for the workload.

Why this answer

SageMaker Inference Recommender provides instance type and configuration recommendations to right-size endpoints, balancing cost and performance. It is the appropriate tool for cost optimization beyond a Savings Plan.

Practice this question →
106
MCQmedium

A company wants to allow a SageMaker model in one AWS account to be accessed by a different AWS account for inference. They need to maintain security and compliance. Which approach meets the requirement?

A.Use AWS PrivateLink to expose the SageMaker endpoint privately and grant access via security groups
B.Attach a resource-based policy to the SageMaker endpoint that grants the other account's IAM role invoke permissions
C.Create an IAM role in the source account and share the role ARN with the target account
D.Share the model artifacts via an S3 bucket with cross-account bucket policies and let the other account deploy independently
AnswerB

Resource-based policies allow cross-account access to the endpoint. The other account's IAM role must have sts:AssumeRole or be allowed by the policy.

Why this answer

Cross-account access can be achieved by using resource-based policies on the SageMaker model or endpoint, combined with appropriate IAM roles in the consuming account.

Practice this question →
107
MCQeasy

A company wants to track the lineage of their ML models, including the training dataset, hyperparameters, and training job used to produce each model version. Which AWS service should they use?

A.SageMaker ML Lineage Tracking
B.Amazon DynamoDB
C.AWS Glue Data Catalog
D.Amazon S3 object tagging
AnswerA

ML Lineage Tracking tracks artifacts, actions, and contexts for full model lineage.

Why this answer

SageMaker ML Lineage Tracking is the correct choice because it is purpose-built to record and query the provenance of ML models, capturing relationships between datasets, training jobs, hyperparameters, and model versions. It creates a directed acyclic graph (DAG) of entities (e.g., artifacts, actions, contexts) that allows you to trace how a specific model version was produced, which directly meets the requirement for lineage tracking.

Exam trap

The trap here is that candidates may confuse general-purpose data storage or cataloging services (like DynamoDB or Glue Data Catalog) with the specialized ML lineage tracking service, overlooking that SageMaker ML Lineage Tracking is the only AWS service designed to model the directed relationships between ML artifacts, actions, and contexts.

How to eliminate wrong answers

Option B (Amazon DynamoDB) is wrong because it is a NoSQL key-value and document database designed for low-latency, scalable data storage, not for tracking ML lineage or modeling the complex relationships between training datasets, hyperparameters, and model versions. Option C (AWS Glue Data Catalog) is wrong because it is a metadata repository for data assets (e.g., tables, schemas, partitions) used in ETL and data cataloging, not for capturing the lineage of ML model training runs or hyperparameters. Option D (Amazon S3 object tagging) is wrong because while tags can label S3 objects with metadata like version or dataset name, they cannot capture the relational graph of lineage (e.g., which training job produced which model from which dataset) and lack query capabilities for tracing provenance across multiple artifacts.

Practice this question →
108
MCQhard

A company deploys a model with SageMaker and wants to monitor for concept drift. They have noticed that the relationship between input features and the target variable has changed, causing model accuracy to degrade. However, the input data distribution remains stable. Which type of drift is this, and what is the most appropriate response strategy?

A.Concept drift; ignore the change as long as input distribution remains stable
B.Data drift; update the baseline statistics and continue monitoring
C.Concept drift; retrain the model with newly collected labeled data
D.Data drift; retrain the model with the latest training data
AnswerC

Concept drift is a change in P(y|x). Retraining with recent labeled data adjusts the model to the new relationship.

Why this answer

This is concept drift because the relationship between input features and the target variable has changed while the input data distribution remains stable. The most appropriate response is to retrain the model with newly collected labeled data that reflects the current relationship, as concept drift requires updating the model's learned mapping from features to labels.

Exam trap

The trap here is that candidates confuse concept drift with data drift, assuming any drift requires updating baseline statistics, when in fact concept drift demands retraining with fresh labeled data to realign the model with the new feature-target relationship.

How to eliminate wrong answers

Option A is wrong because ignoring concept drift will cause continued model accuracy degradation, even if the input distribution is stable; concept drift directly impacts predictive performance. Option B is wrong because this is not data drift (input distribution is stable), and updating baseline statistics would not address the changed feature-target relationship. Option D is wrong because data drift refers to changes in input data distribution, not the feature-target relationship, so retraining with the latest training data under the assumption of data drift is a misdiagnosis.

Practice this question →
109
MCQhard

An organization needs to ensure that all data transmitted between containers in a SageMaker training job is encrypted. In the training job configuration, which setting should they enable?

A.Use a KMS key for data encryption
B.Configure the training job in VPC-only mode
C.Enable inter-container traffic encryption
D.Enable network isolation mode
AnswerC

This encrypts data in transit between containers in the same training job.

Why this answer

SageMaker training jobs support inter-container traffic encryption, which ensures that data transmitted between containers (e.g., distributed training workers) is encrypted in transit. This setting uses TLS to protect the communication channel, meeting the organization's requirement for encrypted data transmission between containers.

Exam trap

The trap here is that candidates often confuse encryption at rest (KMS keys) with encryption in transit, or assume VPC-only mode or network isolation automatically encrypts inter-container traffic, when in fact they only control network boundaries without enabling TLS encryption between containers.

How to eliminate wrong answers

Option A is wrong because using a KMS key for data encryption applies to data at rest (e.g., EBS volumes or S3 buckets), not to data in transit between containers. Option B is wrong because configuring the training job in VPC-only mode controls network access and routing but does not inherently encrypt inter-container traffic; it only restricts traffic to a VPC. Option D is wrong because enabling network isolation mode prevents the training job from accessing the internet but does not encrypt inter-container communication; it focuses on network segmentation, not encryption.

Practice this question →
← PreviousPage 2 of 2 · 109 questions total

Ready to test yourself?

Try a timed practice session using only ML Solution Monitoring, Maintenance, and Security questions.

Start 20-question session