ML Solution Monitoring, Maintenance, and Security

A machine learning engineer is monitoring a deployed model for data drift. The input features are a mix of categorical and numerical columns. The baseline is from the training data. Which SageMaker Model Monitor feature should they enable to detect changes in the distribution of each feature over time?

A team receives alerts that their SageMaker endpoint latency has increased significantly. They check CloudWatch metrics and see Invocations rising, but ModelLatency remains stable. Which metric should they investigate to find the source of the increased latency?

A data scientist wants to track the lineage of models, datasets, and training jobs in SageMaker. Which SageMaker feature should they use to capture these relationships as artifacts and actions?

A financial services company must deploy a SageMaker endpoint that processes sensitive customer data. They require that all traffic between the endpoint and the model containers be encrypted, and that the endpoint cannot be accessed from outside a specific VPC. Which combination of settings should they use?

A team has deployed a real-time inference endpoint and wants to automatically scale based on CPU utilization. Which scaling policy type should they use with Application Auto Scaling for SageMaker endpoints?

A company deploys a model for fraud detection. They need to monitor for bias after deployment, specifically whether the model's false positive rate changes across demographic groups over time. Which SageMaker feature should they use?

A company wants to reduce costs for a production SageMaker endpoint that has predictable traffic patterns. They have purchased a Savings Plan. What additional step can they take to further optimize costs while maintaining performance?

A machine learning engineer is setting up a retraining pipeline that triggers when concept drift is detected. They plan to use CloudWatch Alarms to monitor the model's accuracy metric. When drift is detected, they want to automatically start a SageMaker training job. Which architecture should they use?

A company needs to give a data science team in another AWS account access to deploy a model from a shared model registry. Which approach should they use to grant cross-account access?

A company uses SageMaker Model Monitor for data quality. They notice that monitoring jobs are failing intermittently with constraint violations. Upon review, they see that some features have different data types in production compared to the baseline (e.g., string instead of integer). Which type of drift is this?

A team wants to monitor the number of requests and latency of their SageMaker endpoint using a unified dashboard. Which AWS service should they use to create a custom dashboard with these metrics?

A company uses SageMaker JumpStart to deploy a foundation model for a summarization task. They want to minimize costs while still meeting a latency requirement of under 2 seconds. Which option should they consider?

A company wants to automatically trigger model retraining when SageMaker Model Monitor detects data drift. Which TWO services should they integrate to achieve this automation? (Choose two.)

A company wants to secure data in transit between the client and SageMaker endpoint, and between containers in the same endpoint. Which THREE configurations should they apply? (Choose three.)

A machine learning engineer is setting up model quality monitoring for a binary classification model. They have ground truth labels available in Amazon S3. Which TWO steps are required to configure model quality monitoring? (Choose two.)

A data science team uses Amazon SageMaker Model Monitor to detect data drift in production. They notice that the schema of incoming data (number of features) has changed compared to the training baseline. Which type of monitor is BEST suited to detect this issue?

An ML engineer wants to be notified when the average inference latency of a SageMaker endpoint exceeds 500 ms for 2 consecutive evaluation periods. Which AWS service combination should they use?

A company deploys a model for credit risk assessment on a SageMaker endpoint. To comply with internal policies, they must ensure that the endpoint only allows inference requests from within a specific VPC and that the data is encrypted at rest. Which configuration meets these requirements?

A machine learning team uses SageMaker Pipelines and wants to automatically retrain a model when data drift is detected. They have set up Model Monitor to publish drift violations to CloudWatch. Which approach provides a COMPLETE serverless retraining pipeline triggered by drift detection?

A financial institution uses SageMaker to train and deploy models. They need to track every experiment, model version, and deployment step for audit purposes. Which SageMaker feature should they use to capture the full lineage of artifacts, actions, and contexts?

A company notices that the prediction distribution of their deployed model has shifted significantly from the training data distribution, but the input data distribution remains unchanged. Which type of drift is occurring, and what is the MOST likely cause?

An ML team wants to monitor the cost of their SageMaker endpoints. They have observed that some endpoints are underutilized. Which AWS offering can help them reduce costs by committing to a consistent amount of usage in exchange for a lower price?

A company deploys a model in a different AWS account for production. They want to allow the production account to invoke the model endpoint from a SageMaker notebook in the same account, while keeping the model in the original account. Which configuration is required?

A team wants to use SageMaker Clarify to monitor bias in their production model predictions. They have configured a bias drift monitor. What does SageMaker Clarify compare to detect bias drift?

A machine learning engineer wants to deploy a pre-trained foundation model for text summarization using SageMaker JumpStart. Which of the following is a primary cost consideration when deploying such a model?

A company uses SageMaker endpoints for real-time inference. They want to automatically scale the number of instances based on the number of outstanding requests. Which auto-scaling policy type should they choose?

An organization needs to ensure that all data transmitted between containers in a SageMaker training job is encrypted. In the training job configuration, which setting should they enable?

A company uses SageMaker Model Monitor to detect data drift. They want to receive alerts when drift is detected and automatically trigger a retraining pipeline. Which TWO steps should they implement? (Select TWO.)

A financial services company uses SageMaker Studio. They require that all Studio traffic remains within the corporate network and that user notebooks cannot access the internet. Which TWO configurations should they implement? (Select TWO.)

An ML team uses SageMaker to deploy a model for real-time inference. They want to monitor and improve cost efficiency. Which THREE actions should they take? (Select THREE.)

A machine learning team deploys a model for loan approval. They want to monitor data drift on the real-time endpoint using SageMaker Model Monitor. Which set of actions should they take to set up data quality monitoring?

A financial services company deploys a fraud detection model with a SageMaker endpoint. They need to ensure that all data sent to the endpoint is encrypted in transit and at rest, and that the endpoint cannot be accessed from the public internet. Which combination of settings should they use?

A data scientist notices that a production model's accuracy has degraded over the past week. The training data distribution remains unchanged, but the relationship between features and the target has shifted. Which type of drift is occurring, and which monitoring approach should be used?

A team wants to automatically retrain a model whenever data drift is detected on their SageMaker endpoint. Which AWS service should they use to invoke a retraining pipeline in response to a CloudWatch Alarm?

A company uses SageMaker Model Monitor to track feature attribution drift with SHAP. They notice that the SHAP values have changed significantly for a feature, while the model performance remains stable. What is the MOST likely interpretation?

A machine learning engineer deploys a multi-model endpoint using SageMaker. They need to track which model version was used for each inference request for compliance purposes. Which service should they integrate to capture this lineage?

A company wants to reduce costs for a real-time inference endpoint that experiences predictable traffic spikes during business hours and low traffic at night. Which auto-scaling policy is MOST cost-effective while maintaining performance?

A team trains a model using SageMaker and wants to ensure that the training job cannot access the internet, but needs to access a private S3 bucket in the same VPC. Which configuration should they use?

A data scientist deploys a model and wants to monitor the endpoint's invocation latency. They notice that the CloudWatch metric 'ModelLatency' is high, but 'OverheadLatency' is low. Which statement correctly interprets these metrics?

An organization uses SageMaker Studio and needs to restrict Studio's internet access while allowing users to install custom packages from a private PyPI mirror hosted in a VPC. Which networking configuration should they use?

A machine learning team wants to monitor bias in a deployed model's predictions on an ongoing basis. Which AWS service should they use to schedule bias monitoring jobs and generate reports?

A company uses SageMaker Inference Recommender to select the optimal endpoint configuration. After running the recommender, they receive a recommendation for a specific instance type and initial instance count. What should they do next to optimize costs over time?

A machine learning team wants to detect concept drift in a production model. Which TWO actions should they take? (Choose TWO)

A company needs to secure a SageMaker real-time endpoint such that only authorized applications within a VPC can invoke the model, and all data in transit is encrypted. Which THREE configuration steps should they implement? (Choose THREE)

A data science team wants to automate the retraining of a model when data drift is detected. Which TWO AWS services should they use in combination to achieve this? (Choose TWO)

A machine learning engineer wants to monitor a deployed model for data drift. Which SageMaker feature should they use to automatically detect drift in the input data distribution compared to the training data baseline?

A team has deployed a real-time inference endpoint. They need to monitor the latency experienced by end users, including network overhead. Which CloudWatch metric should they use?

A company wants to automatically trigger a retraining pipeline when concept drift is detected in their deployed model. Which combination of services should they use?

A machine learning team needs to ensure that all model training and inference jobs within SageMaker Studio run in a private network without internet access. The team also requires that inter-container traffic within the same training job be encrypted. Which configurations should they combine?

A data scientist uses SageMaker Model Monitor to track feature attribution drift. Which technique does SageMaker Model Monitor use to compute feature attributions?

A company wants to reduce costs for a SageMaker real-time endpoint that has variable traffic. Which feature allows the endpoint to automatically adjust instance count based on demand?

A hospital deploys a model to predict patient readmission risk. To comply with regulations, they must ensure that the model's predictions do not show bias against any demographic group over time. Which service should they use for ongoing monitoring?

A team uses SageMaker ML Lineage Tracking to capture the metadata of their ML workflow. They want to query the lineage to see which model version was trained from a specific dataset. Which Lineage Tracking entity represents the dataset?

A security team requires that all data used by a SageMaker training job be encrypted at rest using a customer-managed KMS key. The data is stored in an S3 bucket that is already encrypted with SSE-KMS. What additional configuration is needed on the SageMaker training job?

An organization wants to schedule a retraining pipeline to run every Sunday night. Which AWS service should they use to trigger the pipeline on a schedule?

A company uses SageMaker Studio and wants to restrict studio user access to only the VPC. They also need to encrypt the data exchanged between the Studio app and the kernel gateway. Which configuration should they apply?

A team monitors a production endpoint and notices a sudden increase in 5XXError count. Which of the following is the most likely cause?

A machine learning team needs to monitor a deployed model for both data drift and concept drift. Which TWO approaches should they implement? (Select TWO.)

A company wants to ensure that a SageMaker endpoint can only be invoked from within a specific VPC and that the data in transit is encrypted. Which THREE steps should they take? (Select THREE.)

A data science team uses SageMaker to train models. They need to track the lineage of each model, including the dataset used, training job, and hyperparameters. Which TWO SageMaker features can they use together? (Select TWO.)

An ML engineer monitors a SageMaker endpoint for data drift. They set up SageMaker Model Monitor to compare inference data against a baseline created from the training dataset. The monitoring schedule runs daily and reports violations. Which monitoring type should be configured to detect if the distribution of a numerical feature in real-time inference data differs significantly from the training distribution?

A company wants to automate remediation when a SageMaker endpoint's latency exceeds a threshold for more than 5 minutes. The team needs to be notified and a Lambda function should be invoked to scale up the endpoint. Which combination of services should be used?

A machine learning team trains a model in SageMaker and wants to track every step — from dataset version to hyperparameters to final model artifact — for reproducibility and audit compliance. Which SageMaker feature should they use?

A data science team uses SageMaker Studio with a VPC-only mode. They need to access a private S3 bucket in the same VPC to read training data. The SageMaker Studio domain is configured with VPC-only mode. Which configuration ensures the Studio notebook can access the S3 bucket without traversing the public internet?

A company deploys a model for fraud detection. They want to monitor if the model's predictions become less accurate over time due to changes in the underlying data distribution, but they do not have immediate access to ground truth labels. Which type of drift should they monitor as a proxy?

An organization needs to ensure that all data used for inference on a SageMaker endpoint is encrypted at rest. The endpoint uses a SageMaker-provided container. Which configuration should be applied?

A company deploys a real-time inference endpoint with auto-scaling using a target tracking policy based on average Invocations per instance. They notice that during a traffic spike, the endpoint scales out too late, causing increased latency. They want to scale proactively before the spike. Which strategy should they implement?

A team uses SageMaker Clarify to monitor bias drift on a deployed model. They have defined a baseline with training data and set up a monitoring schedule. After one month, they receive a violation report indicating that the post-training metrics have deviated from the baseline. What does this violation indicate?

An ML engineer needs to monitor the operational health of a SageMaker endpoint, specifically the time taken for the container to process an inference request and the overhead added by SageMaker. Which two CloudWatch metrics should they examine?

A company wants to implement a retraining pipeline that automatically triggers when SageMaker Model Monitor detects data drift. The retraining job should use the latest approved pipeline version in SageMaker Pipelines. Which approach meets these requirements?

A financial services company needs to deploy a SageMaker endpoint that only accepts inference requests from within a specific VPC and denies all public traffic. The endpoint must also encrypt data in transit between containers. How should the endpoint be configured?

A company plans to deploy a large foundation model using SageMaker JumpStart. They are concerned about costs because the model will be used intermittently. Which deployment option is MOST cost-effective for intermittent traffic?

An ML team has deployed a model to a SageMaker real-time endpoint and wants to set up automated monitoring for model quality. Which TWO elements are required to configure SageMaker Model Monitor for model quality? (Select TWO.)

A company uses a SageMaker endpoint for real-time inference. To reduce costs, they want to implement auto-scaling based on the number of invocations per instance. They also need to ensure that scaling actions are recorded for audit. Which TWO steps should they take? (Select TWO.)

A company needs to give a data science team from another AWS account access to deploy models to a SageMaker endpoint in the company's account. The company wants to minimize administrative overhead while ensuring security. Which TWO steps should the company take? (Select TWO.)

A machine learning team deploys a fraud detection model on a SageMaker endpoint. The model's predictions are used in real-time. The team wants to monitor for data drift by comparing incoming data distributions against a baseline created from the training data. Which SageMaker capability should they use?

A data scientist notices that a SageMaker endpoint is returning HTTP 5XX errors under high load. The endpoint uses a single ml.m5.large instance. The team wants to reduce these errors without changing the instance type. What is the most cost-effective step?

A financial services company needs to deploy a SageMaker endpoint that processes sensitive customer data. The security policy requires that all data in transit between the endpoint and the application must be encrypted, and that the endpoint cannot be accessed from the public internet. Additionally, model containers must not be able to initiate outbound internet requests. Which combination of settings meets these requirements?

A machine learning engineer observes that model performance on a SageMaker endpoint has degraded over the past week. Ground truth labels are available with a 2-day delay. The engineer wants to automatically trigger a retraining pipeline when prediction quality drops below an acceptable threshold. Which approach is most appropriate?

A company uses SageMaker Studio for collaborative ML development. The security team requires that all SageMaker Studio notebooks run within a VPC and cannot access the public internet. Which configuration should the administrator set?

A data science team wants to track the lineage of models, including datasets, training jobs, and endpoints, for reproducibility and audit. They need a solution that captures relationships between artifacts automatically during training and deployment. Which service should they use?

A company deploys a real-time inference endpoint and wants to be alerted if the number of 4XX errors exceeds 10 per minute over a 5-minute period. Which steps should they take?

A team is using SageMaker Clarify to detect bias drift in a deployed model's predictions. They run weekly bias monitoring jobs. The team wants to be notified when the bias metric for a sensitive feature exceeds a threshold. What is the most efficient method to achieve this?

A company wants to reduce costs for a SageMaker real-time endpoint that receives predictable traffic patterns: high during business hours and low at night. The model is a small PyTorch model. Which cost-saving strategy is most suitable?

A machine learning engineer is deploying a model using a SageMaker endpoint and needs to ensure that the model artifacts are encrypted at rest using a customer-managed KMS key. Which configuration should they set?

A team is monitoring a SageMaker endpoint and notices that the average latency (ModelLatency) is increasing over time, but the number of invocations is steady. They suspect that the model's inference code is becoming slower due to memory leaks. Which metric should they also examine to confirm this hypothesis?

A company wants to share a trained model across multiple AWS accounts for inference. The model is stored in a central account's S3 bucket and needs to be deployed in other accounts' SageMaker endpoints. What is the recommended approach?

A machine learning team needs to automatically retrain a model when concept drift is detected in the deployed endpoint's predictions. Which TWO steps should they take? (Choose TWO.)

A company is deploying a SageMaker real-time endpoint and needs to monitor inference latency. Which THREE metrics are available from SageMaker for this purpose? (Choose THREE.)

A team wants to ensure that their SageMaker training jobs cannot access the internet for security reasons. However, they need to download a public PyTorch package for training. Which TWO steps should they take? (Choose TWO.)

A company has deployed a machine learning model on Amazon SageMaker and wants to automatically detect when the distribution of input features deviates significantly from the training data distribution. Which SageMaker feature should they use?

A machine learning engineer notices that the latency of a SageMaker endpoint has increased over time. They need to identify which component (model inference vs. pre/post-processing) contributes most to the latency. Which CloudWatch metrics should they examine?

A financial services company must deploy a SageMaker endpoint that only accepts traffic from within a VPC and encrypts all data at rest and in transit using customer-managed KMS keys. They also need to prevent inter-container traffic from being visible to other users. Which combination of settings fulfills these requirements?

A data science team wants to automate the retraining of a model whenever SageMaker Model Monitor detects a significant drift in data quality. They need the least amount of custom code. Which approach should they use?

A company wants to track the lineage of their ML models, including the training dataset, hyperparameters, and training job used to produce each model version. Which AWS service should they use?

A company uses SageMaker Model Monitor to detect bias drift in their real-time inference endpoint. They have collected ground truth labels and want to monitor for bias across different demographic groups. Which type of monitoring should they configure?

A company deploys a model with SageMaker and wants to monitor for concept drift. They have noticed that the relationship between input features and the target variable has changed, causing model accuracy to degrade. However, the input data distribution remains stable. Which type of drift is this, and what is the most appropriate response strategy?

A machine learning engineer wants to reduce costs for a SageMaker real-time endpoint that experiences predictable traffic patterns with low traffic at night and high traffic during business hours. Which approach is most cost-effective while maintaining availability?

A company needs to ensure that their SageMaker Studio environment is only accessible from within their corporate network and that all data processed in Studio remains encrypted. Which configuration should they use?

A company wants to deploy a foundation model from SageMaker JumpStart with the lowest possible inference cost, given that latency requirements are flexible. They have a mix of traffic volumes. Which approach should they take?

A company uses SageMaker Model Monitor's feature attribution drift monitoring with SHAP. They receive an alert that the average SHAP value for a particular feature has increased significantly compared to the baseline. The feature's input distribution has not changed. What does this likely indicate?

A company wants to allow a SageMaker model in one AWS account to be accessed by a different AWS account for inference. They need to maintain security and compliance. Which approach meets the requirement?

A company has a SageMaker real-time endpoint that serves predictions. They want to set up automated monitoring and remediation for when the number of 5XX errors exceeds a threshold. Which TWO steps should they take? (Choose TWO.)

A company wants to track the lineage of their ML models for reproducibility and auditability. Which THREE services or features should they use together to achieve this? (Choose THREE.)

A company is deploying a SageMaker endpoint and must meet strict security requirements: no public internet access, all inter-container traffic must be encrypted, and all data at rest must be encrypted with a customer-managed KMS key. Which THREE configurations should they apply? (Choose THREE.)

A machine learning engineer needs to automatically retrain a model whenever SageMaker Model Monitor detects data drift. Which combination of services should be used to trigger the retraining pipeline?

A company has deployed a real-time inference endpoint using SageMaker. The endpoint latency is within acceptable limits, but the team notices that the Invocations metric shows occasional spikes. They want to investigate the source of the spikes. Which CloudWatch metric should they examine to isolate the time spent in SageMaker overhead versus model inference?

A financial services company is deploying a fraud detection model on SageMaker. To comply with regulations, they must ensure that the model's predictions are not biased against protected groups. They plan to monitor bias drift post-deployment using SageMaker Clarify. Which data inputs are required to configure Clarify's bias drift monitoring?

A machine learning team is using SageMaker Studio for model development. They need to restrict all internet access from Studio notebooks and ensure that all data stays within a VPC. Which configuration should they use?

A company wants to implement cost monitoring and optimization for SageMaker endpoints. Which TWO actions should they take? (Select TWO)

A data science team uses SageMaker to train and deploy models. They need to track model lineage, including datasets, training jobs, and model versions, to ensure reproducibility. Which THREE actions should they take? (Select THREE)

A company uses SageMaker Model Monitor to detect data drift in production. The monitoring job compares the current data distribution to a baseline. Which TWO types of drift can SageMaker Model Monitor detect? (Select TWO)

A team wants to secure SageMaker endpoints for a healthcare application. They must ensure data is encrypted at rest and in transit, and that the endpoint can only be accessed from within a VPC. Which THREE steps should they take? (Select THREE)

A company is deploying a foundation model using SageMaker JumpStart. They want to minimize inference costs while maintaining low latency. Which TWO strategies should they consider? (Select TWO)

A machine learning engineer wants to set up a retraining pipeline that triggers when model quality degrades. Which TWO components are essential for this automated retraining pipeline? (Select TWO)

A company uses SageMaker Model Monitor for feature attribution drift monitoring with SHAP. Which THREE prerequisites must be in place before starting the monitoring schedule? (Select THREE)

A company wants to enable cross-account access to a SageMaker model endpoint. The model is in Account A, and Account B needs to invoke it. Which TWO steps are required? (Select TWO)

A machine learning team notices an increase in 5XXError count for a SageMaker endpoint. They want to set up automated remediation. Which THREE actions should they take? (Select THREE)