When implementing drift detection in AWS SageMaker Model Monitor, which metric should be monitored to detect feature attribution shift for structured data?
In Azure Machine Learning, which component should an auditor verify to ensure model registry access is restricted to authorized CI/CD service principals?
During an audit of a model inference endpoint, you observe that 'Model Explanation' logs are missing. Which library or tool was likely omitted from the deployment?
You are managing an Azure Machine Learning workspace and need to ensure that model training jobs are reproducible across environments. Which feature should be configured to capture the exact environment state, including Python packages and system dependencies?
An organization using AWS SageMaker is experiencing model drift in production. You need to implement automated monitoring to trigger alerts when the distribution of input data deviates from the baseline. Which tool should be configured?
In a GCP Vertex AI pipeline, you want to ensure that every model artifact is associated with the specific training data and code version used. Which component provides this lineage tracking?
You are configuring a CI/CD pipeline for ML using GitHub Actions to deploy to a Kubernetes cluster. To ensure sensitive credentials for your container registry are not exposed in logs, what is the best practice?
You are performing an audit on an ML project utilizing Kubeflow. The team is using Katib for hyperparameter tuning. Which configuration step is critical to ensure that individual trials do not starve the production inference service of resources?
A company uses MLflow for model registry and tracking. You need to transition a model from the 'Staging' environment to 'Production' only after it passes an automated integration test. How is this state transition managed in MLflow?
An AI audit team is evaluating a bank's production ML system that processes loan applications. The system uses an automated pipeline that retrains the model every month. The audit team needs to verify that the retraining process does not introduce unintended bias. Which operational control should the audit team examine first?
An AI auditor is reviewing a healthcare organization's ML system that predicts patient readmission risk. The system uses a feature store to serve features to both training and inference pipelines. The auditor notices that the feature store does not enforce point-in-time correctness for training data. What is the most significant operational risk this poses?
An audit team is reviewing a production AI system that performs real-time fraud detection. The system's output is a probability score between 0 and 1, and a threshold of 0.8 is applied to classify transactions as fraudulent. The team discovers that the threshold was set during initial model development and has not been adjusted since. Over the past six months, the false positive rate has increased from 2% to 15%, causing customer friction. The model's AUC-ROC has remained stable at 0.92. What is the MOST likely cause of the increased false positives?
A retail company has deployed an AI-powered recommendation engine. The AI audit team is reviewing the system's operational metrics and finds that the model's prediction latency has gradually increased over three months. Which operational metric should the team analyze to determine if the increase is due to growing model complexity or infrastructure issues?
An AI audit is being conducted for a natural language processing model used in a customer service chatbot. The model was trained on historical chat logs and deployed six months ago. The audit team notices that the model's responses have become increasingly irrelevant and sometimes offensive. The team reviews the model's input pipeline and finds that the tokenization step uses a fixed vocabulary that was built from the training data. However, new slang and abbreviations are frequently used by customers, and these are not in the vocabulary. What is the PRIMARY operational control that should have been implemented to prevent this issue?
An AI audit team is assessing the operational resilience of a fraud detection system that uses a real-time ML model. The system must maintain high availability and handle concept drift. Which two operational practices are most critical for ensuring the system's continued effectiveness? (Choose two.)
An AI audit is examining a computer vision model used for quality control in a manufacturing plant. The model detects defects on assembly lines and triggers alerts for human inspectors. The audit team reviews the model's deployment and finds that it runs on a single GPU server without redundancy. The plant operates 24/7, and any downtime of the model results in unmonitored production, potentially allowing defective products to pass. The team also notes that the model's inference latency is 200ms, which is within the required 500ms. Which of the following is the MOST significant operational risk identified in this scenario?
An AI auditor is reviewing the operational documentation for a computer vision model used in quality control on a manufacturing line. The documentation includes a section on 'model serving infrastructure.' Which element is most important for the auditor to verify regarding the model's ability to handle peak production loads?
An organization has deployed a recommendation model that suggests products to users on its e-commerce platform. The model is retrained weekly using the latest user interaction data. The audit team is reviewing the model's operational logs and finds that the retraining process sometimes fails silently, and the previous week's model continues to serve predictions. There is no alerting mechanism for retraining failures. What is the MOST appropriate control to implement to address this issue?
An audit is being conducted on an AI system that provides real-time predictions for loan approvals. The system uses a gradient boosting model deployed on a Kubernetes cluster. The audit team is evaluating the operational controls for model versioning and rollback. Which TWO of the following practices are essential to ensure that the correct model version is serving predictions and that rollback can be performed safely? (Choose two.)