AI Operations

You are managing an Azure Machine Learning workspace and need to ensure that model training jobs are reproducible across environments. Which feature should be configured to capture the exact environment state, including Python packages and system dependencies?

In SageMaker, what is the role of an 'Endpoint Configuration'?

Which THREE components are critical to monitor in an AI pipeline to ensure operational availability?

An audit team is reviewing a production AI system that performs real-time fraud detection. The system's output is a probability score between 0 and 1, and a threshold of 0.8 is applied to classify transactions as fraudulent. The team discovers that the threshold was set during initial model development and has not been adjusted since. Over the past six months, the false positive rate has increased from 2% to 15%, causing customer friction. The model's AUC-ROC has remained stable at 0.92. What is the MOST likely cause of the increased false positives?

When designing an AI governance framework, which THREE roles should have oversight responsibility?

Which metric would best detect 'Label Drift' in a production model?

Which THREE documents are typically required for an AI Audit trail?

What is the purpose of 'Containerization' (e.g., Docker) in MLOps?

In an AI Audit, what does 'Model Lineage' demonstrate?

Which of the following is an example of an 'operational metric' for an AI system?

What is 'Shadow Deployment' in AI MLOps?

During an audit of an LLM pipeline, you discover that 'System Prompt' versioning is missing. What is the operational risk?

When auditing an AI model, what does 'Concept Drift' signify to an operational team?

When auditing an ML pipeline, what does the 'DAG' (Directed Acyclic Graph) represent?

What is the risk of 'Feature Skew' in an AI pipeline?

Which operational process is required to ensure 'Data Privacy' when using user-generated data for model retraining?

For a production LLM deployment, which operational metric is most critical to monitor to identify 'hallucination' or grounding failure in real-time?

What is the primary purpose of a 'Champion-Challenger' deployment pattern in MLOps?

An organization using AWS SageMaker is experiencing model drift in production. You need to implement automated monitoring to trigger alerts when the distribution of input data deviates from the baseline. Which tool should be configured?

When auditing a Kubernetes-based AI deployment, what does a 'Pod Disruption Budget' (PDB) ensure?

An auditor finds that a model's 'input feature importance' has changed significantly after a retrain. What is the most appropriate action?

Which THREE items should be tracked in a 'Model Registry' for audit readiness?

Which TWO are key 'AI Operational' documentation artifacts?

Which security control is most effective against 'Prompt Injection' attacks in a deployed LLM?

When auditing an AI pipeline, where should 'Environment Variables' be stored to ensure security?

During an audit of a model inference endpoint, you observe that 'Model Explanation' logs are missing. Which library or tool was likely omitted from the deployment?

Which environment should be used to perform 'Model Validation' before moving to production?

When setting up a containerized deployment of an AI model using Docker, what is the primary purpose of a 'multi-stage build'?

Which THREE items must be included in a 'Model Card' to comply with regulatory model risk management standards?

Which TWO are common 'AI Security' threats that require operational oversight?

What is the function of a 'Circuit Breaker' in a microservice-based AI architecture?

An organization experiences 'Data Leakage' in a machine learning project. What is the most likely cause during the training process?

What is the primary function of a 'Version Control System' (like Git) in an MLOps team?

An AI audit is examining a computer vision model used for quality control in a manufacturing plant. The model detects defects on assembly lines and triggers alerts for human inspectors. The audit team reviews the model's deployment and finds that it runs on a single GPU server without redundancy. The plant operates 24/7, and any downtime of the model results in unmonitored production, potentially allowing defective products to pass. The team also notes that the model's inference latency is 200ms, which is within the required 500ms. Which of the following is the MOST significant operational risk identified in this scenario?

An AI auditor is reviewing the operational documentation for a computer vision model used in quality control on a manufacturing line. The documentation includes a section on 'model serving infrastructure.' Which element is most important for the auditor to verify regarding the model's ability to handle peak production loads?

When using DVC (Data Version Control) in an AI project, what is the best practice to ensure the 'data version' is immutable for audit purposes?

An AI audit team is evaluating a bank's production ML system that processes loan applications. The system uses an automated pipeline that retrains the model every month. The audit team needs to verify that the retraining process does not introduce unintended bias. Which operational control should the audit team examine first?

An AI audit team is assessing the operational resilience of a fraud detection system that uses a real-time ML model. The system must maintain high availability and handle concept drift. Which two operational practices are most critical for ensuring the system's continued effectiveness? (Choose two.)

What is the primary objective of a 'Model Inventory' in an enterprise AI audit?

Which AWS SageMaker feature allows auditors to verify that a model was trained on encrypted data?

When an AI system utilizes a 'Feature Store', how does it ensure consistency between training and inference?

In a GCP Vertex AI pipeline, you want to ensure that every model artifact is associated with the specific training data and code version used. Which component provides this lineage tracking?

When auditing a model deployment pipeline, which TWO aspects are critical to verify to ensure compliance with AI governance frameworks?

When deploying an AI model on a public cloud, what is the best practice for 'Secret Management' of API keys?

What is the most effective way to audit 'Data Lineage' in a complex AI ecosystem?

You are performing an audit on an ML project utilizing Kubeflow. The team is using Katib for hyperparameter tuning. Which configuration step is critical to ensure that individual trials do not starve the production inference service of resources?

In an AI Audit, what does 'Model Fairness' assessment typically involve?

What does a 'Model Approval' workflow ensure?

When auditing an AI system, how do you verify 'Model Reproducibility'?

A retail company has deployed an AI-powered recommendation engine. The AI audit team is reviewing the system's operational metrics and finds that the model's prediction latency has gradually increased over three months. Which operational metric should the team analyze to determine if the increase is due to growing model complexity or infrastructure issues?

An AI auditor is reviewing a healthcare organization's ML system that predicts patient readmission risk. The system uses a feature store to serve features to both training and inference pipelines. The auditor notices that the feature store does not enforce point-in-time correctness for training data. What is the most significant operational risk this poses?

In a CI/CD pipeline for AI, where should the 'Model Validation' gate be located?

What is the primary function of an 'AI Pipeline Orchestrator' like Kubeflow or Airflow?

Which THREE of the following are essential components of an MLOps maturity model for ensuring operational stability?

In the context of 'AI Pipelines', what is the role of a 'Model Registry'?

In an AI Audit, what does an 'Input Feature Schema' define?

Which TWO of the following are considered 'Model Metadata' that should be tracked in a registry?

Which of these is a valid reason to 'Roll Back' a model deployment?

You are configuring a CI/CD pipeline for ML using GitHub Actions to deploy to a Kubernetes cluster. To ensure sensitive credentials for your container registry are not exposed in logs, what is the best practice?

Which TWO of the following are primary benefits of implementing a Model Registry in an AI production environment?

Which of the following is a key component of 'MLOps Maturity'?

Which THREE factors influence 'AI Pipeline' latency?

What is the purpose of 'Performance Monitoring' in AI?

What is the primary function of a 'Canary Deployment' in AI model release?

A company uses MLflow for model registry and tracking. You need to transition a model from the 'Staging' environment to 'Production' only after it passes an automated integration test. How is this state transition managed in MLflow?

What is the consequence of 'Pipeline Staleness' in an AI system?

What is the role of an 'A/B Test' in AI production?

In the context of AI regulatory compliance, what is the 'Model Risk Management' (MRM) framework mainly concerned with?

In the context of AI Operations, what does 'CI/CD' stand for?

Which THREE tasks are required for a 'Model Decommissioning' workflow?

What is 'Model Versioning' used for in an AI project?

An audit is being conducted on an AI system that provides real-time predictions for loan approvals. The system uses a gradient boosting model deployed on a Kubernetes cluster. The audit team is evaluating the operational controls for model versioning and rollback. Which TWO of the following practices are essential to ensure that the correct model version is serving predictions and that rollback can be performed safely? (Choose two.)

Which TWO are standard practices for securing AI model endpoints?

Which TWO are common 'AI Operational' challenges?

What is the main benefit of 'Automated Retraining'?

Why must you conduct a 'Bias Audit' on a model that has been fine-tuned on new data?

What is the most effective way to address 'Adversarial Input' during inference?

In SageMaker, what does 'Model Monitor' specifically detect?

Which THREE criteria are essential when selecting an AI deployment strategy?

An organization has deployed a recommendation model that suggests products to users on its e-commerce platform. The model is retrained weekly using the latest user interaction data. The audit team is reviewing the model's operational logs and finds that the retraining process sometimes fails silently, and the previous week's model continues to serve predictions. There is no alerting mechanism for retraining failures. What is the MOST appropriate control to implement to address this issue?

What does a 'Backtesting' report verify in an AI Audit?

What is the role of a 'Deployment Pipeline' in MLOps?

Which TWO items should be included in an automated 'Deployment Log'?

Which TWO are common types of 'AI Model Drift'?

An AI audit is being conducted for a natural language processing model used in a customer service chatbot. The model was trained on historical chat logs and deployed six months ago. The audit team notices that the model's responses have become increasingly irrelevant and sometimes offensive. The team reviews the model's input pipeline and finds that the tokenization step uses a fixed vocabulary that was built from the training data. However, new slang and abbreviations are frequently used by customers, and these are not in the vocabulary. What is the PRIMARY operational control that should have been implemented to prevent this issue?

An auditor notices high 'latency' in an online model deployment. Which component should be inspected first?

What is the function of a 'Feature Store' in an MLOps pipeline?

Which TWO of the following are essential for ensuring 'Model Observability' in an enterprise production environment?

Which of the following is a symptom of 'Overfitting' during the training phase?

What is the primary function of an 'AI Model Gateway'?

What is the purpose of an 'AI Audit trail'?

In Azure Machine Learning, which component should an auditor verify to ensure model registry access is restricted to authorized CI/CD service principals?

If an AI pipeline fails, which component provides the best 'log' to diagnose the failure?

What is the goal of an AI 'Rollout' plan?

If a model's performance drops, and the input data distribution has changed, what operational process must be triggered?

When implementing drift detection in AWS SageMaker Model Monitor, which metric should be monitored to detect feature attribution shift for structured data?