AI Operations

What is the primary purpose of a 'Champion-Challenger' deployment pattern in MLOps?

For a production LLM deployment, which operational metric is most critical to monitor to identify 'hallucination' or grounding failure in real-time?

When implementing drift detection in AWS SageMaker Model Monitor, which metric should be monitored to detect feature attribution shift for structured data?

Which AWS SageMaker feature allows auditors to verify that a model was trained on encrypted data?

In Azure Machine Learning, which component should an auditor verify to ensure model registry access is restricted to authorized CI/CD service principals?

What is the primary function of an 'AI Pipeline Orchestrator' like Kubeflow or Airflow?

In the context of AI Operations, what does 'CI/CD' stand for?

Which TWO of the following are essential for ensuring 'Model Observability' in an enterprise production environment?

Which THREE items must be included in a 'Model Card' to comply with regulatory model risk management standards?

In a CI/CD pipeline for AI, where should the 'Model Validation' gate be located?

When using DVC (Data Version Control) in an AI project, what is the best practice to ensure the 'data version' is immutable for audit purposes?

During an audit of a model inference endpoint, you observe that 'Model Explanation' logs are missing. Which library or tool was likely omitted from the deployment?

When auditing an AI model, what does 'Concept Drift' signify to an operational team?

When designing an AI governance framework, which THREE roles should have oversight responsibility?

Which TWO of the following are considered 'Model Metadata' that should be tracked in a registry?

Which of the following is a symptom of 'Overfitting' during the training phase?

When auditing an ML pipeline, what does the 'DAG' (Directed Acyclic Graph) represent?

Which security control is most effective against 'Prompt Injection' attacks in a deployed LLM?

An auditor notices high 'latency' in an online model deployment. Which component should be inspected first?

What is the function of a 'Feature Store' in an MLOps pipeline?

If a model's performance drops, and the input data distribution has changed, what operational process must be triggered?

What is the primary function of a 'Canary Deployment' in AI model release?

In SageMaker, what is the role of an 'Endpoint Configuration'?

Which environment should be used to perform 'Model Validation' before moving to production?

What is the most effective way to audit 'Data Lineage' in a complex AI ecosystem?

Which TWO are common 'AI Security' threats that require operational oversight?

An organization experiences 'Data Leakage' in a machine learning project. What is the most likely cause during the training process?

Which THREE components are critical to monitor in an AI pipeline to ensure operational availability?

When an AI system utilizes a 'Feature Store', how does it ensure consistency between training and inference?

What is 'Model Versioning' used for in an AI project?

In an AI Audit, what does 'Model Fairness' assessment typically involve?

Which metric would best detect 'Label Drift' in a production model?

What is the consequence of 'Pipeline Staleness' in an AI system?

When auditing a Kubernetes-based AI deployment, what does a 'Pod Disruption Budget' (PDB) ensure?

In the context of AI regulatory compliance, what is the 'Model Risk Management' (MRM) framework mainly concerned with?

What is the primary objective of a 'Model Inventory' in an enterprise AI audit?

Which of these is a valid reason to 'Roll Back' a model deployment?

Which TWO items should be included in an automated 'Deployment Log'?

Which THREE criteria are essential when selecting an AI deployment strategy?

What is the main benefit of 'Automated Retraining'?

In an AI Audit, what does an 'Input Feature Schema' define?

What is 'Shadow Deployment' in AI MLOps?

Which of the following is a key component of 'MLOps Maturity'?

When deploying an AI model on a public cloud, what is the best practice for 'Secret Management' of API keys?

In the context of 'AI Pipelines', what is the role of a 'Model Registry'?

Which of the following is an example of an 'operational metric' for an AI system?

Why must you conduct a 'Bias Audit' on a model that has been fine-tuned on new data?

During an audit of an LLM pipeline, you discover that 'System Prompt' versioning is missing. What is the operational risk?

Which TWO are standard practices for securing AI model endpoints?

Which THREE documents are typically required for an AI Audit trail?

What is the primary function of a 'Version Control System' (like Git) in an MLOps team?

What is the role of an 'A/B Test' in AI production?

What is the purpose of 'Containerization' (e.g., Docker) in MLOps?

In SageMaker, what does 'Model Monitor' specifically detect?

An auditor finds that a model's 'input feature importance' has changed significantly after a retrain. What is the most appropriate action?

Which TWO are common types of 'AI Model Drift'?

Which THREE tasks are required for a 'Model Decommissioning' workflow?

What is the most effective way to address 'Adversarial Input' during inference?

What is the purpose of an 'AI Audit trail'?

If an AI pipeline fails, which component provides the best 'log' to diagnose the failure?

What is the primary function of an 'AI Model Gateway'?

When auditing an AI pipeline, where should 'Environment Variables' be stored to ensure security?

What does a 'Backtesting' report verify in an AI Audit?

Which operational process is required to ensure 'Data Privacy' when using user-generated data for model retraining?

Which TWO are common 'AI Operational' challenges?

Which THREE items should be tracked in a 'Model Registry' for audit readiness?

What is the purpose of 'Performance Monitoring' in AI?

What does a 'Model Approval' workflow ensure?

What is the risk of 'Feature Skew' in an AI pipeline?

What is the role of a 'Deployment Pipeline' in MLOps?

What is the function of a 'Circuit Breaker' in a microservice-based AI architecture?

Which TWO are key 'AI Operational' documentation artifacts?

Which THREE factors influence 'AI Pipeline' latency?

What is the goal of an AI 'Rollout' plan?

When auditing an AI system, how do you verify 'Model Reproducibility'?

In an AI Audit, what does 'Model Lineage' demonstrate?

You are managing an Azure Machine Learning workspace and need to ensure that model training jobs are reproducible across environments. Which feature should be configured to capture the exact environment state, including Python packages and system dependencies?

An organization using AWS SageMaker is experiencing model drift in production. You need to implement automated monitoring to trigger alerts when the distribution of input data deviates from the baseline. Which tool should be configured?

In a GCP Vertex AI pipeline, you want to ensure that every model artifact is associated with the specific training data and code version used. Which component provides this lineage tracking?

You are configuring a CI/CD pipeline for ML using GitHub Actions to deploy to a Kubernetes cluster. To ensure sensitive credentials for your container registry are not exposed in logs, what is the best practice?

Which THREE of the following are essential components of an MLOps maturity model for ensuring operational stability?

You are performing an audit on an ML project utilizing Kubeflow. The team is using Katib for hyperparameter tuning. Which configuration step is critical to ensure that individual trials do not starve the production inference service of resources?

When setting up a containerized deployment of an AI model using Docker, what is the primary purpose of a 'multi-stage build'?

A company uses MLflow for model registry and tracking. You need to transition a model from the 'Staging' environment to 'Production' only after it passes an automated integration test. How is this state transition managed in MLflow?

Which TWO of the following are primary benefits of implementing a Model Registry in an AI production environment?

When auditing a model deployment pipeline, which TWO aspects are critical to verify to ensure compliance with AI governance frameworks?

An AI audit team is evaluating a bank's production ML system that processes loan applications. The system uses an automated pipeline that retrains the model every month. The audit team needs to verify that the retraining process does not introduce unintended bias. Which operational control should the audit team examine first?

An AI auditor is reviewing a healthcare organization's ML system that predicts patient readmission risk. The system uses a feature store to serve features to both training and inference pipelines. The auditor notices that the feature store does not enforce point-in-time correctness for training data. What is the most significant operational risk this poses?

An audit team is reviewing a production AI system that performs real-time fraud detection. The system's output is a probability score between 0 and 1, and a threshold of 0.8 is applied to classify transactions as fraudulent. The team discovers that the threshold was set during initial model development and has not been adjusted since. Over the past six months, the false positive rate has increased from 2% to 15%, causing customer friction. The model's AUC-ROC has remained stable at 0.92. What is the MOST likely cause of the increased false positives?

A retail company has deployed an AI-powered recommendation engine. The AI audit team is reviewing the system's operational metrics and finds that the model's prediction latency has gradually increased over three months. Which operational metric should the team analyze to determine if the increase is due to growing model complexity or infrastructure issues?

An AI audit is being conducted for a natural language processing model used in a customer service chatbot. The model was trained on historical chat logs and deployed six months ago. The audit team notices that the model's responses have become increasingly irrelevant and sometimes offensive. The team reviews the model's input pipeline and finds that the tokenization step uses a fixed vocabulary that was built from the training data. However, new slang and abbreviations are frequently used by customers, and these are not in the vocabulary. What is the PRIMARY operational control that should have been implemented to prevent this issue?

An AI audit team is assessing the operational resilience of a fraud detection system that uses a real-time ML model. The system must maintain high availability and handle concept drift. Which two operational practices are most critical for ensuring the system's continued effectiveness? (Choose two.)

An AI audit is examining a computer vision model used for quality control in a manufacturing plant. The model detects defects on assembly lines and triggers alerts for human inspectors. The audit team reviews the model's deployment and finds that it runs on a single GPU server without redundancy. The plant operates 24/7, and any downtime of the model results in unmonitored production, potentially allowing defective products to pass. The team also notes that the model's inference latency is 200ms, which is within the required 500ms. Which of the following is the MOST significant operational risk identified in this scenario?

An AI auditor is reviewing the operational documentation for a computer vision model used in quality control on a manufacturing line. The documentation includes a section on 'model serving infrastructure.' Which element is most important for the auditor to verify regarding the model's ability to handle peak production loads?

An organization has deployed a recommendation model that suggests products to users on its e-commerce platform. The model is retrained weekly using the latest user interaction data. The audit team is reviewing the model's operational logs and finds that the retraining process sometimes fails silently, and the previous week's model continues to serve predictions. There is no alerting mechanism for retraining failures. What is the MOST appropriate control to implement to address this issue?

An audit is being conducted on an AI system that provides real-time predictions for loan approvals. The system uses a gradient boosting model deployed on a Kubernetes cluster. The audit team is evaluating the operational controls for model versioning and rollback. Which TWO of the following practices are essential to ensure that the correct model version is serving predictions and that rollback can be performed safely? (Choose two.)