AAIR AI Lifecycle Risk Management Practice Test 2 — 15 Questions

Question 1

A company is implementing 'Human-in-the-Loop' (HITL) for its AI decision system. What is the primary risk of relying on HITL?

Accepted Answer

The 'automation bias' or 'rubber stamping' of decisions by human reviewers.. Human fatigue and cognitive bias can lead to poor oversight, potentially creating a false sense of security.

Answer

It requires expensive API calls.

Answer

It causes the model to become slower.

Answer

It forces the model to use a specific programming language.

Question 2

When managing AI lifecycle risk for a model in a regulated industry, which artifact serves as the most important audit trail for the model's provenance?

Accepted Answer

The end-to-end model lineage and version control logs.. The end-to-end model lineage and version control logs provide the complete, traceable record of how the model was built — including data sources, training runs, hyperparameters, code commits, and artifact versions. This is the artifact auditors and regulators rely on to reconstruct the model's provenance and reproduce its behavior. Access logs, latency metrics, and billing statements document operational or financial activity, not the model's origin or derivation.

Answer

The user access logs for the dashboard.

Answer

The model's current latency metrics.

Answer

The cloud provider's billing statement.

Question 3

What is the primary risk objective of 'Model Versioning' in the AI lifecycle?

Accepted Answer

To ensure auditability, reproducibility, and the ability to roll back to a known stable state.. Model versioning exists to provide auditability (who changed what and when), reproducibility (recreate any past model and its results), and rollback capability (revert to a known stable version when a new release degrades). These properties are foundational for incident response, regulatory review, and safe continuous delivery of AI systems.

Answer

To prevent unauthorized users from viewing the code.

Answer

To reduce the amount of disk space used.

Answer

To increase the model's speed in production.

Question 4

You are assessing risk for an AI model that uses 'Transfer Learning'. What is the most critical risk to manage regarding the pre-trained base model?

Accepted Answer

The base model may have been trained on data that contains inherent, unidentifiable biases.. Transfer learning risks include the inheritance of latent biases from the pre-trained weights.

Answer

The base model cannot be used with custom datasets.

Answer

The base model will always run out of memory.

Answer

The base model requires a proprietary operating system.

Question 5

A team notices that an AI model's performance decreases when the input data contains abbreviations that were not present in the training set. This is a risk associated with which lifecycle stage?

Accepted Answer

Data preprocessing and training design.. Abbreviations absent from the training set are a vocabulary/coverage gap introduced during data collection and feature engineering. Because the model never learned representations for those tokens, it cannot generalize to them at inference — this is a data preprocessing and training design risk (B), not an infrastructure, monitoring, or deployment issue. The root cause is upstream of deployment: the training corpus did not represent the input distribution the model now sees.

Answer

Infrastructure provisioning.

Answer

Monitoring phase.

Answer

Deployment phase.

Question 6

When a 'Data Drift' alert is triggered in a production environment, what is the first step in the risk management lifecycle?

Accepted Answer

Perform a root cause analysis to verify if the shift is statistical noise or a genuine change in input patterns.. The first step in the risk management lifecycle when a data drift alert triggers is to perform a root cause analysis to determine whether the drift is due to statistical noise or a genuine change in input patterns. This verification step is crucial before taking any corrective action like retraining or rolling back, as it prevents unnecessary interventions. It aligns with the risk management process of identifying and assessing the risk before mitigating it.

Answer

Immediately retrain the model.

Answer

Delete all current production logs.

Answer

Roll back to the previous model version.

Question 7

A company is integrating an AI model that uses 'Online Inference'. What is the most significant risk regarding model security?

Accepted Answer

The risk of model inversion attacks on the inference endpoint.. Online inference endpoints are exposed to the public/network, creating surface area for adversarial attacks.

Answer

The risk of the training data being corrupted by the user.

Answer

The risk of high database storage costs.

Answer

The risk of the Python interpreter crashing.

Question 8

A team is designing an AI lifecycle monitoring system. Which metric is most effective for detecting 'Out-of-Distribution' (OOD) risks?

Accepted Answer

Anomaly scores derived from the model's embedding space or activation layers.. Out-of-distribution inputs produce anomalous patterns in the model's internal representations — embedding vectors or activation layers — so anomaly scores derived from these spaces (e.g., Mahalanobis distance, reconstruction error, or energy scores) are the most direct signal of OOD risk. These scores detect inputs that differ semantically from training distribution even when surface features look normal. This is the standard OOD detection approach in production ML monitoring.

Answer

Total number of users accessing the model.

Answer

Average request latency.

Answer

The number of lines of code in the model file.

Question 9

In the context of the 'AI Lifecycle', what is the risk of 'Model Decay'?

Accepted Answer

The model's predictions become less accurate because the underlying environment or user behavior changes.. Model decay (also called model drift or concept drift) is the degradation of a model's predictive accuracy over time because the real-world environment, data distributions, or user behavior change after training. The model itself is unchanged, but its assumptions no longer match reality. This is why ongoing monitoring and retraining are essential in the AI lifecycle.

Answer

The model physically degrades over time.

Answer

The model's code becomes deprecated due to language updates.

Answer

The training hardware becomes too slow to run.

Question 10

Which of the following is a primary risk during the 'Deployment' stage of the AI lifecycle?

Accepted Answer

Unexpected performance degradation due to infrastructure differences between development and production.. During deployment, the model moves from a controlled development environment into production, where infrastructure, data distributions, and traffic patterns differ. This shift commonly causes unexpected performance degradation — for example, different hardware, library versions, or input distributions than those used during training and validation. The other options describe activities from earlier lifecycle stages (data preparation, research) or are not AI lifecycle risks at all.

Answer

Cleaning the initial dataset.

Answer

Writing the initial research paper.

Answer

Choosing the wrong programming language.

Question 11

Which THREE factors contribute to 'Data Quality Risk' during the training phase?

Accepted Answer

Incomplete datasets with many missing values.. Data Quality Risk during training arises from defects in the data itself, and option A is correct because incomplete datasets with many missing values degrade the statistical representativeness of the training sample, causing the model to learn from biased or insufficient signal. Option B is correct because systematic errors in target label annotations inject consistent noise into the ground truth, so the model optimizes toward incorrect labels and produces unreliable predictions. Option D is correct because selection bias in the training set creation process means the sampled data does not reflect the real-world distribution the model will encounter, skewing learned patterns. Option C is not a data quality issue but a hardware/resource constraint (GPU memory capacity), and option E is an infrastructure/vendor choice unrelated to the intrinsic quality of the training data.

Answer

The model is too large for the GPU memory.

Answer

Using the wrong cloud vendor.

Question 12

To mitigate 'Training-Serving Skew' in the AI lifecycle, what is the best practice for data processing pipelines?

Accepted Answer

Use a feature store to ensure consistent feature engineering logic for both stages.. Training-serving skew occurs when the feature computation logic differs between the training pipeline and the inference pipeline, producing inconsistent feature values. A feature store (A) centralizes feature definitions and serves the same computed values to both training and online inference, eliminating the divergence. This is the industry-standard mitigation because it enforces a single source of truth for feature engineering.

Answer

Write separate code for training and inference to optimize for speed.

Answer

Use only raw data for both training and inference.

Answer

Train the model on the production server directly.

Question 13

During the 'Training' phase, why is 'Data Splitting' (train/validation/test) crucial for risk management?

Accepted Answer

To evaluate the model's generalization capabilities and mitigate the risk of overfitting.. Data splitting partitions the dataset into training, validation, and test sets so the model is evaluated on data it has never seen. This measures generalization and detects overfitting, where the model memorizes training data but fails on new inputs. It is a foundational risk control because it prevents deploying a model whose performance is illusory.

Answer

To save disk space on the server.

Answer

To increase the training speed.

Answer

To make the model easier to read.

Question 14

A practitioner is managing AI risk in a CI/CD pipeline. What should be the final gate before model promotion to production?

Accepted Answer

Automated performance validation against a hold-out test set with defined threshold gates.. Automated validation of performance metrics against a 'golden dataset' ensures the model meets the required threshold.

Answer

Increasing the memory of the production server.

Answer

Updating the project's documentation.

Answer

A peer review of the training code.

Question 15

Which TWO actions should be taken when a model exhibits significant drift?

Accepted Answer

Perform a root cause analysis to determine the nature of the drift.. Option A is correct because when a model exhibits significant drift, the first step is to perform a root cause analysis to determine whether the drift is due to data quality issues, concept drift, or environmental change, which informs the appropriate remediation. Option B is correct because if the root cause analysis identifies the drift as environmental change (e.g., shifting user behavior or seasonal patterns), retraining the model on updated, representative data is the standard remedy to restore performance. Option C is incorrect because disabling logging removes the observability needed to detect and diagnose drift, and it does not address the underlying model degradation. Option D is incorrect because switching to a simpler architecture is not a targeted response to drift and may reduce performance without resolving the data or concept shift. Option E is incorrect because adding compute resources does not fix drift, since the issue is a mismatch between the model and current data distribution, not insufficient processing power.

Answer

Disable all logging to improve performance.

Answer

Change the model architecture to something simpler.

Answer

Immediately increase the compute resources.