AI Lifecycle Risk Management practice questions

Increase the retraining frequency of the pipeline.

Why it fails: Retraining might mask the drift without identifying the root cause.

Disable the data lineage capture to improve latency.

Why it fails: Disabling lineage reduces visibility into risk, it does not fix detection.

Switch the model to a higher-capacity compute instance.

Why it fails: Compute capacity does not impact the logical drift of the model.

Calculating the total budget for the project.

Why it fails: Financial planning, not risk identification.

Updating the company's social media policy.

Why it fails: Administrative, not AI risk identification.

Choosing the color scheme for the model's web dashboard.

Why it fails: Irrelevant.

After model deployment in production.

Why it fails: Post-deployment assessment detects drift only after poor-quality data has already influenced predictions and business outcomes. Monitoring production data is genuinely useful for ongoing drift detection, yet the stem asks when assessment minimises long-term risk, which requires validating data before it trains the model.

During the final model sign-off.

Why it fails: Sign-off validates the finished model's metrics, not the training data's completeness, accuracy or representativeness; defects are already embedded in learned weights. Sign-off is the right moment to confirm governance approvals and performance thresholds, but data quality must be established upstream of training.

During the decommission phase.

Why it fails: Assessing data quality at decommission cannot mitigate risk already baked into a trained model, since the data has long since been consumed. It is tempting because decommissioning does involve reviewing retained datasets for disposal, but that is a records-retention activity, not a quality gate.

Model monitoring phase.

Why it fails: Monitoring observes live inputs and outputs after training, so it cannot introduce target or future information into the training set. It is tempting because monitoring does surface data quality and drift issues, and would be the correct stage for detecting distribution shift, not for leakage during dataset construction.

Model decommission phase.

Why it fails: Data leakage arises during data collection, preparation or training, when features encode target or future information. Decommissioning only retires artefacts after training, so it cannot introduce leakage. It is tempting because decommissioning governs data retention and disposal, and would be the right concern when scrubbing stored training data or model artefacts at end of life.

Deployment phase.

Why it fails: Deployment serves live inference on new data, so training data is already fixed and cannot absorb future or target information. It is tempting because production data can drift from training distributions, and deployment monitoring would be the right stage for detecting that drift, not leakage.

Disable monitoring until ground truth is available.

Why it fails: Disabling monitoring removes all risk detection, including data drift and data quality checks that need no ground truth. It is tempting when labels lag, yet the correct response is to keep monitoring those ground-truth-independent metrics while delaying only accuracy-based evaluation.

Manually force a model redeployment.

Why it fails: Redeploying the model does not supply delayed ground truth and disrupts serving without improving risk identification. It is tempting as a reset gesture when monitoring looks stale, yet the actual fix is to continue monitoring drift and data quality while ground truth catches up.

Reduce the sampling rate of the inference logs.

Why it fails: Lowering the sampling rate reduces the volume of captured inference data, weakening drift and data quality detection rather than preserving it. It is tempting as a cost or load reduction measure, but sampling is not the bottleneck; delayed ground truth is, and accuracy metrics simply await labels.

Hardware failure in the training cluster.

Why it fails: Hardware failure in a training cluster is an availability concern mitigated by redundancy and checkpointing, not a model-behaviour risk. It would be the critical vigilance area for large offline training jobs; online learning's defining exposure is that the model keeps adapting to live input, so degradation comes from the data, not the hardware.

Data drift from batch updates.

Why it fails: Online learning updates continuously from streaming input, so drift arrives through the live data feed itself, not from periodic batch refreshes. Batch-update drift is the vigilance concern for models retrained on scheduled batches; here the exposure is unbounded, immediate distributional shift in the incoming stream.

Memory exhaustion due to high throughput.

Why it fails: Memory exhaustion from high throughput is a capacity and resource-planning issue addressed by scaling and backpressure, not a model-correctness risk. It would matter for high-volume inference serving; online learning's distinctive danger is that each incoming example immediately alters the model, so harmful or shifting input propagates without review.

Ignore the shift as it may be a temporary market trend.

Why it fails: Ignoring the disparity leaves a potentially discriminatory model in production, breaching fair-lending obligations and allowing harm to continue. Monitoring for drift and bias is precisely what the risk lifecycle demands; the tempting logic is that regional variation may reflect genuine market conditions, but that must be investigated, not assumed.

Immediately shut down the system.

Why it fails: Shutting the system down halts lending entirely and skips the investigation, documentation and impact assessment the risk lifecycle demands at this stage. Immediate deactivation is reserved for confirmed severe harm, not a statistical deviation still requiring root-cause analysis.

Train a secondary model to override the primary one.

Why it fails: A secondary override model adds an ungoverned layer that masks the disparity without diagnosing it, and both models would share the same biased training data. Retraining a replacement is warranted only after the bias source is identified and remediated.

Increase the training epoch count for the new version.

Why it fails: More epochs change how long the model trains, not the class distribution or decision threshold, so edge-case errors persist or worsen through overfitting. Epoch tuning suits convergence problems; imbalance mitigation requires resampling, class weights or threshold adjustment instead.

Directly replace the active production model version.

Why it fails: Overwriting the production version removes any rollback path, so a poor edge-case model immediately affects live traffic with no comparison baseline. Vertex AI Model Registry supports versioning and aliases precisely so a new version can be deployed to a small traffic split and evaluated before promotion.

Update the training data to remove edge case records.

Why it fails: Deleting edge case records reduces the model's exposure to precisely the inputs it must handle, worsening poor performance rather than mitigating it. It might appeal as a quick data-cleanup step, but Vertex AI Model Registry evaluation and staged rollout are the mechanisms for catching regressions before deployment.

Deploy the model with a human-in-the-loop oversight mechanism.

Why it fails: Human-in-the-loop oversight addresses inference-time review, not the training data's underrepresentation, so the model still learns skewed patterns. It is tempting because oversight genuinely mitigates high-stakes errors post-deployment, and would be correct where outputs carry irreversible consequences — but here the defect originates in the dataset itself.

Monitor prediction confidence intervals during the post-deployment phase.

Why it fails: Monitoring confidence intervals is a post-deployment detection control; it observes disparity after the model is live rather than correcting the training data during design. It is tempting because confidence monitoring genuinely surfaces poorly supported predictions, and would be correct for ongoing drift detection — but the stem asks for proactive mitigation at design time.

Increase the regularization parameters in the model architecture.

Why it fails: Regularisation constrains model complexity to reduce overfitting; it cannot manufacture examples of an absent demographic, so the learned bias persists. It is tempting because it is a cheap architectural lever applied during design, and would be correct for variance-driven overfitting — but the problem is skewed class representation, not model capacity.

It guarantees the model will never fail.

Why it fails: No control guarantees a model never fails; explainability surfaces reasoning so failures, drift and bias can be detected and remediated. It is the right control where decisions need justification, auditability or contestability, not where the goal is eliminating model error entirely.

It increases the inference speed of the model.

Why it fails: Explainability documents how a model reaches decisions; it does not alter inference latency, which depends on model architecture, hardware and optimisation. It is tempting because explainability supports auditing, bias detection and regulatory compliance, so it would be the right control when the requirement is transparency or accountability rather than throughput.

It allows the model to train on less data.

Why it fails: Explainability documents how inputs drive outputs; it does not reduce training data requirements, which depend on model complexity and task. It is a risk control because it enables audit, bias detection and challenge of decisions, and would be chosen where regulatory or governance accountability is required.

To automate the retraining pipeline for the model.

Why it fails: A Model Card is a transparency document recording intended use, training data, metrics and limitations; it does not orchestrate retraining. It would be the right artefact when documenting a model's provenance and performance for reviewers, not when automating pipeline execution.

To serve as the primary authentication token for API access.

Why it fails: A Model Card is a documentation artefact describing intended use, training data, evaluation metrics and limitations; it carries no credential or token function. It would be the right artefact when disclosing model provenance and performance to stakeholders, not when authenticating API calls.

To define the deployment strategy in Kubernetes.

Why it fails: A Model Card documents a model's intended use, data, metrics and limitations; it does not specify container orchestration or rollout topology. It would be the right artefact when communicating model characteristics to reviewers, not when defining Kubernetes deployment manifests.

The number of concurrent API requests.

Why it fails: Concurrent API requests measure infrastructure load, not model accuracy, so they cannot reveal drift without ground truth. This metric is tempting because it genuinely indicates scaling pressure and capacity saturation, making it the right choice when diagnosing throughput bottlenecks or provisioning resources. Here, however, drift requires tracking changes in input feature distributions or prediction confidence.

The memory consumption of the container.

Why it fails: Container memory consumption reflects resource utilisation, not predictive accuracy, so it cannot signal drift when labels are unavailable. It is tempting because memory growth can accompany data-volume shifts, and it would be the right metric for capacity planning or detecting leaks in a deployed inference service.

Model accuracy on historical data.

Why it fails: Historical accuracy measures fit to labelled data already seen, so it cannot signal drift when ground truth is unavailable. It is tempting because accuracy is the standard supervised metric, and it would be correct when fresh labelled outcomes exist to score against, such as a scheduled batch evaluation with delayed labels.

Model documentation compliance.

Why it fails: Shadow deployments compare a candidate model's live outputs with the incumbent's to catch behavioural regressions; documentation compliance is a governance process, not a runtime comparison. It is tempting because both support responsible release, and shadow deployment would be correct when assessing accuracy drift before full rollout.

Training data leakage.

Why it fails: Shadow deployments route a copy of live traffic to a candidate model and compare outputs, addressing behavioural divergence before promotion; training data leakage is handled by data governance controls. It is tempting because both concern model safety, and shadow deployment would be correct when validating prediction quality against production.

Hardware infrastructure costs.

Why it fails: Shadow deployments address the risk of a new model degrading production behaviour by running it alongside the live model and comparing outputs before promotion. Infrastructure cost is a deployment concern, but it is not the risk shadow deployment mitigates; cost management would be the focus when scaling serving capacity or choosing instance types.

Validating the hardware security keys.

Why it fails: Hardware security key validation protects cryptographic material and access, an infrastructure control unrelated to model behaviour. Feedback loops occur when predictions influence the data later used for retraining; identifying them needs drift, distribution and correlation monitoring across retraining cycles.

Checking for system uptime.

Why it fails: Uptime checks confirm infrastructure availability, revealing nothing about model behaviour or data. Feedback loops arise when model outputs re-enter training data and amplify bias; detecting them requires monitoring prediction distributions, input drift and outcome correlations over successive retraining cycles.

Analyzing the training pipeline latency.

Why it fails: Training pipeline latency measures how long jobs take to complete, an operational performance metric. Feedback loops emerge when model outputs contaminate subsequent training data, so detection requires monitoring prediction distributions, input drift and label correlation across retraining cycles, not pipeline timing.

Setting a high learning rate.

Why it fails: A high learning rate is a hyperparameter tuning concern affecting model convergence, not the integrity of the training data itself. Data quality risk covers issues such as missing values, mislabelled records, duplicates or sampling bias within the dataset used to train the model.

Using Python instead of R.

Why it fails: Python versus R is a tooling and implementation preference, unrelated to the condition of the data feeding the model. Data quality risk addresses dataset defects — missing fields, inconsistent formats, biased sampling or labelling errors — that degrade model outcomes regardless of language.

Using a GPU that is two generations old.

Why it fails: This is a performance efficiency issue, not a data quality issue.

It requires expensive API calls.

Why it fails: HITL's primary risk is human reviewer latency, inconsistency and fatigue degrading decisions, not API expense. API calls are billed per request regardless of whether a human reviews the output, so HITL adds no API cost. Expensive API calls would matter when choosing between model tiers or high-volume inference.

It causes the model to become slower.

Why it fails: HITL adds review latency to individual decisions, but the primary risk is inconsistent or biased human judgements degrading reliability and scalability, not model speed. Slower inference tempts because human review does add delay; that is a performance trade-off, not the core governance risk.

It forces the model to use a specific programming language.

Why it fails: HITL is an architectural pattern placing a human reviewer in the decision path; it imposes no programming-language constraint on the model. Models are served via APIs or containers in whatever language the platform supports. Language lock-in would be a genuine concern when selecting an inference runtime or SDK for deployment.

The user access logs for the dashboard.

Why it fails: Dashboard access logs record who viewed results, not how the model was trained, so they cannot trace provenance. They are tempting because access logs are a legitimate audit artifact for security and compliance reviews of who touched the system, but provenance demands training data sources, lineage and version history.

The model's current latency metrics.

Why it fails: Latency metrics measure runtime performance, not provenance; they record nothing about training data, lineage or model version, so they cannot evidence how the model was built. They are tempting because latency monitoring is genuinely useful for operational health and drift detection, but audit provenance requires the model card or lineage record.

The cloud provider's billing statement.

Why it fails: A billing statement records consumption and cost, not model lineage, so it evidences nothing about training data or versioning. It is tempting because billing data genuinely supports cost governance and chargeback reporting, but provenance audit trails require the model card, dataset lineage and training records instead.

To prevent unauthorized users from viewing the code.

Why it fails: Versioning tracks which model artefact produced which output, supporting reproducibility, rollback and audit; it grants no access control over code. It tempts because version control repositories often sit behind permissions, so the practice appears to restrict who can read source, which is authorisation, not versioning.

To reduce the amount of disk space used.

Why it fails: Model versioning tracks which artefact produced which output, enabling rollback and audit; storage reduction is a side effect of pruning old versions, not its risk objective. Disk-space savings would be the goal only when retention policy, not traceability, drives the requirement.

To increase the model's speed in production.

Why it fails: Model versioning tracks lineage, reproducibility and rollback across iterations; it does not tune inference latency, which is addressed by optimisation, quantisation or hardware scaling. It is tempting because versioning underpins deployment pipelines, where a specific build may be promoted for performance reasons — but the objective itself remains governance and traceability, not speed.

The base model cannot be used with custom datasets.

Why it fails: Custom datasets are precisely what transfer learning accommodates through fine-tuning or feature extraction on the pre-trained weights. The real risk is that the base model's original training data embeds biases or backdoors that propagate into the adapted model. Inability to use custom data would describe a frozen, non-adaptable model, not transfer learning.

The base model will always run out of memory.

Why it fails: Memory exhaustion is a deployment and sizing concern, not an inherent property of transfer learning; a pre-trained base model's weights are fixed and typically modest relative to training workloads. The genuine risk is inherited bias or poisoned features from the base model's original training data. Out-of-memory scenarios belong to inference capacity planning.

The base model requires a proprietary operating system.

Why it fails: Pre-trained models are distributed as portable weight files consumed by standard frameworks such as PyTorch or TensorFlow, independent of any operating system. The critical risk is provenance: unvetted base models may carry poisoned weights or licence restrictions. OS dependency would concern proprietary inference appliances, not transfer learning itself.

Infrastructure provisioning.

Why it fails: Infrastructure provisioning covers compute, storage and networking resources, not the vocabulary or tokenisation of training data, so unseen abbreviations cannot originate there. It is tempting because provisioning underpins every lifecycle stage, but it would be the answer if the failure involved capacity, configuration or resource availability instead.

Monitoring phase.

Why it fails: Declining accuracy on unseen abbreviations is a data or concept drift issue surfacing during monitoring, but the risk originates in the training data's coverage, not the monitoring stage itself. Monitoring is tempting because it detects the symptom, and would be correct for tracking live performance degradation.

Deployment phase.

Why it fails: Unseen abbreviations are a data-quality issue originating in data preparation, where vocabulary and tokenisation are defined, so the failure traces back to that stage rather than to deployment. Deployment is tempting because performance drops surface in production, but deployment covers release, serving and monitoring, not training-data coverage.