AI0-001 AI Implementation and Operations Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✓

  Establish a shadow deployment that mirrors live traffic to the new model and compares its decisions with the incumbent without affecting customers.

  Why this is correct

  Shadow deployment exposes the new model to real production traffic distributions while isolating customers from its decisions, revealing performance and latency issues that offline evaluation misses. Comparing shadow decisions with the incumbent's output surfaces disagreement patterns and edge cases, providing evidence to approve or block the release before any user impact occurs.

• ✓

  Define alert thresholds on prediction distribution, latency, and error rate that trigger automated rollback to the previous model version.

  Why this is correct

  Post-deployment safety depends on detecting degradation quickly and reverting without manual delay. Alerts on prediction distribution catch silent drift, while latency and error-rate thresholds catch serving failures. An automated rollback path limits customer exposure to a bad model version and provides a deterministic recovery action that the operations team can rely on during incidents.

• ✗

  Increase the model's complexity by adding more layers until offline AUC reaches 0.99.

  Why it's wrong here

  Chasing a very high offline AUC encourages overfitting and does not improve operational safety. A more complex model is harder to explain, slower to serve, and more prone to unstable behavior on shifted inputs. The scenario asks for deployment safety and rapid detection, which complexity increases do not deliver and can actively undermine through latency and interpretability costs.

• ✗

  Publish a model card and require the fraud operations team to acknowledge it before go-live.

  Why it's wrong here

  A model card improves transparency and helps stakeholders understand intended use and limitations, but acknowledgment is a documentation step rather than a mechanism that detects or mitigates production failures. It does not compare live behavior, trigger rollback, or catch drift, so it cannot satisfy the requirement for operational safety and rapid problem detection after release.

• ✗

  Retrain the model weekly on the most recent fraud data regardless of any observed performance change.

  Why it's wrong here

  Unconditional weekly retraining introduces churn, complicates rollback because every week produces a new artifact, and can degrade performance if recent data contains label noise or a temporary fraud pattern. Retraining should be triggered by monitored evidence such as drift or performance decay, not by a fixed cadence that ignores whether the current model remains effective.