CCNA AI Operations Questions

Fine-tuning updates the model's weights using new data, so any bias present in that dataset becomes embedded in the model's learned parameters. A fresh bias audit is therefore required because the original pre-training audit cannot detect biases introduced by the fine-tuning data.

Adversarial training hardens the model against perturbed inputs by exposing it to crafted examples during training, while input filtering rejects suspicious samples at inference time. Together they directly counter the adversarial input constraint, unlike detection-only or post-hoc approaches that cannot prevent manipulation of the inference pipeline.

Model Monitor continuously evaluates endpoint traffic against a baseline and detects both data drift (shifts in input feature distributions) and model drift (degradation in prediction quality), satisfying the requirement to catch changes in incoming data and model behaviour over time.

SLAs define the availability, latency and support guarantees the deployment must meet, so they directly constrain whether a hosting model can satisfy business expectations. This satisfies the selection criteria by quantifying acceptable performance and uptime before committing to an architecture.

Silent failures in retraining mean the model becomes stale without anyone knowing. Automated monitoring and alerting would detect failures and trigger remediation, ensuring the model is updated as intended. This is a fundamental operational control for any automated pipeline and directly addresses the identified gap.

Backtesting replays historical data against the model to confirm it would have produced the same outputs and accuracy over past periods. This verifies historical model performance, satisfying the audit's need to evidence consistent behaviour before deployment rather than projected or live results.

A deployment pipeline automates the sequence of steps that promote a trained model from training into production, including validation, packaging and release. This automation satisfies the MLOps goal of repeatable, reliable delivery, replacing manual handoffs that cause inconsistency and slow iteration.

Recording the deployed model version or ID gives traceability, letting teams map any prediction or incident back to the exact artefact in production. This satisfies the deployment log's audit requirement, since without the version identifier rollback and reproducibility become guesswork.

Concept drift occurs when the statistical relationship between input features and the target variable changes over time, so the same inputs no longer map to the same outputs. It is distinct from data drift, which concerns input distribution shifts, making concept drift one of the two common drift categories.

The fixed vocabulary cannot handle new slang and abbreviations, leading to out-of-vocabulary tokens that degrade model performance. Regular retraining with a rolling window of recent data would update the vocabulary and allow the model to learn new language patterns. This is a fundamental operational control for NLP models in dynamic environments, ensuring the model remains relevant and safe.

Inference latency is most often caused by CPU or memory throttling, so container resource limits and requests should be checked first. Insufficient limits force queuing and slow responses, directly producing the high latency the auditor observed.

A feature store provides a centralised repository that both stores computed features and serves them consistently to training and inference, eliminating duplicated engineering and preventing training-serving skew. This centralised serving and storage is precisely the function the question asks about.

System latency monitoring is essential because model observability requires tracking operational performance alongside prediction quality. Latency directly affects user experience and can signal infrastructure degradation, queue saturation, or resource contention in production inference pipelines. Without it, teams cannot detect performance regressions that occur independently of model accuracy, leaving a critical observability gap unaddressed.

Overfitting occurs when a model memorises training data, fitting noise rather than generalisable patterns. The model scores highly on training data but performs poorly on unseen validation data, producing the characteristic gap of high training accuracy alongside low validation accuracy.

An AI Model Gateway sits between applications and multiple model providers, centralising authentication, rate limiting, prompt logging and content filtering. This brokers every inference request through one controlled point, satisfying the need for consistent governance and security across disparate AI backends rather than per-application credentials.

An AI audit trail records model inputs, outputs, decisions and version changes, creating traceable evidence of how systems behave. This directly delivers transparency and accountability, satisfying the stem's requirement by enabling investigators to reconstruct events, detect bias, and demonstrate compliance with governance obligations.

Workspace Access Control (IAM) holds the role assignments governing who can read or write to the model registry. Auditors verify these assignments to confirm only authorised CI/CD service principals hold registry permissions, satisfying the restriction requirement.

Orchestrator task execution logs record each pipeline step's start, completion, inputs, outputs, and errors, so they pinpoint exactly where the run failed. They satisfy the diagnostic requirement by showing task-level sequencing rather than only model or data-level detail.

A rollout plan governs the controlled, staged release of an AI solution to users, moving it from pilot to production in manageable increments. This satisfies the stem's goal by ensuring adoption, monitoring and risk mitigation occur progressively, rather than exposing the entire user base to an untested deployment at once.

Distribution shift means the learned mapping no longer matches live inputs, so retraining on recent data restores alignment. Monitoring detects drift; retraining is the operational process that updates weights. Without it, accuracy degrades silently as the input distribution diverges from training.

Feature attribution drift applies SHAP values to compare each feature's contribution between baseline and current data, revealing changes in how features influence predictions. Monitoring it satisfies the requirement to detect feature attribution shift for structured datasets in SageMaker Model Monitor.