CCNA Ai Risk Management Questions

Adversarial testing and threat modelling identify weaknesses before attackers exploit them, letting teams remediate design flaws proactively. This satisfies the proactive criterion by addressing risk at the design stage rather than reacting to incidents or breaches after impact.

Assigning a named owner and a review cadence converts an unmanaged residual risk into a governed, documented decision. Because the vendor gives no confidence score, the only reliable feedback loop is clinician override and miss data; routing that evidence to an accountable owner ensures the risk is formally accepted, mitigated, or the model retired, which is exactly what the committee asked for.

This approach applies the core ISACA principle of risk-based decision making: identify the gap, quantify its potential impact, apply compensating controls such as contractual attestations and monitoring, and limit exposure through a pilot. It neither ignores the risk nor abandons the business opportunity, keeping the decision auditable and reversible.

Training and inference data pipelines are core AI System components, so each demands its own threat assessment. They introduce distinct attack surfaces — data poisoning during training, prompt injection or model evasion at inference — that generic infrastructure review misses. Assessing them individually satisfies the stem's requirement to evaluate every AI System component separately.

Unlike rule-based chatbots with deterministic response trees, large language models generate outputs probabilistically, meaning they can produce responses that violate brand guidelines, leak information, or provide harmful advice. This non-determinism is the defining new risk of generative AI in customer-facing contexts and requires controls such as output filtering, guardrails, and human escalation paths.

Continuous monitoring of model performance metrics satisfies NIST AI RMF's MEASURE and MANAGE functions, detecting drift, degradation and emerging risks after deployment. It provides the ongoing evidence needed for GenAI risk frameworks, complementing governance and mapping activities rather than replacing them.

The operational exposure is driven by defective parts that pass inspection. The false negative rate directly measures that failure mode, and translating it into expected escaped defects per shift gives leadership a tangible production risk figure. This metric connects model behavior to business impact, which is what risk communication requires.

Training data collection and ingestion is where attackers can inject corrupted or mislabelled samples before any model exists, directly satisfying the stem's data poisoning constraint. Poisoning manipulates the learned decision boundary at its source, unlike inference-time attacks. Microsoft Entra ID controls access to these pipelines, limiting unauthorised dataset tampering.

Because the retailer cannot inspect the vendor's internals, the risk must be transferred and verified through the contract plus independent assurance. Prohibiting training on submitted content, requiring notification and deletion rights, and validating these through assessment creates enforceable obligations and evidence. This is the standard treatment for third-party AI risk where direct technical control is unavailable.

Strict schema validation rejects malformed or unexpected input before it reaches the model, blocking injection and parameter-tampering attempts. This directly secures the API endpoint by enforcing the expected data structure at the boundary, satisfying the requirement for a technical control on the model's exposed interface.

Model drift occurs when the statistical properties of input data diverge from those used during training, degrading predictive accuracy over time. This definition directly satisfies the stem's requirement by identifying performance decay as the consequence of shifting data distributions, capturing both the cause and effect that distinguish drift from static model error.

When full transparency is unavailable, risk management relies on compensating controls to reduce residual risk to an acceptable level. Contractual audit rights, independent local validation, and mandatory clinician review create layered assurance. Documenting the residual risk with an accountable owner and review cadence makes the acceptance a conscious, defensible governance decision rather than an implicit one.

Threat modelling and initial risk assessment belong to the Design phase because architectural choices, data flows and model selection are still fluid there. Performing this activity before build and deployment lets teams identify attack surfaces and residual risk early, when mitigation is cheapest and least disruptive.

Overfitting occurs when a model learns training data, including noise, too closely, so it performs well on training data but poorly on new inputs. The primary risk is therefore lack of generalisation to unseen data.

Post-hoc explanation techniques combined with a governance process that translates outputs into understandable reasons directly address the legal right-to-explanation requirement without discarding the model. Monitoring explanation fidelity ensures the explanations remain faithful to model behavior over time, making this a balanced and targeted risk management response.

Adversarial debiasing is a proven in-processing technique that trains the model to predict the target while simultaneously minimizing the ability of an adversary to predict the protected attribute from the model's outputs. This directly addresses learned biases that persist even without explicit demographic features. It is more effective than post-hoc removal of attributes because it actively optimizes for fairness during training.

Model inversion attacks often rely on confidence scores to reconstruct training data. By returning only the top-1 class, the attacker loses the fine-grained information needed to infer sensitive attributes. This reduces the attack surface significantly. However, it may not completely eliminate the risk if the attacker can query many times and observe changes, but it is a strong mitigation.

Pinning a specific model version and requiring advance notification contractually addresses the root cause: silent upstream changes. This gives the organization control over when to adopt a new model version and time to re-validate performance. It is the first step because technical monitoring alone cannot prevent an unannounced provider change; only contractual and version-control controls can enforce stability.

This control directly addresses both transparency and contestability. A plain-language explanation helps claimants understand why the recommendation was made, and a defined human review path gives them an actionable remedy. Together they support due process and align with regulatory expectations for automated decision support in insurance, while preserving the human decision-maker's authority.

A dedicated AI incident response process ensures harmful outputs are detected, triaged, contained, and reviewed systematically after deployment. It defines who acts, how quickly, and what containment options exist, such as disabling features or adjusting guardrails. This directly satisfies the requirement for a documented post-deployment handling process and integrates AI risk into existing operational resilience.

Variance-based global sensitivity analysis decomposes output variance into contributions from each input and their interactions, directly quantifying how much each feature drives instability across the entire input space. This gives the analyst a population-level measure of the sensitivity concern rather than anecdotal single-case observations, which is exactly what systematic quantification requires.

The MEASURE function in the NIST AI RMF involves selecting and applying quantitative and qualitative methods, metrics, and tools to analyze and assess AI risks. Applying fairness metrics and robustness testing directly measures the model's discriminatory and adversarial risk, which is exactly the analytical activity MEASURE prescribes after risks have been mapped.

RAG grounds generation in retrieved, verifiable source passages, so the model conditions its output on external evidence rather than relying solely on parametric memory. This constrains fabrication, directly reducing hallucination risk while preserving useful responses.

Recording known limitations and failure modes discloses where the model underperforms or produces unreliable output, so downstream consumers can apply human oversight and mitigation. Model Cards require this transparency element for accountable AI risk management.

AI red teaming is structured, adversarial probing of a deployed or in-development model to surface failure modes such as prompt injection, jailbreaks, bias and unsafe outputs before real attackers exploit them. This proactive, offensive testing distinguishes it from passive monitoring or post-incident forensics.

Structured due diligence is the foundational step for third-party AI risk. It examines how the vendor governs its models, protects data, manages lifecycle changes, and responds to incidents. This gives the risk manager a defensible basis for risk tiering, contractual requirements, and ongoing monitoring. It is proportionate, repeatable, and applicable before any technical deep dive.

Because the vendor will not permit model-level audits, the enterprise cannot reduce model risk at the source. A mandatory human approval gate is a compensating control that converts an automated decision into an assisted one, so an unverifiable model output cannot directly change a customer's credit limit. This preserves accountability and creates an auditable decision record while the model risk remains unmitigated.

XAI exposes how inputs drive outputs, letting auditors trace decisions to specific features. This satisfies the risk-management need to detect hidden bias and assign accountability, since opaque models otherwise prevent root-cause analysis or regulatory justification of adverse outcomes.

Encryption at rest protects stored training datasets from unauthorised disclosure if underlying storage media or backups are compromised, directly satisfying the stem's protection requirement. It defends against physical theft and snapshot exposure, complementing access controls and data-in-transit encryption.

Over-reliance means users treat AI outputs as authoritative without verification, so errors propagate unchecked into decisions. This satisfies the stem's focus on the primary risk: automation bias, where confidence in the tool replaces independent judgement, making incorrect or harmful AI-generated information the direct consequence.

Inherent risk is a function of impact severity and likelihood before controls. A triage tool that misorders critically ill patients can cause direct physical harm or death, which places it in the highest risk tier regardless of model architecture or data volume. This impact-centric view aligns with how AI risk frameworks classify systems by the severity of consequences to individuals and society.

A risk register must capture who owns the risk, how likely it is, its potential impact, and what is being done to mitigate it. These elements enable prioritization, tracking, and accountability. Without them, the register is just a list and cannot support effective risk management. This is the core information needed for any risk management framework, including AI-specific risks.

Performance drift without retraining or pipeline changes usually signals a shift in the relationship between inputs and outcomes, known as concept drift, or a change in input distributions, known as data drift. Comparing current input distributions against the training baseline localizes the cause and informs whether to retrain, recalibrate thresholds, or fix upstream data. This diagnostic step is the correct governance response before altering the model.

The AI Model Inventory provides a centralised register of every AI asset, capturing purpose, owner, data sources and risk classification. This single source of truth satisfies the oversight constraint by letting governance teams identify, assess and monitor AI risks across the organisation consistently.

Capturing a human judgment before the AI output is revealed prevents the anchoring effect that drives automation bias, because the planner's assessment cannot be unconsciously adjusted to match the model. Comparing the independent forecast to the AI output also generates data on where human and machine disagree, which is useful for evaluating both the model and the effectiveness of the control.

Side-channel attacks infer secrets from physical implementation characteristics rather than algorithm flaws. Timing and power consumption patterns leak information about internal computations, enabling an adversary to extract model parameters or data without directly accessing the model.

The DPA contractually specifies how the vendor processes, stores, retains and protects personal data, including subprocessors and breach obligations. It therefore gives the clearest evidence of data handling practices for third-party AI risk assessment, unlike marketing or generic security documentation.

Independent verification of the vendor's model performance tests outputs against your own benchmarks, confirming accuracy claims rather than trusting vendor-reported metrics. This satisfies the third-party AI vendor risk requirement by validating model behaviour objectively before and during use.

This approach follows the risk management cycle: identify and measure the harm, document it with accountable ownership, apply appropriate mitigation, and monitor continuously for regression. Fairness assessment across groups surfaces the actual disparity, mitigation techniques like reweighting or post-processing target the mechanism, and ongoing monitoring ensures the treatment remains effective as data and behavior evolve.

Disclosure of training data sources directly satisfies the provenance constraint, revealing whether data was licensed, scraped or synthetic. This determines bias exposure, intellectual-property liability and regulatory compliance under frameworks such as the EU AI Act. Without it, downstream risk assessments cannot quantify inherited model risk or validate Microsoft Entra ID access governance assumptions.

Poor performance on a protected-class subgroup signals possible underrepresentation or imbalance in the training data. Auditing representation and training balance for that subgroup is the diagnostic step that precedes mitigation, letting you confirm whether the disparity stems from data rather than the model architecture itself.

A PII detection and redaction layer acts as a runtime control that scrubs sensitive identifiers before they enter the model context and before summaries are returned to agents. This directly mitigates the risk of PII appearing in outputs, regardless of whether it originated from the current session or a retrieved document. It complements session isolation by catching residual sensitive data.

Membership inference attacks determine whether a specific record was part of the training set. Success reveals that an individual's data was used for training, exposing sensitive information about those data subjects and potentially their attributes.

Model stealing is an extraction attack where an adversary queries or accesses a deployed model to reconstruct its parameters or weights, producing a functionally equivalent copy. This enables intellectual property theft and removes the need to train the model.

The model's training data no longer reflects the current patient population and treatment environment, so its predictions degrade. This is classic data or concept drift: the statistical relationship between inputs and outcomes has shifted. The clinicians' observation that alerts no longer match judgment is a direct symptom. Addressing it requires monitoring, retraining, or recalibration against current data.

The gap is missing documentation of why each feature is weighted and how the model may be used. Completing the model card and recording feature rationale, intended use, and limitations restores accountability and traceability, which are prerequisites for every other risk treatment. It also makes the model's residual risk visible to the business owner before further security testing or monitoring investments are justified.

The drop in sensitivity is localized, so a targeted response is appropriate. Adjusting decision thresholds at the affected site can restore sensitivity, and enhanced monitoring provides early warning of further degradation. Planning retraining with more representative data addresses the root cause. This approach treats the risk proportionally while maintaining clinical benefit where the model remains safe.

Robust statistical aggregation, such as trimmed means or median-based updates, dilutes or discards malicious contributions before they alter the shared model. This satisfies the stem's collaborative-learning constraint, where many participants submit updates and individual poisoned gradients must be neutralised.

A risk inventory should capture every system whose outputs can materially affect stakeholders or operations, regardless of whether it is generative or predictive, built in-house or procured. This impact-based scope ensures governance resources are directed where harm is possible and supports regulatory reporting. It also enables dependency and concentration analysis across the AI portfolio, which is a core expectation of AI risk management.

Quantified thresholds on named AI risk metrics can be measured continuously by automated monitoring and compared against board-approved tolerances, producing breach alerts rather than retrospective opinions. Mapping thresholds per use case respects differing criticality, and pipeline-fed dashboards let the committee see appetite utilization in near real time. This converts risk appetite from a static statement into an operational control with defined escalation triggers.

AI model transparency provides stakeholders with documented insight into a model's training data, logic, limitations and performance metrics, enabling informed evaluation of its safety and reliability. This directly satisfies the risk management framework's requirement for accountable oversight, letting decision-makers assess residual risk before deployment rather than accepting opaque outputs on trust.

The Map function documents context, purpose, dependencies, and affected parties. That context determines which risks are relevant and what metrics, thresholds, and tolerances are meaningful. Without it, measurement produces numbers disconnected from business impact. Sequencing Map before Measure ensures that testing and monitoring address the risks that actually matter to the model's deployment context, which is why the framework treats mapping as the prerequisite for meaningful measurement.

Defined AI ethics and safety policies establish the boundaries, accountability and acceptable-use rules that govern AI systems. They satisfy the stem's requirement by providing the normative framework against which AI development, deployment and monitoring are directed and assessed within an AI governance programme.

The core risk is that confidential prompts and outputs could become part of the vendor's training corpus. A no-retraining clause, paired with technical evidence such as tenant isolation or zero-retention API configurations, directly mitigates that risk. Contractual language alone is weak unless supported by verifiable technical controls, so combining both is the most defensible action.

Shared model updates can encode training data gradients; an adversary or aggregator can invert them to reconstruct sensitive client records. This gradient leakage is the primary privacy risk in federated learning, distinct from transport or availability concerns.

A bias audit measures model outcomes across protected groups to detect disparate impact, where one population receives systematically worse results than another. It satisfies the objective of surfacing discriminatory effects before deployment, rather than explaining predictions or securing training data.

Verifying the downloaded model's hash against the vendor's published checksum manifest detects any tampering introduced during transfer or hosting, directly addressing poisoning before integration. This cryptographic integrity check confirms the artefact matches the trusted original.

Clearly defined roles establish named ownership for each model, so every decision, output and failure maps to a responsible party. This satisfies accountability's core requirement: a traceable human answerable for the system's behaviour. Without assigned owners, governance collapses into diffuse responsibility, and no audit trail can attribute actions to anyone.

A Model Evaluation Report documents benchmark and behavioural testing results for a third-party AI model, exposing unsuitability or unexpected behaviour before deployment. That evidence directly addresses the risk of adopting a model whose performance or outputs do not match the intended use case.

Distribution shift analysis directly quantifies how the statistical properties of the target deployment population differ from the training population, which is the primary driver of performance degradation when a model is moved to a new environment. This evidence-based assessment allows the committee to estimate the magnitude of risk before deciding on mitigations, retraining, or deployment restrictions.

Continuous change is a core AI-specific characteristic. Models can drift, be updated via retraining or fine-tuning, or adapt through new inputs, meaning a risk assessment performed at deployment may be invalid weeks later. This necessitates continuous monitoring, re-evaluation triggers, and version governance rather than the periodic assessment cycles typical of conventional IT risk.

Indirect prompt injection occurs when the agent ingests external content, such as web pages or documents, that embeds adversarial instructions. Those instructions are then processed as if legitimate, overriding the developer's system prompt and hijacking agent behaviour.

Capturing override reasons turns human oversight into a feedback signal. It documents that a human actively evaluated the output, and the aggregated reasons reveal systematic weaknesses such as a subgroup where the model consistently errs. Feeding those records into periodic review closes the loop between human judgment and model improvement or risk reassessment.

The core failure is that the firm has no contractual or technical mechanism to learn about upstream model changes, leaving it unable to revalidate the system for regulated use. This is a third-party supply chain governance gap: without change notification clauses and version pinning, the firm's own risk assessments become stale the moment the vendor silently updates the model.

Enterprise risk management evaluates risk as a function of impact and likelihood, adjusted for controls. Applying that structure to models means assessing the financial consequence of a wrong decision, the probability of failure, and the strength of mitigating controls. This produces a risk rating that the board can compare across models and against other enterprise risks.

Sentiment drift against a frozen baseline is a practical, vendor-agnostic early warning signal. It does not require source code or training data, it can be automated across a large volume of summaries, and anomaly routing gives the AI risk owner an actionable queue. This directly addresses the regulator's question about how subtle issuer bias would be detected and escalated.

F1 Score is the harmonic mean of precision and recall, so it quantifies the trade-off between false positives and false negatives. From a risk perspective, it reveals whether the model balances missed detections against spurious alerts, informing residual risk decisions.

Comparing production input distributions against training distributions directly quantifies drift, satisfying the stem's requirement for an initial risk assessment step. This statistical comparison reveals which features have shifted and by how much, providing the evidence needed before deciding on retraining, monitoring thresholds, or mitigation. It precedes remediation, making it the correct first action.

This directly addresses the intersection of AI risk and cross-border regulatory compliance. The assessment must verify how the vendor processes, stores, and transfers data across jurisdictions, what visibility the manufacturer has into model behavior, and whether the vendor can meet the data protection requirements of all 12 countries. These factors determine legal exposure and operational continuity.

Defining lifecycle stages with governance gates ensures models are reviewed, validated and approved before deployment and monitored afterwards. This satisfies the MRM policy requirement by assigning accountability at each transition, covering development, validation, deployment and ongoing monitoring rather than leaving risk oversight undefined.

Data privacy concerns protecting personally identifiable information throughout collection, processing and storage, whereas model security concerns preserving the integrity, availability and behaviour of the model itself against tampering, poisoning or extraction. The two address different assets and threat vectors.

Bias arises from unrepresentative or skewed training data, so periodic audits detect disparate outcomes while dataset balancing corrects under-representation across groups. Together they address bias at its data and monitoring roots, satisfying the stem's management requirement continuously.

Human-in-the-loop places a person in the decision path so they can review, approve or override model outputs before consequential actions occur. This directly satisfies the need for oversight and intervention on critical decisions, catching errors that automated thresholds alone would miss.

Outcome-based performance monitoring directly reflects whether the model still delivers value and avoids harmful errors. Rising false dispatches waste field resources and may indicate the model is misclassifying faults. Tracking accuracy and error rates against pre-defined thresholds provides an objective trigger for re-validation or retraining, making it a core monitoring signal.

Adversarial training injects deliberately perturbed samples into the training set, hardening the model's decision boundaries against pixel-level manipulation. This is a technical control applied within the model itself, satisfying the stem's requirement to counter pixel perturbation attacks at the algorithm level.