AI Risk Program Management practice questions

The brand of the coffee machine

Why it fails: Irrelevant.

The number of employees in the break room

Why it fails: Irrelevant.

Ignoring the vendor's feedback

Why it fails: Ignoring vendor feedback leaves third-party AI risks unassessed, since the vendor holds the model, training data and update details you cannot inspect directly. It is tempting because vendor questionnaires feel like marketing, yet in scenarios where you lack audit rights, vendor disclosures are the only evidence available for evaluating their controls.

Allowing the vendor to set their own rules

Why it fails: Letting the vendor define its own rules cedes governance to the supplier, so no independent assurance, contractual controls or monitoring over third-party AI risk exists. It is tempting because vendor-managed frameworks suit low-risk, commoditised tools where the provider already meets recognised standards and your organisation consumes the output without regulatory exposure.

Asking the vendor for their cafeteria menu

Why it fails: A cafeteria menu has no bearing on AI governance, model provenance or data handling, so it cannot mitigate third-party AI risk. It is tempting because vendor due-diligence questionnaires do request documentation, but those artefacts are security certifications, DPAs and model cards, not catering information.

The local weather station

Why it fails: Irrelevant.

The office landscaping team

Why it fails: Irrelevant.

The company's mascot

Why it fails: Irrelevant.

Conduct a one-time audit of all network traffic.

Why it fails: One-time audits are insufficient for ongoing risk management of shadow IT.

Restrict all internet access to AI platforms.

Why it fails: This is overly restrictive and discourages innovation.

Require managers to sign a liability waiver for their own AI projects.

Why it fails: Liability waivers do not manage the underlying risk to the organization.

multiple_choice

Why it fails: Waivers are not a risk management control.

Average model inference latency

Why it fails: Inference latency measures serving performance, not fairness; a biased model can respond quickly. It is tempting because latency is easy to instrument and track as an operational KPI, but bias indicators require disaggregated outcome data across demographic groups, which latency never captures.

Percentage of customer complaints regarding AI decisions

Why it fails: Complaint volume is a lagging indicator: harm has already occurred and reached customers before complaints are logged. It is tempting because complaints are readily available and clearly bias-related, but leading indicators such as disparate impact ratios or subgroup error rates surface bias before deployment.

Number of model updates per quarter

Why it fails: Update frequency measures maintenance cadence, not bias; retraining can introduce or remove bias, so the count carries no directional signal about fairness. It is tempting because frequent updates touch training data, but drift monitoring and disaggregated performance metrics are the genuine leading indicators.

Establish a top-down mandate that ignores business unit input to ensure uniformity.

Why it fails: A top-down mandate without business unit input disconnects AI risk thresholds from the operational realities ERM aggregates, so appetite statements cannot be calibrated or owned locally. It is tempting because uniform rules appear consistent, but ERM alignment requires business units to translate appetite into their own processes.

Focus solely on technical model performance metrics to define the organizational risk appetite.

Why it fails: Technical model metrics such as accuracy or drift describe model behaviour, not the enterprise risk dimensions (financial, regulatory, reputational) that ERM appetite statements quantify. It is tempting because model metrics are measurable and objective, but they cannot alone define organisational risk tolerance across the enterprise.

Create a separate AI-specific risk register that operates independently of the corporate risk register.

Why it fails: An independent AI risk register duplicates and diverges from corporate risk data, so AI risk appetite cannot be aggregated or escalated through ERM. It is tempting because a dedicated register gives AI risks focused visibility, but that works only when it feeds the corporate register rather than operating separately.

The speed of the development team's sprint cycles

Why it fails: Sprint cadence measures delivery tempo, not the likelihood or impact of chatbot harms, so it cannot calibrate risk tolerance. Sprint speed is relevant to release planning; risk appetite requires factors such as data sensitivity, model failure impact and regulatory exposure.

The total cost of GPU infrastructure

Why it fails: GPU infrastructure cost is a budget input, not a measure of risk likelihood or impact, so it cannot define acceptable risk. Cost belongs in financial planning; risk appetite for a chatbot turns on harm severity, data sensitivity and regulatory exposure.

Conducting annual risk assessments

Why it fails: Annual assessments produce a static snapshot, so metrics are stale for up to twelve months and cannot reflect model drift, new deployments or changing risk. It is tempting because periodic assessment satisfies governance and audit cycles, and it would be adequate where the AI system and its operating environment remain stable between reviews.

Setting static thresholds for model accuracy

Why it fails: Static accuracy thresholds cannot adapt as models drift, data shifts or the threat landscape evolves, so the metric stops reflecting current risk. Thresholds are tempting because they give simple pass/fail governance gates for a fixed release; they would suit a one-off model acceptance test, not an ongoing NIST-aligned dynamic programme.

Limiting AI deployment to offline systems

Why it fails: Restricting deployment to offline systems removes the continuous telemetry and live monitoring that dynamic risk metrics depend on, freezing measurement at deployment time. Offline operation is tempting for high-assurance or air-gapped environments where isolation itself is the control; there, static evaluation genuinely suffices.

Head of Legal

Why it fails: The Head of Legal addresses regulatory and contractual compliance, not enterprise risk appetite, so cannot arbitrate alignment against it. Including them is tempting because they assess AI liability and regulatory exposure, which suits compliance reviews; appetite alignment requires the enterprise risk function's mandate.

Chief Information Security Officer

Why it fails: The CISO owns security risk, a subset of enterprise risk appetite, so cannot authoritatively align AI capability to the whole appetite. Including them is tempting because they govern AI security controls and threat exposure, which suits security-focused committees; enterprise-wide appetite alignment sits with the risk owner.

Lead Data Scientist

Why it fails: The Lead Data Scientist owns model development and experimentation, not enterprise risk appetite, so cannot arbitrate alignment against it. Including them is tempting because they translate technical capability into business terms, which suits model design reviews; risk appetite alignment requires the risk function's authority.

The frequency of software patching for the underlying cloud infrastructure.

Why it fails: Patch frequency is an operational control for the hosting platform, not an AI-specific risk dimension; the assessment covers model purpose, training data, bias, transparency and affected stakeholders. It is tempting because unpatched infrastructure causes breaches, which belongs in IT vulnerability management rather than AI risk scoping.

The name of the vendor providing the cloud compute resources.

Why it fails: Vendor identity is a procurement and due-diligence detail; the AI risk assessment examines the model's purpose, data, outputs, affected groups and potential harms. It is tempting because supplier reputation affects third-party risk, which is assessed in vendor onboarding, not in the initial AI-specific risk evaluation.

The specific programming language used for the model development.

Why it fails: Programming language is an implementation detail that does not itself create or mitigate harm; the assessment examines use case, affected populations, data provenance and failure impact. It is tempting because language choice affects maintainability and library security, which matters in code review, not initial AI risk scoping.

AI model training data source list

Why it fails: A training data source list records provenance and inputs, not which AI outcomes are acceptable; the policy needs defined outcome criteria or risk tolerance. It is tempting because data provenance matters for AI risk, and would be correct for a data governance or bias-assessment section instead.

Vendor procurement list

Why it fails: A vendor procurement list names suppliers and purchasing status, giving no statement of which AI outcomes are acceptable or prohibited. It is tempting because procurement controls appear in AI governance, and would be correct for a third-party risk or supply-chain section rather than outcome guidance.

Technical architecture diagram

Why it fails: A technical architecture diagram depicts components and data flows, not the acceptable or prohibited outcomes the policy must state. It is tempting because architecture documentation supports AI governance, and would be correct when documenting system design or control placement rather than defining outcome boundaries.

Centralized control under the Chief Data Officer

Why it fails: Centralising control under the Chief Data Officer concentrates decision rights in one function, so business, legal, security and ethics perspectives are not embedded at each lifecycle stage. A cross-functional committee distributes that accountability. CDO-led control would fit narrower data governance mandates, not enterprise-wide AI risk integration.

External advisory board oversight

Why it fails: An external advisory board provides independent scrutiny but sits outside the delivery organisation, so it cannot enforce risk gates inside the AI development lifecycle. Integration requires internal, cross-functional accountability. External oversight would be correct for independent assurance or regulatory challenge, not for embedding controls in day-to-day development.

Automated AI risk assessment tools alone

Why it fails: Automated risk assessment tools generate findings but cannot own accountability, interpret context or embed risk review into each lifecycle gate. A governance committee needs human decision-makers with defined mandates. Tooling alone would be correct as a supporting control within a broader governance structure, not as the structure itself.

Using different platforms for each business unit

Why it fails: Separate platforms yield divergent scoring methodologies and data schemas, preventing consistent, comparable risk measurement across the five business units. It tempts because per-unit platforms accommodate differing tools and autonomy, and would be correct when units operate independently without any requirement for enterprise-wide risk aggregation.

Allowing each business unit to define its own risk metrics

Why it fails: Unit-defined metrics produce incompatible scales and definitions, so scores cannot be aggregated or compared across the five business units. It tempts because local metrics reflect each unit's context, and would be correct when tailoring thresholds within one unit rather than measuring risk consistently enterprise-wide.

Focusing only on the highest-risk model

Why it fails: Restricting measurement to a single highest-risk model leaves the other four business units unmeasured, so no comparable risk data exists across the portfolio. It tempts because triage prioritises the greatest exposure, and would be correct when resourcing remediation for one critical model rather than establishing consistent measurement across all units.

Frequency of model retraining

Why it fails: Retraining frequency records how often the model is refreshed, not whether its predictions stay accurate between refreshes. A model can be retrained often yet still drift. Reliability over time requires monitoring live accuracy, drift and error rates. Retraining cadence would be the correct KPI when governing MLOps pipeline throughput.

Number of training iterations

Why it fails: Training iterations measure optimisation effort during development, not ongoing reliability in production. Reliability over time requires monitoring live performance against ground truth, such as drift detection or accuracy decay. Iteration counts would be relevant when tuning convergence or comparing training runs.

Total cost of compute resources

Why it fails: Compute cost is a financial efficiency measure, not an indicator of whether the model's predictions remain dependable. Reliability over time needs production performance tracking, such as data drift or accuracy degradation metrics. Compute spend would be the right KPI when optimising infrastructure budget or scaling costs.

Restrict Legal from reviewing AI model outputs

Why it fails: Restricting Legal from reviewing outputs directly contradicts the coordination requirement, since explainability exists precisely to give auditors and legal reviewers visibility into how a model reached its decision. This approach suits security classification controls, where compartmentalising sensitive data from certain groups is the goal, not cross-team AI governance.

Let Legal define the technical parameters of the models

Why it fails: Legal lacks the technical grounding to specify model parameters such as feature weighting or architecture, so this cedes engineering decisions to a non-technical function. It is tempting because Legal legitimately owns regulatory interpretation and disclosure wording, and would correctly define explainability obligations when the task is translating compliance requirements into acceptance criteria.

Only document model logic when a lawsuit occurs

Why it fails: Documenting model logic only after litigation begins fails because explainability must be built into the model lifecycle, not reconstructed retrospectively when evidence is already needed. It is tempting as reactive legal documentation, which suits post-incident audit trails, but coordinating Legal and Data Science requires continuous, proactive explainability artefacts rather than lawsuit-triggered paperwork.

Signing contracts with AI vendors

Why it fails: Vendor contracts transfer some liability but establish no internal governance, inventory or controls, so the programme depends on suppliers rather than the organisation. It is tempting because procurement terms address third-party risk, and it would be correct as one component within a broader AI risk framework covering owned and purchased systems.

Automating all model retraining cycles

Why it fails: Automating retraining addresses model freshness, not risk governance; without documented controls, monitoring and accountability, automation simply scales unmanaged risk. It is tempting because continuous retraining supports performance and drift management, and it would be correct once an AI risk programme with defined policies and owners is already established.

Purchasing third-party AI auditing tools

Why it fails: Buying auditing tools supplies detection capability but no governance structure, ownership or repeatable process, so risk management cannot be sustained once tools change. It is tempting because tooling accelerates assessments, and it would be correct as an enabler layered onto an existing AI risk framework with defined policies and accountability.

System Administrator

Why it fails: System Administrators manage infrastructure, patching and access, not the content-governance decisions that intellectual property risk policies demand. The role is tempting because administrators enforce technical controls, yet IP adherence requires legal and editorial judgement, which sits with the AI governance or compliance owner who defines and reviews acceptable output.

Product Manager

Why it fails: Product Managers own roadmap, requirements and feature prioritisation, not the legal review confirming AI output respects IP policy. The role tempts because product owners often sign off releases, but IP adherence needs the governance or legal function that sets licensing and copyright rules, not the person defining what the product should do.

Data Engineer

Why it fails: Data Engineers build and maintain pipelines, storage and data flows, so they cannot adjudicate whether generated content infringes IP. The role is tempting because engineers implement the filters and metadata that support compliance, yet the accountability for IP risk policy rests with the governance or legal owner who defines permissible use.

They change too quickly to report

Why it fails: Reporting cadence is irrelevant; metrics can be logged and reported at any interval. The actual gap is scope: accuracy, precision and recall describe model behaviour, not fairness, privacy, security, drift or regulatory exposure. Metrics alone would suffice only for periodic model-performance reporting, not holistic risk management.

They are too difficult to calculate for small models

Why it fails: Calculation difficulty is unrelated; performance metrics are computable regardless of model size. Metrics such as accuracy or F1 capture only statistical behaviour on test data, omitting fairness, privacy, security, drift and governance exposure. Metrics alone would suffice only for narrow model-selection comparisons, not enterprise risk management.

They are too subjective

Why it fails: Performance metrics are quantitative and reproducible, so subjectivity is not the defect. They measure predictive quality on a dataset, not operational, ethical, legal or societal risk, which requires separate assessment. Metrics alone would be adequate only where the sole concern is comparative model accuracy under controlled conditions.

It should be ignored if below a specific performance threshold

Why it fails: Ignoring residual risk below a threshold leaves it unowned and unmonitored, so it cannot be escalated or reviewed. Thresholds are tempting because they filter noise in risk registers, but residual risk must still be documented and accepted by the risk owner regardless of magnitude.

It should always be zero for production systems

Why it fails: Residual risk is the exposure remaining after controls, so it is never zero in production; some inherent risk always persists. Zero is tempting as an ideal target, yet the correct treatment is comparing residual risk against appetite and treating or accepting it accordingly.

It should be formally accepted by the risk owner if it exceeds appetite

Why it fails: Acceptance applies when residual risk sits within appetite, not above it. Exceeding appetite demands treatment, escalation or rejection by governance. Formal acceptance is tempting because risk owners do sign off residual risk, but that sign-off occurs only once the exposure is tolerable.

IT Help Desk

Why it fails: The IT Help Desk resolves infrastructure, access and endpoint tickets, and holds no remit over model behaviour, datasets or fairness testing. It is the correct entry point for hardware, account or connectivity faults; bias incidents need the AI governance or data science function that owns the model.

Customer Support

Why it fails: Customer Support handles user-facing complaints and triage, but lacks authority over model training data, evaluation metrics or deployment pipelines, so it cannot investigate bias. It is the right first contact for customer-impacting service faults, not for model fairness incidents requiring data science and governance input.

External Regulators

Why it fails: External regulators are notified only after internal triage and remediation, not on first detection. Reporting bias externally before internal review breaches governance and may trigger penalties. This option suits formal regulatory disclosure once an incident is confirmed and assessed, not the initial coordination step.