Hard Difficulty Questions

Requirements to publish all internal vulnerabilities publicly.

Why it fails: Publishing internal vulnerabilities publicly exposes exploitable weaknesses to attackers and breaches coordinated disclosure norms, undermining the containment and notification phases of an AI incident response plan. It is tempting as a transparency gesture, but controlled internal reporting and regulator notification are the correct choice.

Automated deletion of all logs.

Why it fails: Deleting all logs destroys the forensic evidence an AI incident response plan depends on for root-cause analysis, timeline reconstruction and regulatory reporting, directly contradicting containment and recovery objectives. It is tempting as a blunt "erase the breach" measure, but log retention and integrity, not deletion, are the correct choice.

It is the slow down of the model due to heavy load

Why it fails: Model stealing is an adversary reconstructing a model's functionality or parameters through repeated queries, causing intellectual property loss and enabling further attacks; heavy load is a denial-of-service or resource-exhaustion concern. It is tempting because both degrade service, so performance symptoms get misread as the incident.

It is the unauthorized access to the server room

Why it fails: Model stealing is query-based extraction of a model's parameters or behaviour through its inference API, not physical intrusion. It is tempting because unauthorised server room access is a genuine security incident, but it belongs to physical security controls, whereas model stealing is an AI-specific confidentiality attack on the deployed model.

It is the accidental deletion of model files

Why it fails: Accidental deletion is a data-availability or backup failure, not an adversarial extraction of model parameters, so it cannot constitute model stealing. It is tempting because deletion of model artefacts is genuinely a security incident under asset-management controls, and would be the right answer if the question asked about integrity or availability loss rather than confidentiality.

Ensuring the model's code repository is version-controlled and changes are logged.

Why it fails: Version control is important for reproducibility and auditability, but it does not monitor the model's runtime behavior or data. It cannot detect drift because drift is a data and performance phenomenon, not a code change. While valuable for governance, it is not the most critical element for detecting model drift in production.

Recording the number of API calls made to the model each day.

Why it fails: API call volume indicates usage but does not reveal whether the model's predictions remain accurate or relevant. A model can be called frequently while silently drifting. This metric alone cannot detect changes in data patterns or decision quality, so it is not critical for identifying model drift in a supply chain context.

Reviewing the model's training loss and accuracy metrics on the original test set.

Why it fails: Original test set metrics reflect performance at a fixed point in time and do not capture changes in live data. They cannot detect drift because the test set is static. Continuous monitoring requires comparing live performance to expected behavior, not re-evaluating on historical data that may no longer represent current conditions.

aiplatform.metadata.read

Why it fails: aiplatform.metadata.read grants read access to Vertex AI Metadata Store artefacts and has no bearing on endpoint deployment. It is tempting because metadata visibility supports governance auditing, but preventing unauthorised public endpoint deployment requires denying aiplatform.endpoints.deploy, the permission governing endpoint deployment.

aiplatform.jobs.create

Why it fails: aiplatform.jobs.create governs submitting training or batch prediction jobs, not deploying a base model to a public endpoint. It is tempting because custom training jobs often precede deployment, but restricting public endpoint deployment requires denying aiplatform.endpoints.deploy, which controls endpoint creation and deployment.

aiplatform.models.upload

Why it fails: aiplatform.models.upload governs importing model artefacts into the Vertex AI Model Registry, not deploying a base model to a public endpoint. It is tempting because uploading models is a prerequisite step in many workflows, but blocking public endpoint deployment requires denying aiplatform.endpoints.deploy, which controls endpoint deployment.

Ignoring all software updates.

Why it fails: Ignoring software updates leaves known vulnerabilities unpatched across the AI lifecycle, directly increasing risk. It is tempting because freezing changes can feel like stability, and deferring updates is defensible during a validated model's controlled production window, but blanket ignoring is never a risk control.

Bypassing security reviews for speed.

Why it fails: Bypassing security reviews removes the control that catches unsafe model behaviour, data handling flaws and compliance gaps before deployment. It is tempting because it accelerates delivery, and a lightweight review may be justified for low-risk internal prototypes, but bypassing reviews entirely is not risk management.

Hardening the physical security of the server room

Why it fails: Physical server-room hardening protects hardware housing, not the training supply chain's data, code and model artefacts, which are compromised through repositories, pipelines and third-party datasets. It is tempting because physical controls are foundational for on-premises infrastructure, and would be correct where training hardware sits in an unsecured facility.

Increasing the number of neurons in the model

Why it fails: Neuron count is a model capacity hyperparameter affecting accuracy and compute cost, not a security control over training data, code or weights. It is tempting because larger models are associated with capability, and tuning architecture is correct when the goal is predictive performance rather than supply-chain integrity.

It uses a secondary model to detect malicious input.

Why it fails: Adversarial training augments the training set with adversarial examples so the model learns to classify them correctly; it does not add a separate detector model. It is tempting because detector-based defences are a recognised evasion countermeasure, but that describes adversarial detection, a distinct technique from adversarial training.

It restricts input to a whitelist of authorized users.

Why it fails: Whitelisting authorised users is an access-control mechanism, unrelated to how adversarial training alters model weights. It is tempting because restricting inputs genuinely reduces attack surface, but adversarial training works by retraining on perturbed examples, not by filtering who may submit input.

It adds noise to the weights of the model.

Why it fails: Adding noise to weights is regularisation (weight perturbation), not adversarial training; it hardens the model against random parameter shifts, not crafted evasion inputs. It is tempting because both techniques perturb the model during training to improve robustness, and weight-noise injection would suit a scenario targeting overfitting or unstable gradients rather than adversarial examples.

Access to all employees' private emails.

Why it fails: Blanket access to employees' private emails breaches privacy and data-protection principles, which governance exists to constrain. It is tempting because monitoring can surface misuse, yet lawful, proportionate oversight of work systems — not unrestricted private inbox access — is what governance frameworks actually require.

A mandate to use the most expensive GPUs available.

Why it fails: Hardware cost is a procurement decision, not a governance element; AI governance requires accountability structures, risk management and policy oversight. It is tempting because compute capacity genuinely affects AI capability, so expensive GPUs can appear linked to control, though they address performance rather than governance.

Enable data augmentation

Why it fails: Data augmentation alters training inputs to improve generalisation, not the information returned per query, so it does nothing to stop cloning. It is tempting because augmentation is a common robustness technique. It would be correct when addressing overfitting or limited training data, not adversarial extraction via a public API.

Use a larger model architecture

Why it fails: A larger architecture does not prevent extraction; it may even yield a higher-fidelity clone for the same query budget. It is tempting because capacity is often assumed to confer resistance. It would be correct when the requirement is improved accuracy or representational capacity, not defence against query-based cloning.

Rotate API tokens every 30 days

Why it fails: Rotating API tokens every 30 days limits credential theft, not query-based model cloning; an adversary with a valid token still extracts the model within that window. It is tempting because rotation is a standard API hardening control. It would be correct against stolen or leaked long-lived credentials, not against systematic extraction queries.

Increased latency for API requests

Why it fails: Latency reflects runtime serving performance, whereas poisoning corrupts training data or weights, so symptoms surface as degraded accuracy, skewed predictions or anomalous loss curves during training. Latency monitoring belongs to operational availability and capacity troubleshooting.

Increase in storage costs for the model weights

Why it fails: Storage cost tracks volume and retention, not weight integrity; poisoning alters learned parameters, showing as accuracy drift, unexpected outputs or anomalous validation metrics. Storage growth would be a legitimate capacity-planning concern, unrelated to training-phase tampering.

Rely on the vendor's claims that the model is fair and unbiased.

Why it fails: Vendor claims are not a substitute for independent validation and monitoring. The organization remains accountable for the model's impact, and biases can be context-specific. Trusting vendor assertions without verification exposes the organization to ethical, legal, and reputational risks, and does not provide a control for ongoing detection.

Remove all demographic data from the training set to prevent bias.

Why it fails: Removing demographic data does not eliminate bias and can even worsen it by preventing the model from learning fair representations. It also makes it impossible to measure disparities across groups. This approach is known as fairness through unawareness and is generally ineffective for complex models.

Conduct a one-time bias audit before deployment and document the results.

Why it fails: A one-time pre-deployment audit is insufficient because bias can emerge or change over time due to data drift, population changes, or feedback loops. Ongoing monitoring is necessary to detect new biases. While pre-deployment audits are valuable, they do not satisfy the requirement for continuous detection and mitigation.

The office lease agreement.

Why it fails: An AI audit trail evidences data lineage, model versions, training inputs, validation results and decision logs; a property lease has no bearing on model behaviour or accountability. Lease documents belong in facilities or financial audits, where this option would be the right artefact to cite.

A copy of the employee handbook.

Why it fails: An AI audit trail needs model cards, dataset provenance, hyperparameters and inference logs; the employee handbook covers conduct and HR policy, not algorithmic traceability. Handbooks are relevant to HR compliance reviews, which is the scenario where this document would be the correct choice.

Data masking

Why it fails: Data masking obscures sensitive values in datasets for privacy or test use; it alters content rather than producing a cryptographic digest. Authenticity of an S3 model file requires a signature or hash comparison, such as a checksum or digital signature. Masking would be chosen when sanitising production data before sharing it.

Model quantization

Why it fails: Quantization shrinks a model's weights to lower precision for faster inference and reduced memory, leaving file integrity untouched. Verifying authenticity needs a cryptographic hash or signature checked against a trusted value. Quantization is chosen when optimising model performance for deployment on constrained hardware.

Base64 encoding

Why it fails: Base64 encoding is a reversible transport representation, providing no keyed integrity or origin proof, so it cannot establish a model file's authenticity. It is tempting because encoded files look obfuscated, and would be correct when embedding binary model data inside JSON or XML payloads.

LIME (Local Interpretable Model-agnostic Explanations).

Why it fails: LIME creates local surrogate models to approximate predictions, but its explanations are not guaranteed to be additive or consistent across runs. The sampling process introduces instability, and the surrogate may not faithfully represent the original model. For regulatory adverse action explanations, this inconsistency and lack of theoretical guarantees make it less suitable than a Shapley-based method.

Partial dependence plots.

Why it fails: Partial dependence plots show the average marginal effect of a feature on predictions across the dataset. They are global and can obscure heterogeneous effects for individual applicants. They also assume feature independence, which is often violated in credit data. They do not provide the per-instance, additive explanation needed for adverse action notices.

Global permutation feature importance.

Why it fails: Permutation importance measures how much each feature affects overall model performance across the dataset. It is a global, not local, measure and cannot explain why a specific applicant was denied. Regulators require reasons for the individual decision, so a dataset-level ranking does not satisfy the requirement and may even mislead if feature interactions vary by instance.

Dissolve the central AI risk committee and assign all AI risk decisions to each business unit's existing operational risk function.

Why it fails: Dissolving the central committee removes the enterprise-wide view needed to detect cross-business-unit AI risk concentration and inconsistent treatment. Operational risk functions typically lack AI-specific expertise in areas such as model evaluation, prompt security, and data provenance, so decisions would become inconsistent and the firm would lose the ability to set and enforce enterprise AI risk appetite.

Keep the central committee as the sole approver but add more staff and increase meeting frequency to clear the backlog.

Why it fails: Adding staff and meetings increases throughput but does not change the fundamental design where every pilot, including low-risk ones, must queue for central approval. It also scales cost linearly with AI adoption and leaves the bottleneck in place for future growth. The scenario asks for a governance design change, not a capacity increase within the existing design.

Allow business units to self-certify all AI pilots as low risk and proceed without notifying the central committee.

Why it fails: Self-certification without notification eliminates the central committee's visibility and its ability to validate risk classification. Business units have an incentive to under-classify risk to move faster, and without notification the firm cannot detect misclassification, aggregate exposures, or intervene before harm occurs. This removes oversight rather than redesigning it.

Remove all demographic data from the training set to eliminate the need for explanations.

Why it fails: Removing demographic data does not eliminate the legal right to an explanation and can even worsen fairness by masking proxy variables. The jurisdiction requires explanations for decisions, so reducing data does not satisfy the obligation. This option misunderstands the nature of the legal requirement and could increase discrimination risk through proxies.

Replace the model with a simpler, inherently interpretable model such as a logistic regression or decision tree, accepting a possible accuracy trade-off.

Why it fails: Swapping to an interpretable model may satisfy explainability but is a drastic architectural change that could harm screening accuracy and business value. It is not necessarily the most appropriate response when post-hoc explanation methods or governance controls could satisfy the legal requirement while preserving model performance. The question asks for the best response, and a wholesale replacement is disproportionate without first exploring less disruptive options.

Document the model's architecture and training procedure in detail and provide that documentation to rejected candidates upon request.

Why it fails: Technical documentation of architecture and training procedure is not a candidate-understandable explanation and would not satisfy a right-to-explanation law. Such disclosure may also expose intellectual property and confuse non-technical recipients. The legal requirement is about explaining the individual decision, not about disclosing model internals.

Implement a post-processing step that adjusts the model's decision threshold for each demographic group to achieve statistical parity.

Why it fails: Post-processing threshold adjustment can achieve a fairness metric, but it may not address the root cause of bias in the model's learned representations. It can also lead to legal and ethical concerns if it results in different treatment based on group membership. While sometimes used, it is less effective than in-processing techniques that prevent the model from learning biased patterns in the first place.

Increase the size of the training dataset by collecting more historical loan data to improve the model's overall accuracy.

Why it fails: Adding more historical data will likely reinforce existing biases if the data reflects past discriminatory practices. Accuracy improvement does not equate to fairness improvement; a more accurate model can be more consistently biased. This approach fails to address the underlying issue of biased patterns in the training data and may exacerbate disparate impact.

Remove all demographic attributes from the training dataset and retrain the model.

Why it fails: Removing demographic attributes does not eliminate bias because proxy variables (e.g., zip code, income patterns) can still encode group membership. This approach, known as fairness through unawareness, is widely recognized as insufficient. The model may continue to produce discriminatory outcomes because the underlying historical biases remain in the data and are learned through correlated features.

Membership inference attack testing

Why it fails: Membership inference attacks determine whether a specific data record was part of the training set, but they do not directly identify if sensitive information is leaked in model outputs. While related to privacy, this technique assesses the model's vulnerability to inference rather than detecting actual leakage in generated text. For detecting leakage in outputs, a different approach is needed.

Differential privacy during training

Why it fails: Differential privacy adds noise to training to prevent the model from memorizing individual data points, reducing leakage risk. However, it is a preventive control applied during training, not a detection technique for outputs. The scenario asks for identifying leakage in generated outputs, so a detection method is required.

Output watermarking

Why it fails: Output watermarking embeds a hidden pattern in generated text to trace its origin, not to detect sensitive information leakage. It is useful for provenance and copyright, but it does not scan for or flag confidential data. Implementing watermarking would not satisfy the requirement to identify when the model reveals training data secrets.

Homomorphic encryption

Why it fails: Homomorphic encryption allows computations on encrypted data, but it is typically used for inference or specific cryptographic operations, not for training a model with a formal privacy guarantee. It does not prevent re-identification from the model itself; it only protects data during computation. Applying it to training is computationally expensive and does not address membership inference.

Data masking

Why it fails: Data masking replaces sensitive values with fictitious but realistic data. While it reduces direct exposure, it does not provide a formal privacy guarantee against re-identification from the model. An attacker with auxiliary information could still infer whether a specific individual was in the training set, especially if the masking preserves statistical patterns.

Secure multi-party computation

Why it fails: Secure multi-party computation enables multiple parties to jointly compute a function without revealing their inputs. It protects data during the training process but does not provide a guarantee that the final model cannot be used to re-identify individuals. Once the model is trained, it may still leak information about the training data.

Store the de-identified training corpus in the same object bucket used for model artifacts to simplify lineage tracking.

Why it fails: Co-locating the training corpus with model artifacts expands the blast radius of any credential compromise on that bucket, exposing the raw notes alongside the weights. It does not reduce memorization in the model at all, and it worsens the confidentiality posture by concentrating sensitive data where production pipelines already have broad access.

Increase the model's parameter count so that training examples are distributed across more weights.

Why it fails: Larger models generally memorize more, not less, because they have greater capacity to store rare training sequences. Scaling parameters does not reduce the influence of any single patient note and can make verbatim extraction easier, so this choice works against the stated objective of limiting memorization leakage.

Disable output logging on the summarization service so that generated text cannot be harvested.

Why it fails: Suppressing output logs removes the audit trail needed to investigate leakage and to detect extraction attempts, while doing nothing to stop memorized content from being returned to a caller in real time. The risk is that the model emits patient text, not that the log stores it, so this control reduces oversight without reducing the underlying risk.