AI Security, Ethics and Governance practice questions

Implement role-based access control (RBAC) on the AI model's inference API.

Why it fails: Incorrect. RBAC limits who can query the model but does not prevent the model from returning sensitive data that it has memorized.

Use differential privacy during model training.

Why it fails: Incorrect. Differential privacy provides a mathematical guarantee of privacy by adding noise, but it does not explicitly remove data of opt-out patients; the model may still leak information if not properly calibrated.

Encrypt the training data at rest and in transit.

Why it fails: Incorrect. Encryption protects data confidentiality during storage and transmission, but once the model is trained, it can still output sensitive information from the training data.

Use explainable AI techniques to understand why the chatbot generates certain responses.

Why it fails: Explainability helps understand behavior but does not prevent the chatbot from generating harmful outputs.

Encrypt all chatbot conversations at rest and in transit.

Why it fails: Encryption protects data from unauthorized access but does not prevent the chatbot from generating privacy-violating responses.

Anonymize the training data used to train the chatbot.

Why it fails: Anonymization reduces but does not eliminate the risk of the chatbot generating responses that contain personal information.

Replace the convolutional layers with transformer layers to improve attention.

Why it fails: Changing architecture typically requires full retraining and may not address low-light specifically.

Use adversarial training to add imperceptible perturbations to training images.

Why it fails: Adversarial training improves resistance to malicious perturbations but not necessarily to natural low-light conditions.

Increase the model's depth by adding more convolutional layers.

Why it fails: Increasing depth changes architecture and requires retraining; it may not directly address low-light performance.

Privacy by design

Why it fails: Privacy by design focuses on data protection, not on ensuring fair pricing outcomes.

Accountability

Why it fails: Accountability assigns responsibility but does not directly address the fairness of the pricing algorithm.

Beneficence

Why it fails: Beneficence focuses on promoting good, but does not specifically prevent unfair pricing practices.

Normalizing features to zero mean and unit variance.

Why it fails: Normalization does not identify malicious data; it only transforms values.

Using ensemble methods like random forest for training.

Why it fails: Ensembles improve robustness but are not a detection technique.

Applying PCA to reduce dimensionality.

Why it fails: PCA can obscure patterns of poisoning rather than reveal them.

Enable detailed audit logging

Why it fails: Audit logs help detect breaches but do not prevent them or protect privacy.

Encrypt all data at rest and in transit

Why it fails: Encryption protects confidentiality but does not anonymize the data; re-identification is still possible.

Implement role-based access control

Why it fails: Access control limits who can see data but does not protect against misuse by authorized users.

Set up a human-in-the-loop review process

Why it fails: Human review is slower and costlier; automated filtering is first line.

Create a usage policy for acceptable bot behavior

Why it fails: Policies guide development but are not a technical control.

Sanitize training data to remove toxic examples

Why it fails: Training data sanitization reduces risk but does not guarantee safe outputs in all cases.

Encrypting the model weights

Why it fails: Encrypting weights protects data at rest, not inference-time manipulation; adversarial examples perturb inputs, so the model still returns corrupted predictions. Encryption is the right control for stolen model artefacts or intercepted storage, where confidentiality of the weights themselves is the requirement.

Continuous model monitoring

Why it fails: Monitoring detects drift and anomalous inputs after deployment but does not harden the model against adversarial examples during training or inference. It is tempting because monitoring supports incident response, yet adversarial robustness requires defences such as adversarial training or input sanitisation.

Rate limiting API access

Why it fails: Rate limiting throttles request volume, but an adversarial example succeeds in a single well-crafted query, so the attack still lands. It is the correct control against brute-force, credential-stuffing or denial-of-service abuse of the inference endpoint, not against input perturbation.

The server ran out of memory due to high traffic

Why it fails: Only one request is logged; no high traffic indication.

The model weights were corrupted during loading

Why it fails: No error indicates model loading issue; the crash happened during inference.

The model encountered an unknown data type

Why it fails: The input is text; unknown data type would cause a parsing error, not memory allocation.

Data poisoning

Why it fails: Poisoning affects model training, not inference confidence patterns.

Evasion attack

Why it fails: Evasion causes misclassification, not overconfidence on specific individuals.

Model extraction attack

Why it fails: Extraction aims to steal model parameters, not produce confidence anomalies.

Obtain explicit consent from candidates

Why it fails: Consent provides a lawful basis for processing but does not itself deliver the meaningful information about the logic involved that the right to explanation requires. It is tempting because consent is central to GDPR compliance, yet it would be correct where the lawful basis for processing is in question, not where explainability is the requirement.

Anonymize all candidate data before processing

Why it fails: Anonymising candidate data removes personal data from scope, which eliminates the explanation duty rather than satisfying it; the tool still cannot explain its decisions. It is tempting because anonymisation is a strong privacy control, but it would be correct where data minimisation, not transparency to the candidate, is the requirement.

Always have a human review the AI's recommendations

Why it fails: Human review of recommendations addresses oversight and contestability, not the candidate's right to receive a meaningful explanation of the automated decision's logic. It is tempting because human involvement is a recognised GDPR safeguard, but it would be correct where Article 22 safeguards against solely automated decisions are the requirement.

Cloud infrastructure configuration

Why it fails: Cloud infrastructure configuration is a technical deployment concern, not a governance component such as accountability, oversight, policies or risk management. It is tempting because misconfigured infrastructure creates real AI risk, and configuration management would be the correct answer to a question about securing the hosting environment, but governance frameworks define decision rights and controls.

Model accuracy benchmarks

Why it fails: Accuracy benchmarks measure model performance, not governance structures such as accountability, oversight, policies or risk management. It is tempting because accuracy metrics are central to model validation and monitoring, and they would be the correct component of an MLOps or model risk process, but governance frameworks address decision rights and controls.

Data poisoning of the training dataset

Why it fails: Poisoning would not cause real-time drift and query spikes.

A network hardware failure

Why it fails: Hardware failure does not explain drift or query pattern.

A scheduled model retraining process

Why it fails: Retraining would not cause high query rate or confidence drop.

The policy allows access from a wide private IP range

Why it fails: While the policy allows a wide private IP range, it is still restricted to corporate networks, making this less significant than unlimited inference requests.

The policy does not require multi-factor authentication

Why it fails: Multi-factor authentication is explicitly required in the policy, so this is not a weakness.

The policy grants 'audit_log' access to data scientists

Why it fails: Granting audit_log access to data scientists is a minor access control issue but not as critical as model extraction risk.

Data poisoning corrupted the model, causing NaN outputs

Why it fails: The error is at inference, not model corruption.

The security filter failed to detect an attack and the model returned an error

Why it fails: The filter detected and blocked the attack.

A non-adversarial input caused a NaN error due to missing data

Why it fails: The filter considered it adversarial, overriding the NaN possibility.

Data poisoning

Why it fails: Data poisons the training data to degrade model performance, not cause regional misclassification on benign traffic.

Model inversion

Why it fails: Model inversion extracts training data, not causes misclassification.

Membership inference

Why it fails: Membership inference determines if a record was in training data, not misclassification.

The model architecture is too complex

Why it fails: Complexity doesn't inherently cause bias; it may capture more relationships, but the core issue is proxy variables.

The training data still contains historical hiring bias

Why it fails: While true, the question is about why bias persists after removing gender. Proxies are the more direct cause.

The evaluation metric does not measure fairness

Why it fails: Evaluation metric is important but not the cause of bias persistence; the model still learns biased patterns.

Data poisoning during training

Why it fails: Data poisoning would have occurred before deployment; this scenario focuses on deployment in a new environment.

Model inversion to extract patient data

Why it fails: Model inversion is a privacy risk but not directly related to distribution shift.

Data breach of the inference API

Why it fails: Data breach is a general security concern, not specific to the model's generalization challenge.

Data augmentation

Why it fails: Data augmentation adds noise/transformations but not targeted adversarial perturbations.

Input validation

Why it fails: Input validation checks for malicious inputs but adversarial perturbations are often undetectable.

Feature squeezing

Why it fails: Feature squeezing reduces input precision to remove adversarial perturbations, but it is not always effective.

The auditor role lacks 'evaluate' permission

Why it fails: Auditor permissions are for viewing, not deployment.

Fairness check threshold is set to 0.8, which is too low

Why it fails: Threshold does not cause bypass, it only measures fairness.

External_user role can perform inference, which triggers unfair predictions

Why it fails: Inference does not bypass fairness check.