Trustworthy AI practice questions

Monetize: Quantifying the financial impact of AI deployments to stakeholders.

Why it fails: Monetization is a business outcome, not a pillar of the NIST AI Risk Management Framework. While financial planning is important for any project, the framework focuses specifically on safety, security, and ethics, rather than the commercial success or revenue generation aspects of implementing specific generative AI systems.

Optimize: Ensuring maximum hardware utilization through kernel fusion.

Why it fails: Optimization for hardware performance is a technical engineering goal, not a component of the NIST AI Risk Management Framework. While efficiency is important for sustainability, the framework focuses on risk, reliability, and safety, which are distinct from the computational performance metrics managed by hardware acceleration libraries.

Publicize: Ensuring all training data is released for public scrutiny.

Why it fails: Public disclosure of training data is not a requirement or a pillar of the NIST framework. Often, data must remain private due to IP, privacy, or security constraints. The framework emphasizes transparency in processes and risk assessments, but does not mandate the release of sensitive raw data sets.

max_token_length

Why it fails: The token length setting limits the computational cost and response size of the model. It does not possess any capability to detect or block malicious intent or prompt injection attempts. While useful for preventing resource exhaustion, it provides no protection against adversarial linguistic manipulation or prompt-based attacks.

safety_filter

Why it fails: The safety filter is a broad category that typically checks outputs for prohibited content like hate speech or toxic language. It is generally a downstream check and is less specialized for detecting the complex structural patterns of prompt injection compared to a dedicated defense mechanism designed for input validation.

data_sanitization

Why it fails: Data sanitization is generally used to strip PII or sensitive info from datasets, preventing data leaks or privacy violations. It does not actively monitor for or prevent adversarial prompt attacks aimed at manipulating the LLM logic. It is a data-privacy control, not an adversarial-security control for model input.

Fine-tune the model on a dataset that excludes all copyrighted works.

Why it fails: While curating training data is important, fine-tuning on a non-copyrighted dataset does not erase memorization from the base model. The base model may still regurgitate copyrighted text learned during pretraining. This approach alone is insufficient and does not address the immediate risk during inference, making it a less effective choice than output filtering or differential privacy.

Apply TensorRT-LLM quantization to reduce model size.

Why it fails: Quantization reduces precision to speed up inference and lower memory usage. It does not remove memorized content from the model's parameters and can even exacerbate regurgitation by altering output distributions unpredictably. It is an optimization technique, not a copyright mitigation, and should not be relied upon to prevent verbatim reproduction.

Increase the model's temperature to encourage more creative outputs.

Why it fails: Higher temperature increases randomness, which might reduce exact matches but does not systematically prevent regurgitation. It can also degrade coherence and does not address the root cause of memorization. This is not a reliable technique for copyright protection and could introduce other quality issues, so it is not recommended as a primary mitigation.

Implement k-anonymity on the training dataset to prevent re-identification

Why it fails: K-anonymity ensures that each record is indistinguishable from at least k-1 others, which protects privacy but does not address bias in model outputs. It can even mask bias by homogenizing data. For legal document review, the goal is fair treatment, not just anonymity. Thus, this technique is not appropriate for bias detection or mitigation.

Apply differential privacy during fine-tuning to add noise to the training data

Why it fails: Differential privacy protects individual data points from being memorized, which is important for privacy but does not directly address bias against groups. It can actually obscure bias detection by adding noise. In legal document review, the primary concern is fair treatment across demographics, not individual privacy leakage. Thus, this technique is not the most appropriate for bias mitigation.

Perform data augmentation by generating synthetic legal documents with balanced demographic representations

Why it fails: Data augmentation can increase representation of underrepresented groups, but it does not guarantee that the model will not learn biased associations. Synthetic documents may not capture real-world complexities, and bias can persist. While it can be part of a mitigation strategy, it is not a detection technique and may not be sufficient alone. Therefore, it is not one of the most appropriate methods.

Raise the model's confidence threshold so low-certainty suggestions are suppressed before reaching the clinician.

Why it fails: Thresholding filters which suggestions appear but leaves the accepted high-confidence ones able to proceed without a human gate. A confident wrong recommendation could still be committed, so this tunes precision rather than enforcing clinician authority. It changes what the model proposes, not who approves the resulting clinical action.

Fine-tune the base model on the hospital's historical notes so its summaries match clinician writing style.

Why it fails: Style fine-tuning improves fluency and tone but has no mechanism for human approval or action gating. A model that writes like a clinician is not a model that defers to one, and fine-tuning can even increase overconfidence. This addresses readability, not the governance requirement that a person authorize every clinical decision.

Log every model suggestion and clinician decision to an immutable audit trail with timestamps and user identity.

Why it fails: Audit logging is essential for accountability and post-hoc review, but it only records what happened after the fact. It does not prevent the model from committing an action on its own, so it cannot by itself guarantee that a clinician held final authority. Logging supports oversight; it does not enforce it at the point of decision.

NVIDIA DALI for data preprocessing

Why it fails: DALI accelerates data loading and augmentation but has no explainability capabilities. It cannot generate feature attributions or rationales for model decisions. Using DALI would not help the governance team provide regulators with the required explanations, making it unsuitable for this audit.

NVIDIA NeMo Guardrails

Why it fails: NeMo Guardrails is designed for safety and policy enforcement, not for generating feature attribution explanations. It can block or modify outputs but does not analyze which input features influenced a prediction. For regulatory explainability, a dedicated interpretability tool is needed, so this does not meet the requirement.

NVIDIA TensorRT for model optimization

Why it fails: TensorRT optimizes inference performance but does not include interpretability features. It cannot explain which features influenced a prediction. While it may be part of the deployment stack, it does not address the need for feature attribution, so it is not the right tool for this regulatory requirement.

A retrieval-augmented generation (RAG) pipeline that indexes only approved documents.

Why it fails: RAG restricts the knowledge available to the model but does not prevent the model from answering from its parametric knowledge or from generating a refusal bypass. A user asking about a competitor product could still receive an answer if the model has that information internally. RAG improves factual grounding but is not a hard refusal mechanism for restricted topics.

A dialog rail that defines canonical forms and uses them to match user intents and trigger a refusal flow.

Why it fails: Dialog rails control the conversational flow and can trigger refusals, but without defining canonical forms for the restricted competitor topics, paraphrased or obfuscated requests may not match the intended intent. Canonical forms are the mechanism that lets the rail recognize semantically equivalent phrasings, so configuring only the dialog rail without them leaves gaps in coverage.

An input rail that uses an embedding model and similarity threshold to detect and block restricted topics.

Why it fails: Input rails with embedding similarity can catch many paraphrases, but threshold tuning is brittle: too low and legitimate questions are blocked, too high and rephrased restricted questions slip through. It is a useful layer but not the most reliable enforcement mechanism for a hard compliance rule that must hold across many phrasings.

Increase the model's temperature parameter to maximum to encourage diverse reasoning.

Why it fails: Increasing the temperature enhances randomness and creative generation, which directly contradicts the goal of factual grounding. Higher temperature values lead to greater variance and unpredictability, making the model significantly more prone to hallucinating details that are not supported by the input context or source documentation provided.

Apply fine-tuning on the entire customer support history to memorize expected responses.

Why it fails: Fine-tuning on static datasets creates a 'frozen' knowledge base that becomes outdated quickly. It does not provide the model with a mechanism for real-time verification or external referencing. Relying on memorized patterns often exacerbates hallucinations when the model is asked about current information outside the training distribution.

Restrict the model to only use pre-computed templates for every possible customer query.

Why it fails: While templates offer absolute control over output, they negate the utility of an LLM. This approach fails to address the complexity of natural language understanding required for modern support scenarios. It is not a viable strategy for scaling intelligent conversational systems and lacks the flexibility required for automation.

Apply post-processing filters to censor all sensitive keywords in model output.

Why it fails: Post-processing filters act as a reactive bandage rather than a proactive solution. They are often easily bypassed by adversarial prompts or linguistic variations and do not address the root cause of the bias embedded within the model weights. True mitigation requires addressing the underlying data representational issues.

Increase the model size to allow for better internal alignment with human values.

Why it fails: Scaling the parameter count does not inherently reduce bias; in fact, larger models often memorize and amplify biases more effectively if the input data remains skewed. Simply making a model larger without modifying the training data or objective function does not address the fundamental issue of ethical alignment.

Use an adversarial model to guess the sensitive attributes of the primary model's output.

Why it fails: While this identifies that bias exists, it is a diagnostic tool rather than a mitigation technique. It provides information about model performance but does not provide the structural changes or data balancing required to eliminate the bias. It is insufficient as a standalone approach for bias reduction.

The ability of the model to perform multiple tasks simultaneously without loss of accuracy.

Why it fails: Multitasking capability is a measure of model versatility, not interpretability. While important for efficiency, it does not explain how the model reaches specific decisions or internalize the reasoning process. A multi-tasking model can be just as much of a 'black box' as a single-task model.

The speed at which a model can process training data during the fine-tuning phase.

Why it fails: Training speed is a performance metric related to infrastructure and algorithmic efficiency. It has no correlation with the ability of a human to interpret or audit the model's logic. A fast-training model may still be entirely opaque, offering no explanation for its specific predictions or generative choices.

The process of removing all personal identifiers from the training dataset.

Why it fails: This describes data anonymization or privacy protection, not interpretability. While privacy is a pillar of Trustworthy AI, it is distinct from the requirement that the AI's internal logic and decision-making pathways must be accessible and understandable to developers, auditors, and end-users who need to verify performance.

Automating all processes to eliminate the potential for human error.

Why it fails: Eliminating human oversight is often dangerous in complex or high-stakes environments. AI models can exhibit unpredictable behavior in edge cases, and removing humans entirely creates a single point of failure. Human judgment is necessary for handling nuances and unexpected scenarios that the AI has not been trained to manage.

Restricting human access to the model's internal weights and architecture.

Why it fails: Human oversight should focus on the output and the reasoning process, not necessarily the raw weights. Restricting access to internal documentation often hinders the ability of human reviewers to understand why a model is failing. Transparency and access to information are key components of effective oversight for Trustworthy AI.

Model Explainability.

Why it fails: Explainability would involve the system providing the specific reason or logic behind why a response was blocked, not just the fact that it was blocked. While the log shows a 'safety_strict' action, it does not provide an interpretative breakdown of the model's decision-making process or internal weights.

Data Privacy.

Why it fails: Data privacy focuses on protecting PII and sensitive user information. This log entry reflects a content safety policy regarding toxicity, not a data leakage event or a privacy-related sanitization process. It is a content-moderation action, which is distinct from the protection of user identity or sensitive personal data.

Algorithmic Efficiency.

Why it fails: Efficiency metrics would focus on latency, GPU utilization, or token throughput. The log shows a security intervention, not an optimization or performance metric. While preventing toxic output is necessary, it is a quality and safety concern, not a measure of how efficiently the model utilizes hardware resources during inference.

It ensures that the model can be compressed into a smaller size for edge deployment.

Why it fails: Data provenance is about the history and lineage of the training data, not the compression or quantization techniques used to optimize models for edge devices. These are unrelated technical domains; data provenance supports ethics and auditing, while model compression supports resource efficiency and deployment on constrained hardware.

It speeds up the GPU training process by indexing the data in a vector database.

Why it fails: Data indexing is an engineering optimization for retrieval speed, not a mechanism for tracking the historical provenance or legal status of the data. While indexing helps in RAG systems, it does not provide the historical metadata required for provenance or compliance verification in the context of Trustworthy AI.

It automatically corrects grammatical errors in the training corpus.

Why it fails: This describes data cleaning or pre-processing. Provenance is a descriptive and documentary practice, not an active data-modification process. It does not analyze the content of the data for quality or correctness; it analyzes the history of the data to ensure it meets governance and legal requirements.

A user accidentally providing a very long, complex question that causes a memory error.

Why it fails: This is a system instability or a resource-exhaustion bug, not an adversarial attack. An attack requires intentionality and a malicious goal. A long query causing a crash is a failure of system hardening or resource management, not a breach of the model's behavioral constraints or safety policies.

The model failing to correctly answer a question because the information is not in its training data.

Why it fails: This is a limitation of the model's knowledge base, not an attack. AI models are not omniscient, and failing to provide an answer to an out-of-distribution question is a expected behavior, not a security vulnerability. It does not involve any malicious attempt to force the model into error.

A developer updating the model to use a new, more efficient activation function.

Why it fails: Updating the model architecture is a standard development and maintenance activity. It is the opposite of an attack; it is an improvement effort. It has no malicious intent and does not seek to subvert the model's safety or operational parameters, but rather to improve its performance or reliability.

To increase the total number of data points, thereby lowering training costs.

Why it fails: Diversity is a qualitative requirement, not a quantitative one. Increasing the number of raters without diversity may just reinforce existing biases at a larger scale. The goal of diversity is to improve the quality and fairness of the alignment, not to reduce costs or increase training volume.

To allow the model to learn multiple languages more efficiently during fine-tuning.

Why it fails: Language proficiency is typically handled by the underlying foundation model training and multilingual datasets. While diverse raters may speak different languages, the primary purpose of rater diversity is ethical alignment and bias mitigation, not linguistic performance or the mechanical efficiency of the training process itself.

To optimize the GPU memory usage during the reinforcement learning phase.

Why it fails: Rater demographics have absolutely no impact on GPU memory allocation or hardware utilization. These are entirely separate concerns: one deals with ethical social science and human alignment, while the other deals with computer architecture and resource management. Confusing these two areas is a common error in AI engineering.

Using public, unverified data sources to maximize training diversity.

Why it fails: Using unverified data is a major source of security risk and potential leakage. Public data often contains unscrubbed private information, and using it without vetting can lead to the model incorporating sensitive data into its weights, which is exactly the opposite of the best practice for data protection.

Increasing the learning rate to ensure faster convergence and data masking.

Why it fails: Learning rate is a hyperparameter for optimization and has no relationship with data security or privacy. It cannot 'mask' data. Confusing optimization parameters with security controls is a fundamental misunderstanding of model training; learning rate adjustments have no impact on the privacy or security of the data.

Hard-code all possible responses to prevent the model from learning new patterns.

Why it fails: Hard-coding effectively disables the generative capabilities of the LLM. It prevents the model from 'drifting,' but it also prevents it from functioning as an intelligent assistant. This approach is not a solution to drift but a complete abandonment of the AI's core functionality and utility.

Increase the number of parameters in the model to improve its reasoning capacity.

Why it fails: Increasing model size does nothing to mitigate drift; if the underlying data distribution has shifted, a larger model will simply be a larger, more confident, but still incorrect model. Drift is a temporal and distributional issue, not an issue that can be solved by throwing more parameters at it.

Disable the model's feedback collection to avoid processing incorrect user data.

Why it fails: Disabling feedback removes the very data needed to identify that drift is occurring. You cannot fix what you cannot measure. Ignoring user data is a reactive strategy that ensures the model will eventually fail, as it will be completely disconnected from the actual needs and language of the users.

The model will warn the user about PII but continue processing the request.

Why it fails: A 'strict' mode does not allow for warnings or partial processing. It is a binary, all-or-nothing control mechanism. If an input matches a block list item, the system must reject the request entirely. Warnings are typical of 'permissive' or 'log-only' modes, not a strict security enforcement policy.

The model will automatically redact the PII and then answer the request.

Why it fails: Redaction is a data-handling process, not an enforcement-mode behavior. 'Strict' enforcement means the entire request is blocked, not cleaned. If the intent was to allow the request while scrubbing data, the configuration would need a separate 'sanitization' or 'redaction' policy, not just a strict enforcement block list.

The system logs the violation but permits the model to generate a response.

Why it fails: Logging without blocking is the definition of a 'passive' or 'audit-only' mode. Strict enforcement requires active, immediate intervention. If the system were to permit the response, it would fail to provide the protection that 'strict' enforcement implies, making it an ineffective security control against harmful or forbidden interactions.

Publishing the exact weights of the model to the public internet.

Why it fails: Transparency does not require open-sourcing the weights, which can present significant security and IP risks. Transparency is about the documentation of capabilities, constraints, and data provenance, not the disclosure of sensitive intellectual property or proprietary model weights that could be used for malicious purposes or competitive exploitation.

Eliminating all non-deterministic behaviors to ensure 100% output consistency.

Why it fails: Generative models are inherently probabilistic; forcing 100% consistency removes the core value of an LLM. Transparency means clearly communicating that the model is probabilistic, not suppressing the nature of the model itself. Consistency is a performance goal, not a transparency requirement for Trustworthy AI in generative systems.

Ensuring the model always answers with a neutral tone.

Why it fails: Tone is a style choice, not a measure of transparency. A model can be transparent and opinionated, or opaque and neutral. Transparency relates to the availability of information about the system's construction and performance metrics, not the stylistic or emotional quality of the text generated by the model.

Increasing the model's temperature to make outputs more creative.

Why it fails: Increasing temperature makes the model more random, which is more likely to cause the model to 'guess' or reconstruct parts of its training data if prompted correctly. It does nothing to sanitize the data or protect information; if anything, higher randomness can exacerbate the risk of accidental data leakage.

Training the model on larger datasets to dilute the impact of sensitive info.

Why it fails: Dilution is not a security strategy. The model can still memorize sensitive information even within a massive dataset. Relying on the size of the dataset to hide sensitive information is a dangerous misconception that ignores the ability of LLMs to memorize specific, infrequent training sequences with high fidelity.

Reducing the number of training epochs to stop the model from learning.

Why it fails: Reducing epochs might prevent the model from learning *at all*, rendering it useless. If the goal is a functioning AI, you need enough training time for the model to capture the necessary patterns. Reducing training time is not an information-security strategy for preventing the leakage of sensitive data.