Scenario practice questions

Refer to the exhibit. An engineer is tuning a deployment config. Why is 'enable_cuda_graph' set to true in this JSON configuration?

Exhibit

{
  "model_name": "llama-3-8b",
  "max_batch_size": 128,
  "precision": "fp16",
  "enable_cuda_graph": true
}

Trap 1: To increase the maximum batch size to 256.

CUDA Graphs are unrelated to the batch size limit. The batch size is a memory-constrained parameter, whereas the graph feature is a runtime optimization designed to minimize CPU interaction by grouping GPU task submissions into a single, pre-compiled workload.

Trap 2: To convert the model precision from fp16 to fp8.

Precision conversion is handled by the model compilation step or quantization tools, not by the CUDA Graph feature. The precision field in the JSON is static and independent of the execution graph optimization being enabled in the configuration.

Trap 3: To force the model to use the CPU for inference.

CUDA Graphs are strictly a GPU-side optimization. They cannot be used to offload inference to the CPU. In fact, they are designed to minimize the CPU's influence on the execution pipeline to ensure the GPU maintains peak operational efficiency.

A
To increase the maximum batch size to 256.
Why it fails: CUDA Graphs are unrelated to the batch size limit. The batch size is a memory-constrained parameter, whereas the graph feature is a runtime optimization designed to minimize CPU interaction by grouping GPU task submissions into a single, pre-compiled workload.
B
To reduce CPU overhead during repetitive kernel launches.
By capturing the graph of operations, the driver can execute them with a single launch command. This avoids the overhead of traversing the command queue for every operation, which is highly beneficial for LLM inference where the execution pattern is consistent.
C
To convert the model precision from fp16 to fp8.
Why it fails: Precision conversion is handled by the model compilation step or quantization tools, not by the CUDA Graph feature. The precision field in the JSON is static and independent of the execution graph optimization being enabled in the configuration.
D
To force the model to use the CPU for inference.
Why it fails: CUDA Graphs are strictly a GPU-side optimization. They cannot be used to offload inference to the CPU. In fact, they are designed to minimize the CPU's influence on the execution pipeline to ensure the GPU maintains peak operational efficiency.

Question 2mediummultiple choice

A developer is using a pretrained large language model for a text summarization task. They want to adapt the model to a domain-specific corpus of legal documents but have limited GPU memory and a small labeled dataset. Which fine-tuning approach is most parameter-efficient and suitable for this scenario?

Trap 1: Training the model from scratch on the legal corpus

Training from scratch demands massive computational resources and a large corpus, far beyond what the developer has. It also discards the general language understanding of the pretrained model, which is valuable even for domain-specific tasks. With a small labeled dataset, this approach would almost certainly underperform and is not feasible given the memory constraints.

Trap 2: Freezing all layers and training only the output head

Training only the output head limits adaptation to the new domain because the internal representations remain fixed to the pretraining distribution. For legal text, which differs significantly from general web text, this yields subpar summarization quality. While parameter-efficient, it does not provide the depth of adaptation needed and is less effective than methods that adjust internal representations.

Trap 3: Full fine-tuning of all model parameters

Full fine-tuning updates every weight in the model, requiring substantial GPU memory to store gradients and optimizer states, which exceeds the developer's limited resources. It also risks catastrophic forgetting and overfitting on a small labeled dataset. While it can yield strong performance, it is not parameter-efficient and is impractical under the stated constraints.

A
Low-Rank Adaptation (LoRA)
LoRA injects trainable low-rank matrices into existing layers while freezing the original weights, dramatically reducing the number of trainable parameters and memory footprint. This makes it ideal for limited GPU memory and small datasets, as only the adapters are updated. It also preserves pretrained knowledge, reducing overfitting risk and enabling efficient domain adaptation for legal summarization.
B
Training the model from scratch on the legal corpus
Why it fails: Training from scratch demands massive computational resources and a large corpus, far beyond what the developer has. It also discards the general language understanding of the pretrained model, which is valuable even for domain-specific tasks. With a small labeled dataset, this approach would almost certainly underperform and is not feasible given the memory constraints.
C
Freezing all layers and training only the output head
Why it fails: Training only the output head limits adaptation to the new domain because the internal representations remain fixed to the pretraining distribution. For legal text, which differs significantly from general web text, this yields subpar summarization quality. While parameter-efficient, it does not provide the depth of adaptation needed and is less effective than methods that adjust internal representations.
D
Full fine-tuning of all model parameters
Why it fails: Full fine-tuning updates every weight in the model, requiring substantial GPU memory to store gradients and optimizer states, which exceeds the developer's limited resources. It also risks catastrophic forgetting and overfitting on a small labeled dataset. While it can yield strong performance, it is not parameter-efficient and is impractical under the stated constraints.

Question 3easymultiple choice

Which of the following best defines 'Generalization' in machine learning?

Trap 1: The ability of a model to memorize training data perfectly.

Memorizing training data perfectly is the definition of overfitting, not generalization. If a model only memorizes, it will fail when presented with new inputs that differ even slightly from the training samples. Effective machine learning requires models that capture underlying data patterns, not just specific data points.

Trap 2: The process of reducing the model's parameter count.

Reducing parameter count is known as model compression or pruning. While smaller models might generalize better because they have less capacity to overfit, the term 'generalization' itself refers to the outcome of model performance on unseen data, not the structural method used to achieve that performance.

Trap 3: The speed at which a model converges during training.

Convergence speed relates to the optimization process, such as the learning rate and optimizer choice. While important for developer productivity, it is independent of the model's ability to generalize to new data. A model can converge extremely quickly yet still fail to generalize correctly to unseen data samples.

A
The ability of a model to memorize training data perfectly.
Why it fails: Memorizing training data perfectly is the definition of overfitting, not generalization. If a model only memorizes, it will fail when presented with new inputs that differ even slightly from the training samples. Effective machine learning requires models that capture underlying data patterns, not just specific data points.
B
The model's performance on previously unseen data.
Generalization specifically measures how well a model trained on a subset of data performs on new, independent data. High generalization performance is the hallmark of a successful model, indicating that it has learned the core relationships and features of the domain rather than simply memorizing training examples.
C
The process of reducing the model's parameter count.
Why it fails: Reducing parameter count is known as model compression or pruning. While smaller models might generalize better because they have less capacity to overfit, the term 'generalization' itself refers to the outcome of model performance on unseen data, not the structural method used to achieve that performance.
D
The speed at which a model converges during training.
Why it fails: Convergence speed relates to the optimization process, such as the learning rate and optimizer choice. While important for developer productivity, it is independent of the model's ability to generalize to new data. A model can converge extremely quickly yet still fail to generalize correctly to unseen data samples.

Question 4mediummulti select

Which THREE actions are recommended for establishing a robust 'Human-in-the-Loop' (HITL) system for an AI deployment?

Trap 1: Automating all processes to eliminate the potential for human error.

Eliminating human oversight is often dangerous in complex or high-stakes environments. AI models can exhibit unpredictable behavior in edge cases, and removing humans entirely creates a single point of failure. Human judgment is necessary for handling nuances and unexpected scenarios that the AI has not been trained to manage.

Trap 2: Restricting human access to the model's internal weights and…

Human oversight should focus on the output and the reasoning process, not necessarily the raw weights. Restricting access to internal documentation often hinders the ability of human reviewers to understand why a model is failing. Transparency and access to information are key components of effective oversight for Trustworthy AI.

A
Setting specific confidence thresholds that trigger human review.
Confidence thresholds provide a quantitative way to define when an AI is 'unsure.' By routing low-confidence outputs to human experts, organizations can prevent errors from reaching end-users. This mechanism is vital for maintaining high quality and reliability in automated systems, serving as an essential safety gate for deployment.
B
Automating all processes to eliminate the potential for human error.
Why it fails: Eliminating human oversight is often dangerous in complex or high-stakes environments. AI models can exhibit unpredictable behavior in edge cases, and removing humans entirely creates a single point of failure. Human judgment is necessary for handling nuances and unexpected scenarios that the AI has not been trained to manage.
C
Creating intuitive interfaces for humans to edit or approve AI outputs.
Intervention tools allow humans to efficiently correct or validate AI suggestions. Without intuitive interfaces, the review process becomes a bottleneck, leading to user fatigue or errors. Well-designed UI for HITL processes increases the efficiency of the human expert, making the overall system more responsive and reliable in production.
D
Incorporating expert feedback to refine and improve the model over time.
The value of HITL extends beyond immediate correction; it provides labeled data that can be used for RLHF or fine-tuning. By feeding human-approved outputs back into the training pipeline, the model continuously learns from expert knowledge, reducing future errors and improving the overall quality and reliability of the AI.
E
Restricting human access to the model's internal weights and architecture.
Why it fails: Human oversight should focus on the output and the reasoning process, not necessarily the raw weights. Restricting access to internal documentation often hinders the ability of human reviewers to understand why a model is failing. Transparency and access to information are key components of effective oversight for Trustworthy AI.

Question 5mediummultiple choice

A team is pretraining a large language model on a cluster of NVIDIA GPUs. They observe that the model's training loss decreases steadily for the first few epochs but then suddenly spikes and eventually becomes NaN. They suspect this is due to exploding gradients. Which technique is most appropriate to address this issue?

Trap 1: Reduce the batch size to decrease the variance of gradient…

Reducing batch size increases the variance of gradient estimates, which can make training noisier and potentially worsen instability. While smaller batches sometimes help generalization, they do not reliably prevent exploding gradients. The observed loss spike is better addressed by controlling gradient magnitude directly, not by altering batch statistics.

Trap 2: Increase the learning rate to help the model escape the spiking…

Increasing the learning rate amplifies update steps, which worsens exploding gradients and accelerates divergence to NaN. The scenario already shows instability, so a larger learning rate would likely cause earlier failure. The correct approach is to stabilize updates, not to make them more aggressive. Thus, this option is counterproductive for the described symptom.

Trap 3: Switch from Adam to stochastic gradient descent (SGD) without…

Changing the optimizer might alter dynamics, but SGD without momentum does not inherently prevent exploding gradients; it can still produce large updates if gradients are large. Moreover, such a switch could slow convergence and require extensive retuning. It does not directly address the root cause of gradient explosion, so it is not the most appropriate fix in this situation.

A
Apply gradient clipping to limit the magnitude of gradients during backpropagation.
Gradient clipping caps the norm or value of gradients before the optimizer step, preventing excessively large updates that destabilize training. In this scenario, the sudden loss spike and NaN indicate exploding gradients, so clipping directly mitigates the problem. It is a standard, low-overhead intervention that preserves the model architecture and data pipeline, making it the most appropriate immediate fix.
B
Reduce the batch size to decrease the variance of gradient estimates.
Why it fails: Reducing batch size increases the variance of gradient estimates, which can make training noisier and potentially worsen instability. While smaller batches sometimes help generalization, they do not reliably prevent exploding gradients. The observed loss spike is better addressed by controlling gradient magnitude directly, not by altering batch statistics.
C
Increase the learning rate to help the model escape the spiking region.
Why it fails: Increasing the learning rate amplifies update steps, which worsens exploding gradients and accelerates divergence to NaN. The scenario already shows instability, so a larger learning rate would likely cause earlier failure. The correct approach is to stabilize updates, not to make them more aggressive. Thus, this option is counterproductive for the described symptom.
D
Switch from Adam to stochastic gradient descent (SGD) without momentum.
Why it fails: Changing the optimizer might alter dynamics, but SGD without momentum does not inherently prevent exploding gradients; it can still produce large updates if gradients are large. Moreover, such a switch could slow convergence and require extensive retuning. It does not directly address the root cause of gradient explosion, so it is not the most appropriate fix in this situation.

Question 6mediummultiple choice

A team is fine-tuning an NVIDIA NIM-hosted Llama 3 8B model and wants a single visualization that tracks per-step training loss, learning rate, and GPU memory utilization together, so they can correlate a mid-run loss spike with resource pressure. They need a framework that integrates natively with the NVIDIA NeMo training stack and requires minimal custom plotting code. Which visualization approach best meets these requirements?

Trap 1: Render a t-SNE projection of the model's embedding layer each epoch…

A t-SNE projection of embeddings shows representation structure, not per-step training dynamics such as loss, learning rate, or GPU memory. It cannot correlate a loss spike with resource pressure, and computing it every epoch adds cost without answering the question. This visualization is mismatched to the monitoring goal.

Trap 2: Generate a confusion matrix from the validation set after each…

A confusion matrix summarizes classification outcomes on labeled data, which is not how a generative LLM is typically evaluated, and it does not expose loss, learning rate, or GPU memory trends. It also runs only at checkpoints, so a mid-step loss spike would be missed. This does not satisfy the correlation requirement.

Trap 3: Export per-step metrics to CSV and build a custom Matplotlib…

Post-hoc CSV plotting with Matplotlib works but is not natively integrated with the NeMo training stack and requires custom code for each panel. It also delays diagnosis until training ends, so the team cannot correlate a loss spike with GPU memory pressure in real time. This approach fails the requirement for minimal custom plotting code and live correlation during the run.

A
Render a t-SNE projection of the model's embedding layer each epoch to observe representation drift.
Why it fails: A t-SNE projection of embeddings shows representation structure, not per-step training dynamics such as loss, learning rate, or GPU memory. It cannot correlate a loss spike with resource pressure, and computing it every epoch adds cost without answering the question. This visualization is mismatched to the monitoring goal.
B
Use the NVIDIA NeMo training logs with TensorBoard, logging loss, learning rate, and GPU memory as scalars on the same step axis.
NeMo's Trainer integrates with TensorBoard through its logger, so loss, learning rate, and GPU metrics can be written as scalars keyed by global step. Overlaying them on a shared step axis lets the team visually align a loss spike with memory pressure without writing custom plotting code, satisfying both the integration and minimal-code requirements.
C
Generate a confusion matrix from the validation set after each checkpoint to detect training instability.
Why it fails: A confusion matrix summarizes classification outcomes on labeled data, which is not how a generative LLM is typically evaluated, and it does not expose loss, learning rate, or GPU memory trends. It also runs only at checkpoints, so a mid-step loss spike would be missed. This does not satisfy the correlation requirement.
D
Export per-step metrics to CSV and build a custom Matplotlib multi-panel figure after training completes.
Why it fails: Post-hoc CSV plotting with Matplotlib works but is not natively integrated with the NeMo training stack and requires custom code for each panel. It also delays diagnosis until training ends, so the team cannot correlate a loss spike with GPU memory pressure in real time. This approach fails the requirement for minimal custom plotting code and live correlation during the run.

Question 7mediummultiple choice

A team is fine-tuning a NeMo Megatron GPT model on an internal corpus and observes that validation loss begins rising after epoch three while training loss continues to fall. They want to detect this condition automatically during future experiments without manually watching the curves. Which NeMo callback or mechanism should they configure to stop training when validation loss stops improving?

Trap 1: TensorRT-LLM quantization calibration pass

Quantization calibration is a post-training step that determines scaling factors for reduced-precision inference. It does not monitor training metrics or stop a training job. Applying it here would miss the overfitting signal entirely and would only be relevant after the model is trained and being prepared for deployment.

Trap 2: NeMo ModelCheckpoint with save_top_k set to a negative value

ModelCheckpoint controls which checkpoints are saved based on monitored metrics, but it does not terminate training. Setting save_top_k to a negative value changes retention behavior, not the training loop. The team would still complete all epochs while the model overfits, so this does not satisfy the automatic stopping requirement.

Trap 3: NVIDIA Nsight Compute kernel replay

Nsight Compute is a GPU kernel profiler used to analyze instruction-level performance, not to monitor validation metrics. It cannot observe loss curves or trigger training termination. Configuring it would give the team detailed kernel statistics but no automatic early stopping, so the overfitting condition would continue undetected during the run.

A
NeMo EarlyStopping callback monitoring validation loss
The NeMo EarlyStopping callback watches a monitored metric such as validation loss and halts training when no improvement is seen for a configured patience period. This directly addresses the observed divergence between training and validation loss by ending the run automatically, saving compute and preventing further overfitting in future experiments.
B
TensorRT-LLM quantization calibration pass
Why it fails: Quantization calibration is a post-training step that determines scaling factors for reduced-precision inference. It does not monitor training metrics or stop a training job. Applying it here would miss the overfitting signal entirely and would only be relevant after the model is trained and being prepared for deployment.
C
NeMo ModelCheckpoint with save_top_k set to a negative value
Why it fails: ModelCheckpoint controls which checkpoints are saved based on monitored metrics, but it does not terminate training. Setting save_top_k to a negative value changes retention behavior, not the training loop. The team would still complete all epochs while the model overfits, so this does not satisfy the automatic stopping requirement.
D
NVIDIA Nsight Compute kernel replay
Why it fails: Nsight Compute is a GPU kernel profiler used to analyze instruction-level performance, not to monitor validation metrics. It cannot observe loss curves or trigger training termination. Configuring it would give the team detailed kernel statistics but no automatic early stopping, so the overfitting condition would continue undetected during the run.

Question 8mediummultiple choice

A financial services company is deploying an NVIDIA NIM microservice for an internal LLM assistant that summarizes earnings call transcripts. The security team wants to ensure that the model cannot be coerced via prompt injection into revealing confidential merger discussions embedded in prior context. Which NVIDIA-developed safety mechanism should be integrated directly into the inference pipeline to evaluate prompts and responses against a defined policy at runtime?

Trap 1: NVIDIA Triton Inference Server model ensembles

Triton ensembles orchestrate multiple models into a single inference graph, but they do not inspect prompt content for injection or apply safety policies. They are an infrastructure optimization, not a trust and safety control. Using ensembles alone would leave the confidential merger discussions exposed to prompt injection, so this does not satisfy the security team's requirement.

Trap 2: NVIDIA TensorRT-LLM quantization

TensorRT-LLM quantization reduces model precision to improve latency and memory efficiency. It does not inspect prompts or responses for policy violations and cannot block injection attacks. While valuable for performance, quantization alone does not provide the runtime safety enforcement the security team requires for protecting merger discussions.

Trap 3: NVIDIA DALI preprocessing pipelines

DALI is a data augmentation and preprocessing library for computer vision and audio workloads. It does not evaluate natural language prompts or responses against safety policies, and it has no concept of prompt injection. Applying DALI to an LLM inference pipeline would not prevent the model from revealing confidential context, making it irrelevant to this requirement.

A
NVIDIA Triton Inference Server model ensembles
Why it fails: Triton ensembles orchestrate multiple models into a single inference graph, but they do not inspect prompt content for injection or apply safety policies. They are an infrastructure optimization, not a trust and safety control. Using ensembles alone would leave the confidential merger discussions exposed to prompt injection, so this does not satisfy the security team's requirement.
B
NVIDIA NeMo Guardrails
NeMo Guardrails is the NVIDIA toolkit designed to add programmable safety rails around LLM interactions. It intercepts prompts and responses and enforces policies defined in Colang, allowing the team to block prompt injection attempts and prevent leakage of confidential merger context. Because it runs in the inference pipeline, it is the correct mechanism for runtime policy enforcement in this scenario.
C
NVIDIA TensorRT-LLM quantization
Why it fails: TensorRT-LLM quantization reduces model precision to improve latency and memory efficiency. It does not inspect prompts or responses for policy violations and cannot block injection attacks. While valuable for performance, quantization alone does not provide the runtime safety enforcement the security team requires for protecting merger discussions.
D
NVIDIA DALI preprocessing pipelines
Why it fails: DALI is a data augmentation and preprocessing library for computer vision and audio workloads. It does not evaluate natural language prompts or responses against safety policies, and it has no concept of prompt injection. Applying DALI to an LLM inference pipeline would not prevent the model from revealing confidential context, making it irrelevant to this requirement.

Question 9hardmulti select

A healthcare organization is preparing to deploy an LLM-based clinical documentation assistant. The Trustworthy AI review board requires evidence that the model's outputs are safe and reliable before go-live. Which two practices should the team implement to provide this evidence? (Choose two.)

Trap 1: Increase the model's temperature setting to encourage more creative…

Raising temperature increases randomness, which is the opposite of what a clinical documentation assistant needs. Higher variability makes outputs less deterministic and harder to validate, and it raises hallucination risk in a domain where factual precision is critical. This would undermine rather than support the evidence of safety and reliability the review board requires.

Trap 2: Fine-tune the model on a small curated dataset of ideal clinical…

Fine-tuning on a small curated set can improve style and format but cannot guarantee elimination of unsafe outputs, especially for edge cases not represented in the data. Assuming it does removes the need for validation, which is precisely what the review board wants to see. This practice provides no measurable evidence of safety.

Trap 3: Deploy the model in shadow mode for a single day and rely on…

A one-day shadow deployment with informal feedback is not rigorous evidence. It lacks structured metrics, diverse scenario coverage, and expert review, so it cannot demonstrate safety or reliability to a review board. Shadow deployment can be a useful complement, but alone it is insufficient and the short duration limits the failure modes that would surface.

A
Establish a continuous evaluation harness that scores model outputs against a held-out clinical benchmark and tracks metrics over time.
A continuous evaluation harness provides quantitative, repeatable evidence of model quality on a representative clinical benchmark. Tracking metrics over time detects regressions and drift after updates, giving the review board ongoing assurance rather than a one-time snapshot. This is a core practice for demonstrating reliable performance in a regulated healthcare deployment.
B
Increase the model's temperature setting to encourage more creative and varied clinical documentation.
Why it fails: Raising temperature increases randomness, which is the opposite of what a clinical documentation assistant needs. Higher variability makes outputs less deterministic and harder to validate, and it raises hallucination risk in a domain where factual precision is critical. This would undermine rather than support the evidence of safety and reliability the review board requires.
C
Fine-tune the model on a small curated dataset of ideal clinical notes and assume the fine-tuning eliminates all unsafe outputs.
Why it fails: Fine-tuning on a small curated set can improve style and format but cannot guarantee elimination of unsafe outputs, especially for edge cases not represented in the data. Assuming it does removes the need for validation, which is precisely what the review board wants to see. This practice provides no measurable evidence of safety.
D
Conduct adversarial red-teaming with clinicians to probe for unsafe or biased outputs across diverse patient scenarios.
Red-teaming with domain experts surfaces failure modes that automated benchmarks miss, such as unsafe recommendations in rare clinical contexts or biased language across patient demographics. It produces documented evidence of the model's behavior under adversarial conditions, which directly supports the review board's requirement for demonstrated safety and reliability before deployment.
E
Deploy the model in shadow mode for a single day and rely on informal developer feedback as the sole safety assessment.
Why it fails: A one-day shadow deployment with informal feedback is not rigorous evidence. It lacks structured metrics, diverse scenario coverage, and expert review, so it cannot demonstrate safety or reliability to a review board. Shadow deployment can be a useful complement, but alone it is insufficient and the short duration limits the failure modes that would surface.

Question 10mediummultiple choice

A software company is using an LLM to generate code snippets for developers. During testing, they discover that the model sometimes produces code with security vulnerabilities, such as SQL injection flaws. Which Trustworthy AI principle is most directly violated by this behavior?

Trap 1: Privacy

Privacy focuses on protecting personal data and preventing unauthorized access. SQL injection flaws are about application security, not privacy. Although a SQL injection could lead to data breaches, the root issue is the insecure code generation, which is a security violation.

Trap 2: Transparency

Transparency concerns openness about how the AI system works and its limitations. While important, it does not directly address the generation of insecure code. The scenario highlights a vulnerability in the output, which is a security issue, not a transparency one.

Trap 3: Accountability

Accountability involves assigning responsibility for AI outcomes. The generation of insecure code is a technical flaw that violates security, not accountability. While accountability mechanisms should exist to address such flaws, the principle directly violated is security.

A
Privacy
Why it fails: Privacy focuses on protecting personal data and preventing unauthorized access. SQL injection flaws are about application security, not privacy. Although a SQL injection could lead to data breaches, the root issue is the insecure code generation, which is a security violation.
B
Security
Security in Trustworthy AI ensures that AI systems are resilient to attacks and do not introduce vulnerabilities. Generating code with SQL injection flaws directly compromises the security of the resulting applications. This violates the security principle, which mandates that AI outputs should not create exploitable weaknesses.
C
Transparency
Why it fails: Transparency concerns openness about how the AI system works and its limitations. While important, it does not directly address the generation of insecure code. The scenario highlights a vulnerability in the output, which is a security issue, not a transparency one.
D
Accountability
Why it fails: Accountability involves assigning responsibility for AI outcomes. The generation of insecure code is a technical flaw that violates security, not accountability. While accountability mechanisms should exist to address such flaws, the principle directly violated is security.

Question 11easymultiple choice

A data scientist is preparing a labeled dataset of 50,000 customer support tickets for supervised fine-tuning of an LLM. Each ticket must be assigned exactly one of eight department labels. Which loss function is most appropriate for training this classification head?

Trap 1: Mean squared error loss

Mean squared error measures numeric distance between predicted and target values and is designed for regression, not for selecting among discrete categories. Applying it to a softmax output over eight departments yields weak gradients when predictions are confidently wrong and does not correspond to a proper likelihood, making convergence slower and less reliable than cross-entropy for classification.

Trap 2: Contrastive loss

Contrastive loss pulls embeddings of similar pairs together and pushes dissimilar pairs apart, which is used for representation learning, retrieval, or siamese networks. It does not directly produce a probability distribution over eight fixed department labels, so it is not the correct objective for a single-label classification head on support tickets.

Trap 3: Binary cross-entropy loss

Binary cross-entropy treats each output as an independent yes/no decision applied per label, which suits multi-label problems where several tags can apply simultaneously. Here each ticket has exactly one of eight mutually exclusive departments, so independent sigmoid outputs would fail to enforce that only one label is chosen and would not properly model the competition between classes.

A
Categorical cross-entropy loss
Categorical cross-entropy compares the predicted probability distribution over the eight mutually exclusive department labels with the one-hot true label, penalizing probability mass placed on incorrect classes. It is the standard objective for single-label multi-class classification and produces well-calibrated softmax outputs, making it the right choice when each ticket belongs to exactly one department.
B
Mean squared error loss
Why it fails: Mean squared error measures numeric distance between predicted and target values and is designed for regression, not for selecting among discrete categories. Applying it to a softmax output over eight departments yields weak gradients when predictions are confidently wrong and does not correspond to a proper likelihood, making convergence slower and less reliable than cross-entropy for classification.
C
Contrastive loss
Why it fails: Contrastive loss pulls embeddings of similar pairs together and pushes dissimilar pairs apart, which is used for representation learning, retrieval, or siamese networks. It does not directly produce a probability distribution over eight fixed department labels, so it is not the correct objective for a single-label classification head on support tickets.
D
Binary cross-entropy loss
Why it fails: Binary cross-entropy treats each output as an independent yes/no decision applied per label, which suits multi-label problems where several tags can apply simultaneously. Here each ticket has exactly one of eight mutually exclusive departments, so independent sigmoid outputs would fail to enforce that only one label is chosen and would not properly model the competition between classes.

Question 12hardmulti select

A developer is deploying a large language model using NVIDIA TensorRT-LLM and wants to optimize inference for a production environment with limited GPU memory. Which two techniques can be used to reduce memory footprint while maintaining acceptable performance? (Choose two.)

Trap 1: Using a larger batch size to amortize memory overhead across more…

Increasing batch size typically increases memory usage because more sequences require KV cache storage simultaneously. While it can improve throughput, it does not reduce memory footprint; in fact, it may cause out-of-memory errors on limited GPUs. The goal is to reduce memory, so larger batch size is counterproductive.

Trap 2: Increasing the tensor parallelism degree to spread the model across…

Increasing tensor parallelism spreads the model across multiple GPUs, which reduces per-GPU memory usage but requires more GPUs and introduces communication overhead. The question specifies limited GPU memory, but not necessarily multiple GPUs. Moreover, it does not reduce the total memory footprint; it just distributes it. The goal is to reduce memory footprint, so this is not the best choice.

Trap 3: Storing the model weights on CPU and offloading them to GPU on…

Offloading model weights to CPU and transferring them to GPU on demand introduces significant latency and PCIe bandwidth bottlenecks, severely degrading performance. While it can reduce GPU memory usage, it is not a technique for maintaining acceptable performance in a production environment. TensorRT-LLM is designed for GPU-resident inference; CPU offloading is generally not recommended for latency-sensitive applications.

A
Using a larger batch size to amortize memory overhead across more requests.
Why it fails: Increasing batch size typically increases memory usage because more sequences require KV cache storage simultaneously. While it can improve throughput, it does not reduce memory footprint; in fact, it may cause out-of-memory errors on limited GPUs. The goal is to reduce memory, so larger batch size is counterproductive.
B
Enabling paged KV cache and tuning the maximum sequence length to a realistic value.
Paged KV cache in TensorRT-LLM manages key-value cache memory in fixed-size blocks, reducing fragmentation and allowing more efficient memory usage. Tuning the maximum sequence length to a realistic value prevents over-allocation of KV cache for sequences that are never reached. Together, these techniques can significantly reduce memory footprint while maintaining performance for typical workloads.
C
Increasing the tensor parallelism degree to spread the model across more GPUs.
Why it fails: Increasing tensor parallelism spreads the model across multiple GPUs, which reduces per-GPU memory usage but requires more GPUs and introduces communication overhead. The question specifies limited GPU memory, but not necessarily multiple GPUs. Moreover, it does not reduce the total memory footprint; it just distributes it. The goal is to reduce memory footprint, so this is not the best choice.
D
Quantization of model weights to INT8 or FP8 using TensorRT-LLM's quantization toolkit.
Quantization reduces the precision of model weights, lowering memory usage and enabling faster computations. TensorRT-LLM supports INT8 and FP8 quantization, which can significantly decrease the memory footprint with minimal accuracy loss. This is a standard technique for deploying large models on GPUs with limited memory, as it allows more parameters to fit while maintaining acceptable performance.
E
Storing the model weights on CPU and offloading them to GPU on demand.
Why it fails: Offloading model weights to CPU and transferring them to GPU on demand introduces significant latency and PCIe bandwidth bottlenecks, severely degrading performance. While it can reduce GPU memory usage, it is not a technique for maintaining acceptable performance in a production environment. TensorRT-LLM is designed for GPU-resident inference; CPU offloading is generally not recommended for latency-sensitive applications.

Question 13mediummultiple choice

A data scientist is training a transformer model and observes that the training loss is decreasing while the validation loss is increasing. Which technique should be prioritized to address this specific generalization challenge?

Trap 1: Increase the depth of the transformer architecture layers.

Adding more layers increases the model capacity, which typically exacerbates overfitting in scenarios where validation loss is already diverging. Complex models require more data to converge properly; increasing depth without sufficient data will likely widen the gap between training and validation performance, worsening the existing generalization issue.

Trap 2: Reduce the batch size to increase stochasticity during training.

While smaller batch sizes introduce noise that can help escape local minima, they do not inherently solve overfitting. If the model is already overfitting, increasing the noise level through smaller batches might hinder convergence or make training unstable without directly addressing the model's failure to generalize to validation data.

Trap 3: Decrease the number of training epochs to save time.

Simply stopping early without a validation-based strategy is ineffective. While decreasing epochs is part of early stopping, it must be triggered by validation metrics rather than an arbitrary time constraint. Without a systematic approach, one might stop before the model has learned the relevant features required for good performance.

A
Increase the depth of the transformer architecture layers.
Why it fails: Adding more layers increases the model capacity, which typically exacerbates overfitting in scenarios where validation loss is already diverging. Complex models require more data to converge properly; increasing depth without sufficient data will likely widen the gap between training and validation performance, worsening the existing generalization issue.
B
Implement L2 regularization (weight decay) on the model weights.
L2 regularization penalizes large weights by adding a cost term proportional to the square of the magnitude of coefficients. This forces the model to learn simpler patterns, significantly reducing the variance of the model and effectively mitigating overfitting, which is the primary cause of the divergent loss curves described.
C
Reduce the batch size to increase stochasticity during training.
Why it fails: While smaller batch sizes introduce noise that can help escape local minima, they do not inherently solve overfitting. If the model is already overfitting, increasing the noise level through smaller batches might hinder convergence or make training unstable without directly addressing the model's failure to generalize to validation data.
D
Decrease the number of training epochs to save time.
Why it fails: Simply stopping early without a validation-based strategy is ineffective. While decreasing epochs is part of early stopping, it must be triggered by validation metrics rather than an arbitrary time constraint. Without a systematic approach, one might stop before the model has learned the relevant features required for good performance.

Question 14mediummultiple choice

A developer needs to ensure that an LLM application remains deterministic across multiple runs. Which parameter configuration is most effective?

Trap 1: Increase top_p to 1.0 and set temperature to 0.7.

Setting top_p to 1.0 and temperature to 0.7 introduces high levels of randomness into the output. This configuration maximizes creative variance, making the output highly non-deterministic, which is the opposite of the requirement for consistency and repeatability in automated software testing or production pipelines.

Trap 2: Disable streaming and use a large batch size for inference.

Batch size and streaming affect throughput and latency, not the underlying token selection logic. Changing these parameters will not eliminate the stochasticity of the model's generation process. Determinism relies on sampling configuration, which is independent of how the API delivers the generated content to the client.

Trap 3: Apply top_k filtering with a value of 50.

Top_k filtering limits the pool of candidates but does not force a single path. Even with top_k enabled, the model still selects from the top 50, allowing for variation if the temperature is above zero. Therefore, it does not guarantee the determinism required for consistent testing.

A
Increase top_p to 1.0 and set temperature to 0.7.
Why it fails: Setting top_p to 1.0 and temperature to 0.7 introduces high levels of randomness into the output. This configuration maximizes creative variance, making the output highly non-deterministic, which is the opposite of the requirement for consistency and repeatability in automated software testing or production pipelines.
B
Set temperature to 0.0 and define a fixed seed.
Temperature 0.0 forces the model to choose the most likely token (greedy decoding), while a fixed seed ensures the underlying noise in the sampling process remains constant. Combined, these create a highly deterministic environment where inputs consistently map to the same output tokens, satisfying the requirement.
C
Disable streaming and use a large batch size for inference.
Why it fails: Batch size and streaming affect throughput and latency, not the underlying token selection logic. Changing these parameters will not eliminate the stochasticity of the model's generation process. Determinism relies on sampling configuration, which is independent of how the API delivers the generated content to the client.
D
Apply top_k filtering with a value of 50.
Why it fails: Top_k filtering limits the pool of candidates but does not force a single path. Even with top_k enabled, the model still selects from the top 50, allowing for variation if the temperature is above zero. Therefore, it does not guarantee the determinism required for consistent testing.

Question 15hardmultiple choice

Refer to the exhibit. Given the current configuration, what is the primary risk during high-traffic bursts?

Exhibit

{
  "policy": "strict",
  "quantization": "fp8",
  "max_concurrent_requests": 128,
  "scheduling": "fcfs"
}

Trap 1: FP8 quantization will cause excessive model hallucinations.

FP8 quantization is a standard optimization that preserves accuracy while reducing VRAM usage. It does not introduce hallucinations. The risks associated with FP8 are usually related to precision loss if the model hasn't been calibrated, but it does not inherently impact the logic or factual output.

Trap 2: The request limit of 128 is too low to saturate the GPU.

A limit of 128 concurrent requests is actually quite high for many models. Setting it higher could lead to memory exhaustion (OOM errors) rather than failing to saturate the GPU. The bottleneck is the scheduling policy, not the number of requests the system is configured to permit.

Trap 3: The strict policy setting prevents dynamic batching.

The 'strict' policy usually refers to security or compliance, not the batching algorithm. While potentially restrictive, it is not the cause of performance degradation under load. Performance issues are driven by the FCFS scheduling, which is the incorrect choice for managing diverse, asynchronous inference requests.

A
FP8 quantization will cause excessive model hallucinations.
Why it fails: FP8 quantization is a standard optimization that preserves accuracy while reducing VRAM usage. It does not introduce hallucinations. The risks associated with FP8 are usually related to precision loss if the model hasn't been calibrated, but it does not inherently impact the logic or factual output.
B
Head-of-line blocking due to the FCFS scheduling policy.
FCFS processes requests in the order they arrive regardless of their compute requirements. A single long-generation request will occupy GPU resources, causing all subsequent, potentially short requests to wait. This leads to poor overall system latency and creates a bottleneck during high-traffic bursts, significantly degrading user experience.
C
The request limit of 128 is too low to saturate the GPU.
Why it fails: A limit of 128 concurrent requests is actually quite high for many models. Setting it higher could lead to memory exhaustion (OOM errors) rather than failing to saturate the GPU. The bottleneck is the scheduling policy, not the number of requests the system is configured to permit.
D
The strict policy setting prevents dynamic batching.
Why it fails: The 'strict' policy usually refers to security or compliance, not the batching algorithm. While potentially restrictive, it is not the cause of performance degradation under load. Performance issues are driven by the FCFS scheduling, which is the incorrect choice for managing diverse, asynchronous inference requests.

Question 16mediummultiple choice

A developer is building a retrieval-augmented generation service and needs to embed millions of document chunks and run low-latency similarity search over them on GPU. They want a library that handles both index construction and search with GPU acceleration. Which NVIDIA component should they use?

Trap 1: NVIDIA NeMo Retriever embedding microservices for generating…

NeMo Retriever microservices produce high-quality embeddings, but they are the encoding stage rather than the similarity search index. The developer still needs a component that stores vectors and answers nearest-neighbor queries with GPU acceleration. Using only embedding services leaves the index construction and low-latency search requirement unmet, so it addresses half the pipeline.

Trap 2: NVIDIA Triton Inference Server with an ensemble pipeline that…

Triton orchestrates model execution and ensembles, but it is not a vector index. It can host an embedding model and a reranker behind endpoints, yet similarity search over millions of vectors requires an index structure with GPU-accelerated distance computation. Triton would need an external retrieval library underneath, so it does not itself satisfy the search requirement.

Trap 3: NVIDIA TensorRT-LLM for compiling the embedding model into an…

TensorRT-LLM optimizes generative transformer inference and can accelerate embedding model execution, but it does not build or query a vector index. The scenario's core need is similarity search over millions of stored vectors, which is an index and retrieval problem. Compiling the encoder alone would speed embedding generation while leaving the search stage without a GPU-accelerated index.

A
NVIDIA cuVS, a GPU-accelerated library for vector search and clustering that builds and queries similarity indexes.
cuVS provides GPU-accelerated vector search and clustering primitives, including index build and nearest-neighbor query paths. It is designed exactly for large-scale embedding search where low latency matters, covering both index construction and search. That matches the developer's dual requirement of embedding millions of chunks and serving fast similarity lookups on GPU.
B
NVIDIA NeMo Retriever embedding microservices for generating document and query embeddings at scale.
Why it fails: NeMo Retriever microservices produce high-quality embeddings, but they are the encoding stage rather than the similarity search index. The developer still needs a component that stores vectors and answers nearest-neighbor queries with GPU acceleration. Using only embedding services leaves the index construction and low-latency search requirement unmet, so it addresses half the pipeline.
C
NVIDIA Triton Inference Server with an ensemble pipeline that chains preprocessing and postprocessing steps.
Why it fails: Triton orchestrates model execution and ensembles, but it is not a vector index. It can host an embedding model and a reranker behind endpoints, yet similarity search over millions of vectors requires an index structure with GPU-accelerated distance computation. Triton would need an external retrieval library underneath, so it does not itself satisfy the search requirement.
D
NVIDIA TensorRT-LLM for compiling the embedding model into an optimized engine.
Why it fails: TensorRT-LLM optimizes generative transformer inference and can accelerate embedding model execution, but it does not build or query a vector index. The scenario's core need is similarity search over millions of stored vectors, which is an index and retrieval problem. Compiling the encoder alone would speed embedding generation while leaving the search stage without a GPU-accelerated index.

Question 17mediummultiple choice

A data scientist is analyzing the output of a Llama 3 8B model on a summarization task. The token-level log-probabilities are extracted, and the goal is to visualize how confident the model is in each generated token across the summary. Which visualization is most appropriate for showing the per-token probability distribution and identifying tokens where the model is uncertain?

Trap 1: A bar chart of the top-5 predicted tokens for the entire summary,…

Aggregating top-5 predictions over the whole summary destroys the per-position information needed to see where uncertainty occurs. A bar chart of aggregated counts shows which tokens were frequently predicted, not how confident the model was at each step. It cannot reveal token-level uncertainty across the summary.

Trap 2: A t-SNE scatter plot of the hidden states of all generated tokens,…

t-SNE visualizes high-dimensional embeddings in 2D and preserves local neighborhood structure, but it does not directly represent the model's output probability for each token. Colored by token ID, it would show clustering of hidden states, not confidence. It cannot answer which tokens the model is uncertain about during generation.

Trap 3: A confusion matrix comparing predicted tokens to ground-truth…

A confusion matrix is designed for classification with a fixed, finite label set, not for open-ended text generation where the label space is the whole vocabulary. It would collapse token identities into an enormous matrix and obscure the sequential confidence pattern. It does not show per-token probability or position, so it fails this scenario.

A
A bar chart of the top-5 predicted tokens for the entire summary, aggregated across all positions.
Why it fails: Aggregating top-5 predictions over the whole summary destroys the per-position information needed to see where uncertainty occurs. A bar chart of aggregated counts shows which tokens were frequently predicted, not how confident the model was at each step. It cannot reveal token-level uncertainty across the summary.
B
A t-SNE scatter plot of the hidden states of all generated tokens, colored by token ID.
Why it fails: t-SNE visualizes high-dimensional embeddings in 2D and preserves local neighborhood structure, but it does not directly represent the model's output probability for each token. Colored by token ID, it would show clustering of hidden states, not confidence. It cannot answer which tokens the model is uncertain about during generation.
C
A line chart plotting the log-probability of each generated token against its position in the output sequence.
Plotting log-probability versus token position directly shows per-token confidence and highlights dips where the model is uncertain, which is exactly the goal. This line chart is a standard way to inspect generation quality token by token, and it scales to long sequences without losing individual token information.
D
A confusion matrix comparing predicted tokens to ground-truth tokens for the entire summary.
Why it fails: A confusion matrix is designed for classification with a fixed, finite label set, not for open-ended text generation where the label space is the whole vocabulary. It would collapse token identities into an enormous matrix and obscure the sequential confidence pattern. It does not show per-token probability or position, so it fails this scenario.

Question 18easymultiple choice

Which of the following scenarios best represents an 'Adversarial Attack' against an LLM?

Trap 1: A user accidentally providing a very long, complex question that…

This is a system instability or a resource-exhaustion bug, not an adversarial attack. An attack requires intentionality and a malicious goal. A long query causing a crash is a failure of system hardening or resource management, not a breach of the model's behavioral constraints or safety policies.

Trap 2: The model failing to correctly answer a question because the…

This is a limitation of the model's knowledge base, not an attack. AI models are not omniscient, and failing to provide an answer to an out-of-distribution question is a expected behavior, not a security vulnerability. It does not involve any malicious attempt to force the model into error.

Trap 3: A developer updating the model to use a new, more efficient…

Updating the model architecture is a standard development and maintenance activity. It is the opposite of an attack; it is an improvement effort. It has no malicious intent and does not seek to subvert the model's safety or operational parameters, but rather to improve its performance or reliability.

A
A user accidentally providing a very long, complex question that causes a memory error.
Why it fails: This is a system instability or a resource-exhaustion bug, not an adversarial attack. An attack requires intentionality and a malicious goal. A long query causing a crash is a failure of system hardening or resource management, not a breach of the model's behavioral constraints or safety policies.
B
A user inputting a specifically engineered prompt to bypass content filters.
This is a classic adversarial attack, such as 'jailbreaking,' where the user manipulates the prompt to trick the model into producing restricted content. By crafting specific contexts, the user forces the model to ignore its safety training, which is a direct threat to the trustworthiness of the application.
C
The model failing to correctly answer a question because the information is not in its training data.
Why it fails: This is a limitation of the model's knowledge base, not an attack. AI models are not omniscient, and failing to provide an answer to an out-of-distribution question is a expected behavior, not a security vulnerability. It does not involve any malicious attempt to force the model into error.
D
A developer updating the model to use a new, more efficient activation function.
Why it fails: Updating the model architecture is a standard development and maintenance activity. It is the opposite of an attack; it is an improvement effort. It has no malicious intent and does not seek to subvert the model's safety or operational parameters, but rather to improve its performance or reliability.