NCA-GENL Practice Test 25 — 15 Questions

Question 1

Refer to the exhibit. You are experimenting with a model and find the validation loss is increasing while training loss decreases. Which parameter should you adjust first?

Accepted Answer

Increase dropout. The divergence between training loss and validation loss is a classic sign of overfitting. Increasing the 'dropout' value is a highly effective way to introduce noise during training, preventing the model from relying on specific neuron activations. This forces more robust feature learning and is a standard first-line defense in the experimentation process to bridge the gap between training and validation performance.

Answer

Change activation function to ReLU

Answer

Change optimizer to SGD

Answer

Decrease weight decay

Question 2

A developer is building a retrieval-augmented generation (RAG) system for an internal knowledge base. The system must answer questions using company documents that are updated frequently. Which component is primarily responsible for retrieving the most relevant document chunks to include in the LLM's context?

Accepted Answer

The vector database and its similarity search.. Retrieval-augmented generation separates knowledge retrieval from generation. The vector database stores embeddings of document chunks, and a similarity search matches the query embedding to the most relevant chunks. Those chunks are then inserted into the LLM's context. This design allows the knowledge base to be updated independently of the model, which is essential for frequently changing internal documents.

Answer

The fine-tuning dataset used to adapt the LLM.

Answer

The LLM's attention mechanism.

Answer

The tokenizer used to preprocess the query.

Question 3

You are analyzing the output of an LLM inference endpoint that returns a JSON payload containing a top-k token probability distribution for a single generated step. Which visualization most directly communicates the model's confidence ranking across the returned tokens?

Accepted Answer

A horizontal bar chart with tokens on the y-axis sorted by probability descending.. Ranking data is best shown with sorted bar lengths because position and length are preattentively processed. A descending horizontal bar chart lets a reviewer instantly see the top token and the margin over runners-up, which is exactly the confidence information the JSON payload contains. Other chart types either distort magnitude judgments or plot against irrelevant dimensions.

Answer

A scatter plot of probability versus vocabulary index position.

Answer

A line chart plotting probability against token string length.

Answer

A pie chart showing each token's probability as a slice of the total mass.

Question 4

You are analyzing the quality of a synthetic data generation pipeline for an LLM. You want to ensure the synthetic data does not suffer from 'mode collapse' compared to the real-world dataset. Which visualization technique is most effective for comparing the diversity of the two datasets?

Accepted Answer

A scatter plot of embedding density. Mode collapse is the phenomenon where a generative model produces a limited subset of variations. To detect this, you can compute embeddings for both real and synthetic data and plot them using a density-based approach. If the synthetic density plot is concentrated in small areas compared to the broad coverage of the real data, it indicates mode collapse. This comparison is vital for validating that synthetic data preserves the distribution of the original corpus.

Answer

A bar chart of the number of documents

Answer

A line chart of training loss

Answer

A histogram of average word count

Question 5

A data analyst is preparing a dashboard for an LLM inference service. The service logs per-request latency in milliseconds. The team wants a single chart that lets an on-call engineer quickly see both the typical latency and how often requests exceed the service-level objective of 500 ms. Which visualization best supports that goal?

Accepted Answer

A histogram of latency with a vertical marker at 500 ms. A histogram shows the latency distribution in ordered bins, so the typical range and the tail are both visible. Adding a vertical marker at 500 ms turns the chart into a threshold monitor, making SLO exceedance immediately apparent. Mean-only line charts, pie charts, and scatter plots each omit either the distribution shape or the threshold context needed for fast operational decisions.

Answer

A pie chart of requests grouped by latency bucket

Answer

A time-series line chart of mean latency only

Answer

A scatter plot of request size versus latency

Question 6

A developer is using the NVIDIA NeMo framework to fine-tune a large language model. They want to reduce GPU memory usage during training without significantly sacrificing model quality. Which technique should they apply?

Accepted Answer

Enable gradient checkpointing. Gradient checkpointing reduces memory by storing only a subset of activations and recomputing the rest during backpropagation. This allows training larger models or using bigger batches on the same GPU. Other options either increase memory usage or do not affect it, making gradient checkpointing the correct choice for memory-constrained fine-tuning.

Answer

Increase the micro batch size

Answer

Disable mixed precision training

Answer

Use a higher learning rate

Question 7

Which approach is most effective for preventing a model from leaking proprietary information included in its training set?

Accepted Answer

Implementing rigorous data pre-processing and output filtering.. To prevent the leakage of proprietary training data, organizations must employ data-sanitization techniques, such as PII scrubbing and filtering, before training begins. Additionally, during inference, techniques like output-guarding and strict prompt-management help ensure the model does not recover and emit sensitive information. This multi-layered strategy is crucial for Trustworthy AI as it protects the confidentiality of corporate intellectual property while still allowing for the benefits of LLM-based productivity.

Answer

Increasing the model's temperature to make outputs more creative.

Answer

Training the model on larger datasets to dilute the impact of sensitive info.

Answer

Reducing the number of training epochs to stop the model from learning.

Question 8

When fine-tuning a Large Language Model using PEFT (Parameter-Efficient Fine-Tuning) techniques like LoRA, what is the primary technical advantage being leveraged?

Accepted Answer

Weight matrices are frozen and low-rank adaptors are trained.. LoRA injects trainable rank decomposition matrices into the transformer layers while keeping pre-trained weights frozen. This approach significantly reduces the number of parameters requiring gradient updates, which saves memory and compute. This is essential for developers working with limited GPU hardware, allowing them to adapt massive models to specific tasks without full-parameter fine-tuning, which would be computationally prohibitive for most enterprise-grade infrastructure deployments.

Answer

Full parameter updates are performed on every layer.

Answer

The entire model is converted to INT8 precision during training.

Answer

All model activations are stored in CPU memory.

Question 9

When designing a scalable inference microservice, which factor most significantly impacts the 'Time to First Token' (TTFT) for concurrent users?

Accepted Answer

The prefill phase efficiency and GPU throughput.. TTFT is primarily driven by the time taken to process the input prompt and initialize the decoding state. In a multi-user environment, resource contention and efficient scheduling become the main inhibitors. Optimizing the prefill phase, which is compute-bound, is critical for achieving low TTFT, allowing users to perceive the application as responsive even during high load periods.

Answer

The total number of output tokens generated.

Answer

The network latency between the user and the load balancer.

Answer

The number of active vector databases connected to the service.

Question 10

Refer to the exhibit. Given the current configuration, what is the primary risk during high-traffic bursts?

Accepted Answer

Head-of-line blocking due to the FCFS scheduling policy.. The 'fcfs' (First-Come, First-Served) scheduling policy is suboptimal for LLM inference because it lacks priority handling. In high-traffic scenarios, long requests block shorter ones, leading to 'head-of-line blocking'. This causes latency spikes for all users. For robust production systems, implementing continuous batching or priority-aware scheduling is necessary to ensure that the GPU utilization remains high while maintaining acceptable response times for individual requests.

Answer

FP8 quantization will cause excessive model hallucinations.

Answer

The request limit of 128 is too low to saturate the GPU.

Answer

The strict policy setting prevents dynamic batching.

Question 11

A machine learning engineer is evaluating a fine-tuned LLM for a customer-facing summarization task. The model produces fluent summaries, but the team needs to detect when the model generates content that is not supported by the source document. Which TWO evaluation approaches are appropriate for measuring factual consistency between the generated summary and the source? (Choose two.)

Accepted Answer

Have human annotators label each summary sentence as supported or not supported by the source document.. Factual consistency requires checking whether the summary's claims are supported by the source document. Entailment-based metrics like FactCC or NLI models directly test this relationship, and human annotation provides a reliable ground truth. Metrics like BLEU, ROUGE, or perplexity measure overlap or fluency and can be fooled by fluent hallucinations, so they are not appropriate for this specific goal.

Answer

Compute BLEU score between the generated summary and a single human-written reference summary.

Answer

Calculate ROUGE-L recall against the source document.

Answer

Measure perplexity of the generated summary under the fine-tuned model.

Question 12

When fine-tuning a model for domain-specific tasks, which THREE metrics should you monitor during the training phase to ensure the experiment is progressing healthily?

Accepted Answer

Training Loss. Monitoring training dynamics is crucial for detecting issues early. Training loss confirms the optimizer is reducing error, while validation loss catches overfitting before it becomes permanent. Gradient norm tracking is the most reliable way to identify instability—such as the infamous 'exploding gradients'—which is common in complex LLM training. Together, these metrics form the dashboard necessary for a data scientist to make informed decisions about when to stop, adjust, or continue their experiments.

Answer

Model disk usage

Answer

Total number of GPUs used

Question 13

A healthcare AI team is using NVIDIA NeMo to fine-tune an LLM for clinical note summarization. During evaluation, they notice the model generates different summaries for the same patient note when the note includes demographic descriptors, even though clinical content is identical. The team wants to quantify this behavior systematically before deployment. Which approach should they use to measure the model's sensitivity to demographic attributes?

Accepted Answer

Run a counterfactual fairness test by swapping demographic terms and measuring output divergence.. Counterfactual fairness testing is the established method to detect whether a model's output changes when only a protected attribute is altered. It provides a quantitative measure of demographic sensitivity, which is essential before deploying a clinical summarization model. The other options either change model behavior without measuring bias or fail to isolate the demographic variable, so they do not meet the evaluation requirement.

Answer

Fine-tune the model again on a larger dataset without demographic terms.

Answer

Increase the temperature setting to produce more diverse summaries.

Answer

Apply TensorRT-LLM INT8 quantization to reduce model variance.

Question 14

Which NVIDIA framework is specifically designed to facilitate the deployment of optimized LLMs as microservices with standardized APIs?

Accepted Answer

NVIDIA NIM. NVIDIA NIM (NVIDIA Inference Microservices) is the standard framework for packaging optimized AI models as containerized microservices. By providing standardized APIs, NIM allows developers to easily integrate state-of-the-art models into their applications. This abstraction simplifies the deployment lifecycle, ensuring that developers can focus on building application features rather than managing the complexities of model optimization, serving infrastructure, and hardware-specific performance tuning.

Answer

NVIDIA TensorRT-LLM

Answer

NVIDIA CUDA-X

Answer

NVIDIA NeMo

Question 15

A researcher is using NVIDIA's 'TensorRT-LLM' to optimize an LLM. During the experimentation phase, they observe the model's accuracy drops significantly after quantization. What is the most appropriate next step?

Accepted Answer

Use a representative calibration dataset for quantization.. Post-training quantization often introduces errors that degrade model accuracy. To mitigate this, techniques like 'Quantization-Aware Training' (QAT) or using a calibration dataset are essential. These methods help the model adapt to the lower precision format during or after the process. Mastering these techniques is critical for delivering high-performance, resource-efficient models that maintain their accuracy in production environments.

Answer

Revert to full FP32 training without any optimization.

Answer

Increase the model's hidden dimension size.

Answer

Switch to a smaller base model architecture.