NCA-GENL Practice Test 26 — 15 Questions

Question 1

When experimenting with synthetic data generation to improve model performance, what is the most important risk to monitor?

Accepted Answer

The introduction of model bias or hallucinations.. Synthetic data can inadvertently contain artifacts or biases present in the teacher model. Over time, 'model collapse'—where the model learns from its own generated noise—can occur, leading to a degradation in performance. Regular evaluation on a held-out, human-verified dataset is essential to ensure that the synthetic data is actually improving the model's ability to reason, rather than just forcing it to mirror the stylistic flaws of the synthetic data source.

Answer

The increase in training time.

Answer

The file format of the training data.

Answer

The cost of disk storage.

Question 2

A developer is building a retrieval-augmented generation service and needs to embed millions of document chunks and run low-latency similarity search over them on GPU. They want a library that handles both index construction and search with GPU acceleration. Which NVIDIA component should they use?

Accepted Answer

NVIDIA cuVS, a GPU-accelerated library for vector search and clustering that builds and queries similarity indexes.. The task splits into encoding and retrieval, and the retrieval half demands an index plus fast nearest-neighbor search. cuVS supplies GPU-accelerated vector search and clustering, handling index build and query for large embedding sets. Embedding microservices, TensorRT-LLM, and Triton each address adjacent concerns such as encoding, generation optimization, or orchestration, but none provides the vector index and search capability the scenario requires.

Answer

NVIDIA NeMo Retriever embedding microservices for generating document and query embeddings at scale.

Answer

NVIDIA Triton Inference Server with an ensemble pipeline that chains preprocessing and postprocessing steps.

Answer

NVIDIA TensorRT-LLM for compiling the embedding model into an optimized engine.

Question 3

In the context of analyzing LLM output safety, what does a 'confusion matrix' help identify?

Accepted Answer

Patterns of misclassification in safety filters.. A confusion matrix provides a clear breakdown of True Positives, False Positives, True Negatives, and False Negatives for classification tasks, such as content moderation. By visualizing where the model misclassifies safe vs. unsafe content, developers can identify bias or systematic errors in safety filters. This is vital for fine-tuning the model's safety boundaries and ensuring that the deployment adheres to strict ethical and security guidelines while minimizing false rejections of legitimate user queries.

Answer

The total number of tokens generated.

Answer

The latency of the safety filter response.

Answer

The distribution of model parameters.

Question 4

When using NVIDIA Riva for speech-to-text applications, which component provides the real-time transcription service based on streaming audio inputs?

Accepted Answer

Riva Speech API (ASR).. Riva Speech API is designed for high-performance, low-latency streaming applications. It uses optimized neural networks to transcribe audio in real-time as it arrives. By leveraging GPU acceleration, Riva ensures that the transcription remains synchronized with the audio feed, which is critical for live transcription services, voice assistants, and other interactive applications that require immediate feedback from the system to the user.

Answer

The TensorRT Model Optimizer.

Answer

The NVIDIA NeMo Guardrails Engine.

Answer

The Triton Model Repository.

Question 5

A data scientist observes that the model's loss plateaus early during fine-tuning. Which visualization would best help diagnose if the model is suffering from 'catastrophic forgetting'?

Accepted Answer

A line chart comparing original and new task performance.. Catastrophic forgetting occurs when a model loses the ability to perform tasks it previously mastered while learning new ones. A line chart comparing the model's performance on the original evaluation set versus the new training set over time is the best visualization. Seeing performance on the original tasks plummet while the new task performance improves confirms the issue, allowing developers to adjust training parameters like lower learning rates or replay buffers to maintain overall performance.

Answer

A histogram of activation values.

Answer

A pie chart showing weight distribution.

Answer

A heat map of the training loss per sample.

Question 6

During development of a RAG application using NVIDIA NeMo Guardrails, why is it important to define specific 'canonical forms' in the configuration?

Accepted Answer

To standardize user inputs for easier rule matching.. Canonical forms in NeMo Guardrails act as a bridge between diverse user inputs and the system's internal logic. By mapping many variations of a user query to a single canonical form, developers simplify the dialog management rules. This ensures consistent responses and predictable guardrail behavior, which is crucial for maintaining safety and accuracy in enterprise LLM applications where ambiguous user intent could lead to potentially dangerous or incorrect AI outputs.

Answer

To increase the training speed of the underlying LLM.

Answer

To force the LLM to output only JSON-formatted data.

Answer

To bypass the authentication module of the application.

Question 7

During an LLM experimentation phase using NVIDIA NeMo, an ML engineer needs to systematically track hyperparameters, dataset lineage, and evaluation artifacts to meet strict auditing standards. Which TWO actions should the engineer take to achieve comprehensive experiment tracking?

Accepted Answer

Integrate an experiment tracking framework like MLflow to log hyperparameters and metrics automatically.. Effective experiment tracking in enterprise generative AI requires systematic logging of both code configurations and artifact lineage. Integrating MLflow or NeMo-native logging captures hyperparameters, while artifact stores maintain dataset versions and model weights, ensuring full traceability and regulatory compliance during model evaluation.

Answer

Rely solely on manual terminal output logs stored in local temporary directories.

Answer

Disable all logging mechanisms to maximize GPU execution speed and prevent I/O bottlenecks.

Answer

Store all intermediate checkpoints in volatile system memory without persisting to disk.

Question 8

A data scientist is analyzing the output of a Llama 3 8B model on a summarization task. The token-level log-probabilities are extracted, and the goal is to visualize how confident the model is in each generated token across the summary. Which visualization is most appropriate for showing the per-token probability distribution and identifying tokens where the model is uncertain?

Accepted Answer

A line chart plotting the log-probability of each generated token against its position in the output sequence.. The scenario asks for per-token confidence across a generated summary. A line chart of log-probability by token position preserves the sequential order and directly displays the model's confidence at each step. The other options either aggregate away position information or use dimensionality reduction that does not represent output probabilities.

Answer

A bar chart of the top-5 predicted tokens for the entire summary, aggregated across all positions.

Answer

A t-SNE scatter plot of the hidden states of all generated tokens, colored by token ID.

Answer

A confusion matrix comparing predicted tokens to ground-truth tokens for the entire summary.

Question 9

Refer to the exhibit. What is the effect of the 'enforcement_mode: strict' configuration on the AI application?

Accepted Answer

The request is rejected if it triggers any items in the block list.. Setting the enforcement mode to 'strict' indicates that the system will block any input that matches the defined block list, with zero tolerance for ambiguity. In the context of Trustworthy AI, this is a high-security posture that prioritizes safety over user experience. It ensures that any attempt to elicit prohibited information results in an immediate and non-negotiable rejection, effectively preventing the model from acting upon harmful or sensitive user requests.

Answer

The model will warn the user about PII but continue processing the request.

Answer

The model will automatically redact the PII and then answer the request.

Answer

The system logs the violation but permits the model to generate a response.

Question 10

Which of the following activation functions is most commonly used in hidden layers of deep neural networks to mitigate the vanishing gradient problem?

Accepted Answer

ReLU. The Rectified Linear Unit (ReLU) is the standard activation function for deep networks. Unlike Sigmoid or Tanh, which saturate at high and low values, ReLU maintains a constant gradient of 1 for all positive inputs. This effectively prevents the gradient from vanishing during backpropagation across many layers, allowing for the training of much deeper architectures without needing complex initialization or normalization schemes.

Answer

Sigmoid

Answer

Tanh

Answer

Linear

Question 11

Which of the following scenarios best represents an 'Adversarial Attack' against an LLM?

Accepted Answer

A user inputting a specifically engineered prompt to bypass content filters.. An adversarial attack occurs when a user intentionally crafts inputs designed to deceive the model into ignoring its safety guidelines or producing unintended behavior. This is a primary concern for Trustworthy AI because it highlights the fragility of models when faced with malicious inputs. Recognizing these patterns is essential for developers to implement robust defensive guardrails that maintain the system's integrity under hostile conditions.

Answer

A user accidentally providing a very long, complex question that causes a memory error.

Answer

The model failing to correctly answer a question because the information is not in its training data.

Answer

A developer updating the model to use a new, more efficient activation function.

Question 12

When developing with NVIDIA NeMo, which component is primarily responsible for scaling the training of massive LLMs across multiple GPU nodes?

Accepted Answer

PyTorch Lightning and Megatron-Core.. NVIDIA NeMo leverages PyTorch Lightning and the Megatron-Core library to handle distributed training complexities. This framework is essential because LLMs are too large to fit into a single GPU's memory. By using techniques like tensor parallelism, pipeline parallelism, and data parallelism, NeMo enables developers to train models with hundreds of billions of parameters efficiently across clusters, ensuring consistent performance and scalability in high-performance computing environments.

Answer

The NeMo Data Augmentation Module.

Answer

The TensorRT Model Parser.

Answer

The NVIDIA Driver API.

Question 13

An AI researcher is fine-tuning a Llama-3 model using NeMo Framework and notices high GPU memory usage during training. Which experimentation technique is most effective for reducing memory footprint without sacrificing model quality?

Accepted Answer

Enable gradient checkpointing. Gradient checkpointing is a standard technique in large model experimentation that trades computation time for memory efficiency. By storing only a subset of activations during the forward pass and recomputing others during the backward pass, it enables training larger models or larger batch sizes within the same VRAM constraints. This is critical for scaling experiments when hardware resources are restricted during initial prototyping phases.

Answer

Increase the learning rate significantly

Answer

Reduce the batch size to one

Answer

Switch to a smaller model architecture

Question 14

A data science team at a retail company is building a neural network to predict customer churn from tabular data with mixed numerical and categorical features. They want the model to output a probability between 0 and 1, and they are training with a standard gradient descent optimizer. Which loss function is most appropriate for this binary classification task?

Accepted Answer

Binary cross-entropy loss. Binary cross-entropy is the standard loss for binary classification with a sigmoid output. It directly optimizes the log-likelihood of the correct class, yielding strong gradients when predictions are wrong and well-calibrated probabilities. The other losses either assume multiclass targets, produce non-probabilistic outputs, or suffer from vanishing gradients with sigmoid units.

Answer

Categorical cross-entropy loss

Answer

Mean squared error loss

Answer

Hinge loss

Question 15

An engineer is designing an experiment to measure how prompt phrasing affects the output quality of a deployed LLM. They will test several prompt templates against a fixed evaluation set and want the comparison to be valid. Which two practices are required? (Choose two.)

Accepted Answer

Hold the model, decoding parameters, and evaluation dataset constant across all prompt templates.. A valid prompt experiment isolates the prompt template as the only variable. Keeping the model, decoding parameters, and evaluation dataset fixed means any quality difference can be attributed to phrasing, and scoring all outputs with the same rubric ensures the measurement itself does not shift. Changing datasets, retraining, or varying temperature introduces confounds that make the comparison meaningless.

Answer

Retrain the model after each prompt template so it adapts to the new phrasing.

Answer

Use a different evaluation dataset for each prompt template so the results are not correlated.

Answer

Increase the temperature for each successive template to explore more diverse outputs.