NCA-GENL Practice Test 19 — 15 Questions

Question 1

An AI governance team is preparing an NVIDIA-hosted LLM for a regulated financial service. They need a documented, repeatable method to detect whether the model produces systematically different approval recommendations for otherwise identical applicants across demographic groups. Which practice best meets this need?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Conduct a structured bias evaluation using counterfactual test cases that vary only protected attributes and compare approval rates across groups.. Counterfactual testing is the standard way to isolate disparate treatment: by changing only the protected attribute across otherwise identical applicants, any shift in recommendation is attributable to that attribute, and group-level approval rates quantify the disparity. Leaderboards, self-reports, and latency monitoring cannot produce a reproducible, documented fairness metric tied to matched inputs, so they fail the governance requirement.

Answer

Ask the model to self-report whether its own recommendations are biased and record the responses.

Answer

Run the model through a public leaderboard benchmark and publish the aggregate accuracy score.

Answer

Monitor production traffic for anomalous latency spikes that might indicate unequal treatment of certain requests.

Question 2

A retail company wants its customer-facing LLM assistant to refuse requests for medical advice, legal advice, and instructions for dangerous activities. The team needs a runtime mechanism that inspects both user input and model output and can block or rewrite disallowed content without retraining the base model. Which NVIDIA component is designed for this purpose?

Accepted Answer

NVIDIA NeMo Guardrails. The requirement is runtime inspection and control of both user input and model output against topical policies, without retraining. NeMo Guardrails is the NVIDIA component purpose-built for defining and enforcing such conversational rails. The other options address inference speed, model serving infrastructure, or performance profiling, none of which can express or enforce content policies in a live assistant.

Answer

NVIDIA TensorRT-LLM

Answer

NVIDIA Triton Inference Server

Answer

NVIDIA Nsight Systems

Question 3

A data scientist is working with a dataset that has a highly skewed distribution, with one class representing only 2% of the samples. They are training a binary classifier and notice that the model predicts the majority class almost exclusively. Which technique is most appropriate to address this issue?

Accepted Answer

Apply class weighting in the loss function. Class imbalance causes models to favor the majority class because the loss function is dominated by those samples. Applying class weights adjusts the loss to penalize minority class errors more heavily, which balances the influence of each class during training. This is a direct and effective technique for improving minority class detection.

Answer

Remove the minority class samples

Answer

Use a simpler model architecture

Answer

Increase the learning rate

Question 4

A machine learning engineer is evaluating a generative language model for a chatbot application. They notice that the model frequently generates repetitive phrases and gets stuck in loops. Which decoding strategy is most likely to reduce this repetition?

Accepted Answer

Nucleus sampling with a repetition penalty. Repetition in generated text often stems from decoding strategies that favor high-probability tokens. Nucleus sampling with a repetition penalty addresses this by dynamically truncating the probability distribution and penalizing repeated tokens, encouraging diverse and non-repetitive outputs. Greedy and beam search lack such penalties, and top-k with small k restricts diversity.

Answer

Top-k sampling with a small k

Answer

Greedy search

Answer

Beam search with a large beam width

Question 5

An AI team is preparing to release an LLM-powered legal research assistant. Before launch, they want to quantify how often the model produces confident but unsupported legal citations. Which evaluation approach most directly measures this failure mode?

Accepted Answer

Run a benchmark of legal queries and score each generated citation against an authoritative case-law database to compute a hallucination rate.. Directly testing whether generated citations exist and support their propositions requires checking them against an authoritative case-law source. Aggregating those checks yields a concrete hallucination rate that can be compared across versions and prompts. Response length, user surveys, and perplexity all measure adjacent qualities but cannot verify factual grounding, so they would not quantify the specific failure mode.

Answer

Measure the average response length in tokens across a sample of legal queries.

Answer

Compare the model's perplexity on a held-out set of legal documents to its perplexity on general web text.

Answer

Survey the legal team to ask whether they generally trust the assistant's answers.

Question 6

A global retailer uses an NVIDIA-powered LLM to generate product descriptions in multiple languages. The compliance team requires that the model's outputs do not contain culturally insensitive or legally restricted terms in any target market. Which evaluation practice should be implemented to detect such issues before deployment?

Accepted Answer

Conduct red teaming exercises with native speakers to probe for culturally insensitive or legally restricted outputs.. Red teaming with native speakers is the most effective way to identify culturally insensitive or legally restricted terms because it leverages human judgment and local expertise. Automated benchmarks and sentiment analysis lack the contextual understanding needed for multi-language compliance. Adjusting temperature does not filter content and may introduce more risk.

Answer

Increase the model's temperature during generation to produce more varied descriptions, reducing the chance of restricted terms.

Answer

Run a standard benchmark like MMLU to measure the model's general language understanding across languages.

Answer

Use automated sentiment analysis to flag negative tones in the generated descriptions.

Question 7

A developer is building a document-summarization service on NVIDIA NIM for LLMs and wants to stream partial tokens to the client while the model is still generating. The NIM endpoint exposes an OpenAI-compatible /chat/completions route. Which request parameter should the developer set to receive incremental token deltas rather than one complete response body?

Accepted Answer

Set "stream": true in the JSON request body and consume the server-sent event chunks.. Streaming on an OpenAI-compatible NIM chat completions endpoint is controlled by the stream boolean in the request body, which switches the transport to server-sent events carrying incremental delta objects. The other parameters alter candidate count, content negotiation, or scoring metadata, and none of them cause tokens to be emitted before generation completes.

Answer

Set "logprobs": true and reconstruct partial text from the token log probabilities.

Answer

Set the SSE header "Accept: text/event-stream" on the request without changing the body.

Answer

Set "n": 2 so the endpoint returns two candidate completions the client can merge.

Question 8

A team is evaluating a generative large language model for a customer support chatbot. They need to ensure the model produces factually accurate and contextually appropriate responses while avoiding harmful or biased outputs. Which two techniques are most effective for aligning the model's behavior with these safety and quality requirements? (Choose two.)

Accepted Answer

Supervised fine-tuning on curated, high-quality demonstrations. RLHF and supervised fine-tuning on curated demonstrations are central alignment techniques. RLHF uses human preferences to optimize for safety and helpfulness, while supervised fine-tuning directly teaches desired responses. Temperature, parameter count, and batch size do not specifically improve factual accuracy or reduce harmful outputs.

Answer

Using a larger batch size during pretraining

Answer

Reducing the model's parameter count

Answer

Increasing the model's temperature during inference

Question 9

A data scientist is pretraining a 12-layer transformer encoder on a corpus of legal contracts. To prevent the model from simply copying each token to its output during masked language modeling, the team needs a strategy that forces the model to learn bidirectional context. Which masking approach should they apply?

Accepted Answer

Mask 15% of input tokens at random, replacing 80% with [MASK], 10% with a random token, and 10% with the original token.. The standard masked language modeling objective masks a small percentage of tokens and requires the model to reconstruct them from bidirectional context. Using mostly [MASK] with some random and unchanged tokens balances learning signal and reduces pretraining-finetuning mismatch. This is the correct way to force bidirectional understanding in a transformer encoder.

Answer

Mask the final token in every sequence and train the model to predict only that token.

Answer

Replace 50% of tokens with [MASK] and leave the rest unchanged.

Answer

Randomly shuffle the order of tokens within each sequence before feeding them to the model.

Question 10

An enterprise is preparing an LLM-based document assistant for internal use and must demonstrate accountability for Trustworthy AI to its auditors. Which two practices most directly provide verifiable accountability for the assistant's behavior? (Choose two.)

Accepted Answer

Maintain an immutable audit log that records prompts, retrieved sources, model versions, and generated responses for each interaction.. Verifiable accountability rests on two complementary pillars: a durable record of what the system did, and a governed process that assigns responsibility for what it is allowed to do. Immutable logging supplies the evidentiary trail, while documented roles and approval gates supply the human ownership and change control. Together they let auditors reconstruct decisions and confirm that modifications were reviewed by accountable parties before reaching users.

Answer

Publish the assistant's system prompt on the company intranet so employees understand how the model is instructed to behave.

Answer

Collect user satisfaction ratings after each interaction and display the rolling average on a dashboard for leadership.

Answer

Run a one-time red teaming exercise before launch and archive the final report in the project repository.

Question 11

A data scientist is preparing a dataset of 50,000 customer support conversations to fine-tune a large language model. The conversations vary widely in length, and many exceed the model's maximum context window. The team wants to preserve conversational coherence while avoiding truncation that removes critical resolution details. Which preprocessing strategy is most appropriate?

Accepted Answer

Chunk conversations into overlapping segments of at most N tokens, preserving order and overlap between chunks. Long conversations that exceed the context window must be divided into smaller pieces while retaining meaning. Overlapping chunks preserve the order of turns and keep cross-boundary context visible, so the model can still learn from resolutions that depend on earlier messages. Random sampling and head truncation lose critical information, while padding does nothing for conversations that are already too long.

Answer

Randomly sample a fixed number of tokens from each conversation and discard the rest

Answer

Truncate each conversation to the first N tokens that fit the context window

Answer

Pad every conversation with a special token until it reaches the maximum context window

Question 12

A developer is building a customer support chatbot using NVIDIA NIM microservices. They need the model to always respond in a formal tone and never mention competitor products. Where should these directives be placed in the API request to ensure consistent behavior across all user interactions?

Accepted Answer

In the 'system' role message at the beginning of the messages array. The system role message provides persistent, high-priority instructions that shape the model's behavior throughout the conversation. By placing tone and content restrictions there, the developer ensures they are applied consistently to every response. Other roles or parameters do not carry the same authoritative weight for enforcing global rules.

Answer

In the 'assistant' role message pre-filled with a sample response

Answer

In the 'temperature' parameter set to a low value

Answer

In the 'user' role message appended to each turn

Question 13

A machine learning engineer is fine-tuning a pre-trained language model on a small domain-specific dataset. She notices that the model quickly achieves high accuracy on the training set but performs poorly on the validation set. She wants to mitigate this overfitting without collecting more data. Which technique is most appropriate?

Accepted Answer

Apply L2 regularization. L2 regularization is a classic method to combat overfitting by penalizing large weights, which encourages the model to learn simpler patterns that generalize better. When fine-tuning on a small dataset, it is particularly effective because it constrains the model's capacity to memorize noise, improving validation performance.

Answer

Increase the learning rate

Answer

Train for more epochs

Answer

Use a larger batch size

Question 14

An enterprise is deploying an LLM-based document summarization system for internal legal contracts. The security team wants to implement measures to detect and mitigate prompt injection attacks that could cause the model to leak confidential information. Which TWO measures should be implemented? (Choose two.)

Accepted Answer

Implement output filtering that redacts any confidential legal terms or entity names before displaying the summary.. Input rails and output filtering together provide a robust defense against prompt injection. Input rails block malicious prompts before they reach the model, while output filtering redacts sensitive information even if an injection succeeds. The other options either do not address the attack vector or are not runtime mitigations suitable for a deployed system.

Answer

Fine-tune the model on a dataset of adversarial prompts to teach it to ignore malicious instructions.

Answer

Increase the model's temperature to make its responses less predictable and harder for attackers to exploit.

Answer

Restrict the model's access to the internet to prevent it from fetching external malicious content.

Question 15

A team is deploying a large language model for real-time inference on an NVIDIA GPU. They observe that the first few inference requests have high latency, but subsequent requests are much faster. What is the most likely explanation for this behavior?

Accepted Answer

CUDA kernels are being compiled and cached. The initial high latency is due to just-in-time compilation of CUDA kernels, which occurs on the first execution of each operation. Once compiled, the kernels are cached for reuse, making subsequent inferences faster. This is a well-known behavior in deep learning frameworks and explains the warm-up effect.

Answer

The model is being quantized on the fly

Answer

The GPU is warming up its clock speed

Answer

The model weights are being loaded from disk