NCA-GENL Practice Test 21 — 15 Questions

Question 1

A team fine-tunes a Llama model with NVIDIA NeMo Framework and must serve it behind an OpenAI-compatible endpoint with no Python glue code. They want the adapter weights kept separate from the base model so several adapters can share one loaded base. Which deployment approach fits these constraints?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Deploy the base model on NVIDIA NIM and load the fine-tuned adapter at runtime through multi-LoRA support in the NIM container.. The constraints are an OpenAI-compatible endpoint, no custom Python service code, and adapter weights stored separately from a shared base model. NIM's multi-LoRA serving satisfies all three: the container exposes standard APIs and loads adapters against one resident base. Merged engines, custom wrappers, and per-adapter Python backends each break at least one stated requirement.

Answer

Export the merged model to a TensorRT-LLM engine and serve it through NVIDIA Triton Inference Server with the TensorRT-LLM backend.

Answer

Package the fine-tuned model inside a custom FastAPI service that wraps the NeMo checkpoint and exposes chat completions.

Answer

Convert the adapter to ONNX and deploy it with NVIDIA Triton Inference Server using the Python backend for each adapter.

Question 2

A global bank must demonstrate to regulators that its LLM-based loan-advisory chatbot treats applicants from different regions and demographic groups equitably. The compliance team asks for an evaluation approach that quantifies outcome disparities across protected groups and produces evidence suitable for audit. Which approach best meets this requirement?

Accepted Answer

Run structured bias and fairness evaluations that measure outcome metrics across defined demographic subgroups and retain the results as audit artifacts. Regulatory proof of equitable treatment requires measured outcomes per protected group, documented and retained. Structured fairness evaluation supplies subgroup disparity metrics tied to the deployed chatbot, creating repeatable audit evidence. Vendor model cards, aggregate satisfaction scores, and larger datasets describe inputs or overall performance but cannot show whether specific groups receive different advisory outcomes, which is what the compliance team must demonstrate.

Answer

Rely on the vendor's model card, which states the base model was trained on diverse data

Answer

Increase the size of the fine-tuning dataset to include more loan applications

Answer

Monitor average user satisfaction scores across the entire customer base

Question 3

A machine learning engineer is deploying a transformer-based language model for real-time translation. They observe that inference latency is too high for the required throughput. The model uses standard multi-head self-attention. Which modification is most likely to reduce latency without significantly degrading translation quality?

Accepted Answer

Use a sparse attention mechanism. Sparse attention reduces the quadratic cost of self-attention by limiting each token's attention to a subset of positions, which lowers computation and latency. Increasing heads, hidden dimension, or layers all raise computational load and latency. Sparse attention is a targeted optimization for real-time inference while preserving translation quality.

Answer

Increase the number of attention heads

Answer

Increase the model's hidden dimension

Answer

Add more layers to the transformer

Question 4

A developer is preparing a RAG service that calls an NVIDIA-hosted LLM endpoint and must reduce hallucinations for questions whose answers are absent from the retrieved context. Which two practices should be applied in the application layer? (Choose two.)

Accepted Answer

Instruct the model in the system prompt to answer only from the supplied context and to state explicitly when the context is insufficient.. Hallucination in RAG is controlled by constraining generation and by refusing to generate when retrieval fails. A grounding system prompt gives the model permission to abstain and limits it to supplied evidence, while a relevance threshold stops irrelevant chunks from ever reaching the prompt. Together they address both the model's behavior and the quality of its input.

Answer

Increase the number of retrieved chunks to the model's full context limit so the answer is guaranteed to be somewhere in the prompt.

Answer

Cache every generated response and replay it for semantically similar questions to keep the answers consistent over time.

Answer

Set the sampling temperature to its maximum so the model explores a wider range of candidate answers and avoids repeating a single phrasing.

Question 5

A developer is building a Retrieval-Augmented Generation (RAG) pipeline using NVIDIA NIM microservices. They need to ensure that the retriever returns the most relevant documents for a given query. Which two components should they optimize? (Choose two.)

Accepted Answer

The embedding model used to vectorize documents and queries. Retrieval quality in RAG depends heavily on the embedding model that encodes semantic meaning and the vector database that efficiently finds nearest neighbors. Optimizing both ensures that the most relevant documents are surfaced. Other parameters like temperature or attention heads affect generation, not retrieval, and tokenizer vocabulary is fixed.

Answer

The LLM's temperature setting

Answer

The tokenizer's vocabulary size

Answer

The number of attention heads in the LLM

Question 6

A machine learning engineer is evaluating a generative LLM for a customer-facing question-answering system. The model produces fluent answers, but during testing it confidently states incorrect facts about company policies. The team wants a metric that specifically measures whether the model's output is supported by the provided source documents. Which evaluation approach is most appropriate?

Accepted Answer

Faithfulness or groundedness evaluation that checks whether claims in the answer are entailed by the retrieved source passages. The team needs to know whether generated answers are supported by the source documents, which is precisely what faithfulness or groundedness evaluation measures. It checks entailment between answer claims and retrieved passages, catching confident hallucinations. Perplexity, BLEU, and ROUGE assess fluency or surface overlap with references, none of which guarantee that a stated policy fact is actually present in the source material.

Answer

Perplexity measured on a held-out set of company documents

Answer

ROUGE score computed against a large corpus of historical customer emails

Answer

BLEU score computed between the model's answers and reference answers

Question 7

A team is serving a 70B-parameter model with NVIDIA Triton Inference Server and TensorRT-LLM. Under concurrent load, GPU memory is exhausted because each request reserves its own large KV cache. Which Triton feature should the team enable to share KV cache blocks across requests that have common prompt prefixes?

Accepted Answer

Paged KV cache with prefix caching enabled in the TensorRT-LLM backend.. Prefix caching combined with paged KV cache in the TensorRT-LLM backend lets Triton reuse cache blocks for requests sharing a prompt prefix, directly reducing memory consumed by duplicate caches. Batching, sequence scheduling, and instance groups tune throughput or state handling but do not deduplicate KV cache blocks.

Answer

Dynamic batching with a larger maximum batch size.

Answer

Model instance groups with multiple instances per GPU.

Answer

Sequence batching with a higher maximum queue delay.

Question 8

A developer is preparing a container for an LLM microservice that will run on an NVIDIA GPU node and must be deployable through NVIDIA NIM. They want the image to be portable across supported GPU generations while still using NVIDIA's optimized inference stack. Which two practices should they follow? (Choose two.)

Accepted Answer

Base the service on an NGC-hosted NIM container image and pin a specific version tag.. Portable NIM deployment rests on using the NGC-published, version-pinned NIM image and exposing the GPU through the NVIDIA Container Toolkit. Bundling weights or drivers into the image, or shipping one SKU-specific TensorRT engine, either breaks the supported deployment model or destroys the cross-generation portability the team requires.

Answer

Compile the model into a TensorRT engine for one exact GPU SKU and ship only that engine.

Answer

Install a full CUDA toolkit and matching driver inside the image for maximum portability.

Answer

Bake the model weights directly into the image layer during the build step.

Question 9

A developer is packaging a fine-tuned Llama 3 model as a TensorRT-LLM engine for an on-premises inference service. The model was trained with a custom tokenizer that adds four new special tokens beyond the base vocabulary. When the engine is built and the service is started, the model outputs garbled text and repeats the same fragment regardless of the prompt. Which action should the developer take to resolve this?

Accepted Answer

Rebuild the TensorRT-LLM engine using the fine-tuned checkpoint's updated tokenizer vocabulary and matching special-token IDs.. Garbled, looping text from a fine-tuned model almost always traces back to a tokenizer or vocabulary mismatch between training and inference. Because the fine-tuned checkpoint added special tokens, the TensorRT-LLM engine must be compiled against the checkpoint's tokenizer metadata so token IDs align with the embedding table. Rebuilding the engine with the updated vocabulary is the only option that restores correct prompt encoding.

Answer

Convert the fine-tuned checkpoint back to a Hugging Face format and serve it through a generic Python HTTP wrapper instead of a compiled engine.

Answer

Increase the engine's maximum batch size and rebuild the engine so the additional special tokens can be processed in parallel.

Answer

Set the sampling temperature to zero and add a repetition penalty in the runtime generation config to stabilize the output.

Question 10

A developer is profiling a TensorRT-LLM serving deployment and notices that throughput collapses once concurrent requests exceed a small number of users, even though GPU compute utilization stays low. The model uses paged KV cache and continuous batching. Which factor most likely explains the bottleneck?

Accepted Answer

The KV cache memory pool is too small, so the scheduler cannot admit more concurrent sequences and requests queue while the GPU idles.. Paged KV cache allocates GPU memory in blocks per sequence, and the in-flight batch size is bounded by how many blocks the memory pool can hold. When that pool is too small, the scheduler queues new requests and the GPU sits underutilized, which matches the described symptom. Missing batching, thermal throttling, and GPU tokenization do not explain low compute utilization with a hard concurrency ceiling.

Answer

The tokenizer runs on the GPU and competes with the model for SM cycles, capping the batch size.

Answer

TensorRT-LLM lacks continuous batching support and therefore serializes every request regardless of the KV cache size.

Answer

The GPU is thermally throttled, which reduces clock speed and therefore limits throughput at high concurrency.

Question 11

A team is serving a 70B-parameter LLM with TensorRT-LLM on a node with four GPUs. During load testing they observe that increasing concurrent requests improves throughput up to a point, then latency spikes sharply and GPU memory utilization sits near the limit. Profiling shows the KV cache is being paged out and recomputed. Which change most directly addresses this bottleneck?

Accepted Answer

Reduce the maximum sequence length and configure a KV cache size that fits in remaining GPU memory, potentially with quantized cache.. Sharp latency growth with memory near the limit and profiler evidence of KV cache paging and recomputation means the cache working set no longer fits. Capping maximum sequence length and explicitly sizing the KV cache, optionally with quantized cache, keeps the cache resident and eliminates recomputation. In-flight batching remains useful, but it must operate within a cache budget that the deployment actually fits.

Answer

Enable in-flight batching and increase the maximum batch size so more requests share each forward pass.

Answer

Switch the deployment from tensor parallelism across four GPUs to pipeline parallelism to reduce per-GPU memory pressure.

Answer

Increase the tensor-parallel degree beyond four GPUs so the model weights and cache are spread across more devices.

Question 12

A research team is designing a decoder-only transformer LLM for long-document question answering. They want to reduce the quadratic computational cost of self-attention so that training on sequences of 32,000 tokens is feasible on their GPU cluster. Which two techniques are appropriate for this goal? (Choose two.)

Accepted Answer

Sparse attention patterns that restrict each token to a subset of positions. Sparse attention lowers the number of attention interactions, and IO-aware exact attention kernels like FlashAttention reduce memory traffic and speed up the exact computation. Both directly target the quadratic cost of self-attention for long sequences. Increasing heads, enlarging the vocabulary, or swapping normalization layers does not change the sequence-length-squared scaling and therefore does not make 32,000-token training feasible.

Answer

Using a larger vocabulary size for the tokenizer

Answer

Replacing layer normalization with batch normalization in the transformer blocks

Answer

Increasing the number of attention heads while keeping the same total hidden dimension

Question 13

A developer is writing a Python service that calls a locally hosted NVIDIA NIM microservice for a Llama model. They want to keep the client code portable so the same class can later target NVIDIA's hosted API endpoints without rewrites. Which client approach fits this goal?

Accepted Answer

Use the OpenAI Python client pointed at the NIM base_url, since NIM exposes OpenAI-compatible /v1/chat/completions routes.. Portability comes from targeting the OpenAI-compatible HTTP interface that NIM exposes, letting a single client class switch between a local container URL and NVIDIA hosted endpoints by configuration alone. Direct TensorRT-LLM bindings and the Triton gRPC client both couple code to a specific runtime or protocol, while GPU telemetry tools cannot serve inference requests.

Answer

Invoke nvidia-smi from a subprocess and parse GPU utilization to infer model outputs.

Answer

Use the Triton client library with the gRPC protocol and hardcode the model name and version in every call site.

Answer

Call the TensorRT-LLM C++ runtime directly from Python with ctypes and load the engine file inside the service.

Question 14

A developer is writing an application that streams chat completions from an NVIDIA-hosted NIM endpoint. Users report that the interface freezes until the entire answer is ready, even though the endpoint supports token streaming. Which client-side change fixes the perceived latency?

Accepted Answer

Enable streaming on the request and render each incremental delta as it is received instead of awaiting the full response body.. Streaming endpoints deliver the completion incrementally, so the client must both request streaming and render each delta as it arrives. That shifts perceived latency from full generation time down to time to first token. Capping tokens, disabling streaming, or merely threading the call all leave the user staring at a blank interface until the whole answer exists.

Answer

Send the request from a background thread and update the user interface only after the full response has been parsed.

Answer

Lower the maximum token limit so the model finishes generating the complete answer sooner.

Answer

Set the request parameter that disables streaming to false and read server-sent events incrementally as each chunk arrives.

Question 15

A developer is packaging a NeMo-based LLM application into a container for deployment on an NVIDIA GPU node. Which two practices are required to ensure the container can access the GPU and run inference efficiently? (Choose two.)

Accepted Answer

Install the NVIDIA Container Toolkit on the host and run the container with the --gpus flag or equivalent runtime configuration.. GPU access in containers requires the NVIDIA Container Toolkit on the host plus a runtime flag, and efficient NeMo inference requires matching CUDA runtime, driver, and framework versions. Bundling drivers, disabling persistence mode, or shrinking shared memory does not enable GPU access and can degrade performance.

Answer

Set the container's shared memory size to the minimum allowed value to conserve RAM.

Answer

Disable NVIDIA persistence mode on the host to reduce memory overhead.

Answer

Bake the full NVIDIA data center driver into the container image.