NCA-GENL Software Development Practice Test 6 — 15 Questions

Question 1

A developer is preparing a container for an LLM microservice that will run on an NVIDIA GPU node and must be deployable through NVIDIA NIM. They want the image to be portable across supported GPU generations while still using NVIDIA's optimized inference stack. Which two practices should they follow? (Choose two.)

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Base the service on an NGC-hosted NIM container image and pin a specific version tag.. Portable NIM deployment rests on using the NGC-published, version-pinned NIM image and exposing the GPU through the NVIDIA Container Toolkit. Bundling weights or drivers into the image, or shipping one SKU-specific TensorRT engine, either breaks the supported deployment model or destroys the cross-generation portability the team requires.

Answer

Compile the model into a TensorRT engine for one exact GPU SKU and ship only that engine.

Answer

Install a full CUDA toolkit and matching driver inside the image for maximum portability.

Answer

Bake the model weights directly into the image layer during the build step.

Question 2

A team is building a RAG assistant and wants to reduce hallucinated citations. They plan to have the LLM return structured output that names the source document chunk used for each claim. Which implementation strategy most directly improves the reliability of that structured output?

Accepted Answer

Pass the retrieved chunks with stable identifiers in the prompt and constrain generation to a JSON schema that references those identifiers.. Reliable attribution comes from making the citation a constrained choice among known identifiers. Supplying stable chunk IDs in context and enforcing a JSON schema means the model selects from real evidence instead of generating citation text. The application can then validate each returned identifier against the retrieved set, converting an unverifiable prose claim into a programmatically checkable reference.

Answer

Retrieve more chunks and place them all in the prompt without identifiers.

Answer

Increase the model temperature so the model explores more citation candidates.

Answer

Ask the model to cite sources in a free-text bibliography appended after the answer.

Question 3

A developer is packaging a NeMo-based LLM application into a container for deployment on an NVIDIA GPU node. Which two practices are required to ensure the container can access the GPU and run inference efficiently? (Choose two.)

Accepted Answer

Install the NVIDIA Container Toolkit on the host and run the container with the --gpus flag or equivalent runtime configuration.. GPU access in containers requires the NVIDIA Container Toolkit on the host plus a runtime flag, and efficient NeMo inference requires matching CUDA runtime, driver, and framework versions. Bundling drivers, disabling persistence mode, or shrinking shared memory does not enable GPU access and can degrade performance.

Answer

Set the container's shared memory size to the minimum allowed value to conserve RAM.

Answer

Disable NVIDIA persistence mode on the host to reduce memory overhead.

Answer

Bake the full NVIDIA data center driver into the container image.

Question 4

A developer is writing an application that streams chat completions from an NVIDIA-hosted NIM endpoint. Users report that the interface freezes until the entire answer is ready, even though the endpoint supports token streaming. Which client-side change fixes the perceived latency?

Accepted Answer

Enable streaming on the request and render each incremental delta as it is received instead of awaiting the full response body.. Streaming endpoints deliver the completion incrementally, so the client must both request streaming and render each delta as it arrives. That shifts perceived latency from full generation time down to time to first token. Capping tokens, disabling streaming, or merely threading the call all leave the user staring at a blank interface until the whole answer exists.

Answer

Send the request from a background thread and update the user interface only after the full response has been parsed.

Answer

Lower the maximum token limit so the model finishes generating the complete answer sooner.

Answer

Set the request parameter that disables streaming to false and read server-sent events incrementally as each chunk arrives.

Question 5

A team is serving a 70B-parameter LLM with TensorRT-LLM on a node with four GPUs. During load testing they observe that increasing concurrent requests improves throughput up to a point, then latency spikes sharply and GPU memory utilization sits near the limit. Profiling shows the KV cache is being paged out and recomputed. Which change most directly addresses this bottleneck?

Accepted Answer

Reduce the maximum sequence length and configure a KV cache size that fits in remaining GPU memory, potentially with quantized cache.. Sharp latency growth with memory near the limit and profiler evidence of KV cache paging and recomputation means the cache working set no longer fits. Capping maximum sequence length and explicitly sizing the KV cache, optionally with quantized cache, keeps the cache resident and eliminates recomputation. In-flight batching remains useful, but it must operate within a cache budget that the deployment actually fits.

Answer

Enable in-flight batching and increase the maximum batch size so more requests share each forward pass.

Answer

Switch the deployment from tensor parallelism across four GPUs to pipeline parallelism to reduce per-GPU memory pressure.

Answer

Increase the tensor-parallel degree beyond four GPUs so the model weights and cache are spread across more devices.

Question 6

A developer is using the NVIDIA API Catalog to experiment with a hosted LLM. They want to send a prompt and receive a completion. Which endpoint should they use?

Accepted Answer

/v1/chat/completions. The /v1/chat/completions endpoint is designed for conversational LLMs and accepts a structured messages array. It is the primary interface for interacting with hosted models on NVIDIA API Catalog. Other endpoints serve different purposes like listing models or generating embeddings, and the legacy completions endpoint is less suitable for chat-based models.

Answer

/v1/models

Answer

/v1/completions

Answer

/v1/embeddings

Question 7

A developer is writing a Python service that calls a locally hosted NVIDIA NIM microservice for a Llama model. They want to keep the client code portable so the same class can later target NVIDIA's hosted API endpoints without rewrites. Which client approach fits this goal?

Accepted Answer

Use the OpenAI Python client pointed at the NIM base_url, since NIM exposes OpenAI-compatible /v1/chat/completions routes.. Portability comes from targeting the OpenAI-compatible HTTP interface that NIM exposes, letting a single client class switch between a local container URL and NVIDIA hosted endpoints by configuration alone. Direct TensorRT-LLM bindings and the Triton gRPC client both couple code to a specific runtime or protocol, while GPU telemetry tools cannot serve inference requests.

Answer

Invoke nvidia-smi from a subprocess and parse GPU utilization to infer model outputs.

Answer

Use the Triton client library with the gRPC protocol and hardcode the model name and version in every call site.

Answer

Call the TensorRT-LLM C++ runtime directly from Python with ctypes and load the engine file inside the service.

Question 8

A developer is tuning a TensorRT-LLM deployment of a long-context chat model and observes that GPU memory is exhausted under concurrent requests, causing requests to be rejected. They want to reduce KV cache memory pressure without retraining the model. (Choose two.)

Accepted Answer

Enable paged KV cache so cache blocks are allocated dynamically instead of reserving a contiguous buffer per sequence.. KV cache memory dominates long-context serving, so the effective levers are how cache memory is allocated and how many bytes each cached token consumes. Paged KV cache removes fragmentation by allocating blocks on demand, and KV cache quantization reduces per-token storage precision. Both are runtime or build-time configuration changes that need no retraining, unlike architectural pruning, and both increase the number of concurrent sequences the GPU can hold.

Answer

Reduce the model's hidden dimension by pruning attention heads and rebuild the engine from the pruned checkpoint.

Answer

Disable continuous batching so each request is processed to completion before the next one begins.

Answer

Increase the maximum number of batched tokens so more requests are processed simultaneously in a single forward pass.

Question 9

A developer is writing an application that calls an NVIDIA-hosted LLM endpoint and needs to keep multi-turn context across several user messages. Which payload structure should the application send to the chat completions API?

Accepted Answer

A 'messages' array of objects, each with a 'role' and 'content', ordered from system to user and assistant turns.. Chat completions APIs from NVIDIA expect an ordered messages array with role and content fields, preserving system, user, and assistant turns. This structure gives the model the turn boundaries it needs for coherent multi-turn dialogue, unlike a flat prompt string or incomplete history.

Answer

A 'context' object mapping user IDs to their previous messages.

Answer

A 'history' array containing only the assistant's prior responses.

Answer

A single 'prompt' string containing all previous turns concatenated.

Question 10

A developer is deploying a large language model using NVIDIA TensorRT-LLM and wants to optimize inference for a production environment with limited GPU memory. Which two techniques can be used to reduce memory footprint while maintaining acceptable performance? (Choose two.)

Accepted Answer

Enabling paged KV cache and tuning the maximum sequence length to a realistic value.. Quantization and paged KV cache with tuned sequence length are effective techniques to reduce GPU memory footprint in TensorRT-LLM. Quantization lowers precision of weights, and paged KV cache optimizes memory allocation for key-value pairs. Both help fit larger models on limited GPUs while preserving performance. Other options either increase memory usage or introduce latency, making them unsuitable for the scenario.

Answer

Using a larger batch size to amortize memory overhead across more requests.

Answer

Increasing the tensor parallelism degree to spread the model across more GPUs.

Answer

Storing the model weights on CPU and offloading them to GPU on demand.

Question 11

A developer is writing a Python client for an NVIDIA-hosted LLM endpoint and needs the model to answer every request in a strict JSON schema without extra prose. Where should the formatting contract be expressed so it applies consistently across all requests from the service?

Accepted Answer

In a system-role message describing the required schema and the rule against extra prose.. A service-wide formatting contract belongs in the system-role message, which the endpoint treats as standing instructions for the whole conversation and which the client can reuse on every call. User-turn reminders, assistant-role content, and custom headers either dilute the instruction, misrepresent the turn, or are simply ignored.

Answer

In the assistant-role message, since the assistant produces the JSON output.

Answer

In an HTTP header on the request, such as X-Response-Schema.

Answer

Appended to each user message as a trailing reminder sentence.

Question 12

A developer needs to serve a quantized Llama model on an NVIDIA GPU and wants the runtime to automatically select the fastest available execution kernels for the detected GPU architecture. Which approach aligns with the NVIDIA inference stack for this requirement?

Accepted Answer

Compile the model with TensorRT-LLM to produce an engine that contains architecture-tuned kernels and let the runtime pick the best ones for the GPU present.. TensorRT-LLM compiles model graphs into engines containing kernels tuned for specific GPU architectures, and at runtime it selects from the available tactics for the hardware present. Building the engine for the deployment GPU ensures the fastest kernels are candidates. This gives the developer optimized execution without hand-authoring CUDA code or losing performance to eager framework dispatch.

Answer

Write custom CUDA kernels for each target GPU and dispatch them manually at runtime.

Answer

Convert the model to ONNX and rely on the default CPU execution provider.

Answer

Run the model in eager execution mode on PyTorch without any compilation step.

Question 13

A developer is containerizing an inference service built with TensorRT-LLM and NVIDIA NIM for a Kubernetes cluster. They want the deployment to start reliably and use the GPU efficiently. Which two practices should they follow? (Choose two.)

Accepted Answer

Request nvidia.com/gpu resources in the pod spec and ensure the NVIDIA GPU Operator or device plugin is installed so the scheduler can allocate the device.. Reliable GPU-backed startup requires that Kubernetes can schedule the device through the NVIDIA device plugin or GPU Operator, and that the heavy engine artifacts are already present locally rather than downloaded at boot. Restart loops, bypassing the NVIDIA container runtime, and CPU-only emulation all prevent the engine from loading or waste resources instead of improving reliability and GPU efficiency.

Answer

Disable the NVIDIA container runtime so the pod uses the host's default runtime and accesses the GPU through /dev/nvidia devices directly.

Answer

Set the pod's restartPolicy to Always and rely on CrashLoopBackOff to warm the GPU driver on each restart.

Answer

Pin the pod to a CPU-only node pool and use CUDA emulation to run the TensorRT-LLM engine.

Question 14

A developer has a working TensorRT-LLM engine and wants to expose it through NVIDIA Triton Inference Server so that multiple client applications can call it over HTTP and gRPC with a stable interface. Which Triton feature should they configure to serve the TensorRT-LLM engine as a backend?

Accepted Answer

The TensorRT-LLM backend, declaring the engine in the model repository with a config.pbtxt that sets the backend and engine directory.. Triton's TensorRT-LLM backend is designed specifically to load TensorRT-LLM engines from the model repository and serve them over the standard HTTP and gRPC endpoints. Declaring the engine directory and backend in config.pbtxt gives clients a stable interface while retaining optimized in-flight batching and KV cache management. Custom backends and ensembles add complexity without providing the required serving capability.

Answer

An ensemble model that chains a preprocessing model, the TensorRT-LLM engine, and a postprocessing model.

Answer

A Python backend model that loads the engine and implements the inference logic in a custom script.

Answer

A custom C++ backend compiled against the TensorRT-LLM libraries and registered with Triton at startup.

Question 15

A developer is using TensorRT-LLM to build a chatbot and wants to reduce the memory footprint of the KV cache during inference. Which technique should they use?

Accepted Answer

Enable paged KV cache. Paged KV cache is a memory management technique in TensorRT-LLM that divides the KV cache into fixed-size blocks, allowing non-contiguous storage and dynamic allocation. This reduces fragmentation and enables more sequences to fit in memory. It is the recommended approach to minimize KV cache memory footprint during inference.

Answer

Disable quantization

Answer

Increase the beam width

Answer

Use a larger batch size