NCA-GENL Software Development Practice Question
Exhibit
{
"model_name": "llama-3",
"precision": "fp16",
"max_batch_size": 8
}Refer to the exhibit. If a developer increases the 'max_batch_size' in the JSON configuration, what is the primary expected trade-off in the system's performance metrics?
⚠ Common exam trap
Candidates often confuse throughput with latency, mistakenly believing that increasing the maximum batch size improves responsiveness for every individual user rather than causing initial requests to queue longer.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Higher latency for individual requests.
Increasing the batch size allows the GPU to process more requests in parallel, which typically increases total throughput. However, this often leads to higher individual request latency for earlier requests as they wait for the buffer to fill. This is a classic throughput-vs-latency trade-off in GPU computing. For LLMs, this balance is crucial, as too large a batch can lead to memory exhaustion or unacceptable delays for users waiting for the initial tokens.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Reduced total system throughput.
Why it's wrong here
Increasing the batch size generally improves total system throughput because it allows the GPU to better utilize its massive parallel cores. Throughput increases by packing more operations into a single kernel launch, which reduces the overhead per request, making this an unlikely outcome in most deployment scenarios.
- ✗
Increased GPU memory fragmentation.
Why it's wrong here
Batch size affects memory usage but not necessarily fragmentation. Fragmentation is typically handled by memory allocators like PagedAttention. Increasing batch size will increase the total memory demand for the KV cache and activation buffers, but it is not the primary driver of fragmentation in a properly configured server.
- ✓
Higher latency for individual requests.
Why this is correct
As the maximum batch size increases, the system may wait longer to accumulate enough requests to fill the batch. This increased wait time at the start of the inference pipeline results in higher latency for the first few requests, which is a standard trade-off for maximizing overall throughput.
- ✗
Improved accuracy of the underlying model.
Why it's wrong here
Batch size does not affect the mathematical accuracy of the model's output. The inference results remain identical regardless of whether a request is processed in a batch of 1 or a batch of 8; the only changes are in performance metrics like latency, throughput, and memory consumption on the GPU.
About these practice questions
One of 367 original NCA-GENL practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
JA
Written and reviewed by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
Last reviewed September 2026 · checked against the official NVIDIA exam blueprint
This NCA-GENL practice question is part of Courseiva's free NVIDIA certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the NCA-GENL exam.