NCA-GENL Software Development Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✓

  Compare the time to first token (TTFT) and inter-token latency measured before and after the update to isolate whether the delay is in prefill or decode.

  Why this is correct

  Splitting latency into time to first token and inter-token latency immediately separates prefill (prompt processing) from decode (token generation). A long pause before the first token with fast subsequent tokens points at prefill, while slow steady output points at decode. This measurement requires no code change and directly targets the reported symptom, making it the correct first diagnostic step for the NIM deployment.

• ✗

  Increase the tensor-parallel degree of the NIM container so the model weights are sharded across more GPUs.

  Why it's wrong here

  Tensor parallelism shards weights and attention heads across GPUs and can raise throughput, but applying it before measuring where the latency lives risks a costly redeploy that may not touch the bottleneck. Low GPU utilization does not by itself prove the model needs more shards; it can also indicate host-side or scheduling overhead. Diagnosis must precede this configuration change.

• ✗

  Switch the client from streaming to a single non-streaming request so the full response is timed end to end.

  Why it's wrong here

  Removing streaming discards the very signal that distinguishes prefill from decode: the pause before the first token. A non-streaming wall-clock measurement collapses both phases into one number and hides where the two seconds are spent. It also changes the client behavior under test, so the comparison against the previous 300 ms baseline becomes invalid.

• ✗

  Enable dynamic batching in the NIM microservice so concurrent requests are grouped into larger inference batches.

  Why it's wrong here

  Dynamic batching improves aggregate throughput under concurrency, but it can actually add latency to individual requests because a request waits for the batch window to close. The scenario describes a single slow response with low GPU utilization, which is not a throughput starvation symptom. Turning on batching first would confound the measurement rather than explain the regression.