NCP-GENL Model Optimization Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✓

  Set the maximum input length and maximum sequence length to accommodate 8K tokens and size the KV cache pool accordingly.

  Why this is correct

  TensorRT-LLM engines have fixed maximum input and sequence length settings, and the KV cache pool must be large enough to hold the longest sequence the engine was built for. Raising these limits and sizing the pool for 8K tokens lets long prompts allocate their blocks, while shorter requests continue to use only the blocks they need.

• ✗

  Enable INT8 KV cache quantization so each token's cached keys and values consume fewer bytes.

  Why it's wrong here

  Quantizing the KV cache does shrink bytes per token and can help fit longer contexts, but it introduces accuracy risk and still requires the engine's maximum sequence length to permit 8K tokens. If the built-in limit is below 8K, requests are rejected before cache quantization can help.

• ✗

  Enable chunked context prefill so long prompts are processed in multiple smaller prefill passes.

  Why it's wrong here

  Chunked context prefill reduces peak activation memory during prefill for very long prompts and improves scheduling, but it does not raise the engine's maximum sequence length or enlarge the KV cache pool. The capacity error is about how many token blocks the sequence may hold, which prefill chunking alone does not change.

• ✗

  Reduce the maximum batch size to 1 so all KV cache blocks are available to a single long request.

  Why it's wrong here

  Lowering max batch size frees pool capacity for one sequence but cripples throughput and still fails if the per-sequence block requirement exceeds the pool sized for the original maximum sequence length. The capacity error stems from the sequence-length limit and pool sizing, not from concurrent batch occupancy.