NCP-GENL LLM Architecture Practice Test 2 — 15 Questions

Question 1

What is the function of the 'Masked' component in a Decoder-only Transformer's self-attention during training?

Accepted Answer

To prevent the model from attending to future tokens in the sequence.. The mask ensures that each token can only attend to itself and the tokens that precede it in the sequence. This autoregressive property is essential for training decoder models, as it prevents the model from 'cheating' by looking at future tokens. This ensures that the model learns to predict the next word based solely on the context that would be available during actual inference.

Answer

To filter out stop words from the input text.

Answer

To increase the randomness of the model's predictions.

Answer

To reduce the computation of the attention mechanism by 50%.

Question 2

Which architectural component is responsible for projecting the model's hidden states back into the vocabulary space to predict the next token?

Accepted Answer

The LM Head linear layer.. The language model head (LM Head) is a final linear projection layer that maps the high-dimensional hidden states from the Transformer blocks into a vector representing the probability distribution over the entire vocabulary. This projection is the final step in the forward pass, converting the learned internal representation into actionable predictions that can be sampled to generate text.

Answer

The Feed-Forward Network.

Answer

The Softmax normalization layer.

Answer

The positional embedding layer.

Question 3

A team is pretraining a decoder-only Transformer LLM on a large corpus of code and natural language. They observe that the model's training loss decreases smoothly, but during generation it sometimes produces degenerate repetition, and attention entropy on long sequences collapses. They suspect the issue is related to the positional encoding scheme. Which architectural change is most likely to mitigate the attention entropy collapse while preserving the model's ability to generalize to sequences longer than those seen during pretraining?

Accepted Answer

Replace learned absolute positional embeddings with Rotary Positional Embeddings (RoPE) applied to queries and keys in every self-attention layer.. The symptoms point to a positional encoding that does not generalize beyond the pretraining sequence length. Rotary Positional Embeddings inject relative position directly into the attention computation by rotating queries and keys, so attention scores depend on relative offsets rather than absolute indices. This preserves attention entropy on longer sequences and reduces degenerate repetition, making it the appropriate architectural change for this scenario.

Answer

Increase the number of attention heads while keeping the total hidden dimension constant, without changing the positional encoding.

Answer

Apply Layer Normalization before the self-attention sublayer instead of after it, leaving the positional encoding unchanged.

Answer

Add a sinusoidal positional encoding to the input embeddings and freeze those embeddings during training.

Question 4

A developer is building a decoder-only generative model and wants to prevent the model from attending to future tokens during training so that each position can only use information from itself and earlier positions. Which architectural mechanism should they implement in the self-attention layer?

Accepted Answer

A causal attention mask that sets attention scores for future positions to negative infinity before the softmax.. Decoder-only generative models must not see future tokens during training, otherwise next-token prediction becomes trivial and the model will not generalize to autoregressive inference. A causal attention mask accomplishes this by making future positions invisible to the softmax, ensuring each position only conditions on itself and earlier tokens.

Answer

A padding mask that ignores tokens added to equalize sequence lengths across a batch.

Answer

A learned positional embedding added to the token embeddings at the input layer.

Answer

A residual connection that adds the attention output back to the input hidden state.

Question 5

A team is training a large language model and wants to reduce the memory used by the optimizer without changing the model architecture. They are using Adam and notice that optimizer state consumes more GPU memory than the model weights. Which technique should they apply to reduce optimizer memory while keeping the model architecture unchanged?

Accepted Answer

Switch to a memory-efficient optimizer such as 8-bit Adam or Adafactor that stores reduced-precision or factored optimizer state.. Adam maintains two full-precision moment estimates for every trainable parameter, so optimizer state can be several times the size of the model weights. Memory-efficient optimizers such as 8-bit Adam or Adafactor reduce this footprint by quantizing or factoring the state, directly lowering the dominant memory cost without changing the model architecture.

Answer

Replace the feed-forward activation with a gated variant to reduce activation memory.

Answer

Reduce the number of attention heads while keeping the hidden size constant.

Answer

Increase the batch size proportionally to the number of GPUs so optimizer states are shared across more tokens.

Question 6

An engineer is deploying a 70B parameter model and needs to serve many concurrent users on a single GPU with limited memory. They want to store the attention keys and values for past tokens efficiently so that generation does not recompute them at every step. Which technique should they implement?

Accepted Answer

A key-value cache that stores the projected keys and values for previously processed tokens.. Autoregressive decoding generates one token at a time, and without caching the model would recompute keys and values for the entire prefix at every step, wasting compute. A key-value cache stores those projections so each step only processes the new token, which is the standard way to make long-context generation practical.

Answer

Quantization-aware training that reduces weight precision before deployment.

Answer

Speculative decoding that uses a smaller draft model to propose tokens.

Answer

Gradient checkpointing that recomputes activations during the backward pass.

Question 7

A team is deploying a 13B-parameter decoder-only LLM on a single NVIDIA A100 40GB GPU for a real-time chatbot. During load, the process runs out of memory even though the model weights in FP16 require roughly 26GB. The team wants to reduce GPU memory usage with minimal impact on output quality and no change to the model architecture. Which technique is most appropriate?

Accepted Answer

Apply INT8 weight-only quantization using NVIDIA TensorRT-LLM. The bottleneck is static weight storage on a single 40GB device. Weight-only INT8 quantization halves the parameter footprint while preserving the architecture and offering strong quality retention, and TensorRT-LLM provides a supported path for this on NVIDIA hardware. The other choices target attention compute, KV cache paging, or multi-GPU partitioning, none of which reduce the resident weight memory on one GPU.

Answer

Enable tensor parallelism across the single GPU's SMs

Answer

Switch the attention implementation to FlashAttention-2

Answer

Increase the KV cache block size to 32 tokens

Question 8

A team is pretraining a 13B-parameter decoder-only LLM on English text using byte-pair encoding with a 50,000-token vocabulary. They observe that the model produces fluent but repetitive continuations and that the average log-probability assigned to ground-truth tokens plateaus early. The training loss curve shows the model is underfitting rather than overfitting. Which architectural change is most likely to improve the model's capacity to capture long-range dependencies?

Accepted Answer

Increase the number of transformer layers and the hidden dimension proportionally.. Underfitting in a large decoder-only LLM indicates the model lacks sufficient capacity for the data distribution. Scaling both depth and width increases the parameter count and the number of nonlinear transformations applied to the residual stream, which improves the model's ability to capture long-range dependencies. The other options either rebalance existing capacity, change positional encoding without adding parameters, or reduce computation without improving fit.

Answer

Reduce the vocabulary size to 32,000 tokens to lower the softmax computation cost.

Answer

Increase the number of attention heads while keeping the hidden dimension fixed.

Answer

Replace the learned positional embeddings with sinusoidal absolute positional encodings.

Question 9

A team is fine-tuning a pretrained decoder-only model on a small domain-specific dataset. They observe that the model quickly overfits and loses general language ability. They want to update only a small number of additional parameters while keeping the base weights frozen. Which approach should they use?

Accepted Answer

Low-Rank Adaptation (LoRA), which injects trainable low-rank matrices into existing layers while freezing the base weights.. Parameter-efficient fine-tuning methods such as LoRA freeze the pretrained weights and train only small injected matrices, which reduces optimizer memory and limits drift from the base model. This is well suited to small domain datasets where full fine-tuning would overfit and degrade general language ability.

Answer

Full fine-tuning with a very low learning rate and early stopping.

Answer

Increasing the dropout rate in the feed-forward layers during fine-tuning.

Answer

Freezing the embedding layer only and training all transformer blocks.

Question 10

A team is deploying a 70B-parameter decoder-only LLM on an NVIDIA H100 GPU node. During generation they observe that the KV cache grows linearly with sequence length and is consuming most of the available HBM, forcing them to limit batch size. They want to reduce KV cache memory without retraining the model from scratch. Which architectural change should they apply?

Accepted Answer

Replace multi-head attention with Grouped Query Attention (GQA), where multiple query heads share a single key/value head.. Grouped Query Attention reduces the number of key/value heads relative to query heads, so the KV cache that must be stored for each token shrinks by the grouping factor. Because the cache dominates HBM during long-context decoding, this architectural change directly relieves memory pressure and permits larger batches, and it can be adopted via uptraining rather than a full retrain.

Answer

Switch the feed-forward network activation from SwiGLU to ReLU to reduce parameter count.

Answer

Apply rotary positional embeddings to the key and query vectors instead of learned absolute embeddings.

Answer

Increase the number of attention heads while keeping the head dimension constant.

Question 11

A developer is inspecting a decoder-only Transformer and notices that during training, the model attends to future tokens in the sequence, causing the loss to drop unrealistically fast but generation to be incoherent. Which architectural mechanism is missing or misconfigured?

Accepted Answer

Causal attention mask in the self-attention layers. Autoregressive language models require a causal mask that sets attention scores to negative infinity for positions after the current token. When that mask is absent, the training objective becomes trivial because the target token is visible in the input context, producing the fast loss drop and incoherent generation. Positional encodings, dropout, and normalization placement do not enforce temporal directionality.

Answer

Pre-layer normalization placement before attention

Answer

Dropout regularization on attention weights

Answer

Rotary positional embedding frequency base

Question 12

An ML engineer is evaluating Mixture-of-Experts (MoE) routing for a large decoder-only model to increase capacity without proportionally increasing compute per token. Which TWO statements accurately describe how top-k token routing behaves in such an architecture? (Choose two.)

Accepted Answer

Routing decisions are computed per token, allowing different tokens in the same sequence to be processed by different expert subsets. Sparse MoE replaces a single feed-forward block with many expert blocks plus a router that, for each token, selects the top-k experts to run. This yields token-level dynamic routing and keeps per-token FLOPs tied to k and expert size rather than the total expert population. Dense averaging, static assignment, and automatic k growth all contradict the sparse, learned, token-specific routing that defines the architecture.

Answer

Increasing the number of experts proportionally increases the number of activated experts per token

Answer

The router assigns tokens to experts once at initialization and keeps the assignment fixed throughout training

Answer

All experts receive every token, but their outputs are averaged with learned scalar weights

Question 13

A team is pre-training a 13B-parameter decoder-only LLM on a cluster of NVIDIA GPUs. They observe that gradient norms spike sharply during the first few hundred steps, destabilizing training. They want to keep the standard post-layer-normalization placement but stabilize early optimization. Which architectural technique should they apply?

Accepted Answer

Initialize all residual output projections with a small scaling factor such as 1/sqrt(2N).. Post-layer-normalization Transformers suffer from residual stream variance that grows with depth, producing large gradients early in training. Scaling residual branch outputs at initialization by a depth-dependent factor keeps variance bounded, directly stabilizing optimization without changing normalization placement or the optimizer schedule. Optimizer tweaks and activation swaps do not address the architectural root cause.

Answer

Apply learning rate warmup over the first several thousand steps.

Answer

Insert RMSNorm immediately after the token embedding layer only.

Answer

Replace GELU activations in the feed-forward network with ReLU.

Question 14

A research team is pretraining a decoder-only LLM and observes that gradient magnitudes in the earliest layers are extremely small while later layers train normally, causing slow convergence. They are using post-layer normalization. Which architectural change is most likely to improve gradient flow to the early layers?

Accepted Answer

Move layer normalization to before each sublayer (pre-LN) and add a final normalization before the output projection. Post-LN places normalization inside the residual branch, so gradients must pass through normalization at every layer, attenuating them in deep stacks. Pre-LN normalizes the sublayer input and leaves the residual stream unnormalized, giving a direct gradient highway from the loss to early parameters. That structural change, plus a final normalization before the output head, is the recognized fix for the described symptom.

Answer

Replace layer normalization with batch normalization across the sequence dimension

Answer

Remove residual connections to shorten the gradient path

Answer

Increase the number of attention heads while keeping head dimension constant

Question 15

An engineer is reviewing the attention implementation of a decoder-only LLM used for chat. During inference with a KV cache, generated tokens must not attend to future positions. Which mechanism enforces this constraint inside scaled dot-product attention?

Accepted Answer

A causal mask that sets attention scores for future positions to negative infinity before the softmax.. Causal masking is the standard mechanism that makes decoder-only attention autoregressive. By adding negative infinity to scores of future positions before softmax, those positions receive zero weight, so each token can only attend to itself and prior tokens. Other listed techniques affect numerical stability or positional encoding but do not enforce the temporal constraint.

Answer

Dropping the scaling factor 1/sqrt(d_k) so that scores decay for distant tokens.

Answer

Applying LayerNorm to the query and key projections before computing attention scores.

Answer

Shifting the position IDs of cached keys so they appear before the current token.