CCNA LLM Fundamentals Questions

Chunking strategy (size and overlap) directly affects how well the retrieval step can find relevant passages. Too large or poorly split chunks can dilute semantic meaning.

BPE reduces vocabulary size by representing words as subword units, and it can handle out-of-vocabulary words by breaking them into known subwords. Fixed-length tokens and losslessness are not advantages of BPE.

Beam search considers multiple candidate sequences and selects the one with the highest overall probability, which often results in more fluent and accurate translations than greedy decoding, but at higher computational cost.

Pre-training is the initial phase where a model learns general language patterns from a large corpus. Fine-tuning adapts the pre-trained model to a specific task using a smaller labeled dataset.

Embedding models (e.g., Cohere Embed, OpenAI text-embedding-ada) are specialized to produce dense vector representations. Generation models (like GPT) produce text, not embeddings.

BERTScore correlates with human judgment but is not perfect; it may favor certain styles. Additionally, BERTScore may be inflated if the reference summaries are similar to the model's training data.

RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

Retrieval-Augmented Generation (RAG) directly addresses the problem of stale training data by dynamically retrieving current documents from an external knowledge base at inference time. This allows the model to generate answers grounded in up-to-date information without requiring retraining or a larger model, making it the most effective and practical solution for a question-answering system.

Hallucinations are a primary limitation of large language models because they cause the model to generate text that is factually incorrect, nonsensical, or not grounded in the training data. This occurs due to the probabilistic nature of token prediction, where the model prioritizes fluency and coherence over factual accuracy, especially when the prompt lacks sufficient context or the model is asked to recall specific facts not well-represented in its training.

ROUGE-L measures the longest common subsequence (LCS) between generated and reference summaries, focusing on lexical (word-level) overlap. It does not assess semantic meaning, fluency, or factual correctness. Human evaluators often prefer summaries that are coherent and capture key ideas, even if they use different wording, which explains why Model Y can score lower on ROUGE-L but be preferred 60% of the time.

Scaling prevents the dot products from growing too large in magnitude, which would push softmax into regions with extremely small gradients.

Self-attention allows each token to attend to every other token in the sequence, capturing contextual relationships regardless of distance.

Temperature >1 flattens the probability distribution, making low-probability tokens more likely. top_k=50 restricts to top 50 tokens, but top_p=1.0 imposes no cumulative probability cutoff. The combination yields diverse but potentially incoherent outputs.

Top-k sampling and top-p (nucleus) sampling introduce randomness by sampling from a restricted token set. Temperature modifies the probability distribution and controls randomness but is a generation parameter, not a standalone sampling strategy. Greedy decoding and beam search are deterministic or near-deterministic.

Models like Code Llama (a variant of Llama) are specifically fine-tuned on code and are known for strong code generation capabilities. While other models can generate code, Code Llama is the best fit among the options.

BERTScore measures semantic similarity using contextual embeddings, while ROUGE-L measures n-gram overlap. Higher BERTScore indicates better semantic faithfulness even without exact phrase matches.

BERTScore computes cosine similarity between contextual embeddings, capturing semantic meaning better than surface-level n-gram metrics.

Decoder-only models use masked self-attention (causal) and generate tokens left-to-right. They cannot use bidirectional context because future tokens are masked.

Byte-Pair Encoding (BPE) is a subword tokenization method that starts with individual characters and merges the most frequent pairs iteratively until a vocabulary size is reached.

Document chunking is essential because it breaks large documents into smaller, manageable pieces that can be individually indexed and retrieved. Without chunking, the retrieval phase would either miss relevant context or return overly large documents that exceed the LLM's context window, reducing accuracy.

For retrieval tasks, embedding quality is best measured by semantic similarity metrics like BERTScore, which correlate with how well embeddings capture meaning. Perplexity measures language model fluency, not embedding quality.

Self-attention computes attention scores between all pairs of tokens, enabling the model to capture dependencies across the entire sequence.

Few-shot prompting—providing explicit examples of the desired JSON format in the prompt—directly guides the LLM's output structure without requiring retraining. This technique leverages in-context learning, where the model infers the required schema from the examples, making it the most effective and efficient method for format adherence.

Cosine similarity measures the cosine of the angle between two vectors, is commonly used for comparing embedding vectors, and ranges from -1 to 1, where 1 indicates identical direction.

The self-attention mechanism allows each token in the input sequence to attend to every other token, computing a weighted sum of their representations. This enables the model to capture long-range dependencies directly without the sequential processing constraints of RNNs, and because the attention scores for all tokens can be computed simultaneously, the mechanism supports parallel processing of the entire sequence.

OCI Generative AI offers models including Cohere Command/Embed and Meta Llama. Mistral and GPT are not mentioned in the context of OCI's available models, and BERT is an encoder-only model not typically offered as a generation model.

RAG provides factual grounding, and reducing temperature makes outputs more deterministic, reducing fabricated details.

RAG can ground generation in a curated corpus, and with strict retrieval settings (e.g., only retrieving from the legal corpus), the model will not use any outside knowledge, reducing hallucinations.

Top-p (nucleus) sampling cuts off the tail of the probability distribution where cumulative probability exceeds p, allowing dynamic vocabulary size.

BLEU is a precision-based metric (not recall). It uses modified n-gram precision with a brevity penalty. It correlates reasonably well with human judgment at the corpus level but has known limitations such as not capturing semantic equivalence.

RAG retrieves factual information from an external knowledge base to ground the generation, reducing hallucinations. The other options do not address factual accuracy.

The self-attention mechanism computes attention scores between all pairs of tokens, enabling the model to dynamically focus on relevant parts of the input. Positional encoding adds order information, multi-head attention runs multiple attention heads in parallel, and the feed-forward network processes each position independently.