CCNA LLM Fundamentals Questions

RAG grounds answers in retrieved documents, and fine-tuning on domain-specific data reduces errors. Increasing context window alone does not reduce hallucinations; greedy decoding reduces creativity but not factual errors.

ROUGE is recall-oriented, measuring how much of the reference content appears in the generated summary, which aligns with capturing key facts. BLEU is precision-oriented, and BERTScore measures semantic similarity but is not specifically recall-focused.

Hallucination is the term used when an LLM produces content that is not grounded in training data or provided context, appearing confident but being incorrect.

RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

ROUGE-L measures longest common subsequence overlap, focusing on word order recall. It does not capture coherence, factual consistency, or fluency. High ROUGE-L with low human scores suggests the summaries are lexically similar but not well-formed.

Most LLM APIs charge based on the number of input and output tokens. The token count directly affects cost. Model size, context window, and temperature are related but the direct billing metric is token count.

Top-p (nucleus) sampling selects from a dynamically chosen set of tokens whose cumulative probability exceeds p, allowing diversity while avoiding the long tail of improbable tokens.

Embedding models are specifically designed to produce dense vector representations that capture semantic meaning. They are distinct from generation models. For semantic search, embeddings from an embedding model are compared using cosine similarity.

Providing few-shot examples of desired behavior in the system prompt (in-context learning) can guide the model toward safe responses. Fine-tuning would require retraining, and prompt engineering is broader; few-shot examples are a specific, effective technique.

Content filtering and human review are direct mitigations. Training from scratch is impractical; modifying sampling does not address bias; increasing context window is irrelevant.

RAG addresses hallucinations by grounding the model's output in retrieved documents that contain factual information. Knowledge cutoff, bias, and context length constraints are separate issues that RAG may partially help with, but its primary purpose is to reduce factual errors.

RAG retrieves relevant, current documents at inference time, providing up-to-date context without modifying the model parameters.

RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

Fine-tuning on a curated dataset of secure code examples can teach the model to avoid common vulnerability patterns while retaining its general coding ability. RAG with security docs could also help, but fine-tuning directly addresses the model's behavior more comprehensively.

High BLEU and low ROUGE indicate that the generated text has high precision (many n-grams match reference) but low recall (missing many reference n-grams). This often occurs when the output is short or overly cautious.

LLMs can produce hallucinations (factual errors), have a knowledge cutoff date, and can exhibit bias from training data. They do not inherently lack creativity, and context length is a limitation but not a 'lack of understanding'.

Temperature sampling with a higher temperature (>1) increases the probability of less likely tokens, promoting creativity and diversity.

Lowering temperature reduces randomness, making outputs more deterministic and less prone to errors, while still allowing some variation.

ROUGE, BLEU, and BERTScore are all used for summarization evaluation. Perplexity measures model confidence, and cosine similarity is for embedding comparison.

RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

ROUGE measures recall-oriented overlap between generated and reference summaries, suitable for summarization. BERTScore uses semantic similarity via embeddings. BLEU is for translation, perplexity for language modeling, and human evaluation is qualitative.

OCI Data Science with a model deployment endpoint is the correct choice because it provides a managed infrastructure for hosting and serving LLMs within a specific OCI region, ensuring all processing and data remain within that region to meet strict data residency requirements. The model deployment endpoint runs on dedicated compute resources in the chosen region, allowing low-latency inference while adhering to regional data boundaries.

Positional encoding provides information about token order and is added to the input embeddings. It is used in encoder-decoder and decoder-only models. It does not replace attention, and it is not learned for each token position in the original formulation (it uses fixed sine/cosine functions).

Using a smaller model from the same family (e.g., switching from Llama 3 70B to Llama 3 8B) reduces the number of parameters and computational resources required per inference, directly lowering cost. Smaller models often retain strong performance on many tasks, especially when the task complexity does not demand the full capacity of the larger model, thus maintaining output quality while reducing token processing costs.

Byte-Pair Encoding (BPE) starts with characters and merges the most frequent pairs to create subword units. WordPiece uses a similar likelihood-based approach. SentencePiece is a framework that can use BPE or unigram.

ROUGE-L measures the longest common subsequence, emphasizing recall of key content (n-grams in order). BLEU measures n-gram precision, penalizing short or non-fluent outputs. A high ROUGE-L but low BLEU suggests the summaries capture the main ideas but use different phrasing or are too short.

Generate embeddings for both sentences, then compute cosine similarity. Normalizing vectors ensures cosine similarity equals the dot product. Training a new model is unnecessary, and using a generation model is not appropriate for embeddings.

RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

Large language models (LLMs) are known to generate factually incorrect information, often called 'hallucinations.' This occurs because LLMs are probabilistic models that predict the next token based on training data patterns, not verified facts. They lack a grounding mechanism to validate outputs against real-world truth, making hallucinations a fundamental limitation.

Low chunk overlap can cause loss of context, improper prompt instructions may lead the model to ignore retrieved content, and a small top-K retrieval may miss relevant documents. A large embedding dimension improves retrieval precision, and a high temperature increases randomness but does not cause ignoring documents.

Temperature sampling scales the logits before applying softmax, controlling the randomness of token selection. A temperature > 1 increases diversity, reducing repetition, while still allowing the model to generate coherent text.

An encoder-only embedding model like Cohere Embed is designed to convert text into dense vector representations (embeddings) that capture semantic meaning, which is exactly what is needed for semantic search over a large corpus of legal documents. These models use a bidirectional transformer architecture to encode context from both directions, producing fixed-size embeddings that can be efficiently compared using cosine similarity or other distance metrics.

ROUGE is recall-oriented and measures overlap of n-grams, making it a standard metric for summarization. BERTScore measures semantic similarity using embeddings, which can capture meaning even when wording differs. BLEU is more for translation, perplexity measures fluency, and human evaluation is qualitative but not a metric.

Encoder-decoder architectures like T5 are designed for sequence-to-sequence tasks. The encoder processes the input sequence, and the decoder generates the output sequence, making it ideal for translation.

BLEU (Bilingual Evaluation Understudy) is the most suitable metric for evaluating a translation model's output against multiple reference translations because it measures n-gram precision between the candidate translation and one or more reference translations. It directly quantifies how many words and phrases in the candidate match those in the references, making it the standard metric for machine translation tasks.

If the retriever returns irrelevant chunks or the generation model ignores the context, the answers will be off-topic. Checking retrieval relevance and prompt instruction is key.

Retrieval-Augmented Generation (RAG) with a vector database is the most suitable approach because it allows the LLM to answer questions about a frequently updated private codebase without retraining. RAG retrieves relevant code snippets from a vector index at query time, ensuring the model always has access to the latest code without the cost and latency of hourly fine-tuning.

The 'Max tokens' parameter controls the maximum length of the generated output. When a model cuts off mid-sentence, it means the token limit has been reached before the model could complete its response. Increasing this value allows the model to generate more tokens, thus completing the translation.

Chunking splits documents, embedding converts chunks to vectors, retrieval fetches relevant chunks, and generation produces the answer.

In-context learning does not update model weights; it provides examples in the prompt at inference time. Fine-tuning updates the model weights through additional training on a dataset.

Multi-head attention allows the model to attend to information from different representation subspaces at different positions, capturing a richer understanding.

Pre-training trains a model on a large, general corpus to learn language representations; fine-tuning adapts the pre-trained model to a specific task or domain using a smaller labeled dataset.

Top-p (nucleus) sampling selects from the smallest set of tokens whose cumulative probability exceeds p. It adapts to the model's confidence, allowing diversity while maintaining relevance. Greedy decoding is deterministic, temperature scales all probabilities, top-k fixes the number of candidates, and beam search explores multiple sequences but tends to produce safe outputs.

Self-attention computes attention scores between all pairs of positions, enabling the model to weigh the importance of different input tokens.

RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

BERTScore uses contextual embeddings, captures semantic similarity, and correlates better with human judgment than n-gram metrics.

Encoder-decoder models like T5 or BART are designed for sequence-to-sequence tasks such as summarization, and can handle long inputs with their encoder.

RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

SentencePiece is language-agnostic, works directly on raw text without requiring pre-tokenisation (e.g., whitespace splitting), and handles languages like Chinese and Korean where word boundaries are not obvious. BPE and WordPiece typically require pre-tokenisation, making them less suitable for CJK languages.

Chunking splits documents into manageable pieces, and embedding converts them into vectors for similarity search.

Enabling frequency penalty directly reduces the likelihood of the model repeating the same tokens by subtracting a penalty proportional to the token's existing frequency in the generated text. This is the most targeted way to combat repetitive output without altering the core sampling parameters (temperature and top-p) that control randomness.

Hallucination in LLMs refers to the generation of content that is plausible-sounding but factually incorrect or nonsensical. This occurs when the model's probabilistic next-token prediction produces statements that are not grounded in its training data or real-world facts, often due to overgeneralization or lack of factual recall mechanisms.

Smaller models typically have lower latency, model families (Llama, Code Llama) are designed for code, and deployment choices affect latency. Context window size and fine-tuning cost are secondary or not directly relevant to latency.

Higher dimensions can capture more nuanced semantics but require more storage and may increase retrieval latency.

In OCI Generative AI, pricing is primarily based on the total number of tokens processed, which includes both input (prompt) and output (generated) tokens. Processing large batches of legal documents generates high token counts due to lengthy text inputs and verbose outputs, directly increasing cost. The number of API requests alone does not determine cost—a single request with many tokens costs more than many requests with few tokens.

BERT uses an encoder-only architecture, which processes the entire input sequence bidirectionally. This makes it well-suited for tasks like classification, NER, and QA where understanding the full context is important.

Embedding models convert text into dense vectors that capture semantic meaning. Cosine similarity between query and document embeddings is then used to find similar documents.

ROUGE-L measures n-gram overlap, which may not capture semantic quality. Human evaluators often prefer summaries that are fluent, coherent, and concise, even if they use different wording. The discrepancy indicates that ROUGE-L alone is insufficient for evaluation.

Fine-tuning a pre-trained encoder-only model like BERT on the labeled dataset is a standard approach for classification tasks, offering good performance with relatively modest data and compute.

Hallucination is the generation of false information; RAG grounds responses in retrieved factual documents, reducing hallucinations.

Cosine similarity measures the cosine of the angle between two vectors, which is equivalent to the dot product of the vectors after they have been normalized to unit length (L2 norm = 1). Option C correctly describes this process: first normalize each embedding vector to unit length, then compute the dot product. This is the standard approach because raw embedding vectors from models like OCI's AI services may not be unit vectors, and the dot product alone does not account for magnitude differences.

GPT is a decoder-only model. BERT is encoder-only. T5 is encoder-decoder.

Chunking determines how documents are split into pieces. If chunks are too small, key information may be split across chunks, causing incomplete retrieval. Adjusting chunk size and overlap can improve completeness.

Fine-tuning with 500 examples may lead to overfitting or catastrophic forgetting, especially when the tasks are diverse. In-context learning with a few examples per task is more flexible and leverages the model's pre-trained knowledge. The small dataset size makes fine-tuning risky.

Greedy decoding always selects the token with highest probability, producing the same output for a given input. Temperature, top-k, and top-p introduce randomness.

The self-attention mechanism computes a weighted sum of all input token representations, where the weights are determined by the pairwise compatibility (attention scores) between tokens. This allows each token to dynamically attend to every other token in the sequence, capturing global dependencies without the limitations of fixed local windows or recurrence.

Since self-attention processes tokens in parallel without inherent order, positional encoding adds information about the position of each token in the sequence.

Positional encoding is essential in Transformer models because the self-attention mechanism processes all tokens in parallel and has no inherent notion of sequence order. By adding positional encodings (often sinusoidal or learned) to the input embeddings, the model can distinguish between tokens at different positions, enabling it to capture word order and relative positions. Without this, the model would treat the sequence as a bag of tokens, losing all sequential context.