CCNA Oci Genai Llm Fundamentals Questions

In OCI Generative AI, pricing is primarily based on the total number of tokens processed, which includes both input (prompt) and output (generated) tokens. Processing large batches of legal documents generates high token counts due to lengthy text inputs and verbose outputs, directly increasing cost. The number of API requests alone does not determine cost—a single request with many tokens costs more than many requests with few tokens.

The model's knowledge cutoff date determines the temporal boundary of its training data. Since the LLM was fine-tuned on company data but not retrained, it lacks exposure to events occurring after its cutoff, making it unable to generate accurate answers about recent events. This is the most direct and common cause for such temporal knowledge gaps in LLMs.

BERT uses an encoder-only architecture, which processes the entire input sequence bidirectionally. This makes it well-suited for tasks like classification, NER, and QA where understanding the full context is important.

Embedding models convert text into dense vectors that capture semantic meaning. Cosine similarity between query and document embeddings is then used to find similar documents.

ROUGE-L measures n-gram overlap, which may not capture semantic quality. Human evaluators often prefer summaries that are fluent, coherent, and concise, even if they use different wording. The discrepancy indicates that ROUGE-L alone is insufficient for evaluation.

Fine-tuning a pre-trained encoder-only model like BERT on the labeled dataset is a standard approach for classification tasks, offering good performance with relatively modest data and compute.

Hallucination is the generation of false information; RAG grounds responses in retrieved factual documents, reducing hallucinations.

Option C is correct because BPE (Byte Pair Encoding) tokenizers trained specifically on code corpora learn subword units that align closely with programming language syntax (e.g., common keywords, operators, and variable patterns), resulting in fewer tokens for a given code snippet compared to general-purpose tokenizers. This reduces API calls by encoding more semantic meaning per token, directly minimizing token count.

Cosine similarity measures the cosine of the angle between two vectors, which is equivalent to the dot product of the vectors after they have been normalized to unit length (L2 norm = 1). Option C correctly describes this process: first normalize each embedding vector to unit length, then compute the dot product. This is the standard approach because raw embedding vectors from models like OCI's AI services may not be unit vectors, and the dot product alone does not account for magnitude differences.

GPT is a decoder-only model. BERT is encoder-only. T5 is encoder-decoder.

ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is the most appropriate metric because it measures the overlap of n-grams between the generated summary and reference summaries, directly aligning with the company's requirement. Unlike BLEU, which focuses on precision, ROUGE emphasizes recall, making it better suited for summarization tasks where capturing all key content from the reference is critical.

Chunking determines how documents are split into pieces. If chunks are too small, key information may be split across chunks, causing incomplete retrieval. Adjusting chunk size and overlap can improve completeness.

Fine-tuning with 500 examples may lead to overfitting or catastrophic forgetting, especially when the tasks are diverse. In-context learning with a few examples per task is more flexible and leverages the model's pre-trained knowledge. The small dataset size makes fine-tuning risky.

Greedy decoding always selects the token with highest probability, producing the same output for a given input. Temperature, top-k, and top-p introduce randomness.

The self-attention mechanism computes a weighted sum of all input token representations, where the weights are determined by the pairwise compatibility (attention scores) between tokens. This allows each token to dynamically attend to every other token in the sequence, capturing global dependencies without the limitations of fixed local windows or recurrence.

Greedy decoding, which is equivalent to setting temperature=0, always selects the token with the highest probability at each step. This deterministic behavior causes the model to produce the exact same output sequence for any input prompt, as there is no randomness or variation in token selection. The lack of diversity is a direct consequence of eliminating all stochasticity from the generation process.

Since self-attention processes tokens in parallel without inherent order, positional encoding adds information about the position of each token in the sequence.

Positional encoding is essential in Transformer models because the self-attention mechanism processes all tokens in parallel and has no inherent notion of sequence order. By adding positional encodings (often sinusoidal or learned) to the input embeddings, the model can distinguish between tokens at different positions, enabling it to capture word order and relative positions. Without this, the model would treat the sequence as a bag of tokens, losing all sequential context.

Chunking strategy (size and overlap) directly affects how well the retrieval step can find relevant passages. Too large or poorly split chunks can dilute semantic meaning.

Fine-tuning an encoder-decoder model like T5 on the 500 sentence pairs is the best approach because it leverages the model's pre-trained knowledge of language structure and translation patterns, then adapts it to the specific legal domain with a small but relevant parallel corpus. This transfer learning method requires far less data than training from scratch and typically outperforms zero-shot prompting for specialized, low-resource translation tasks.

BPE reduces vocabulary size by representing words as subword units, and it can handle out-of-vocabulary words by breaking them into known subwords. Fixed-length tokens and losslessness are not advantages of BPE.

The self-attention mechanism is the core component of the Transformer architecture that computes attention scores between every pair of words in the input sequence. These scores determine how much each word should influence the representation of the current word, allowing the model to dynamically weigh the importance of different words regardless of their positional distance. This mechanism is what enables the Transformer to capture long-range dependencies and contextual relationships in parallel.

Beam search considers multiple candidate sequences and selects the one with the highest overall probability, which often results in more fluent and accurate translations than greedy decoding, but at higher computational cost.

Pre-training is the initial phase where a model learns general language patterns from a large corpus. Fine-tuning adapts the pre-trained model to a specific task using a smaller labeled dataset.

Embedding models (e.g., Cohere Embed, OpenAI text-embedding-ada) are specialized to produce dense vector representations. Generation models (like GPT) produce text, not embeddings.

BERTScore correlates with human judgment but is not perfect; it may favor certain styles. Additionally, BERTScore may be inflated if the reference summaries are similar to the model's training data.

RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

Retrieval-Augmented Generation (RAG) directly addresses the problem of stale training data by dynamically retrieving current documents from an external knowledge base at inference time. This allows the model to generate answers grounded in up-to-date information without requiring retraining or a larger model, making it the most effective and practical solution for a question-answering system.

Hallucinations are a primary limitation of large language models because they cause the model to generate text that is factually incorrect, nonsensical, or not grounded in the training data. This occurs due to the probabilistic nature of token prediction, where the model prioritizes fluency and coherence over factual accuracy, especially when the prompt lacks sufficient context or the model is asked to recall specific facts not well-represented in its training.

ROUGE-L measures the longest common subsequence (LCS) between generated and reference summaries, focusing on lexical (word-level) overlap. It does not assess semantic meaning, fluency, or factual correctness. Human evaluators often prefer summaries that are coherent and capture key ideas, even if they use different wording, which explains why Model Y can score lower on ROUGE-L but be preferred 60% of the time.

Scaling prevents the dot products from growing too large in magnitude, which would push softmax into regions with extremely small gradients.

Positional encoding is essential in the Transformer architecture because the self-attention mechanism processes all tokens in parallel and has no inherent sense of order. By adding sinusoidal or learned positional vectors to the input embeddings, the model gains information about the relative or absolute position of each token in the sequence, enabling it to understand word order and sequence structure.

Self-attention allows each token to attend to every other token in the sequence, capturing contextual relationships regardless of distance.

Temperature >1 flattens the probability distribution, making low-probability tokens more likely. top_k=50 restricts to top 50 tokens, but top_p=1.0 imposes no cumulative probability cutoff. The combination yields diverse but potentially incoherent outputs.

Temperature sampling and top-p (nucleus) sampling both introduce randomness by adjusting the probability distribution. Greedy decoding and beam search are deterministic or near-deterministic. Top-k sampling also introduces randomness but top-p is more dynamic.

Models like Code Llama (a variant of Llama) are specifically fine-tuned on code and are known for strong code generation capabilities. While other models can generate code, Code Llama is the best fit among the options.

BERTScore measures semantic similarity using contextual embeddings, while ROUGE-L measures n-gram overlap. Higher BERTScore indicates better semantic faithfulness even without exact phrase matches.

BERTScore computes cosine similarity between contextual embeddings, capturing semantic meaning better than surface-level n-gram metrics.

Byte-Pair Encoding (BPE) is a subword tokenization algorithm that iteratively merges the most frequent pairs of bytes or characters into new tokens. Its main advantage over word-level tokenization is that it can represent any word, including rare or unseen words, as a sequence of subword tokens, thereby eliminating the out-of-vocabulary (OOV) problem. This allows models like GPT and BERT to handle arbitrary input without requiring a fixed word vocabulary.

Decoder-only models use masked self-attention (causal) and generate tokens left-to-right. They cannot use bidirectional context because future tokens are masked.

Option A is correct because embedding each ticket into a dense vector space and computing cosine similarity between all pairs is both efficient and effective for detecting duplicate intents. This approach leverages pre-trained embedding models (e.g., sentence-transformers) that capture semantic similarity, and pairwise cosine similarity scales well for 10,000 tickets (approximately 50 million comparisons) using optimized matrix operations. It avoids the quadratic cost of LLM inference per pair while preserving high accuracy for intent matching.

Byte-Pair Encoding (BPE) is a subword tokenization method that starts with individual characters and merges the most frequent pairs iteratively until a vocabulary size is reached.

Reducing temperature to 0.2 makes the output distribution more peaked, favoring high-probability tokens, which reduces syntax errors. Combining this with top-p=0.9 (nucleus sampling) limits the sampling pool to the smallest set of tokens whose cumulative probability reaches 0.9, further filtering out low-probability tokens that often cause invalid syntax. This balance preserves some creativity while ensuring syntactically valid code.

Document chunking is essential because it breaks large documents into smaller, manageable pieces that can be individually indexed and retrieved. Without chunking, the retrieval phase would either miss relevant context or return overly large documents that exceed the LLM's context window, reducing accuracy.

For retrieval tasks, embedding quality is best measured by semantic similarity metrics like BERTScore, which correlate with how well embeddings capture meaning. Perplexity measures language model fluency, not embedding quality.

Self-attention computes attention scores between all pairs of tokens, enabling the model to capture dependencies across the entire sequence.

Option D is correct because the generation model in a RAG pipeline is designed to leverage retrieved context but can still produce outputs not grounded in that context, a phenomenon known as hallucination. This occurs when the model relies on its parametric knowledge or statistical patterns rather than strictly adhering to the retrieved documents, especially if the prompt does not enforce strict grounding or the model's training biases override the context.

The self-attention mechanism is the core component of the Transformer architecture that computes attention scores between every pair of tokens in the input sequence. These scores determine how much each token should influence the representation of every other token, allowing the model to dynamically weigh the importance of different tokens when generating an output. This is achieved through scaled dot-product attention, where queries, keys, and values are derived from the input embeddings.

Option B is correct because few-shot prompting—providing explicit examples of the desired JSON format in the prompt—directly guides the LLM's output structure without requiring retraining. This technique leverages in-context learning, where the model infers the required schema from the examples, making it the most effective and efficient method for format adherence.

Option B is correct because large language models are trained on static datasets that have a fixed cutoff date, after which they have no knowledge of new events, publications, or data. This is an inherent architectural limitation: the model's parameters are frozen at the end of training, so it cannot learn or incorporate information beyond that point without retraining or fine-tuning.

Option D is correct because human-in-the-loop evaluation with fairness-focused rubrics directly addresses bias by incorporating human judgment to detect and correct biased outputs. This approach allows reviewers to systematically assess responses against predefined fairness criteria, catching subtle biases that automated methods might miss. It is a standard practice in responsible AI deployment for high-stakes applications like hiring.

Cosine similarity measures the cosine of the angle between two vectors, is commonly used for comparing embedding vectors, and ranges from -1 to 1, where 1 indicates identical direction.

Tokenization is the first step in processing text for LLMs, where raw input is split into smaller units called tokens (words, subwords, or characters). This is essential because models like GPT or BERT operate on discrete tokens, not raw strings, and tokenization defines the model's vocabulary and input structure.

The self-attention mechanism allows each token in the input sequence to attend to every other token, computing a weighted sum of their representations. This enables the model to capture long-range dependencies directly without the sequential processing constraints of RNNs, and because the attention scores for all tokens can be computed simultaneously, the mechanism supports parallel processing of the entire sequence.

OCI Generative AI offers models including Cohere Command/Embed and Meta Llama. Mistral and GPT are not mentioned in the context of OCI's available models, and BERT is an encoder-only model not typically offered as a generation model.

RAG provides factual grounding, and reducing temperature makes outputs more deterministic, reducing fabricated details.

Option A is correct because embedding each chunk into a dense vector using an embedding model is a fundamental step in a RAG pipeline. The embedding model converts text chunks into high-dimensional vector representations that capture semantic meaning, enabling efficient similarity search during retrieval. Without this step, the system cannot compare query intent with document content in a vector space.

RAG can ground generation in a curated corpus, and with strict retrieval settings (e.g., only retrieving from the legal corpus), the model will not use any outside knowledge, reducing hallucinations.

Retrieval-Augmented Generation (RAG) is the most appropriate approach because it allows the chatbot to answer questions from the policy documents without retraining the model. By indexing the documents in a vector store and retrieving relevant chunks at query time, RAG handles monthly updates by simply re-indexing the new documents, avoiding the cost and complexity of fine-tuning or retraining.

BERT uses WordPiece and GPT-2 uses BPE; both are subword tokenization methods. SentencePiece is used by models like T5 and Llama.

Top-p (nucleus) sampling cuts off the tail of the probability distribution where cumulative probability exceeds p, allowing dynamic vocabulary size.

BLEU is a precision-based metric (not recall). It uses modified n-gram precision with a brevity penalty. It correlates reasonably well with human judgment at the corpus level but has known limitations such as not capturing semantic equivalence.

RAG retrieves factual information from an external knowledge base to ground the generation, reducing hallucinations. The other options do not address factual accuracy.

The self-attention mechanism computes attention scores between all pairs of tokens, enabling the model to dynamically focus on relevant parts of the input. Positional encoding adds order information, multi-head attention runs multiple attention heads in parallel, and the feed-forward network processes each position independently.

Greedy decoding selects the token with the highest probability at each step, making it deterministic and often leading to repetitive outputs because it always chooses the most likely next token without considering future alternatives. This contrasts with stochastic methods that introduce randomness or explore multiple paths.

The self-attention mechanism is the core component of the transformer architecture that enables the model to dynamically assign weights to each word in a sentence relative to every other word. This allows the model to capture contextual relationships and dependencies, such as determining which words are most relevant to the current word being processed, regardless of their distance in the sequence.