CCNA Oci Genai Llm Fundamentals Questions

RAG grounds answers in retrieved documents, and fine-tuning on domain-specific data reduces errors. Increasing context window alone does not reduce hallucinations; greedy decoding reduces creativity but not factual errors.

ROUGE is recall-oriented, measuring how much of the reference content appears in the generated summary, which aligns with capturing key facts. BLEU is precision-oriented, and BERTScore measures semantic similarity but is not specifically recall-focused.

Hallucination is the term used when an LLM produces content that is not grounded in training data or provided context, appearing confident but being incorrect.

RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

ROUGE-L measures longest common subsequence overlap, focusing on word order recall. It does not capture coherence, factual consistency, or fluency. High ROUGE-L with low human scores suggests the summaries are lexically similar but not well-formed.

Most LLM APIs charge based on the number of input and output tokens. The token count directly affects cost. Model size, context window, and temperature are related but the direct billing metric is token count.

Top-p (nucleus) sampling selects from a dynamically chosen set of tokens whose cumulative probability exceeds p, allowing diversity while avoiding the long tail of improbable tokens.

Embedding models are specifically designed to produce dense vector representations that capture semantic meaning. They are distinct from generation models. For semantic search, embeddings from an embedding model are compared using cosine similarity.

Providing few-shot examples of desired behavior in the system prompt (in-context learning) can guide the model toward safe responses. Fine-tuning would require retraining, and prompt engineering is broader; few-shot examples are a specific, effective technique.

Content filtering and human review are direct mitigations. Training from scratch is impractical; modifying sampling does not address bias; increasing context window is irrelevant.

Perplexity measures how well a language model predicts a sequence, but it does not directly assess code correctness or syntactic validity. Model A's lower perplexity (1.5) indicates it is more confident in its predictions, yet it produces more syntactically incorrect code, showing that perplexity alone is not a reliable indicator of code generation quality. Task-specific metrics like BLEU (for n-gram overlap) or human review are necessary to evaluate functional and syntactic correctness, making option C the most valid conclusion.

RAG directly addresses the hallucination problem by grounding LLM outputs in retrieved, factual documents from an external knowledge base. Instead of relying solely on the model's parametric memory, RAG fetches relevant context at inference time, reducing the likelihood of generating plausible-sounding but incorrect facts.

RAG addresses hallucinations by grounding the model's output in retrieved documents that contain factual information. Knowledge cutoff, bias, and context length constraints are separate issues that RAG may partially help with, but its primary purpose is to reduce factual errors.

RAG retrieves relevant, current documents at inference time, providing up-to-date context without modifying the model parameters.

RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

Fine-tuning on a curated dataset of secure code examples can teach the model to avoid common vulnerability patterns while retaining its general coding ability. RAG with security docs could also help, but fine-tuning directly addresses the model's behavior more comprehensively.

High BLEU and low ROUGE indicate that the generated text has high precision (many n-grams match reference) but low recall (missing many reference n-grams). This often occurs when the output is short or overly cautious.

Option A is correct because encoder-only models like BERT employ bidirectional attention, allowing each token to attend to all other tokens in both directions, which is ideal for tasks requiring full context understanding such as classification or named entity recognition (NER). In contrast, decoder-only models like GPT use causal (masked) attention, where each token can only attend to previous tokens, making them suitable for autoregressive text generation.

LLMs can produce hallucinations (factual errors), have a knowledge cutoff date, and can exhibit bias from training data. They do not inherently lack creativity, and context length is a limitation but not a 'lack of understanding'.

Pre-training involves training a large language model from scratch on a vast, general corpus of unlabeled data to learn language patterns, grammar, and world knowledge. Fine-tuning then takes this pre-trained model and further trains it on a smaller, labeled dataset specific to a downstream task, such as sentiment analysis or question answering, adapting the model's weights for that particular objective. Option C correctly captures this fundamental distinction in the LLM workflow.

Temperature sampling with a higher temperature (>1) increases the probability of less likely tokens, promoting creativity and diversity.

Lowering temperature reduces randomness, making outputs more deterministic and less prone to errors, while still allowing some variation.

Oracle's OCI Generative AI service natively provides Oracle's own LLM models, such as the OCI Generative AI models, which are hosted entirely within OCI's infrastructure. This ensures that all data processing and storage remain within OCI, addressing the organization's data privacy concerns. Third-party models like Gemini, GPT-4, and Claude are not natively available in OCI Generative AI and would require external API calls, violating the data residency requirement.

ROUGE, BLEU, and BERTScore are all used for summarization evaluation. Perplexity measures model confidence, and cosine similarity is for embedding comparison.

Option B is correct because embedding models transform input data (text, images, etc.) into a fixed-size vector representation (a single vector per input) that captures semantic meaning, while generation models (like GPT or LLaMA) produce variable-length sequences of tokens as output, predicting one token at a time. This fundamental architectural difference defines their distinct roles: embeddings for similarity search and retrieval, generation for producing coherent text.

RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

ROUGE measures recall-oriented overlap between generated and reference summaries, suitable for summarization. BERTScore uses semantic similarity via embeddings. BLEU is for translation, perplexity for language modeling, and human evaluation is qualitative.

OCI Data Science with a model deployment endpoint is the correct choice because it provides a managed infrastructure for hosting and serving LLMs within a specific OCI region, ensuring all processing and data remain within that region to meet strict data residency requirements. The model deployment endpoint runs on dedicated compute resources in the chosen region, allowing low-latency inference while adhering to regional data boundaries.

Positional encoding provides information about token order and is added to the input embeddings. It is used in encoder-decoder and decoder-only models. It does not replace attention, and it is not learned for each token position in the original formulation (it uses fixed sine/cosine functions).

Option B is correct because using a smaller model from the same family (e.g., switching from Llama 3 70B to Llama 3 8B) reduces the number of parameters and computational resources required per inference, directly lowering cost. Smaller models often retain strong performance on many tasks, especially when the task complexity does not demand the full capacity of the larger model, thus maintaining output quality while reducing token processing costs.

Byte-Pair Encoding (BPE) starts with characters and merges the most frequent pairs to create subword units. WordPiece uses a similar likelihood-based approach. SentencePiece is a framework that can use BPE or unigram.

ROUGE-L measures the longest common subsequence, emphasizing recall of key content (n-grams in order). BLEU measures n-gram precision, penalizing short or non-fluent outputs. A high ROUGE-L but low BLEU suggests the summaries capture the main ideas but use different phrasing or are too short.

Generate embeddings for both sentences, then compute cosine similarity. Normalizing vectors ensures cosine similarity equals the dot product. Training a new model is unnecessary, and using a generation model is not appropriate for embeddings.

A large context window (e.g., 128K tokens) is critical for code generation because it allows the model to process entire code files, long dependency chains, and multi-file projects in a single pass. This reduces the risk of syntax errors caused by missing imports or incomplete function definitions, and it supports the generation of coherent, syntactically correct code across multiple languages without truncation.

RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

Option C is correct because large language models (LLMs) are known to generate factually incorrect information, often called 'hallucinations.' This occurs because LLMs are probabilistic models that predict the next token based on training data patterns, not verified facts. They lack a grounding mechanism to validate outputs against real-world truth, making hallucinations a fundamental limitation.

Lowering the temperature increases randomness, reducing the likelihood of exact copying. Greedy decoding and beam search both favor high-probability tokens, which can lead to copy behavior. Top-k and top-p control sampling but do not directly address copying if temperature is too low.

Low chunk overlap can cause loss of context, improper prompt instructions may lead the model to ignore retrieved content, and a small top-K retrieval may miss relevant documents. A large embedding dimension improves retrieval precision, and a high temperature increases randomness but does not cause ignoring documents.

ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is the correct metric because it focuses on recall of n-grams, measuring how many of the reference summary's n-grams appear in the generated summary. This makes it the standard metric for summarization evaluation, as it directly assesses content coverage.

Temperature sampling scales the logits before applying softmax, controlling the randomness of token selection. A temperature > 1 increases diversity, reducing repetition, while still allowing the model to generate coherent text.

An encoder-only embedding model like Cohere Embed is designed to convert text into dense vector representations (embeddings) that capture semantic meaning, which is exactly what is needed for semantic search over a large corpus of legal documents. These models use a bidirectional transformer architecture to encode context from both directions, producing fixed-size embeddings that can be efficiently compared using cosine similarity or other distance metrics.

Byte-Pair Encoding (BPE) is a subword tokenization algorithm that iteratively merges the most frequent byte pairs, creating a fixed-size vocabulary of common subwords. For English text, BPE efficiently represents common words as single tokens and splits rare words into meaningful subword units, resulting in fewer tokens overall compared to other methods. This directly reduces token usage and associated costs when processing large documents with an LLM on OCI.

Cosine similarity measures the angle between vectors, making it ideal for comparing semantic similarity regardless of vector magnitude. High-dimensional embeddings from models like text-embedding-ada-002 or all-MiniLM-L6-v2 are specifically trained to produce isotropic vector spaces where cosine similarity correlates with semantic relatedness. This property is fundamental for tasks like clustering or retrieval-augmented generation (RAG) where relative direction, not absolute distance, matters.

Option A is correct because generating synthetic labeled examples using an LLM is a data augmentation technique that expands the small dataset without requiring new human annotations. This approach leverages the LLM's ability to produce diverse, class-consistent text, which helps the model generalize better and reduces overfitting when training on only 200 examples per class.

Option B is correct because BERTScore leverages pre-trained contextual embeddings from BERT to compute token-level similarity between candidate and reference texts, allowing it to capture semantic equivalence even when surface forms differ. This makes it more robust to paraphrases than BLEU, which relies on exact n-gram matching and cannot recognize synonymous or rephrased content.

ROUGE is recall-oriented and measures overlap of n-grams, making it a standard metric for summarization. BERTScore measures semantic similarity using embeddings, which can capture meaning even when wording differs. BLEU is more for translation, perplexity measures fluency, and human evaluation is qualitative but not a metric.

Encoder-decoder architectures like T5 are designed for sequence-to-sequence tasks. The encoder processes the input sequence, and the decoder generates the output sequence, making it ideal for translation.

BLEU (Bilingual Evaluation Understudy) is the most suitable metric for evaluating a translation model's output against multiple reference translations because it measures n-gram precision between the candidate translation and one or more reference translations. It directly quantifies how many words and phrases in the candidate match those in the references, making it the standard metric for machine translation tasks.

If the retriever returns irrelevant chunks or the generation model ignores the context, the answers will be off-topic. Checking retrieval relevance and prompt instruction is key.

ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is the most appropriate metric because it specifically measures recall of key information by comparing n-gram overlap between the generated summary and a reference summary. This aligns directly with the practitioner's goal of evaluating how well the LLM-generated summary captures the essential content from the human-written reference.

Retrieval-Augmented Generation (RAG) with a vector database is the most suitable approach because it allows the LLM to answer questions about a frequently updated private codebase without retraining. RAG retrieves relevant code snippets from a vector index at query time, ensuring the model always has access to the latest code without the cost and latency of hourly fine-tuning.

The 'Max tokens' parameter controls the maximum length of the generated output. When a model cuts off mid-sentence, it means the token limit has been reached before the model could complete its response. Increasing this value allows the model to generate more tokens, thus completing the translation.

Chunking splits documents, embedding converts chunks to vectors, retrieval fetches relevant chunks, and generation produces the answer.

Option A is correct because the transformer decoder-only architecture, such as GPT, employs masked self-attention (also known as causal attention) to ensure that each token can only attend to previous tokens in the sequence. This masking prevents the model from 'seeing' future tokens during training and generation, which is essential for maintaining the autoregressive property where predictions depend only on past context.

Encoder-only models like BERT are well-suited for classification tasks (e.g., sentiment analysis) because they produce contextualized representations for each token and can be fine-tuned with a classification head. BERT supports multiple languages via multilingual variants.

In-context learning does not update model weights; it provides examples in the prompt at inference time. Fine-tuning updates the model weights through additional training on a dataset.

Multi-head attention allows the model to attend to information from different representation subspaces at different positions, capturing a richer understanding.

Pre-training trains a model on a large, general corpus to learn language representations; fine-tuning adapts the pre-trained model to a specific task or domain using a smaller labeled dataset.

Top-p (nucleus) sampling selects from the smallest set of tokens whose cumulative probability exceeds p. It adapts to the model's confidence, allowing diversity while maintaining relevance. Greedy decoding is deterministic, temperature scales all probabilities, top-k fixes the number of candidates, and beam search explores multiple sequences but tends to produce safe outputs.

BERTScore is correct because it leverages contextual embeddings from BERT to compute semantic similarity between generated and reference summaries, capturing meaning beyond exact n-gram overlap. This makes it highly effective for evaluating text summarization models where paraphrasing and semantic equivalence are critical.

Self-attention computes attention scores between all pairs of positions, enabling the model to weigh the importance of different input tokens.

RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

BERTScore uses contextual embeddings, captures semantic similarity, and correlates better with human judgment than n-gram metrics.

Encoder-decoder models like T5 or BART are designed for sequence-to-sequence tasks such as summarization, and can handle long inputs with their encoder.

RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

SentencePiece is language-agnostic, works directly on raw text without requiring pre-tokenisation (e.g., whitespace splitting), and handles languages like Chinese and Korean where word boundaries are not obvious. BPE and WordPiece typically require pre-tokenisation, making them less suitable for CJK languages.

Chunking splits documents into manageable pieces, and embedding converts them into vectors for similarity search.

Enabling frequency penalty directly reduces the likelihood of the model repeating the same tokens by subtracting a penalty proportional to the token's existing frequency in the generated text. This is the most targeted way to combat repetitive output without altering the core sampling parameters (temperature and top-p) that control randomness.

Option B is correct because hallucination in LLMs refers to the generation of content that is plausible-sounding but factually incorrect or nonsensical. This occurs when the model's probabilistic next-token prediction produces statements that are not grounded in its training data or real-world facts, often due to overgeneralization or lack of factual recall mechanisms.

Smaller models typically have lower latency, model families (Llama, Code Llama) are designed for code, and deployment choices affect latency. Context window size and fine-tuning cost are secondary or not directly relevant to latency.

Option B is correct because a well-documented limitation of large language models (LLMs) is their tendency to generate plausible-sounding but factually incorrect or nonsensical information, commonly referred to as 'hallucinations'. This occurs because LLMs are trained to predict the next token based on statistical patterns in their training data, not to verify facts or maintain a ground-truth database. This limitation is a core challenge in deploying LLMs for tasks requiring high factual accuracy, such as legal or medical advice.

Higher dimensions can capture more nuanced semantics but require more storage and may increase retrieval latency.