1Z0-1127-25 LLM Fundamentals Practice Test 3 — 15 Questions

Question 1

A team is building a code generation assistant and needs to choose between fine-tuning a base LLM or using in-context learning with a few examples. They have 500 high-quality code examples. The assistant must generate code for a wide variety of tasks. Which approach is BETTER and why?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

In-context learning, because it allows the model to adapt to each task dynamically without risking catastrophic forgetting. Fine-tuning with 500 examples may lead to overfitting or catastrophic forgetting, especially when the tasks are diverse. In-context learning with a few examples per task is more flexible and leverages the model's pre-trained knowledge. The small dataset size makes fine-tuning risky.

Answer

Fine-tuning, because it reduces inference cost compared to providing examples each time

Answer

Fine-tuning, because it permanently encodes the examples into the model weights

Answer

In-context learning, because it requires no additional training infrastructure

Question 2

A developer is building a text generation application using OCI Generative AI and wants to control the creativity of the output. Which THREE sampling parameters can they adjust? (Choose three.)

Accepted Answer

Top-k. Temperature scales the logits before softmax. Top-k limits the sampling pool to the k most likely tokens. Top-p (nucleus) sampling selects from tokens whose cumulative probability exceeds p. Beam search is a decoding strategy (not a sampling parameter), max tokens controls output length, and frequency penalty reduces repetition.

Answer

Max tokens

Answer

Beam search width

Question 3

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. Retrieval-Augmented Generation (RAG) is the most appropriate approach because it allows the chatbot to answer questions from the policy documents without retraining the model. By indexing the documents in a vector store and retrieving relevant chunks at query time, RAG handles monthly updates by simply re-indexing the new documents, avoiding the cost and complexity of fine-tuning or retraining.

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Answer

Fine-tune a base LLM on the policy documents monthly

Answer

Train a custom model from scratch on the policy documents each month

Question 4

A developer notices that a text generation model produces repetitive phrases when using greedy decoding. Which sampling strategy would best introduce controlled randomness to reduce repetition while maintaining coherence?

Accepted Answer

Apply temperature sampling with temperature > 1. Temperature sampling with temperature > 1 flattens the softmax distribution, giving lower-probability tokens more chance of being selected. This introduces controlled randomness that breaks the deterministic loops greedy decoding falls into, while the model still favors high-probability tokens enough to maintain coherence. It is the standard remedy for repetitive greedy output.

Answer

Use greedy decoding but with a repetition penalty

Answer

Increase the beam width in beam search

Answer

Disable top-k sampling

Question 5

A team is evaluating two embedding models for a similarity search task. Model A has a higher BERTScore on a reference dataset. Model B has a lower perplexity on the same dataset. Which model is likely better for retrieval?

Accepted Answer

Model A, because BERTScore directly measures semantic similarity of embeddings. For retrieval tasks, embedding quality is best measured by semantic similarity metrics like BERTScore, which correlate with how well embeddings capture meaning. Perplexity measures language model fluency, not embedding quality.

Answer

Both are equally good for retrieval

Answer

Model B, because lower perplexity indicates better language modeling, which improves retrieval

Answer

Neither metric is relevant for retrieval tasks

Question 6

A data scientist is building a text summarization system using an LLM. They want to evaluate the model's output against human-written summaries. Which TWO metrics are most appropriate for this evaluation? (Choose two.)

Accepted Answer

ROUGE. ROUGE is recall-oriented and measures overlap of n-grams, making it a standard metric for summarization. BERTScore measures semantic similarity using embeddings, which can capture meaning even when wording differs. BLEU is more for translation, perplexity measures fluency, and human evaluation is qualitative but not a metric.

Answer

Human evaluation rubrics

Answer

BLEU

Answer

Perplexity

Question 7

An organization is deploying an LLM for document question answering. They want to reduce hallucinations and ensure answers are grounded in provided documents. Which THREE techniques should they implement? (Choose three.)

Accepted Answer

Fine-tune the model on a corpus of in-domain documents. Option B is correct because fine-tuning the model on an in-domain corpus adapts its weights to the target document style, terminology, and factual patterns, which improves grounding and reduces the tendency to hallucinate outside the provided domain. Option C is correct because a low temperature such as 0.1 makes the sampling distribution sharply peaked, so the model selects high-probability, more deterministic tokens and avoids the creative, low-probability outputs that drive hallucination. Option E is correct because Retrieval-Augmented Generation retrieves relevant passages from the provided documents and injects them into the prompt, grounding the answer in actual source text rather than relying solely on parametric memory. Option A does not belong because a longer context window only increases how much text can be supplied; without retrieval or grounding mechanisms it does not ensure the model actually attends to or uses the relevant documents. Option D does not belong because a high temperature such as 1.5 flattens the distribution and increases randomness, which raises rather than reduces hallucination.

Answer

Use a longer context window to include more document text

Answer

Set a high temperature (e.g., 1.5) for sampling

Question 8

A developer is comparing different foundation models for a text completion API on OCI. Which TWO of the following are model families available through OCI Generative AI service? (Choose two.)

Accepted Answer

Meta Llama. OCI Generative AI offers models including Cohere Command/Embed and Meta Llama. Mistral and GPT are not mentioned in the context of OCI's available models, and BERT is an encoder-only model not typically offered as a generation model.

Answer

OpenAI GPT

Answer

BERT

Answer

Mistral

Question 9

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. Retrieval-Augmented Generation (RAG) with documents indexed in a vector store is the most appropriate approach because it allows the chatbot to retrieve relevant document chunks at query time without retraining the model. When documents update monthly, only the vector store needs re-indexing, not the LLM, making it cost-effective and current.

Answer

Fine-tune a base LLM on the policy documents monthly

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Answer

Train a custom model from scratch on the policy documents each month

Question 10

Which TWO of the following are advantages of using Byte-Pair Encoding (BPE) tokenization compared to word-level tokenization?

Accepted Answer

Smaller vocabulary size. Option B is correct because BPE builds its vocabulary from the most frequent character and subword merges rather than from every distinct whole word, so it typically yields a much smaller vocabulary size than word-level tokenization, which must include an entry for each unique word. Option E is correct because BPE can represent any unseen or rare word by splitting it into known subword units (e.g., 'unhappiness' into 'un', 'happi', 'ness'), effectively eliminating the out-of-vocabulary problem that plagues word-level tokenizers. Option A is not marked correct because lossless Unicode handling is not a guaranteed property of BPE itself; it depends on the underlying character/byte encoding and preprocessing, and word-level tokenizers can also preserve Unicode if configured properly. Option C is wrong because BPE does not enforce a fixed token length per input; sequence lengths vary with the text and merge rules. Option D is wrong because BPE often produces longer token sequences than word-level tokenization (splitting words into multiple subwords), which can increase, not reduce, sequence length and inference cost.

Answer

Guaranteed lossless encoding of all Unicode characters

Answer

Fixed token length for every input

Answer

Faster inference due to reduced sequence length

Question 11

A data scientist is evaluating an LLM's performance on a summarization task. Which TWO metrics are most suitable for this evaluation?

Accepted Answer

ROUGE. ROUGE (D) is correct because it measures n-gram and longest-common-subsequence overlap between a generated summary and reference summaries, which directly targets summarization quality. BERTScore (E) is correct because it uses contextual embeddings to compute semantic similarity between generated and reference summaries, capturing meaning beyond exact word overlap. Perplexity (A) is not suitable here since it measures a language model's token-level predictive uncertainty, not summary quality against references. Human evaluation (B) is valuable but is not a metric in the same automated sense and is not among the two marked correct. BLEU (C) is designed primarily for machine translation and correlates poorly with summarization quality.

Answer

Perplexity

Answer

Human evaluation

Answer

BLEU

Question 12

Which TWO of the following sampling strategies introduce randomness into text generation?

Accepted Answer

Top-k sampling. Top-k sampling (D) is correct because it restricts the candidate pool to the k most probable tokens and then randomly samples from that set according to their probabilities, which explicitly introduces randomness into the output. Top-p (nucleus) sampling (E) is also correct because it samples randomly from the smallest set of tokens whose cumulative probability exceeds p, again injecting stochasticity into token selection. By contrast, beam search (A) is a deterministic search that keeps the top-b sequences by cumulative log-probability and returns the best one, and greedy decoding (B) deterministically picks the single highest-probability token at each step, so neither introduces randomness. Temperature sampling (C) is not marked correct here; although temperature can scale logits and affect randomness, the two strategies explicitly identified as introducing randomness in this question are top-k and top-p.

Answer

Beam search

Answer

Greedy decoding

Answer

Temperature sampling

Question 13

A team wants to compare the semantic similarity between two sentences using embeddings. Which THREE steps are required?

Accepted Answer

Normalize the resulting vectors to unit length. Option E is correct because semantic similarity with embeddings requires first converting each sentence into a dense vector using an embedding model, which produces the numerical representations that can be compared. Option B is correct because normalizing the resulting vectors to unit length ensures that cosine similarity reflects only directional (semantic) alignment and is not distorted by differing vector magnitudes. Option C is correct because cosine similarity is the standard metric for comparing embedding vectors, computed as the dot product of the normalized vectors, yielding a value from -1 to 1 that indicates semantic closeness. Option A is incorrect because text generation models produce text rather than the fixed embedding vectors needed for direct similarity computation. Option D is incorrect because training a new neural network is unnecessary; pretrained embedding models already provide the representations required for this comparison.

Answer

Use a text generation model to compare the sentences

Answer

Train a new neural network on the two sentences

Question 14

A practitioner is choosing a model for a code generation assistant that must run on OCI with low latency. Which THREE considerations are most important?

Accepted Answer

Model size (number of parameters). For a low-latency code generation assistant on OCI, model size (A) is critical because parameter count directly drives inference latency and the GPU memory/compute footprint required, so smaller or distilled models generally respond faster. Whether the model is offered as a managed endpoint versus requiring self-hosting on OCI GPU shapes (B) matters because managed endpoints (e.g., OCI Generative AI or a model catalog service) reduce operational overhead and can be provisioned close to the consumer for lower network latency, whereas self-hosting adds deployment and scaling latency. Choosing a model specifically fine-tuned for code such as Code Llama or StarCoder (D) is important because code-specialized models produce more accurate completions with fewer tokens and retries, which directly improves perceived latency and quality for a coding assistant. Fine-tuning cost on internal codebases (C) is a training-time expense that does not govern runtime latency, and context window length (E) affects how much input can be processed but is not inherently a latency driver, so neither is among the three most important considerations here.

Answer

The cost of fine-tuning the model on internal codebases

Answer

The model's context window length

Question 15

A solutions architect is designing a document-summarization assistant on OCI Generative AI. The source contracts are 60 pages long, and the model must preserve details from both the first and last pages. The architect must choose an architecture that natively attends to every token in the input in a single forward pass. Which model architecture should be selected?

Accepted Answer

An encoder-only Transformer, because it builds bidirectional contextual embeddings over the full input. Bidirectional self-attention is the defining trait of encoder-only Transformers, letting every token in a long contract attend to every other token, including those at the far end of the document. That global, non-causal attention is what preserves first-page and last-page details in a single forward pass, which is precisely the property the architect needs for whole-document summarization.

Answer

A decoder-only Transformer, because causal masking guarantees the model reads the entire prompt before generating

Answer

A convolutional sequence model, because sliding kernels compress the entire document into one fixed vector

Answer

A retrieval-augmented pipeline, because chunking the contract into passages and embedding each chunk removes the context-length problem