1Z0-1127-25 LLM Fundamentals Practice Test 8 — 15 Questions

Question 1

Which evaluation metric is designed to measure the overlap of n-grams between a generated summary and a reference summary, focusing on recall of content words?

Accepted Answer

ROUGE. ROUGE-N measures n-gram recall (how many n-grams from the reference appear in the generated text). BLEU measures n-gram precision. BERTScore leverages contextual embeddings. Perplexity measures likelihood under the model.

Answer

Perplexity

Answer

BERTScore

Answer

BLEU

Question 2

A developer is building a code generation assistant and needs to ensure the LLM follows a specific output format (e.g., JSON). Which approach is MOST effective for achieving format adherence without retraining?

Accepted Answer

Provide a few-shot example of the desired JSON format in the prompt. Few-shot prompting—providing explicit examples of the desired JSON format in the prompt—directly guides the LLM's output structure without requiring retraining. This technique leverages in-context learning, where the model infers the required schema from the examples, making it the most effective and efficient method for format adherence.

Answer

Lower the temperature to 0 to reduce output variability

Answer

Fine-tune the model on a dataset of JSON code examples

Answer

Increase the context window to include more code context

Question 3

Which tokenization algorithm is commonly used by models like GPT and BERT, and works by merging frequently occurring character pairs iteratively?

Accepted Answer

Byte-Pair Encoding (BPE). Byte-Pair Encoding (BPE) starts with characters and merges the most frequent pairs to create subword units. WordPiece uses a similar likelihood-based approach. SentencePiece is a framework that can use BPE or unigram.

Answer

Morpheme-based tokenization

Answer

SentencePiece

Answer

WordPiece

Question 4

Which sampling strategy selects the token with the highest probability at each step, resulting in deterministic and often repetitive outputs?

Accepted Answer

Greedy decoding. Greedy decoding selects the token with the highest probability at each step, making it deterministic and often leading to repetitive outputs because it always chooses the most likely next token without considering future alternatives. This contrasts with stochastic methods that introduce randomness or explore multiple paths.

Answer

Beam search

Answer

Temperature sampling

Answer

Top-k sampling

Question 5

A developer is using the OCI Generative AI service and notices that the cost per API call is higher than expected. Which factor contributes MOST to the cost of an LLM inference call?

Accepted Answer

Number of input and output tokens. Most LLM APIs charge based on the number of input and output tokens. The token count directly affects cost. Model size, context window, and temperature are related but the direct billing metric is token count.

Answer

Model size (number of parameters)

Answer

Context window size

Answer

Temperature setting

Question 6

When evaluating a summarization model, the team notices that the ROUGE-L score is high but human evaluators rate the summaries poorly for coherence. What does this discrepancy MOST likely indicate?

Accepted Answer

ROUGE-L is not sensitive to coherence and fluency, only to n-gram overlap. ROUGE-L measures longest common subsequence overlap, focusing on word order recall. It does not capture coherence, factual consistency, or fluency. High ROUGE-L with low human scores suggests the summaries are lexically similar but not well-formed.

Answer

The model is suffering from hallucination

Answer

The reference summaries are too short

Answer

The human evaluators are biased

Question 7

Which TWO of the following are characteristics of decoder-only models like GPT? (Select TWO)

Accepted Answer

They use masked self-attention to prevent attending to future tokens. Decoder-only models use masked self-attention (causal) and generate tokens left-to-right. They cannot use bidirectional context because future tokens are masked.

Answer

They process input through an encoder and a decoder

Answer

They use bidirectional self-attention

Answer

They are ideal for tasks requiring full bidirectional context like NER

Question 8

A team is building a RAG pipeline on OCI. Which THREE steps are essential components of a standard RAG pipeline? (Select THREE)

Accepted Answer

Embedding each chunk into a dense vector using an embedding model. Option E is correct because chunking documents into passages is the foundational ingestion step in a RAG pipeline, splitting source content into retrievable units that fit the embedding model's context window. Option A is correct because each chunk must be converted into a dense vector via an embedding model so it can be indexed and compared semantically in vector space. Option B is correct because retrieval ranks stored chunk vectors against the query embedding, typically using cosine similarity, to fetch the most relevant context for the generator. Option C is not part of a standard RAG pipeline, since RAG augments a pre-existing LLM rather than pretraining a custom model from scratch on the corpus. Option D is also unnecessary, as RAG supplies retrieved chunks at inference time through the prompt context and does not require fine-tuning the generation model on those chunks.

Answer

Training a custom LLM from scratch on the document corpus

Answer

Fine-tuning the generation model on the retrieved chunks

Question 9

Which component of the transformer architecture allows the model to weigh the importance of different words in a sentence when processing input?

Accepted Answer

Self-attention mechanism. The self-attention mechanism is the core component of the transformer architecture that enables the model to dynamically assign weights to each word in a sentence relative to every other word. This allows the model to capture contextual relationships and dependencies, such as determining which words are most relevant to the current word being processed, regardless of their distance in the sequence.

Answer

Layer normalization

Answer

Positional encoding

Answer

Feed-forward neural network

Question 10

An organization is evaluating two LLMs for a code generation task. Model A has a perplexity of 1.5 on the validation set, and Model B has a perplexity of 3.0. However, Model A generates more syntactically incorrect code. Which conclusion is MOST valid?

Accepted Answer

Perplexity alone is insufficient; evaluate with task-specific metrics like BLEU or human review. Perplexity measures how well a language model predicts a sequence, but it does not directly assess code correctness or syntactic validity. Model A's lower perplexity (1.5) indicates it is more confident in its predictions, yet it produces more syntactically incorrect code, showing that perplexity alone is not a reliable indicator of code generation quality. Task-specific metrics like BLEU (for n-gram overlap) or human review are necessary to evaluate functional and syntactic correctness, making option C the most valid conclusion.

Answer

Model A's perplexity indicates overfitting, so Model B is preferable

Answer

Model B is better because higher perplexity correlates with more diverse outputs

Answer

Model A is better because lower perplexity always indicates higher quality outputs

Question 11

A developer is using the Cohere Command model for text generation and wants to ensure the output is deterministic for testing purposes. Which sampling strategy should they use?

Accepted Answer

Greedy decoding. Greedy decoding always selects the token with highest probability, producing the same output for a given input. Temperature, top-k, and top-p introduce randomness.

Answer

Top-k sampling with k=50

Answer

Temperature sampling with temperature=0.7

Answer

Top-p (nucleus) sampling with p=0.9

Question 12

Which of the following is a known limitation of large language models?

Accepted Answer

They sometimes produce plausible but incorrect information (hallucinations). A well-documented limitation of large language models (LLMs) is their tendency to generate plausible-sounding but factually incorrect or nonsensical information, commonly referred to as 'hallucinations'. This occurs because LLMs are trained to predict the next token based on statistical patterns in their training data, not to verify facts or maintain a ground-truth database. This limitation is a core challenge in deploying LLMs for tasks requiring high factual accuracy, such as legal or medical advice.

Answer

They can generate text in multiple languages

Answer

They support few-shot learning via in-context examples

Answer

They can be fine-tuned for domain-specific tasks

Question 13

A data scientist needs to compare two summaries generated by different models against a reference summary. Which metric focuses on recall of n-grams and is commonly used for summarization evaluation?

Accepted Answer

ROUGE. ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is the correct metric because it focuses on recall of n-grams, measuring how many of the reference summary's n-grams appear in the generated summary. This makes it the standard metric for summarization evaluation, as it directly assesses content coverage.

Answer

BLEU

Answer

Perplexity

Answer

BERTScore

Question 14

A team is building a system to detect duplicate customer support tickets. They have a dataset of 10,000 resolved tickets and want to find pairs with similar intent. Which approach would be MOST efficient and effective?

Accepted Answer

Embed each ticket using an embedding model and compute cosine similarity between all pairs. Embedding each ticket into a dense vector space and computing cosine similarity between all pairs is both efficient and effective for detecting duplicate intents. This approach leverages pre-trained embedding models (e.g., sentence-transformers) that capture semantic similarity, and pairwise cosine similarity scales well for 10,000 tickets (approximately 50 million comparisons) using optimized matrix operations. It avoids the quadratic cost of LLM inference per pair while preserving high accuracy for intent matching.

Answer

Use a long-context LLM to process all tickets in a single prompt

Answer

Use a generation model like Cohere Command to compare tickets one pair at a time

Answer

Fine-tune a generation model on a classification task to predict duplicates

Question 15

In the transformer architecture, what is the primary purpose of positional encoding?

Accepted Answer

To provide the model with information about the order of tokens. Since self-attention processes tokens in parallel without inherent order, positional encoding adds information about the position of each token in the sequence.

Answer

To normalize the input embeddings

Answer

To reduce the number of parameters in the model

Answer

To enable multi-head attention