1Z0-1127-25 LLM Fundamentals Practice Test 7 — 15 Questions

Question 1

A research team is comparing two LLMs for a translation task. Model A uses greedy decoding, Model B uses beam search with width=5. Both models are otherwise identical. Which statement about their outputs is MOST likely true?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Model B will generally produce more fluent and accurate translations. Beam search considers multiple candidate sequences and selects the one with the highest overall probability, which often results in more fluent and accurate translations than greedy decoding, but at higher computational cost.

Answer

Model A will have higher BLEU scores than Model B

Answer

Model A will produce more diverse translations

Answer

Model B will have lower latency than Model A

Question 2

What is the primary difference between pre-training and fine-tuning in the context of large language models?

Accepted Answer

Pre-training trains from scratch, fine-tuning updates all weights on a new dataset. Pre-training trains a model on a large, general corpus to learn language representations; fine-tuning adapts the pre-trained model to a specific task or domain using a smaller labeled dataset.

Answer

Pre-training uses a smaller dataset, fine-tuning uses a larger dataset

Answer

Pre-training produces embeddings, fine-tuning produces text generation

Answer

Pre-training is unsupervised, fine-tuning is always supervised

Question 3

A data scientist is debugging a RAG system where the generated answers are not relevant to the retrieved documents. Which TWO factors are MOST likely causing this issue?

Accepted Answer

The retriever is returning irrelevant chunks due to poor embeddings or low similarity threshold. If the retriever returns irrelevant chunks or the generation model ignores the context, the answers will be off-topic. Checking retrieval relevance and prompt instruction is key.

Answer

The context window of the generation model is smaller than the retrieved chunks

Answer

The temperature is set too low, making outputs deterministic

Answer

The chunking strategy produces chunks that are too small, losing context

Question 4

A team is evaluating two LLMs for a summarization task. Model X has a BERTScore of 0.85, Model Y has a BERTScore of 0.82. However, human evaluators prefer Model Y. Which TWO reasons could explain this discrepancy?

Accepted Answer

BERTScore is based on BERT embeddings, which may not fully capture summary-specific qualities like conciseness or readability. BERTScore correlates with human judgment but is not perfect; it may favor certain styles. Additionally, BERTScore may be inflated if the reference summaries are similar to the model's training data.

Answer

BERTScore uses precision only, so it misses recall aspects

Answer

Human evaluators were not given clear criteria for evaluation

Answer

Model Y was fine-tuned on a different dataset, causing distribution shift

Question 5

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

Answer

Train a custom model from scratch on the policy documents each month

Answer

Fine-tune a base LLM on the policy documents monthly

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Question 6

A data scientist is fine-tuning a Llama 2 model on a custom dataset for a summarization task. After fine-tuning, the model produces summaries that are too similar to the input text, often copying sentences verbatim. Which adjustment is MOST likely to reduce copying and improve abstractive summarization?

Accepted Answer

Increase the temperature to 0.8 and use top-p sampling. Increasing temperature (e.g., to 0.8) adds randomness to token selection, making the model less likely to choose the most probable tokens that lead to verbatim copying from the input. Top-p sampling further restricts the sampling pool to a cumulative probability mass, balancing diversity and coherence. In contrast, greedy decoding and beam search always select high-probability tokens, which encourages copying. Reducing context window may lose important information, not fix copying. Thus, higher temperature with top-p sampling is the best adjustment to promote abstractive summarization.

Answer

Switch to greedy decoding

Answer

Increase the beam search width

Answer

Reduce the context window size

Question 7

An LLM generates a response that contains a plausible-sounding but factually incorrect statement about a historical event. This is an example of which known limitation?

Accepted Answer

Hallucination. Hallucination in LLMs refers to the generation of content that is plausible-sounding but factually incorrect or nonsensical. This occurs when the model's probabilistic next-token prediction produces statements that are not grounded in its training data or real-world facts, often due to overgeneralization or lack of factual recall mechanisms.

Answer

Knowledge cutoff

Answer

Bias in training data

Answer

Context length constraint

Question 8

Which evaluation metric is designed to measure the overlap of n-grams between a generated summary and a reference summary, focusing on recall of content words?

Accepted Answer

ROUGE. ROUGE-N measures n-gram recall (how many n-grams from the reference appear in the generated text). BLEU measures n-gram precision. BERTScore leverages contextual embeddings. Perplexity measures likelihood under the model.

Answer

Perplexity

Answer

BERTScore

Answer

BLEU

Question 9

A developer is building a code generation assistant and needs to ensure the LLM follows a specific output format (e.g., JSON). Which approach is MOST effective for achieving format adherence without retraining?

Accepted Answer

Provide a few-shot example of the desired JSON format in the prompt. Few-shot prompting—providing explicit examples of the desired JSON format in the prompt—directly guides the LLM's output structure without requiring retraining. This technique leverages in-context learning, where the model infers the required schema from the examples, making it the most effective and efficient method for format adherence.

Answer

Lower the temperature to 0 to reduce output variability

Answer

Fine-tune the model on a dataset of JSON code examples

Answer

Increase the context window to include more code context

Question 10

Which tokenization algorithm is commonly used by models like GPT and BERT, and works by merging frequently occurring character pairs iteratively?

Accepted Answer

Byte-Pair Encoding (BPE). Byte-Pair Encoding (BPE) starts with characters and merges the most frequent pairs to create subword units. WordPiece uses a similar likelihood-based approach. SentencePiece is a framework that can use BPE or unigram.

Answer

Morpheme-based tokenization

Answer

SentencePiece

Answer

WordPiece

Question 11

A developer is using the OCI Generative AI service and notices that the cost per API call is higher than expected. Which factor contributes MOST to the cost of an LLM inference call?

Accepted Answer

Number of input and output tokens. Most LLM APIs charge based on the number of input and output tokens. The token count directly affects cost. Model size, context window, and temperature are related but the direct billing metric is token count.

Answer

Model size (number of parameters)

Answer

Context window size

Answer

Temperature setting

Question 12

When evaluating a summarization model, the team notices that the ROUGE-L score is high but human evaluators rate the summaries poorly for coherence. What does this discrepancy MOST likely indicate?

Accepted Answer

ROUGE-L is not sensitive to coherence and fluency, only to n-gram overlap. ROUGE-L measures longest common subsequence overlap, focusing on word order recall. It does not capture coherence, factual consistency, or fluency. High ROUGE-L with low human scores suggests the summaries are lexically similar but not well-formed.

Answer

The model is suffering from hallucination

Answer

The reference summaries are too short

Answer

The human evaluators are biased

Question 13

Which TWO of the following are characteristics of decoder-only models like GPT? (Select TWO)

Accepted Answer

They use masked self-attention to prevent attending to future tokens. Decoder-only models use masked self-attention (causal) and generate tokens left-to-right. They cannot use bidirectional context because future tokens are masked.

Answer

They process input through an encoder and a decoder

Answer

They use bidirectional self-attention

Answer

They are ideal for tasks requiring full bidirectional context like NER

Question 14

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

Answer

Train a custom model from scratch on the policy documents each month

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Answer

Fine-tune a base LLM on the policy documents monthly

Question 15

A developer is using the Cohere Command model for text generation and wants to ensure the output is deterministic for testing purposes. Which sampling strategy should they use?

Accepted Answer

Greedy decoding. Greedy decoding always selects the token with highest probability, producing the same output for a given input. Temperature, top-k, and top-p introduce randomness.

Answer

Top-k sampling with k=50

Answer

Temperature sampling with temperature=0.7

Answer

Top-p (nucleus) sampling with p=0.9