1Z0-1127-25 LLM Fundamentals Practice Test 9 — 15 Questions

Question 1

A machine learning engineer is evaluating the performance of a translation model using BLEU score. Which THREE statements about BLEU are correct? (Choose three.)

Accepted Answer

BLEU includes a brevity penalty to penalize outputs that are too short. BLEU is a precision-based metric (not recall). It uses modified n-gram precision with a brevity penalty. It correlates reasonably well with human judgment at the corpus level but has known limitations such as not capturing semantic equivalence.

Answer

BLEU measures recall of n-grams by comparing the output to the reference

Answer

BLEU is a recall-oriented metric

Question 2

A developer is building a text generation application using OCI Generative AI and wants to control the creativity of the output. Which THREE sampling parameters can they adjust? (Choose three.)

Accepted Answer

Top-k. Temperature scales the logits before softmax. Top-k limits the sampling pool to the k most likely tokens. Top-p (nucleus) sampling selects from tokens whose cumulative probability exceeds p. Beam search is a decoding strategy (not a sampling parameter), max tokens controls output length, and frequency penalty reduces repetition.

Answer

Max tokens

Answer

Beam search width

Question 3

Which component of the Transformer architecture allows each token to consider the relevance of every other token in the input sequence?

Accepted Answer

Self-attention. Self-attention computes attention scores between all pairs of tokens, enabling the model to capture dependencies across the entire sequence.

Answer

Multi-head attention

Answer

Feed-forward network

Answer

Positional encoding

Question 4

A data scientist is comparing BLEU, ROUGE, and BERTScore to evaluate a summarization model. The client cares most about whether the summary captures all key facts from the source document. Which metric is most aligned with this requirement?

Accepted Answer

ROUGE. ROUGE is recall-oriented, measuring how much of the reference content appears in the generated summary, which aligns with capturing key facts. BLEU is precision-oriented, and BERTScore measures semantic similarity but is not specifically recall-focused.

Answer

Perplexity

Answer

BERTScore

Answer

BLEU

Question 5

A developer notices that a text generation model produces repetitive phrases when using greedy decoding. Which sampling strategy would best introduce controlled randomness to reduce repetition while maintaining coherence?

Accepted Answer

Apply temperature sampling with temperature > 1. Temperature sampling scales the logits before applying softmax, controlling the randomness of token selection. A temperature > 1 increases diversity, reducing repetition, while still allowing the model to generate coherent text.

Answer

Use greedy decoding but with a repetition penalty

Answer

Increase the beam width in beam search

Answer

Disable top-k sampling

Question 6

Which tokenization algorithm is commonly used in models like GPT and BERT and builds tokens by merging the most frequent pairs of characters or subwords iteratively?

Accepted Answer

Byte-Pair Encoding (BPE). Byte-Pair Encoding (BPE) is a subword tokenization method that starts with individual characters and merges the most frequent pairs iteratively until a vocabulary size is reached.

Answer

WordPiece

Answer

SentencePiece

Answer

Unigram tokenization

Question 7

An organization needs to deploy a model that can both understand and generate text, such as for a translation task where the input is in English and output is in French. Which model architecture is most suitable?

Accepted Answer

Encoder-decoder (e.g., T5). Encoder-decoder architectures like T5 are designed for sequence-to-sequence tasks. The encoder processes the input sequence, and the decoder generates the output sequence, making it ideal for translation.

Answer

Decoder-only (e.g., GPT)

Answer

Encoder-only (e.g., BERT)

Answer

Mixture of Experts (MoE)

Question 8

A team is deploying a chatbot that must never output harmful or biased statements. They plan to use a pre-trained LLM with in-context learning. Which additional measure is MOST effective at reducing harmful outputs without retraining?

Accepted Answer

Include few-shot examples in the system prompt that demonstrate appropriate responses. Providing few-shot examples of desired behavior in the system prompt (in-context learning) can guide the model toward safe responses. Fine-tuning would require retraining, and prompt engineering is broader; few-shot examples are a specific, effective technique.

Answer

Apply a larger context window to include more safety instructions

Answer

Fine-tune the model on a curated dataset of safe dialogues

Answer

Use beam search with a high beam width

Question 9

A developer wants to compare two sentences for semantic similarity using embeddings. Which distance or similarity metric is most commonly used for dense vector representations?

Accepted Answer

Cosine similarity. Cosine similarity measures the cosine of the angle between two vectors, is commonly used for comparing embedding vectors, and ranges from -1 to 1, where 1 indicates identical direction.

Answer

Jaccard similarity

Answer

Manhattan distance

Answer

Euclidean distance

Question 10

Which of the following is a known limitation of large language models where the model generates plausible-sounding but factually incorrect information?

Accepted Answer

Hallucination. Hallucination is the term used when an LLM produces content that is not grounded in training data or provided context, appearing confident but being incorrect.

Answer

Bias

Answer

Context window limitation

Answer

Knowledge cutoff

Question 11

A team is evaluating two embedding models for a similarity search task. Model A has a higher BERTScore on a reference dataset. Model B has a lower perplexity on the same dataset. Which model is likely better for retrieval?

Accepted Answer

Model A, because BERTScore directly measures semantic similarity of embeddings. For retrieval tasks, embedding quality is best measured by semantic similarity metrics like BERTScore, which correlate with how well embeddings capture meaning. Perplexity measures language model fluency, not embedding quality.

Answer

Both are equally good for retrieval

Answer

Model B, because lower perplexity indicates better language modeling, which improves retrieval

Answer

Neither metric is relevant for retrieval tasks

Question 12

A company's AI system uses RAG to answer customer questions. Users often get incomplete answers because the retrieved chunks do not contain all relevant information. Which step in the RAG pipeline is most likely the issue?

Accepted Answer

Chunking strategy (chunk size and overlap). Chunking determines how documents are split into pieces. If chunks are too small, key information may be split across chunks, causing incomplete retrieval. Adjusting chunk size and overlap can improve completeness.

Answer

Retrieval top-k setting

Answer

Generation model temperature

Answer

Embedding model selection

Question 13

An ML engineer is selecting a pre-trained model for a code generation task. The model must be able to generate syntactically correct code in multiple programming languages. Which model family is BEST suited for this task?

Accepted Answer

Meta Llama (Code Llama variant). Models like Code Llama (a variant of Llama) are specifically fine-tuned on code and are known for strong code generation capabilities. While other models can generate code, Code Llama is the best fit among the options.

Answer

BERT

Answer

Cohere Command

Answer

Mistral

Question 14

A data scientist is building a text summarization system using an LLM. They want to evaluate the model's output against human-written summaries. Which TWO metrics are most appropriate for this evaluation? (Choose two.)

Accepted Answer

ROUGE. ROUGE is recall-oriented and measures overlap of n-grams, making it a standard metric for summarization. BERTScore measures semantic similarity using embeddings, which can capture meaning even when wording differs. BLEU is more for translation, perplexity measures fluency, and human evaluation is qualitative but not a metric.

Answer

Human evaluation rubrics

Answer

BLEU

Answer

Perplexity

Question 15

An organization is deploying an LLM for document question answering. They want to reduce hallucinations and ensure answers are grounded in provided documents. Which THREE techniques should they implement? (Choose three.)

Accepted Answer

Fine-tune the model on a corpus of in-domain documents. RAG retrieves relevant document chunks and conditions the generation on them, reducing hallucination. Fine-tuning on the document domain can improve grounding. Using a lower temperature (closer to 0) makes the model more deterministic and less likely to fabricate. Higher temperature increases hallucination risk, and longer context window alone does not guarantee grounding.

Answer

Use a longer context window to include more document text

Answer

Set a high temperature (e.g., 1.5) for sampling