1Z0-1127-25 LLM Fundamentals Practice Test 1 — 15 Questions

Question 1

Which of the following best describes the difference between an encoder-only model (e.g., BERT) and a decoder-only model (e.g., GPT)?

Accepted Answer

Encoder-only uses bidirectional attention and is suited for classification or NER; decoder-only uses causal attention and is suited for text generation. Encoder-only models like BERT employ bidirectional attention, allowing each token to attend to all other tokens in both directions, which is ideal for tasks requiring full context understanding such as classification or named entity recognition (NER). In contrast, decoder-only models like GPT use causal (masked) attention, where each token can only attend to previous tokens, making them suitable for autoregressive text generation.

Answer

Encoder-only is trained for text generation; decoder-only is trained for classification

Answer

Both use the same attention pattern but differ in number of layers

Answer

Encoder-only uses causal attention; decoder-only uses bidirectional attention

Question 2

A practitioner wants to evaluate an LLM-generated summary against a human-written reference using a metric that focuses on recall of key information. Which metric is most appropriate?

Accepted Answer

ROUGE. ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is the most appropriate metric because it specifically measures recall of key information by comparing n-gram overlap between the generated summary and a reference summary. This aligns directly with the practitioner's goal of evaluating how well the LLM-generated summary captures the essential content from the human-written reference.

Answer

BLEU

Answer

Perplexity

Answer

Cosine similarity

Question 3

What is the main advantage of using Byte-Pair Encoding (BPE) over word-level tokenization?

Accepted Answer

It can represent any word as a sequence of subword tokens, including rare or unseen words. Byte-Pair Encoding (BPE) is a subword tokenization algorithm that iteratively merges the most frequent pairs of bytes or characters into new tokens. Its main advantage over word-level tokenization is that it can represent any word, including rare or unseen words, as a sequence of subword tokens, thereby eliminating the out-of-vocabulary (OOV) problem. This allows models like GPT and BERT to handle arbitrary input without requiring a fixed word vocabulary.

Answer

It produces fixed-length token sequences

Answer

It reduces the number of tokens by merging all letters into single tokens

Answer

It eliminates the need for a vocabulary altogether

Question 4

An OCI user notices that their Llama 3 model generates the same output sequence regardless of the input prompt when using default generation parameters. Which setting is most likely causing this lack of diversity?

Accepted Answer

Greedy decoding (temperature=0). Greedy decoding, which is equivalent to setting temperature=0, always selects the token with the highest probability at each step. This deterministic behavior causes the model to produce the exact same output sequence for any input prompt, as there is no randomness or variation in token selection. The lack of diversity is a direct consequence of eliminating all stochasticity from the generation process.

Answer

Top-k sampling with k=50

Answer

Temperature sampling with temperature=0.8

Answer

Top-p (nucleus) sampling with p=0.9

Question 5

A developer is building a code generation assistant and wants to minimize the number of API calls to the OCI Generative AI service. Which tokenization approach results in the lowest token count for a given code snippet?

Accepted Answer

BPE tokenizer trained on code corpora. BPE (Byte Pair Encoding) tokenizers trained specifically on code corpora learn subword units that align closely with programming language syntax (e.g., common keywords, operators, and variable patterns), resulting in fewer tokens for a given code snippet compared to general-purpose tokenizers. This reduces API calls by encoding more semantic meaning per token, directly minimizing token count.

Answer

WordPiece tokenizer

Answer

SentencePiece tokenizer with unigram LM

Answer

Character-level tokenization

Question 6

Which of the following is a known limitation of large language models that Retrieval-Augmented Generation (RAG) aims to address?

Accepted Answer

Hallucination of facts. RAG directly addresses the hallucination problem by grounding LLM outputs in retrieved, factual documents from an external knowledge base. Instead of relying solely on the model's parametric memory, RAG fetches relevant context at inference time, reducing the likelihood of generating plausible-sounding but incorrect facts.

Answer

High cost of training

Answer

Slow inference speed

Answer

Lack of multilingual support

Question 7

Which of the following best describes the difference between pre-training and fine-tuning?

Accepted Answer

Pre-training trains a model from scratch on a general corpus; fine-tuning adapts the model to a specific task. Pre-training involves training a large language model from scratch on a vast, general corpus of unlabeled data to learn language patterns, grammar, and world knowledge. Fine-tuning then takes this pre-trained model and further trains it on a smaller, labeled dataset specific to a downstream task, such as sentiment analysis or question answering, adapting the model's weights for that particular objective. Option C correctly captures this fundamental distinction in the LLM workflow.

Answer

Pre-training uses labeled data; fine-tuning uses unlabeled data

Answer

Pre-training requires a GPU; fine-tuning can be done on CPU

Answer

Pre-training is only for encoder models; fine-tuning is only for decoder models

Question 8

A data scientist needs to evaluate the quality of a text summarization model. Which TWO metrics are appropriate for this task?

Accepted Answer

BERTScore. BERTScore is correct because it leverages contextual embeddings from BERT to compute semantic similarity between generated and reference summaries, capturing meaning beyond exact n-gram overlap. This makes it highly effective for evaluating text summarization models where paraphrasing and semantic equivalence are critical.

Answer

Perplexity

Answer

BLEU

Answer

F1 score

Question 9

In the context of LLMs, what is the primary function of tokenization?

Accepted Answer

To split text into manageable pieces (tokens) that the model can understand. Tokenization is the first step in processing text for LLMs, where raw input is split into smaller units called tokens (words, subwords, or characters). This is essential because models like GPT or BERT operate on discrete tokens, not raw strings, and tokenization defines the model's vocabulary and input structure.

Answer

To assign positional encodings to each word

Answer

To convert tokens into dense vector representations

Answer

To remove stop words and punctuation from the input

Question 10

Which of the following best describes the difference between an embedding model and a generation model?

Accepted Answer

Embedding models output a single vector per input; generation models output a sequence of tokens. Embedding models transform input data (text, images, etc.) into a fixed-size vector representation (a single vector per input) that captures semantic meaning, while generation models (like GPT or LLaMA) produce variable-length sequences of tokens as output, predicting one token at a time. This fundamental architectural difference defines their distinct roles: embeddings for similarity search and retrieval, generation for producing coherent text.

Answer

Embedding models cannot be used with RAG; generation models can

Answer

Embedding models are used only for classification; generation models are used only for translation

Answer

Embedding models require fine-tuning; generation models do not

Question 11

Which tokenization algorithm is used by models like BERT and GPT-2?

Accepted Answer

Byte-Pair Encoding (BPE). The question asks which tokenization algorithm is used by models like BERT and GPT-2. While BERT uses WordPiece, GPT-2 uses Byte-Pair Encoding (BPE). Since GPT-2 is one of the models cited, BPE is the correct answer for that model. WordPiece is a distractor that applies only to BERT.

Answer

SentencePiece

Answer

WordPiece

Answer

Unigram Language Model

Question 12

A company needs to evaluate a text summarization model. They have reference summaries and want a metric that measures overlap of n-grams. Which metric is MOST appropriate?

Accepted Answer

ROUGE. ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is the most appropriate metric because it measures the overlap of n-grams between the generated summary and reference summaries, directly aligning with the company's requirement. Unlike BLEU, which focuses on precision, ROUGE emphasizes recall, making it better suited for summarization tasks where capturing all key content from the reference is critical.

Answer

BLEU

Answer

Perplexity

Answer

BERTScore

Question 13

An OCI practitioner observes that an LLM consistently generates incorrect answers for questions about recent events (last 6 months). The model was fine-tuned on company data but not retrained recently. What is the MOST likely root cause?

Accepted Answer

The model's knowledge cutoff date is before the recent events. The model's knowledge cutoff date determines the temporal boundary of its training data. Since the LLM was fine-tuned on company data but not retrained, it lacks exposure to events occurring after its cutoff, making it unable to generate accurate answers about recent events. This is the most direct and common cause for such temporal knowledge gaps in LLMs.

Answer

The context window is too short

Answer

The sampling strategy is too greedy

Answer

The fine-tuning dataset was too small

Question 14

Which component of the Transformer architecture allows the model to weigh the importance of different words in a sequence when processing a given word?

Accepted Answer

Self-attention mechanism. The self-attention mechanism is the core component of the Transformer architecture that computes attention scores between every pair of words in the input sequence. These scores determine how much each word should influence the representation of the current word, allowing the model to dynamically weigh the importance of different words regardless of their positional distance. This mechanism is what enables the Transformer to capture long-range dependencies and contextual relationships in parallel.

Answer

Feed-forward neural network

Answer

Positional encoding

Answer

Layer normalization

Question 15

An ML engineer notices that when using temperature sampling with temperature=0.8 for code generation, the model sometimes produces syntactically incorrect code. The engineer needs to ensure syntactically valid outputs while maintaining some creativity. Which combination of sampling parameters is MOST appropriate?

Accepted Answer

Reduce temperature to 0.2 and use top-p=0.9. Reducing temperature to 0.2 makes the output distribution more peaked, favoring high-probability tokens, which reduces syntax errors. Combining this with top-p=0.9 (nucleus sampling) limits the sampling pool to the smallest set of tokens whose cumulative probability reaches 0.9, further filtering out low-probability tokens that often cause invalid syntax. This balance preserves some creativity while ensuring syntactically valid code.

Answer

Increase temperature to 1.2 and use top-k=50

Answer

Use beam search with width=5

Answer

Set temperature=1.0 and use greedy decoding