1Z0-1127-25 LLM Fundamentals Practice Test 10 — 15 Questions

Question 1

A data scientist is comparing BLEU, ROUGE, and BERTScore to evaluate a summarization model. The client cares most about whether the summary captures all key facts from the source document. Which metric is most aligned with this requirement?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

ROUGE. ROUGE is recall-oriented, measuring how much of the reference content appears in the generated summary, which aligns with capturing key facts. BLEU is precision-oriented, and BERTScore measures semantic similarity but is not specifically recall-focused.

Answer

Perplexity

Answer

BERTScore

Answer

BLEU

Question 2

Which tokenization algorithm is commonly used in models like GPT and BERT and builds tokens by merging the most frequent pairs of characters or subwords iteratively?

Accepted Answer

Byte-Pair Encoding (BPE). Byte-Pair Encoding (BPE) is a subword tokenization method that starts with individual characters and merges the most frequent pairs iteratively until a vocabulary size is reached.

Answer

WordPiece

Answer

SentencePiece

Answer

Unigram tokenization

Question 3

An organization needs to deploy a model that can both understand and generate text, such as for a translation task where the input is in English and output is in French. Which model architecture is most suitable?

Accepted Answer

Encoder-decoder (e.g., T5). Encoder-decoder architectures like T5 are designed for sequence-to-sequence tasks. The encoder processes the input sequence, and the decoder generates the output sequence, making it ideal for translation.

Answer

Decoder-only (e.g., GPT)

Answer

Encoder-only (e.g., BERT)

Answer

Mixture of Experts (MoE)

Question 4

A team is deploying a chatbot that must never output harmful or biased statements. They plan to use a pre-trained LLM with in-context learning. Which additional measure is MOST effective at reducing harmful outputs without retraining?

Accepted Answer

Include few-shot examples in the system prompt that demonstrate appropriate responses. Few-shot examples embedded in the system prompt provide the model with concrete demonstrations of desired safe behavior at inference time, which is the essence of in-context learning. This steers the model's output distribution toward the demonstrated patterns without any weight updates, making it the most effective no-retraining mitigation. It also allows rapid iteration — safety examples can be swapped or expanded without redeploying the model.

Answer

Apply a larger context window to include more safety instructions

Answer

Fine-tune the model on a curated dataset of safe dialogues

Answer

Use beam search with a high beam width

Question 5

A developer wants to compare two sentences for semantic similarity using embeddings. Which distance or similarity metric is most commonly used for dense vector representations?

Accepted Answer

Cosine similarity. Cosine similarity measures the cosine of the angle between two vectors, focusing on orientation rather than magnitude. For dense embeddings from models like Word2Vec, BERT, or sentence transformers, cosine similarity is the standard metric because it captures semantic similarity regardless of vector length.

Answer

Jaccard similarity

Answer

Manhattan distance

Answer

Euclidean distance

Question 6

Which of the following is a known limitation of large language models where the model generates plausible-sounding but factually incorrect information?

Accepted Answer

Hallucination. Hallucination refers to the phenomenon where a large language model generates output that is fluent and plausible but factually incorrect or nonsensical. This is a known limitation because models are trained to predict the next word based on patterns, not to verify facts.

Answer

Bias

Answer

Context window limitation

Answer

Knowledge cutoff

Question 7

A company's AI system uses RAG to answer customer questions. Users often get incomplete answers because the retrieved chunks do not contain all relevant information. Which step in the RAG pipeline is most likely the issue?

Accepted Answer

Chunking strategy (chunk size and overlap). Chunking determines how documents are split into pieces. If chunks are too small, key information may be split across chunks, causing incomplete retrieval. Adjusting chunk size and overlap can improve completeness.

Answer

Retrieval top-k setting

Answer

Generation model temperature

Answer

Embedding model selection

Question 8

An ML engineer is selecting a pre-trained model for a code generation task. The model must be able to generate syntactically correct code in multiple programming languages. Which model family is BEST suited for this task?

Accepted Answer

Meta Llama (Code Llama variant). Models like Code Llama (a variant of Llama) are specifically fine-tuned on code and are known for strong code generation capabilities. While other models can generate code, Code Llama is the best fit among the options.

Answer

BERT

Answer

Cohere Command

Answer

Mistral

Question 9

A developer is debugging a RAG pipeline where the LLM frequently ignores retrieved documents and produces hallucinations. Which THREE factors could contribute to this problem?

Accepted Answer

The prompt does not explicitly instruct the model to base its answer on the provided context. Low chunk overlap can cause loss of context, improper prompt instructions may lead the model to ignore retrieved content, and a small top-K retrieval may miss relevant documents. A large embedding dimension improves retrieval precision, and a high temperature increases randomness but does not cause ignoring documents.

Answer

The generation temperature is set to 0

Answer

The embedding model's output dimension is too large

Question 10

Which TWO of the following are characteristics of the transformer decoder-only architecture (e.g., GPT)?

Accepted Answer

It uses masked self-attention to prevent attending to future tokens. Option A is correct because decoder-only architectures such as GPT apply a causal (masked) self-attention mechanism, where the attention mask sets future positions to -inf before the softmax so each token can only attend to itself and preceding tokens. Option B is correct because these models generate text autoregressively: at each step the model predicts the next token from the previously generated sequence, appends it, and repeats until an end-of-sequence token or length limit is reached. Option C is wrong because bidirectional self-attention is characteristic of encoder-only models like BERT, not decoder-only GPT models, which must remain causal to preserve the autoregressive factorization. Option D is wrong because although training can parallelize across positions via teacher forcing, generation itself is inherently sequential, producing one token per forward pass. Option E is wrong because cross-attention between encoder and decoder layers belongs to encoder-decoder architectures such as the original Transformer or T5, whereas decoder-only models have no separate encoder stack to attend to.

Answer

It uses bidirectional self-attention to capture context from both directions

Answer

It processes all tokens in parallel during generation

Answer

It implements cross-attention between encoder and decoder layers

Question 11

Which THREE of the following are known limitations of LLMs that practitioners must account for?

Accepted Answer

Knowledge cutoff: the model only knows information up to its training data date. Option A is correct because LLMs are trained on a fixed corpus with a knowledge cutoff date, so they cannot know events, facts, or updates that occurred after that point unless given external context (e.g., via RAG or tools). Option B is correct because LLMs learn statistical patterns from training data that may encode societal, historical, or representational biases, which the model can reproduce or amplify in its outputs. Option C is correct because LLMs generate tokens probabilistically to maximize plausibility rather than truth, so they can produce fluent, confident, but factually incorrect statements (hallucinations). Option D is not a known limitation, since LLMs are widely used for creative text generation such as stories, poems, and brainstorming. Option E is also incorrect, as LLMs demonstrably model syntax and grammar statistically, enabling coherent and grammatically structured language production.

Answer

Inability to produce creative text

Answer

Complete lack of understanding of language syntax

Question 12

Which TWO of the following are true about positional encoding in transformer models?

Accepted Answer

It provides the model with information about the order of tokens in a sequence. Option B is correct because positional encoding exists specifically to inject sequence-order information into a transformer, which otherwise processes all tokens in parallel and has no inherent notion of token position. Option C is correct because the standard implementation adds the positional encoding vector element-wise to the token embedding vector before the combined representation is fed into the transformer layers (e.g., in the original 'Attention Is All You Need' architecture). Option A is incorrect because the canonical sinusoidal positional encodings are fixed deterministic functions of position, not learned per-position parameters, and even learned variants are not 'independent for each position in the training data.' Option D is incorrect because positional encoding does not replace self-attention; self-attention remains the core mechanism for contextual mixing, and positional encoding only supplements it with order information. Option E is incorrect because positional encoding is used in encoder-only, decoder-only, and encoder-decoder transformer architectures alike, not just BERT-style encoders.

Answer

It is learned independently for each position in the training data

Answer

It replaces the need for self-attention

Answer

It is only used in encoder-only architectures like BERT

Question 13

A developer is configuring an OCI Generative AI text generation endpoint for a customer support assistant. They set the following parameters: temperature = 0.9, top_p = 0.95, top_k = 50, and max_tokens = 512. During testing, the assistant produces responses that are creative but frequently drift off-topic and include irrelevant tangents. The developer wants to make the output more focused and deterministic while keeping the same model and prompt. Which change to the inference parameters is most likely to achieve this?

Accepted Answer

Lower the temperature to a value near 0.2 and reduce top_p to around 0.7.. The drift stems from high sampling entropy caused by a high temperature and a wide nucleus. Reducing temperature makes the distribution peakier, and lowering top_p restricts sampling to the most probable tokens. Together these settings produce more focused, repeatable responses without changing the model or prompt, directly solving the developer's problem.

Answer

Set temperature to 1.0 and top_p to 1.0 to let the model use its full probability distribution as trained.

Answer

Increase max_tokens to 1024 to give the model more room to complete its thoughts.

Answer

Increase top_k to 100 so the model can choose from a larger vocabulary during sampling.

Question 14

A team is evaluating an OCI Generative AI model for a document summarization task where the input documents frequently exceed the model's context window. They plan to use a retrieval-augmented generation (RAG) pipeline with an OCI Generative AI embedding model and a vector store. Which two statements correctly describe how chunking and embedding choices affect the quality of the generated summaries? (Choose two.)

Accepted Answer

Using smaller chunks with overlap helps preserve context across chunk boundaries and can improve retrieval relevance for summarization.. Effective RAG summarization depends on coherent chunking with overlap and consistent embedding models for indexing and querying. Overlap preserves cross-boundary context, and using the same embedding model ensures similarity scores are meaningful. These two choices directly improve the relevance of retrieved context, which in turn improves summary quality.

Answer

Increasing the embedding dimension always improves retrieval accuracy regardless of the domain or corpus size.

Answer

Storing chunk embeddings in a vector store eliminates the need to pass retrieved text to the LLM for summarization.

Answer

Chunk size should always be set to the model's maximum context window to minimize the number of chunks stored.

Question 15

A solutions architect is explaining the difference between fine-tuning and prompt engineering on OCI Generative AI to a customer. The customer wants to adapt a base model to consistently follow a proprietary output format for internal reports without changing the model weights. Which approach should the architect recommend?

Accepted Answer

Use prompt engineering with few-shot examples that demonstrate the desired output format.. When the goal is to enforce an output format without modifying model weights, prompt engineering with few-shot examples is the appropriate technique. It leverages in-context learning to demonstrate the desired structure, is quick to iterate, and avoids the cost and complexity of fine-tuning or retraining.

Answer

Deploy a custom model endpoint with a higher temperature to encourage the model to adopt the format.

Answer

Retrain the model from scratch using the customer's report corpus to ensure the format is learned.

Answer

Fine-tune the base model with a dataset of formatted reports to bake the format into the weights.