1Z0-1127-25 LLM Fundamentals Practice Test 3 — 15 Questions

Question 1

An AI engineer is deploying a RAG pipeline using OCI Generative AI. They notice the generated answers sometimes include information not present in the retrieved documents. What is the MOST likely cause?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

The generation model is hallucinating because it does not rely solely on the retrieved context. The generation model in a RAG pipeline is designed to leverage retrieved context but can still produce outputs not grounded in that context, a phenomenon known as hallucination. This occurs when the model relies on its parametric knowledge or statistical patterns rather than strictly adhering to the retrieved documents, especially if the prompt does not enforce strict grounding or the model's training biases override the context.

Answer

The chunking strategy splits sentences across chunks

Answer

The embedding model is too small to represent document semantics

Answer

The context window of the generation model is exceeded

Question 2

A data scientist is evaluating BERTScore to compare model-generated summaries with reference summaries. Which TWO statements about BERTScore are correct?

Accepted Answer

BERTScore can capture paraphrases better than BLEU because of its semantic nature. BERTScore leverages pre-trained contextual embeddings from BERT to compute token-level similarity between candidate and reference texts, allowing it to capture semantic equivalence even when surface forms differ. This makes it more robust to paraphrases than BLEU, which relies on exact n-gram matching and cannot recognize synonymous or rephrased content.

Answer

BERTScore uses n-gram overlap to measure recall

Answer

BERTScore is fully deterministic and always yields the same result as ROUGE

Answer

BERTScore requires the generation model's training data to compute scores

Question 3

A team is designing a text classification system using OCI Generative AI. They have a small labeled dataset of 200 examples per class. Which THREE techniques can help improve model performance without requiring additional labeled data?

Accepted Answer

Generate synthetic labeled examples using an LLM and include them in training. Generating synthetic labeled examples using an LLM is a data augmentation technique that expands the small dataset without requiring new human annotations. This approach leverages the LLM's ability to produce diverse, class-consistent text, which helps the model generalize better and reduces overfitting when training on only 200 examples per class.

Answer

Increase the context window of the model to include all training examples

Answer

Fine-tune a large decoder-only model on the 200 examples

Question 4

An organization is concerned about bias in their LLM-powered hiring assistant. Which TWO actions are MOST effective in mitigating bias?

Accepted Answer

Implement human-in-the-loop evaluation with fairness-focused rubrics. Human-in-the-loop evaluation with fairness-focused rubrics directly addresses bias by incorporating human judgment to detect and correct biased outputs. This approach allows reviewers to systematically assess responses against predefined fairness criteria, catching subtle biases that automated methods might miss. It is a standard practice in responsible AI deployment for high-stakes applications like hiring.

Answer

Use a larger context window to include more examples

Answer

Increase the temperature parameter to introduce more randomness

Answer

Use only encoder-only models like BERT for classification

Question 5

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. Retrieval-Augmented Generation (RAG) is the most appropriate approach because it allows the chatbot to answer questions from the policy documents without retraining the model. By indexing the documents in a vector store and retrieving relevant chunks at query time, RAG handles monthly updates by simply re-indexing the new documents, avoiding the cost and complexity of fine-tuning or retraining.

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Answer

Fine-tune a base LLM on the policy documents monthly

Answer

Train a custom model from scratch on the policy documents each month

Question 6

Which TWO of the following are characteristics of the transformer decoder-only architecture (e.g., GPT)?

Accepted Answer

It uses masked self-attention to prevent attending to future tokens. The transformer decoder-only architecture, such as GPT, employs masked self-attention (also known as causal attention) to ensure that each token can only attend to previous tokens in the sequence. This masking prevents the model from 'seeing' future tokens during training and generation, which is essential for maintaining the autoregressive property where predictions depend only on past context.

Answer

It uses bidirectional self-attention to capture context from both directions

Answer

It processes all tokens in parallel during generation

Answer

It implements cross-attention between encoder and decoder layers

Question 7

What is the primary purpose of the self-attention mechanism in a Transformer model?

Accepted Answer

To compute a weighted sum of all token representations based on pairwise relevance. Self-attention allows each token to attend to every other token in the sequence, capturing contextual relationships regardless of distance.

Answer

To generate token embeddings in parallel

Answer

To reduce the dimensionality of token embeddings

Answer

To encode positional information of tokens

Question 8

A company needs to generate embeddings for a large corpus of legal documents to enable semantic search. Which type of model should they use?

Accepted Answer

An encoder-only embedding model like Cohere Embed. An encoder-only embedding model like Cohere Embed is designed to convert text into dense vector representations (embeddings) that capture semantic meaning, which is exactly what is needed for semantic search over a large corpus of legal documents. These models use a bidirectional transformer architecture to encode context from both directions, producing fixed-size embeddings that can be efficiently compared using cosine similarity or other distance metrics.

Answer

A decoder-only generation model like GPT

Answer

A text-to-speech model

Answer

A machine translation model

Question 9

Which of the following sampling strategies selects tokens based on a cumulative probability threshold from the highest probability tokens?

Accepted Answer

Top-p (nucleus) sampling. Top-p (nucleus) sampling cuts off the tail of the probability distribution where cumulative probability exceeds p, allowing dynamic vocabulary size.

Answer

Top-k sampling

Answer

Greedy decoding

Answer

Temperature sampling

Question 10

An OCI Generative AI practitioner observes that a Cohere Command model generates responses with outdated information about a recent event. The model was fine-tuned six months ago. Which technique should be applied to incorporate new knowledge without retraining the model?

Accepted Answer

Implement a RAG pipeline that indexes the latest documents into a vector store and retrieves relevant passages at query time. RAG retrieves relevant, current documents at inference time, providing up-to-date context without modifying the model parameters.

Answer

Use a longer context window and include all new articles in the prompt

Answer

Fine-tune the model again with the new data

Answer

Increase the temperature parameter to encourage more creative outputs

Question 11

When using an LLM for code generation, a developer notices the model occasionally produces syntactically incorrect code. Which approach is most likely to reduce syntax errors while still allowing diverse output?

Accepted Answer

Set temperature to 0 and use greedy decoding. Lowering temperature reduces randomness, making outputs more deterministic and less prone to errors, while still allowing some variation.

Answer

Use top-k sampling with k=100

Answer

Increase the context window size

Answer

Increase the temperature to 1.5

Question 12

In a Transformer model, what is the role of positional encoding?

Accepted Answer

To provide information about the position of each token in the sequence. Positional encoding is essential in Transformer models because the self-attention mechanism processes all tokens in parallel and has no inherent notion of sequence order. By adding positional encodings (often sinusoidal or learned) to the input embeddings, the model can distinguish between tokens at different positions, enabling it to capture word order and relative positions. Without this, the model would treat the sequence as a bag of tokens, losing all sequential context.

Answer

To reduce the number of parameters in the model

Answer

To enable the model to process tokens in parallel

Answer

To encode the semantic meaning of each token

Question 13

An LLM is being used to answer customer queries about a product catalog. The answers are fluent but sometimes include plausible-sounding but incorrect product details. What is this phenomenon called, and which technique is most effective to mitigate it?

Accepted Answer

Hallucination; use Retrieval-Augmented Generation (RAG) with the catalog indexed. Hallucination is the generation of false information; RAG grounds responses in retrieved factual documents, reducing hallucinations.

Answer

Knowledge cutoff; fine-tune the model on the catalog

Answer

Bias amplification; increase temperature

Answer

Overfitting; reduce the model size

Question 14

Which of the following metrics is most suitable for evaluating a translation model's output against multiple reference translations?

Accepted Answer

BLEU. BLEU (Bilingual Evaluation Understudy) is the most suitable metric for evaluating a translation model's output against multiple reference translations because it measures n-gram precision between the candidate translation and one or more reference translations. It directly quantifies how many words and phrases in the candidate match those in the references, making it the standard metric for machine translation tasks.

Answer

ROUGE

Answer

Perplexity

Answer

BERTScore

Question 15

An OCI user is comparing two embedding models: one with 768 dimensions and another with 1024 dimensions. Which of the following trade-offs is most relevant?

Accepted Answer

Higher dimensions may capture more fine-grained semantic information but require more storage and slower similarity search. Higher dimensions can capture more nuanced semantics but require more storage and may increase retrieval latency.

Answer

The 1024-dimensional model always yields better accuracy with no additional cost

Answer

The 768-dimensional model is always faster and more storage-efficient

Answer

Dimension size has no impact on performance or storage