1Z0-1127-25 LLM Fundamentals Practice Test 4 — 15 Questions

Question 1

A data scientist is building a RAG pipeline on OCI. Which TWO components are essential for the retrieval step?

Accepted Answer

Embedding model to convert chunks into vectors. Chunking splits documents into manageable pieces, and embedding converts them into vectors for similarity search.

Answer

Fine-tuned generation model

Answer

Beam search decoder

Answer

Human feedback loop

Question 2

A team wants to reduce hallucinations in their LLM-powered question-answering system. Which TWO techniques are most effective?

Accepted Answer

Implementing RAG to retrieve relevant documents. RAG provides factual grounding, and reducing temperature makes outputs more deterministic, reducing fabricated details.

Answer

Switching to a smaller model

Answer

Using a larger context window

Answer

Increasing the temperature to 1.5

Question 3

An OCI practitioner is comparing BERTScore with traditional n-gram metrics (ROUGE, BLEU) for evaluating summarization. Which THREE statements about BERTScore are true?

Accepted Answer

BERTScore uses pre-trained contextual embeddings from BERT. BERTScore uses contextual embeddings, captures semantic similarity, and correlates better with human judgment than n-gram metrics.

Answer

BERTScore does not require a reference text

Answer

BERTScore computes similarity based on exact n-gram overlap

Question 4

A data scientist wants to compare the semantic similarity between two sentences generated by an LLM. Which evaluation metric is most suitable for this purpose?

Accepted Answer

BERTScore. BERTScore computes cosine similarity between contextual embeddings, capturing semantic meaning better than surface-level n-gram metrics.

Answer

ROUGE-L

Answer

BLEU

Answer

Perplexity

Question 5

Which component of the Transformer architecture allows the model to focus on different parts of the input sequence when generating each output token?

Accepted Answer

Self-attention mechanism. Self-attention computes attention scores between all pairs of positions, enabling the model to weigh the importance of different input tokens.

Answer

Positional encoding

Answer

Feed-forward network

Answer

Layer normalization

Question 6

An organization wants to deploy a model that can summarize long financial reports (5000+ tokens) without losing context. Which model architecture is best suited for this requirement?

Accepted Answer

Encoder-decoder model (e.g., T5). Encoder-decoder models like T5 or BART are designed for sequence-to-sequence tasks such as summarization, and can handle long inputs with their encoder.

Answer

Mixture-of-experts model

Answer

Decoder-only model (e.g., GPT)

Answer

Encoder-only model (e.g., BERT)

Question 7

A practitioner is using a Cohere Command model on OCI for a translation task. They notice that the output is often incomplete and cuts off mid-sentence. Which parameter should they adjust to address this?

Accepted Answer

Max tokens. The 'Max tokens' parameter controls the maximum length of the generated output. When a model cuts off mid-sentence, it means the token limit has been reached before the model could complete its response. Increasing this value allows the model to generate more tokens, thus completing the translation.

Answer

Temperature

Answer

Frequency penalty

Answer

Top-p

Question 8

In the self-attention mechanism, what is the role of the 'scaling factor' (division by sqrt(d_k)) in the softmax computation?

Accepted Answer

To prevent the softmax from saturating and producing small gradients. Scaling prevents the dot products from growing too large in magnitude, which would push softmax into regions with extremely small gradients.

Answer

To make the attention mechanism translation invariant

Answer

To increase the variance of attention scores

Answer

To ensure the sum of attention weights equals 1

Question 9

A team wants to use an LLM to answer questions about a private codebase that is updated hourly. They cannot afford to fine-tune every hour. Which OCI feature or approach is most suitable?

Accepted Answer

Implement Retrieval-Augmented Generation (RAG) with a vector database. Retrieval-Augmented Generation (RAG) with a vector database is the most suitable approach because it allows the LLM to answer questions about a frequently updated private codebase without retraining. RAG retrieves relevant code snippets from a vector index at query time, ensuring the model always has access to the latest code without the cost and latency of hourly fine-tuning.

Answer

Use a long-context model with full codebase in prompt

Answer

Fine-tune a model on the codebase daily

Answer

Use a smaller model with faster inference

Question 10

An OCI user observes that their Mistral model produces very repetitive text when temperature is set to 0.9 and top-p to 1.0. Which adjustment is most likely to reduce repetition?

Accepted Answer

Enable frequency penalty. Enabling frequency penalty directly reduces the likelihood of the model repeating the same tokens by subtracting a penalty proportional to the token's existing frequency in the generated text. This is the most targeted way to combat repetitive output without altering the core sampling parameters (temperature and top-p) that control randomness.

Answer

Decrease top-p to 0.5

Answer

Increase temperature to 1.5

Answer

Set top-k to 1

Question 11

Which of the following sampling strategies is most likely to produce the most diverse and creative text?

Accepted Answer

Temperature sampling with temperature=1.2. Temperature sampling with a higher temperature (>1) increases the probability of less likely tokens, promoting creativity and diversity.

Answer

Beam search with width 5

Answer

Greedy decoding

Answer

Top-p sampling with p=0.1

Question 12

An OCI user wants to reduce the cost of running a generative AI model while maintaining output quality. Which THREE strategies can help achieve this?

Accepted Answer

Use a smaller model from the same family. Using a smaller model from the same family (e.g., switching from Llama 3 70B to Llama 3 8B) reduces the number of parameters and computational resources required per inference, directly lowering cost. Smaller models often retain strong performance on many tasks, especially when the task complexity does not demand the full capacity of the larger model, thus maintaining output quality while reducing token processing costs.

Answer

Use greedy decoding instead of sampling

Answer

Increase max tokens to ensure complete answers

Question 13

Which TWO components are essential in a Retrieval-Augmented Generation (RAG) pipeline?

Accepted Answer

Chunking the documents into smaller pieces. Chunking splits documents, embedding converts chunks to vectors, retrieval fetches relevant chunks, and generation produces the answer.

Answer

Fine-tuning the LLM on the documents

Answer

Knowledge distillation

Answer

Beam search decoding

Question 14

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

Answer

Train a custom model from scratch on the policy documents each month

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Answer

Fine-tune a base LLM on the policy documents monthly

Question 15

Which of the following best describes the role of the self-attention mechanism in a Transformer model?

Accepted Answer

It computes a weighted sum of all input token representations, where weights depend on pairwise compatibility between tokens. The self-attention mechanism computes a weighted sum of all input token representations, where the weights are determined by the pairwise compatibility (attention scores) between tokens. This allows each token to dynamically attend to every other token in the sequence, capturing global dependencies without the limitations of fixed local windows or recurrence.

Answer

It encodes the order of tokens in the sequence

Answer

It applies a convolutional filter over local windows of tokens

Answer

It replaces the need for positional encoding by using recurrence