1Z0-1127 LLM Fundamentals Practice Test 3 — 15 Questions

Question 1

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

Answer

Train a custom model from scratch on the policy documents each month

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Answer

Fine-tune a base LLM on the policy documents monthly

Question 2

Which of the following best describes the role of the self-attention mechanism in a Transformer model?

Accepted Answer

It computes a weighted sum of all input token representations, where weights depend on pairwise compatibility between tokens. The self-attention mechanism computes a weighted sum of all input token representations, where the weights are determined by the pairwise compatibility (attention scores) between tokens. This allows each token to dynamically attend to every other token in the sequence, capturing global dependencies without the limitations of fixed local windows or recurrence.

Answer

It encodes the order of tokens in the sequence

Answer

It applies a convolutional filter over local windows of tokens

Answer

It replaces the need for positional encoding by using recurrence

Question 3

A data scientist is using OCI Generative AI to process a large batch of legal documents. The total cost is higher than expected. Which factor is most likely the primary driver of cost?

Accepted Answer

The total number of tokens processed (input + output). In OCI Generative AI, pricing is primarily based on the total number of tokens processed, which includes both input (prompt) and output (generated) tokens. Processing large batches of legal documents generates high token counts due to lengthy text inputs and verbose outputs, directly increasing cost. The number of API requests alone does not determine cost—a single request with many tokens costs more than many requests with few tokens.

Answer

The number of API requests made

Answer

The choice of sampling strategy (e.g., top-k vs greedy)

Answer

The latency of the inference endpoint

Question 4

A researcher is evaluating two LLMs for a summarization task. Model A achieves a ROUGE-L score of 0.45 and a BERTScore of 0.92. Model B achieves a ROUGE-L score of 0.50 and a BERTScore of 0.88. Which model is likely better for producing summaries that are semantically faithful to the source, even if not using the exact same words?

Accepted Answer

Model A because BERTScore is higher. BERTScore measures semantic similarity using contextual embeddings, while ROUGE-L measures n-gram overlap. Higher BERTScore indicates better semantic faithfulness even without exact phrase matches.

Answer

Neither model is acceptable because ROUGE-L is below 0.6

Answer

Both are equally good because the scores are close

Answer

Model B because ROUGE-L is higher

Question 5

A developer is implementing a text generation pipeline using OCI Generative AI and needs to produce diverse, creative outputs for a marketing campaign. Which sampling strategy should they choose?

Accepted Answer

Top-p sampling with p=0.9. Top-p (nucleus) sampling selects from a dynamically chosen set of tokens whose cumulative probability exceeds p, allowing diversity while avoiding the long tail of improbable tokens.

Answer

Beam search

Answer

Temperature sampling with temperature=0.0

Answer

Greedy decoding

Question 6

Which of the following is a recognized limitation of large language models?

Accepted Answer

They can generate factually incorrect information (hallucinations). Option C is correct because large language models (LLMs) are known to generate factually incorrect information, often called 'hallucinations.' This occurs because LLMs are probabilistic models that predict the next token based on training data patterns, not verified facts. They lack a grounding mechanism to validate outputs against real-world truth, making hallucinations a fundamental limitation.

Answer

They always require internet access

Answer

They cannot perform translation between languages

Answer

They can only process numerical data

Question 7

A team is building a code generation assistant using OCI Generative AI. They notice that the model occasionally produces code with subtle security vulnerabilities. Which approach would most effectively reduce this risk without compromising the assistant's usefulness?

Accepted Answer

Fine-tune the model on a dataset of secure code examples and security best practices. Fine-tuning on a curated dataset of secure code examples can teach the model to avoid common vulnerability patterns while retaining its general coding ability. RAG with security docs could also help, but fine-tuning directly addresses the model's behavior more comprehensively.

Answer

Use a larger context window to include all project files in every prompt

Answer

Switch to a model with more parameters

Answer

Use greedy decoding to reduce randomness in code generation

Question 8

In the context of LLMs, what is the primary function of tokenization?

Accepted Answer

To split text into manageable pieces (tokens) that the model can understand. Tokenization is the first step in processing text for LLMs, where raw input is split into smaller units called tokens (words, subwords, or characters). This is essential because models like GPT or BERT operate on discrete tokens, not raw strings, and tokenization defines the model's vocabulary and input structure.

Answer

To assign positional encodings to each word

Answer

To convert tokens into dense vector representations

Answer

To remove stop words and punctuation from the input

Question 9

A company wants to build a sentiment analysis system for customer reviews. They have a labeled dataset of 10,000 reviews. Which approach is most cost-effective and likely to yield good performance?

Accepted Answer

Fine-tune a pre-trained BERT model on the labeled dataset. Fine-tuning a pre-trained encoder-only model like BERT on the labeled dataset is a standard approach for classification tasks, offering good performance with relatively modest data and compute.

Answer

Use GPT-4 with a prompt and no fine-tuning

Answer

Use a simple bag-of-words model with logistic regression

Answer

Train a Transformer model from scratch on the reviews

Question 10

An OCI user observes that their embedding model returns vectors that are not normalized, and they want to compute cosine similarity between two text embeddings. What should they do?

Accepted Answer

Normalize the vectors to unit length, then compute the dot product. Cosine similarity measures the cosine of the angle between two vectors, which is equivalent to the dot product of the vectors after they have been normalized to unit length (L2 norm = 1). Option C correctly describes this process: first normalize each embedding vector to unit length, then compute the dot product. This is the standard approach because raw embedding vectors from models like OCI's AI services may not be unit vectors, and the dot product alone does not account for magnitude differences.

Answer

Compute the Euclidean distance between the vectors

Answer

Compute the L1 norm of the difference

Answer

Compute the dot product directly

Question 11

A developer is using OCI Generative AI for a question-answering system. The model frequently provides outdated information because the training data cutoff is over a year old. Which approach would most effectively address this issue?

Accepted Answer

Implement a Retrieval-Augmented Generation (RAG) pipeline that retrieves up-to-date documents from an external knowledge base. Retrieval-Augmented Generation (RAG) directly addresses the problem of stale training data by dynamically retrieving current documents from an external knowledge base at inference time. This allows the model to generate answers grounded in up-to-date information without requiring retraining or a larger model, making it the most effective and practical solution for a question-answering system.

Answer

Increase the context window to include more of the user's prompt

Answer

Fine-tune the model on a dataset that includes recent information up to today

Answer

Switch to a larger model that has a more recent knowledge cutoff

Question 12

Which of the following best describes the difference between an embedding model and a generation model?

Accepted Answer

Embedding models output a single vector per input; generation models output a sequence of tokens. Option B is correct because embedding models transform input data (text, images, etc.) into a fixed-size vector representation (a single vector per input) that captures semantic meaning, while generation models (like GPT or LLaMA) produce variable-length sequences of tokens as output, predicting one token at a time. This fundamental architectural difference defines their distinct roles: embeddings for similarity search and retrieval, generation for producing coherent text.

Answer

Embedding models cannot be used with RAG; generation models can

Answer

Embedding models are used only for classification; generation models are used only for translation

Answer

Embedding models require fine-tuning; generation models do not

Question 13

A machine learning engineer is designing a RAG pipeline in OCI to improve the accuracy of an LLM-based FAQ bot. Which TWO components are essential for the retrieval phase? (Select TWO.)

Accepted Answer

Document chunking. Document chunking is essential because it breaks large documents into smaller, manageable pieces that can be individually indexed and retrieved. Without chunking, the retrieval phase would either miss relevant context or return overly large documents that exceed the LLM's context window, reducing accuracy.

Answer

Tokenization before the generation step

Answer

Text generation model

Answer

A reranker model

Question 14

A data scientist is evaluating an LLM's performance on a summarization task. They observe that the model produces fluent summaries but often misses key information. Which TWO metrics would best capture this issue? (Select TWO.)

Accepted Answer

Human evaluation with a rubric for completeness. ROUGE-L measures recall of the longest common subsequence, capturing information coverage. Human evaluation can assess completeness. BLEU emphasizes precision and fluency. BERTScore measures semantic similarity but not directly the presence of key points. Perplexity measures model confidence, not recall.

Answer

BLEU score

Answer

Perplexity

Answer

BERTScore

Question 15

An organization wants to use OCI Generative AI for a multilingual translation task. They need high quality and must avoid biases present in the training data. Which THREE strategies should they consider? (Select THREE.)

Accepted Answer

Implement a human-in-the-loop review process to catch biased translations. Fine-tuning on high-quality parallel corpora improves accuracy. Using models designed for translation (e.g., encoder-decoder) often yields better results. Implementing human-in-the-loop review catches biases. Increasing temperature may reduce bias but also reduces quality; it is not a primary strategy for bias mitigation. RAG is not directly applicable to translation as it requires retrieved documents in the target language.

Answer

Use a RAG pipeline to retrieve canonical translations from a trusted database

Answer

Increase the temperature parameter to 1.5 to reduce repetitive biases