1Z0-1127-25 LLM Fundamentals Practice Test 5 — 15 Questions

Question 1

A data scientist is using OCI Generative AI to process a large batch of legal documents. The total cost is higher than expected. Which factor is most likely the primary driver of cost?

Accepted Answer

The total number of tokens processed (input + output). In OCI Generative AI, pricing is primarily based on the total number of tokens processed, which includes both input (prompt) and output (generated) tokens. Processing large batches of legal documents generates high token counts due to lengthy text inputs and verbose outputs, directly increasing cost. The number of API requests alone does not determine cost—a single request with many tokens costs more than many requests with few tokens.

Answer

The number of API requests made

Answer

The choice of sampling strategy (e.g., top-k vs greedy)

Answer

The latency of the inference endpoint

Question 2

A researcher is evaluating two LLMs for a summarization task. Model A achieves a ROUGE-L score of 0.45 and a BERTScore of 0.92. Model B achieves a ROUGE-L score of 0.50 and a BERTScore of 0.88. Which model is likely better for producing summaries that are semantically faithful to the source, even if not using the exact same words?

Accepted Answer

Model A because BERTScore is higher. BERTScore measures semantic similarity using contextual embeddings, while ROUGE-L measures n-gram overlap. Higher BERTScore indicates better semantic faithfulness even without exact phrase matches.

Answer

Neither model is acceptable because ROUGE-L is below 0.6

Answer

Both are equally good because the scores are close

Answer

Model B because ROUGE-L is higher

Question 3

A developer is implementing a text generation pipeline using OCI Generative AI and needs to produce diverse, creative outputs for a marketing campaign. Which sampling strategy should they choose?

Accepted Answer

Top-p sampling with p=0.9. Top-p (nucleus) sampling selects from a dynamically chosen set of tokens whose cumulative probability exceeds p, allowing diversity while avoiding the long tail of improbable tokens.

Answer

Beam search

Answer

Temperature sampling with temperature=0.0

Answer

Greedy decoding

Question 4

Which of the following is a recognized limitation of large language models?

Accepted Answer

They can generate factually incorrect information (hallucinations). Large language models (LLMs) are known to generate factually incorrect information, often called 'hallucinations.' This occurs because LLMs are probabilistic models that predict the next token based on training data patterns, not verified facts. They lack a grounding mechanism to validate outputs against real-world truth, making hallucinations a fundamental limitation.

Answer

They always require internet access

Answer

They cannot perform translation between languages

Answer

They can only process numerical data

Question 5

A team is building a code generation assistant using OCI Generative AI. They notice that the model occasionally produces code with subtle security vulnerabilities. Which approach would most effectively reduce this risk without compromising the assistant's usefulness?

Accepted Answer

Fine-tune the model on a dataset of secure code examples and security best practices. Fine-tuning on a curated dataset of secure code examples can teach the model to avoid common vulnerability patterns while retaining its general coding ability. RAG with security docs could also help, but fine-tuning directly addresses the model's behavior more comprehensively.

Answer

Use a larger context window to include all project files in every prompt

Answer

Switch to a model with more parameters

Answer

Use greedy decoding to reduce randomness in code generation

Question 6

A company wants to build a sentiment analysis system for customer reviews. They have a labeled dataset of 10,000 reviews. Which approach is most cost-effective and likely to yield good performance?

Accepted Answer

Fine-tune a pre-trained BERT model on the labeled dataset. Fine-tuning a pre-trained encoder-only model like BERT on the labeled dataset is a standard approach for classification tasks, offering good performance with relatively modest data and compute.

Answer

Use GPT-4 with a prompt and no fine-tuning

Answer

Use a simple bag-of-words model with logistic regression

Answer

Train a Transformer model from scratch on the reviews

Question 7

An OCI user observes that their embedding model returns vectors that are not normalized, and they want to compute cosine similarity between two text embeddings. What should they do?

Accepted Answer

Normalize the vectors to unit length, then compute the dot product. Cosine similarity measures the cosine of the angle between two vectors, which is equivalent to the dot product of the vectors after they have been normalized to unit length (L2 norm = 1). Option C correctly describes this process: first normalize each embedding vector to unit length, then compute the dot product. This is the standard approach because raw embedding vectors from models like OCI's AI services may not be unit vectors, and the dot product alone does not account for magnitude differences.

Answer

Compute the Euclidean distance between the vectors

Answer

Compute the L1 norm of the difference

Answer

Compute the dot product directly

Question 8

A developer is using OCI Generative AI for a question-answering system. The model frequently provides outdated information because the training data cutoff is over a year old. Which approach would most effectively address this issue?

Accepted Answer

Implement a Retrieval-Augmented Generation (RAG) pipeline that retrieves up-to-date documents from an external knowledge base. Retrieval-Augmented Generation (RAG) directly addresses the problem of stale training data by dynamically retrieving current documents from an external knowledge base at inference time. This allows the model to generate answers grounded in up-to-date information without requiring retraining or a larger model, making it the most effective and practical solution for a question-answering system.

Answer

Increase the context window to include more of the user's prompt

Answer

Fine-tune the model on a dataset that includes recent information up to today

Answer

Switch to a larger model that has a more recent knowledge cutoff

Question 9

A machine learning engineer is designing a RAG pipeline in OCI to improve the accuracy of an LLM-based FAQ bot. Which TWO components are essential for the retrieval phase? (Select TWO.)

Accepted Answer

Document chunking. Document chunking is essential because it breaks large documents into smaller, manageable pieces that can be individually indexed and retrieved. Without chunking, the retrieval phase would either miss relevant context or return overly large documents that exceed the LLM's context window, reducing accuracy.

Answer

Tokenization before the generation step

Answer

Text generation model

Answer

A reranker model

Question 10

A data scientist is evaluating an LLM's performance on a summarization task. They observe that the model produces fluent summaries but often misses key information. Which TWO metrics would best capture this issue? (Select TWO.)

Accepted Answer

Human evaluation with a rubric for completeness. ROUGE-L measures recall of the longest common subsequence, capturing information coverage. Human evaluation can assess completeness. BLEU emphasizes precision and fluency. BERTScore measures semantic similarity but not directly the presence of key points. Perplexity measures model confidence, not recall.

Answer

BLEU score

Answer

Perplexity

Answer

BERTScore

Question 11

An organization wants to use OCI Generative AI for a multilingual translation task. They need high quality and must avoid biases present in the training data. Which THREE strategies should they consider? (Select THREE.)

Accepted Answer

Implement a human-in-the-loop review process to catch biased translations. Fine-tuning on high-quality parallel corpora improves accuracy. Using models designed for translation (e.g., encoder-decoder) often yields better results. Implementing human-in-the-loop review catches biases. Increasing temperature may reduce bias but also reduces quality; it is not a primary strategy for bias mitigation. RAG is not directly applicable to translation as it requires retrieved documents in the target language.

Answer

Use a RAG pipeline to retrieve canonical translations from a trusted database

Answer

Increase the temperature parameter to 1.5 to reduce repetitive biases

Question 12

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

Answer

Train a custom model from scratch on the policy documents each month

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Answer

Fine-tune a base LLM on the policy documents monthly

Question 13

What is the primary purpose of the self-attention mechanism in a transformer model?

Accepted Answer

To process tokens in parallel while modeling long-range dependencies. The self-attention mechanism allows each token in the input sequence to attend to every other token, computing a weighted sum of their representations. This enables the model to capture long-range dependencies directly without the sequential processing constraints of RNNs, and because the attention scores for all tokens can be computed simultaneously, the mechanism supports parallel processing of the entire sequence.

Answer

To reduce the number of parameters in the model

Answer

To convert tokens into fixed-length vectors

Answer

To ensure the model is autoregressive

Question 14

A data scientist is fine-tuning a Llama 2 7B model on a custom dataset using OCI Data Science. After training, the model generates fluent but factually incorrect statements about the new domain. Which post-training technique would BEST address this issue without retraining?

Accepted Answer

Use a retrieval-augmented generation (RAG) pipeline. RAG retrieves factual information from an external knowledge base to ground the generation, reducing hallucinations. The other options do not address factual accuracy.

Answer

Decrease the temperature to 0.1

Answer

Switch to a larger model like Llama 2 70B

Answer

Apply top-p sampling with p=0.9

Question 15

A developer is implementing a text generation pipeline and wants to produce diverse, creative outputs. They set temperature=1.2, top_k=50, and top_p=1.0. What is the MOST likely effect of this combination?

Accepted Answer

The output will be diverse and creative, but may occasionally be incoherent or off-topic. Temperature >1 flattens the probability distribution, making low-probability tokens more likely. top_k=50 restricts to top 50 tokens, but top_p=1.0 imposes no cumulative probability cutoff. The combination yields diverse but potentially incoherent outputs.

Answer

The output will be identical to greedy decoding because top_p=1.0 disables sampling

Answer

The output will be mostly factual because top_k filters out unlikely tokens

Answer

The output will be highly deterministic and repetitive