1Z0-1127-25 LLM Fundamentals Practice Test 2 — 15 Questions

Question 1

What is the primary difference between pre-training and fine-tuning in the context of large language models?

Accepted Answer

Pre-training trains from scratch, fine-tuning updates all weights on a new dataset. Pre-training trains a model from scratch on a large corpus, updating all weights via self-supervised objectives like masked language modeling or next-token prediction. Fine-tuning then adapts the pre-trained model to a specific task by updating weights (often all, sometimes a subset) on a smaller labeled dataset. Option A correctly captures that pre-training starts from random initialization and fine-tuning updates weights on a new dataset.

Answer

Pre-training uses a smaller dataset, fine-tuning uses a larger dataset

Answer

Pre-training produces embeddings, fine-tuning produces text generation

Answer

Pre-training is unsupervised, fine-tuning is always supervised

Question 2

An ML engineer is choosing an LLM for a code generation assistant. The model must generate syntactically correct code, handle multiple programming languages, and be cost-efficient. Which THREE characteristics should the engineer prioritize?

Accepted Answer

Model with a large context window (e.g., 128K tokens). Option A is correct because a large context window (e.g., 128K tokens) lets the assistant ingest entire files, multiple related modules, or long documentation in one prompt, which improves the model's ability to generate syntactically consistent and context-aware code across a large codebase. Option C is correct because high pass@k on coding benchmarks such as HumanEval directly measures functional correctness of generated programs, which is the core requirement for a code generation assistant. Option D is correct because training on a diverse multilingual code corpus enables the model to handle multiple programming languages and varied syntax, idioms, and libraries, satisfying the multi-language requirement. Option B is not appropriate because text summarization optimization targets a different task and does not specifically improve code synthesis or syntactic correctness. Option E is not appropriate because maximizing parameter count regardless of performance ignores cost-efficiency and does not guarantee better coding ability; larger models typically increase inference cost and latency.

Answer

Model optimized for text summarization tasks

Answer

Model with the highest number of parameters regardless of performance

Question 3

A team is evaluating two LLMs for a summarization task. Model X has a BERTScore of 0.85, Model Y has a BERTScore of 0.82. However, human evaluators prefer Model Y. Which TWO reasons could explain this discrepancy?

Accepted Answer

BERTScore is based on BERT embeddings, which may not fully capture summary-specific qualities like conciseness or readability. BERTScore correlates with human judgment but is not perfect; it may favor certain styles. Additionally, BERTScore may be inflated if the reference summaries are similar to the model's training data.

Answer

BERTScore uses precision only, so it misses recall aspects

Answer

Human evaluators were not given clear criteria for evaluation

Answer

Model Y was fine-tuned on a different dataset, causing distribution shift

Question 4

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. Retrieval-Augmented Generation (RAG) with a vector store allows the chatbot to retrieve relevant policy documents at query time and generate answers without retraining the model. This approach is ideal for monthly document updates because the vector store can be updated independently, and the LLM does not need fine-tuning.

Answer

Train a custom model from scratch on the policy documents each month

Answer

Fine-tune a base LLM on the policy documents monthly

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Question 5

A data scientist is fine-tuning a Llama 2 model on a custom dataset for a summarization task. After fine-tuning, the model produces summaries that are too similar to the input text, often copying sentences verbatim. Which adjustment is MOST likely to reduce copying and improve abstractive summarization?

Accepted Answer

Increase the temperature to 0.8 and use top-p sampling. The model is copying input text verbatim because decoding is too deterministic — likely using greedy or low-temperature sampling. Increasing temperature to 0.8 and using top-p (nucleus) sampling introduces controlled randomness, encouraging the model to explore alternative phrasings and produce more abstractive summaries rather than extractive copies.

Answer

Switch to greedy decoding

Answer

Increase the beam search width

Answer

Reduce the context window size

Question 6

A machine learning engineer needs to select an embedding model to compute semantic similarity between customer reviews. Which property is MOST important for the embedding model to produce useful similarity scores?

Accepted Answer

The embedding model should produce high-dimensional vectors that can be compared using cosine similarity. Cosine similarity measures the angle between vectors, making it ideal for comparing semantic similarity regardless of vector magnitude. High-dimensional embeddings from specialized embedding models (e.g., those available in OCI GenAI) are specifically trained to produce isotropic vector spaces where cosine similarity correlates with semantic relatedness. This property is fundamental for tasks like clustering or retrieval-augmented generation (RAG) where relative direction, not absolute distance, matters.

Answer

The embedding model should be a generative model like GPT

Answer

The embedding model should be fine-tuned on a classification task

Answer

The embedding model should have a large context window to encode long reviews

Question 7

An organization wants to use an LLM for translation tasks but is concerned about data privacy and wants to keep all data within OCI. Which model family is natively available in OCI Generative AI service?

Accepted Answer

Oracle's own models (e.g., OCI Generative AI models). OCI Generative AI service natively provides Oracle's own LLM models (e.g., OCI Generative AI models) that are hosted entirely within OCI infrastructure, ensuring data privacy. While the service also offers native access to models from Cohere and Meta's Llama, using Oracle's own models guarantees full data residency within OCI without reliance on third-party API calls. Third-party models like Gemini, GPT-4, and Claude are not natively available in OCI Generative AI.

Answer

Gemini from Google

Answer

GPT-4 from OpenAI

Answer

Claude from Anthropic

Question 8

Which THREE of the following are known limitations of large language models? (Select THREE)

Accepted Answer

Knowledge cutoff (lack of information after a certain date). Option B is correct because LLMs are trained on a fixed corpus, so their parametric knowledge stops at a training cutoff date and they cannot natively know events after it. Option C is correct because LLMs learn statistical patterns from training data, and any societal, sampling, or labeling bias present in that data is reflected and can be amplified in their outputs. Option D is correct because LLMs generate probable token sequences rather than verified facts, so they can produce fluent but factually incorrect or fabricated content (hallucination). Option A is not a limitation but a capability some systems approximate via retrieval or tools; base LLMs lack real-time awareness, yet the option as phrased describes a feature, not a limitation. Option E is incorrect because context windows are finite (e.g., thousands to a few million tokens depending on the model), so 'unlimited context window' is false.

Answer

Real-time awareness of current events

Answer

Unlimited context window

Question 9

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

Answer

Train a custom model from scratch on the policy documents each month

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Answer

Fine-tune a base LLM on the policy documents monthly

Question 10

A data scientist wants to reduce the cost of token usage when summarizing large documents using an LLM on OCI. Which tokenization approach is MOST likely to lower token count for English text?

Accepted Answer

Use a model that tokenizes with Byte-Pair Encoding (BPE). Byte-Pair Encoding (BPE) is a subword tokenization algorithm that iteratively merges the most frequent byte pairs, creating a fixed-size vocabulary of common subwords. For English text, BPE efficiently represents common words as single tokens and splits rare words into meaningful subword units, resulting in fewer tokens overall compared to other methods. This directly reduces token usage and associated costs when processing large documents with an LLM on OCI.

Answer

Use a model that tokenizes with WordPiece

Answer

Use a model that tokenizes with SentencePiece

Answer

Use a model that tokenizes at the character level

Question 11

Which of the following best describes the difference between pre-training and fine-tuning?

Accepted Answer

Pre-training learns general language representations; fine-tuning adapts to a specific task. Pre-training is the initial phase where a model learns general language patterns from a large corpus. Fine-tuning adapts the pre-trained model to a specific task using a smaller labeled dataset.

Answer

Pre-training uses labeled data; fine-tuning uses unlabeled data

Answer

Fine-tuning requires more data than pre-training

Answer

Pre-training is done on a single task; fine-tuning is done on multiple tasks

Question 12

A data scientist is evaluating BERTScore to compare model-generated summaries with reference summaries. Which TWO statements about BERTScore are correct?

Accepted Answer

BERTScore can capture paraphrases better than BLEU because of its semantic nature. Option B is correct because BERTScore matches tokens in the candidate and reference via contextual embeddings and cosine similarity, so semantically equivalent paraphrases that share few n-grams still score highly, unlike BLEU's surface n-gram precision. Option E is correct because BERTScore's core mechanism is greedy token matching over pre-trained contextual embeddings (e.g., BERT, RoBERTa) to compute precision, recall, and F1 of semantic similarity. Option A is wrong because BERTScore does not use n-gram overlap; that describes BLEU/ROUGE-style metrics. Option C is wrong because BERTScore is not equivalent to ROUGE and, while deterministic given fixed embeddings and tokenization, it is a distinct embedding-based metric. Option D is wrong because BERTScore only needs a pre-trained embedding model and the candidate/reference texts, not the generation model's training data.

Answer

BERTScore uses n-gram overlap to measure recall

Answer

BERTScore is fully deterministic and always yields the same result as ROUGE

Answer

BERTScore requires the generation model's training data to compute scores

Question 13

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. Retrieval-Augmented Generation (RAG) with a vector store is most appropriate because it allows the chatbot to retrieve relevant passages from updated policy documents at query time without retraining the model. Monthly document updates only require re-indexing, not model retraining.

Answer

Train a custom model from scratch on the policy documents each month

Answer

Fine-tune a base LLM on the policy documents monthly

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Question 14

Which component of the Transformer architecture allows the model to weigh the importance of different tokens in the input sequence when generating each output token?

Accepted Answer

Self-attention mechanism. The self-attention mechanism allows the Transformer to compute attention scores between all tokens in the input sequence, enabling each token to weigh the importance of other tokens when producing its representation. This is the core innovation that captures long-range dependencies without recurrence.

Answer

Feed-forward neural network

Answer

Multi-head attention

Answer

Positional encoding

Question 15

A developer is using OCI Generative AI with a Cohere Command model for text generation. They want the output to be more creative and diverse, but still relevant. Which sampling strategy should they use?

Accepted Answer

Top-p (nucleus) sampling. Top-p (nucleus) sampling dynamically selects the smallest set of tokens whose cumulative probability exceeds a threshold p, balancing creativity and relevance by excluding unlikely tokens. This makes it ideal for generating diverse yet coherent text, unlike fixed strategies like top-k or greedy decoding.

Answer

Temperature sampling (temperature > 1)

Answer

Top-k sampling

Answer

Greedy decoding