1Z0-1127-25 LLM Fundamentals Practice Test 11 — 15 Questions

Question 1

A data scientist is preparing a 10,000-document corpus to fine-tune a base LLM hosted in OCI Generative AI. They want the model to reliably answer questions about these proprietary documents without the model memorizing the entire corpus verbatim. Which training approach should they use?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Supervised fine-tuning (SFT) on question-answer pairs derived from the corpus. Supervised fine-tuning on curated question-answer pairs derived from the corpus is the correct training approach because it directly teaches the model to produce the desired answers from domain content without memorizing documents verbatim. Continued pre-training on raw text does not teach question-answering behavior, RAG does not modify weights, and RLHF requires preference data the scenario does not provide.

Answer

Continued pre-training on the raw, unlabeled corpus without any instruction formatting

Answer

Reinforcement learning from human feedback (RLHF) using only the document corpus as reward signal

Answer

Retrieval-augmented generation (RAG) with an embedding model and a vector store

Question 2

A data scientist is preparing a corpus of 50,000 customer support tickets to pre-train a small domain-specific LLM on OCI. The tickets vary in length, and the scientist wants the model to learn statistical patterns of token sequences without any human-annotated labels. Which training objective is most appropriate for this corpus?

Accepted Answer

Causal language modeling, where the model predicts the next token given all previous tokens in the sequence.. The corpus is unlabeled and the goal is to learn token-sequence statistics for a generative model. Causal language modeling fits because it uses raw text and trains next-token prediction, which is the foundational objective for decoder-only LLMs. Alternatives require labels or target understanding rather than generation, and thus do not match the scenario.

Answer

Masked language modeling, where random tokens are replaced with a mask and the model predicts them from bidirectional context.

Answer

Supervised fine-tuning with human preference labels ranking pairs of responses.

Answer

Reinforcement learning from human feedback using a reward model trained on ticket resolution outcomes.

Question 3

A developer is evaluating an LLM for a summarization task and notices the model produces different outputs for the same prompt across separate API calls, even when temperature is set to 0. Which statement best explains this behavior?

Accepted Answer

Non-determinism can still occur due to floating-point arithmetic and parallel computation on GPUs. Temperature 0 makes sampling greedy, but it does not eliminate nondeterminism from floating-point arithmetic, GPU kernel scheduling, or batching effects. These low-level sources can shift logits enough to change the selected token. Beam search, top_p settings, and response caching do not explain the observed variation.

Answer

Temperature 0 guarantees deterministic output only when top_p is also set to 1

Answer

The OCI Generative AI service caches previous responses and returns them in random order

Answer

The model uses beam search by default, which introduces randomness across calls

Question 4

A developer is preparing prompts for an OCI Generative AI chat model. They notice that adding a short instruction such as 'Answer in one sentence' at the start of the prompt changes the model's output style, even though the model weights are not updated. Which concept explains this behavior?

Accepted Answer

In-context learning, because the model conditions on the instruction within the prompt to shape its response.. The model weights are unchanged, so the behavior must come from the prompt itself. In-context learning describes exactly this: the decoder conditions on instruction tokens and shifts its output distribution accordingly. Fine-tuning, RAG, and RLHF all involve external training or retrieval steps that are absent in this scenario.

Answer

Reinforcement learning from human feedback, because the model has been aligned to follow formatting instructions.

Answer

Fine-tuning, because the model adapts its parameters to the new instruction.

Answer

Retrieval-augmented generation, because the model retrieves relevant instructions from a vector store.

Question 5

An engineer is building a document summarization pipeline on OCI Generative AI. They need the model to produce summaries in a strict JSON schema with keys 'title' and 'summary' and no additional text. Which decoding or generation technique is most appropriate to enforce this structure?

Accepted Answer

Use constrained decoding with a grammar or JSON schema on the OCI Generative AI endpoint.. Constrained decoding is the only listed method that actively restricts token generation to sequences that satisfy a formal grammar or JSON schema. By integrating the schema into the decoding process, the model cannot emit tokens that would violate the required structure. Sampling adjustments and beam search only influence probability, not validity, so they cannot guarantee the strict JSON contract the pipeline requires.

Answer

Increase the temperature to 1.0 and add 'return only JSON' to the prompt.

Answer

Set top_p to 0.1 and rely on the model's pretraining to follow JSON patterns.

Answer

Use beam search with a large beam width to find the most likely valid JSON sequence.

Question 6

A solution architect is choosing an open model on OCI Generative AI to power a long-document analysis pipeline where each input can exceed 100,000 tokens. Which two factors most directly determine whether a given model can handle these inputs? (Choose two.)

Accepted Answer

The model's maximum context length (context window). The maximum context length sets the hard token limit per request, and the positional encoding scheme determines how well the model generalizes to long sequences. Parameter count, vocabulary size, and temperature do not directly govern the ability to process 100,000-token inputs, so they are not the primary factors.

Answer

The number of parameters in the model

Answer

The default temperature setting exposed in the OCI console

Answer

The tokenizer's vocabulary size

Question 7

A team is evaluating a decoder-only LLM for a summarization task and wants to understand the role of tokenization in model behavior. Which two statements about tokenization in LLMs are correct? (Choose two.)

Accepted Answer

Subword tokenization allows rare words to be represented as combinations of more frequent subword units.. Tokenization maps text to integer IDs consumable by embeddings and, through subword schemes, represents rare words compositionally. It occurs before the transformer and its granularity varies, affecting context usage and cost. Claims of one-token-per-word or tokenization inside attention contradict how the inference pipeline actually works.

Answer

Tokenization guarantees that every word in any language maps to exactly one token.

Answer

Because token counts are fixed per word, tokenization has no effect on context window usage.

Answer

Tokenization is performed by the attention mechanism inside each transformer layer at inference time.

Question 8

A team is evaluating an LLM for a retrieval-augmented generation (RAG) application on OCI. They observe that when the retrieved context is long, the model sometimes ignores information placed in the middle of the context window, even though it answers correctly when the same information appears at the beginning or end. Which phenomenon best explains this behavior?

Accepted Answer

The 'lost in the middle' effect due to positional bias in attention.. The described behavior matches the well-documented 'lost in the middle' effect, where LLMs exhibit a U-shaped recall curve across long contexts. Information at the start and end of the input receives stronger attention than content buried in the middle. For RAG on OCI, this means retrieved chunks should be ordered so the most relevant passages appear near the beginning or end of the prompt, improving answer fidelity without changing the model or retriever.

Answer

Catastrophic forgetting during pretraining.

Answer

Insufficient embedding dimensionality in the vector store.

Answer

Tokenization mismatch between the query and the retrieved documents.

Question 9

A developer configures an OCI Generative AI model with temperature set to 0.1 and top_p set to 0.95 for a factual Q&A assistant. Users report that answers are still occasionally inconsistent across identical prompts. Which explanation best accounts for this behavior?

Accepted Answer

The model is deterministic only when temperature is exactly 0, and even then hardware nondeterminism can produce minor variations.. Any temperature above zero enables stochastic sampling, so identical prompts can produce different outputs. Even at temperature zero, GPU floating-point and batching can introduce tiny nondeterminism. Top_p does not disable sampling, weights are not updated at inference, and context overflow would manifest differently, making the nondeterminism explanation the best fit.

Answer

The context window is being exceeded, causing the model to drop earlier tokens and change its answer.

Answer

Top_p of 0.95 disables sampling entirely and forces greedy decoding, which should be fully reproducible.

Answer

The model is retraining on each request, so its weights change between calls.

Question 10

A team is comparing two approaches for a customer support assistant. Approach 1 fine-tunes a base model on 5,000 historical support tickets. Approach 2 uses few-shot prompting with examples embedded in each request. The team wants the fastest path to a working prototype with minimal infrastructure. Which approach should they choose and why?

Accepted Answer

Approach 2, because few-shot prompting requires no training and can be iterated quickly. Few-shot prompting is the fastest path to a working prototype because it needs no training pipeline, labeled dataset for weight updates, or dedicated model hosting. Fine-tuning can improve accuracy later but adds latency, cost, and complexity that conflict with the stated goal of minimal infrastructure and quick iteration.

Answer

Approach 2, because few-shot prompting eliminates the need for any labeled data

Answer

Approach 1, because few-shot prompting cannot handle multi-turn conversations

Answer

Approach 1, because fine-tuning always produces higher accuracy than prompting

Question 11

An enterprise is deploying an LLM assistant that must answer questions grounded in a frequently updated internal knowledge base. The team wants to minimize hallucinations and avoid retraining the model whenever documents change. Which approach best meets these requirements?

Accepted Answer

Use retrieval-augmented generation to fetch relevant passages from a vector store and include them in the prompt.. Retrieval-augmented generation grounds responses in retrieved, up-to-date passages without changing model weights, so document updates only require re-indexing. Fine-tuning is costly and repeated, higher temperature worsens accuracy, and pasting the whole knowledge base is infeasible and dilutes relevance. RAG therefore best satisfies both grounding and freshness requirements.

Answer

Increase the model's temperature so it explores more possibilities and finds the correct answer.

Answer

Expand the context window and paste the entire knowledge base into every prompt.

Answer

Fine-tune the base model on the entire knowledge base each time documents are updated.

Question 12

A developer is choosing a foundation model on OCI Generative AI for a task that requires generating SQL queries from natural language. The model must be able to follow instructions and produce code-like output. Which model family characteristic is most relevant to this requirement?

Accepted Answer

The model is instruction-tuned and supports code generation.. For text-to-SQL, the model must interpret a natural language instruction and emit structured query syntax. Instruction tuning aligns the model to follow directives, and code-aware pretraining or fine-tuning provides familiarity with SQL grammar and patterns. Together these characteristics predict success better than parameter count, architecture, or context length alone. On OCI Generative AI, selecting a model documented for instruction following and code tasks is the practical approach.

Answer

The model has the largest possible parameter count.

Answer

The model uses a decoder-only architecture.

Answer

The model has a very large context window.

Question 13

An engineer notices that an LLM occasionally generates answers that are fluent and confident but contradict the provided source document. The team wants to reduce this behavior without retraining the model. Which technique is most appropriate?

Accepted Answer

Lower the temperature and instruct the model to answer only from the provided document. Lowering temperature reduces random sampling that can pull the model away from the source, and an explicit instruction to answer only from the provided document constrains generation to the given context. These prompt-level changes reduce contradictions without retraining, unlike raising temperature, changing model size, or removing the source.

Answer

Remove the source document from the prompt to avoid confusing the model

Answer

Add more parameters to the model by switching to a larger variant

Answer

Increase the temperature to encourage more diverse outputs

Question 14

A data scientist is preparing a dataset to fine-tune a foundation model on OCI Generative AI for a domain-specific Q&A task. They want to reduce the risk of the model hallucinating outside the provided domain knowledge. Which two data preparation practices are most appropriate? (Choose two.)

Accepted Answer

Add explicit examples where the correct response is 'I don't know' for out-of-scope questions.. Reducing hallucination in a fine-tuned model requires teaching it both the domain knowledge and the boundaries of that knowledge. Diverse, representative examples ensure the model learns the full scope of valid questions, while explicit 'I don't know' examples give it a learned abstention behavior for out-of-scope inputs. Together these practices improve reliability. Unlabeled text, easy-only filtering, and terminology removal do not provide the corrective signal needed.

Answer

Maximize dataset size by including as much unlabeled web text as possible.

Answer

Use only questions that the model already answers correctly to avoid confusing it.

Answer

Remove all examples that contain domain-specific terminology to simplify the text.

Question 15

A developer is configuring an OCI Generative AI text-generation call and wants the model to produce a deterministic, repeatable answer for the same prompt every time, with no creative variation. Which parameter should be set to zero?

Accepted Answer

Temperature. Temperature divides the logits before softmax; at zero the highest-scoring token receives essentially all the probability, producing greedy decoding. Identical prompts then yield identical completions, which is exactly the repeatable behavior requested. Other sampling controls only reshape which tokens are eligible and do not eliminate randomness on their own.

Answer

Max tokens

Answer

Frequency penalty

Answer

Top-p