1Z0-1127 Fundamentals of Large Language Models Practice Test 2 — 15 Questions

Question 1

A research team is experimenting with few-shot prompting to improve a model's performance on a complex reasoning task. They find that the model's performance degrades when the few-shot examples are too similar to each other. What is the likely cause and best remedy?

Accepted Answer

The examples lack diversity, causing the model to overfit to a narrow pattern. Use more diverse examples.. When few-shot examples are too similar, the model overfits to a narrow pattern, reducing its ability to generalize to the diverse reasoning paths required by the task. This is a known limitation of in-context learning: the model treats the examples as a template rather than as diverse demonstrations. Using more diverse examples exposes the model to a wider range of reasoning patterns, improving robustness.

Answer

The model has not seen enough examples. Increase the number of few-shot examples.

Answer

The examples are presented in a confusing order. Reorder them by difficulty.

Answer

The temperature is too low, making the model too deterministic. Increase temperature slightly.

Question 2

Which TWO of the following are common applications of large language models in enterprise settings?

Accepted Answer

Summarizing lengthy legal documents.. Option A is correct because large language models (LLMs) excel at abstractive summarization, which involves condensing lengthy legal documents into concise summaries while preserving key facts and legal reasoning. This is a common enterprise application for legal departments, as LLMs can process large volumes of text and generate coherent, context-aware summaries without requiring manual reading.

Answer

Performing real-time signal processing for audio streams.

Answer

Replacing relational databases for data storage.

Answer

Enhancing low-resolution images through super-resolution.

Question 3

Which THREE of the following are known limitations of large language models that practitioners must consider?

Accepted Answer

Hallucination of facts not present in the input.. Option A is correct because large language models (LLMs) are prone to hallucination, where they generate plausible-sounding but factually incorrect information that was not present in the input. This occurs because LLMs are next-token predictors without a built-in fact-checking mechanism, and they can invent details, citations, or events to maintain coherence. Practitioners must implement retrieval-augmented generation (RAG) or external verification to mitigate this risk.

Answer

Limited to processing only one language at a time.

Answer

Inability to process inputs longer than a few hundred tokens.

Question 4

Refer to the exhibit. A developer ran the OCI CLI command shown and received the JSON output. What does the output indicate about the model's confidence and why?

Accepted Answer

The model is highly confident the text is positive, as indicated by the 0.98 score.. Option D is correct because the JSON output shows a sentiment score of 0.98 for 'positive', which is very close to 1.0, indicating the model is highly confident that the text is positive. In sentiment analysis models, scores represent probabilities for each class, and a value near 1.0 for one class with much lower scores for others reflects strong confidence.

Answer

The model is uncertain because all scores are roughly equal.

Answer

The model is neutral because the neutral score is lowest.

Answer

The model is unsure because the scores are probabilities that sum to 1.

Question 5

Refer to the exhibit. A developer runs the OCI CLI command and receives the output. However, the text "Hello, how are you?" is actually a mix of English and French words. Why does the model assign only 0.03 to French?

Accepted Answer

The text is overwhelmingly English, so the model assigns a low probability to French.. Option A is correct because the model's output shows a probability distribution over languages, and the text is predominantly English with only a few French words. The model assigns a low probability (0.03) to French because the overwhelming majority of tokens are English, making the text far more likely to be classified as English. This reflects how language identification models evaluate the overall composition of the input.

Answer

The model is limited to identifying a single language per query.

Answer

The model cannot detect multiple languages in a single text.

Answer

The model's scores are normalized to sum to 1, so a high English score forces low others.

Question 6

You are a machine learning engineer at a large e-commerce company. You have been tasked with deploying a large language model to power a customer service chatbot that handles product returns and refunds. The model will answer customer queries based on a knowledge base of return policies and FAQs. The company has strict requirements: (1) responses must be factually accurate and grounded in the knowledge base, (2) the system must be cost-effective, and (3) latency should be under 2 seconds per response. You decide to use a pre-trained LLM from OCI Data Science and implement retrieval-augmented generation (RAG). You have two options for the retriever: a dense embedding-based retriever (e.g., using OCI AI Language embeddings) or a sparse keyword-based retriever (e.g., BM25). You also need to decide on the generation model size: a 7B parameter model or a 70B parameter model. You run a pilot test: with the dense retriever + 7B model, average latency is 1.8 seconds and accuracy is 85%. With the sparse retriever + 7B model, latency is 1.2 seconds but accuracy drops to 75%. With the 70B model (any retriever), latency exceeds 5 seconds. Which combination should you choose to meet all requirements?

Accepted Answer

Dense retriever + 7B model.. Option D (dense retriever + 7B model) is correct because it meets all three requirements: factual accuracy (85% accuracy from dense retrieval grounding), latency under 2 seconds (1.8 seconds), and cost-effectiveness (7B model is cheaper to run than 70B). The dense retriever provides better semantic matching for nuanced return policy queries, while the 7B model keeps inference fast and affordable.

Answer

Sparse retriever + 70B model.

Answer

Dense retriever + 70B model.

Answer

Sparse retriever + 7B model.

Question 7

A healthcare startup is building an AI assistant to help doctors draft clinical notes from patient-physician conversations. They have a large language model that is fine-tuned on medical data. During testing, they notice the model occasionally generates plausible-sounding but incorrect medical recommendations. The startup wants to deploy the assistant to assist doctors, not replace them. They have the following options: (A) Deploy the model as-is and rely on doctors to catch errors, (B) Add a disclaimer that the model may make mistakes, (C) Implement a fact-checking pipeline that cross-references outputs with a trusted medical knowledge base before presenting to doctors, (D) Reduce the model's temperature to 0 to ensure deterministic outputs. Which option best balances safety and utility?

Accepted Answer

Implement a fact-checking pipeline that cross-references outputs with a trusted medical knowledge base.. Option C is correct because it directly addresses the factual accuracy issue by validating outputs. Option A is wrong because relying on doctors to catch all errors is unsafe and burdensome. Option B is wrong because a disclaimer does not prevent harm. Option D is wrong because deterministic outputs do not guarantee correctness; the model can still be confidently wrong.

Answer

Add a disclaimer that the model may make mistakes.

Answer

Deploy the model as-is and rely on doctors to catch errors.

Answer

Reduce the model's temperature to 0 to ensure deterministic outputs.

Question 8

A healthcare startup is building a chatbot to answer patient inquiries using a large language model (LLM) deployed on OCI Data Science AI Quick Actions. The chatbot must comply with HIPAA regulations, so all patient data must remain within the OCI tenancy and never be sent to third-party APIs. The team has fine-tuned a Llama 2 7B model on de-identified medical records using OCI Data Science notebooks. The model is deployed as a managed endpoint via AI Quick Actions. Early testing shows that the chatbot sometimes generates responses containing specific patient names or dates of birth that were present in the fine-tuning dataset. Moreover, the model occasionally hallucinates medication dosages that are not medically accurate. Which course of action should the team take to address both issues while maintaining HIPAA compliance?

Accepted Answer

Re-fine-tune the model using differential privacy to limit memorization of training data, and implement retrieval-augmented generation (RAG) with a curated medical knowledge base to ground medication-related responses.. Option D is correct because it addresses both memorization of PII and hallucination of medication dosages while maintaining HIPAA compliance. Differential privacy during fine-tuning limits the model's ability to memorize specific patient data, and retrieval-augmented generation (RAG) grounds responses in a curated medical knowledge base, reducing hallucinations without sending data outside the OCI tenancy.

Answer

Deploy a rule-based post-processing script that checks each response against a list of known patient names and medication dosages, and rejects any response containing them.

Answer

Switch to a larger model (e.g., Llama 2 70B) to improve accuracy and reduce hallucinations, and apply output filtering to remove any detected PII from responses.

Answer

Increase the fine-tuning dataset size with more varied de-identified records to reduce overfitting, and apply a temperature setting of 0 to make outputs deterministic.

Question 9

A company wants to build a customer support chatbot using OCI Generative AI. They have a large number of historical support tickets. Which approach is most effective for leveraging this data to improve the chatbot's responses?

Accepted Answer

Fine-tune the Cohere Command model on the historical tickets using OCI Data Science.. Fine-tuning the Cohere Command model on the historical support tickets using OCI Data Science is the most effective approach because it adapts the model's weights to the specific domain language, terminology, and resolution patterns found in the company's data. This supervised learning process creates a specialized model that can generate accurate, context-aware responses for customer support queries, unlike generic prompting methods that lack deep domain adaptation.

Answer

Use a pre-loaded prompt template from the OCI console.

Answer

Increase the temperature parameter to 1.0 to encourage diverse responses.

Answer

Use zero-shot prompting with the base model and include few-shot examples in the prompt.

Question 10

A developer is using the OCI Generative AI API to generate text. The responses are often too short and incomplete. Which parameter adjustment is most likely to produce longer, more complete responses?

Accepted Answer

Increase the max_tokens parameter.. The max_tokens parameter controls the maximum number of tokens (words or subwords) the model can generate in a single response. By increasing max_tokens, the model is allowed to produce longer sequences, which directly addresses the issue of responses being too short and incomplete. In the OCI Generative AI API, this is the primary parameter for capping output length.

Answer

Decrease the max_tokens parameter.

Answer

Increase the top_p parameter.

Answer

Decrease the frequency_penalty parameter.

Question 11

During fine-tuning of a large language model on OCI, you notice that the model's performance on the validation set is not improving after several epochs, but the training loss continues to decrease. What is the most likely cause?

Accepted Answer

The model is overfitting to the training data.. When training loss decreases but validation performance stagnates or worsens, the model is overfitting to the training data. It memorizes the training examples but fails to generalize. A high learning rate might cause divergence, not this pattern. Too small training data can contribute to overfitting but is not the direct symptom. An unrepresentative validation set could cause mismatch, but the described pattern is classic overfitting.

Answer

The learning rate is too high.

Answer

The validation set is not representative.

Answer

The training data is too small.

Question 12

A user wants to use OCI Generative AI to generate marketing copy. They want the output to be more creative and varied. Which parameter should they adjust?

Accepted Answer

Increase the temperature parameter.. Increasing the temperature parameter makes the model's output more random and diverse, which is ideal for creative tasks like generating marketing copy. A higher temperature (e.g., 0.7–1.0) increases the probability of sampling less likely tokens, leading to more varied and imaginative text. Setting temperature to 0 would make the output deterministic and repetitive, which is the opposite of what the user wants.

Answer

Set temperature to 0.

Answer

Decrease the temperature parameter.

Answer

Increase the max_tokens parameter.

Question 13

An organization is concerned about the safety of generated content. Which OCI feature allows them to define custom policies to block inappropriate outputs?

Accepted Answer

Content filtering and safety controls in Generative AI. Option B is correct because OCI Generative AI includes built-in content filtering and safety controls that allow organizations to define custom policies to block inappropriate or harmful outputs. These controls operate at the model inference layer, enabling fine-grained filtering based on categories such as toxicity, hate speech, or personally identifiable information (PII). This directly addresses the concern about generated content safety.

Answer

OCI IAM policies

Answer

OCI Audit logs

Answer

OCI Vault

Question 14

A data scientist is using the OCI Generative AI SDK to create embeddings for a large corpus of legal documents. They want to perform semantic search. Which endpoint should they use?

Accepted Answer

/v1/embed. The /v1/embed endpoint is specifically designed to generate vector embeddings from input text, which are numerical representations that capture semantic meaning. For semantic search over a large corpus of legal documents, embeddings must be created to enable similarity comparisons, making this the correct choice.

Answer

/v1/classify

Answer

/v1/generate

Answer

/v1/chat

Question 15

Which of the following best describes the role of attention in transformer models?

Accepted Answer

It allows the model to focus on relevant parts of the input sequence when generating output.. Option C is correct because the attention mechanism in transformer models dynamically computes a weighted sum of all input tokens, allowing the model to focus on the most relevant parts of the input sequence when generating each output token. This is achieved through scaled dot-product attention, which assigns higher weights to tokens that are more contextually important, enabling the model to capture long-range dependencies effectively.

Answer

It assigns equal weight to all words in the input.

Answer

It is used only during training, not inference.

Answer

It replaces the need for positional encoding.