1Z0-1127-25 Practice Test 16 — 15 Questions

Question 1

Which THREE of the following are known limitations of LLMs that practitioners must account for?

Accepted Answer

Knowledge cutoff: the model only knows information up to its training data date. LLMs can produce hallucinations (factual errors), have a knowledge cutoff date, and can exhibit bias from training data. They do not inherently lack creativity, and context length is a limitation but not a 'lack of understanding'.

Answer

Inability to produce creative text

Answer

Complete lack of understanding of language syntax

Question 2

A data scientist is evaluating an LLM's performance on a summarization task. Which TWO metrics are most suitable for this evaluation?

Accepted Answer

ROUGE. ROUGE measures recall-oriented overlap between generated and reference summaries, suitable for summarization. BERTScore uses semantic similarity via embeddings. BLEU is for translation, perplexity for language modeling, and human evaluation is qualitative.

Answer

Perplexity

Answer

Human evaluation

Answer

BLEU

Question 3

Which TWO of the following are true about positional encoding in transformer models?

Accepted Answer

It provides the model with information about the order of tokens in a sequence. Positional encoding provides information about token order and is added to the input embeddings. It is used in encoder-decoder and decoder-only models. It does not replace attention, and it is not learned for each token position in the original formulation (it uses fixed sine/cosine functions).

Answer

It is learned independently for each position in the training data

Answer

It replaces the need for self-attention

Answer

It is only used in encoder-only architectures like BERT

Question 4

A team wants to compare the semantic similarity between two sentences using embeddings. Which THREE steps are required?

Accepted Answer

Normalize the resulting vectors to unit length. Generate embeddings for both sentences, then compute cosine similarity. Normalizing vectors ensures cosine similarity equals the dot product. Training a new model is unnecessary, and using a generation model is not appropriate for embeddings.

Answer

Use a text generation model to compare the sentences

Answer

Train a new neural network on the two sentences

Question 5

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

Answer

Fine-tune a base LLM on the policy documents monthly

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Answer

Train a custom model from scratch on the policy documents each month

Question 6

A company needs to classify customer support tickets into 20 categories. They have a labeled dataset of 50,000 examples. They want to use OCI Generative AI Embedding API to generate embeddings, then train a classifier. Which input type should they use for the embedding API when processing the training examples?

Accepted Answer

classification. For training a classifier, the 'classification' input type is optimized to generate embeddings that work well for classification tasks. Other types are for different use cases.

Answer

search_query

Answer

clustering

Answer

search_document

Question 7

A practitioner is choosing a model for a code generation assistant that must run on OCI with low latency. Which THREE considerations are most important?

Accepted Answer

Model size (number of parameters). Smaller models typically have lower latency, model families (Llama, Code Llama) are designed for code, and deployment choices affect latency. Context window size and fine-tuning cost are secondary or not directly relevant to latency.

Answer

The cost of fine-tuning the model on internal codebases

Answer

The model's context window length

Question 8

A company is using OCI Generative AI for a RAG-based code assistant. They index source code repositories into a vector store. Developers report that the assistant often suggests deprecated APIs or outdated code snippets, even though the latest code is in the repository. The index was built a week ago and has not been updated. They plan to set up incremental updates. However, they notice that even after re-indexing the latest commits, the issue persists. What is the most likely oversight?

Accepted Answer

The vector store is not configured to overwrite existing vectors for updated documents.. If the vector store does not overwrite or update vectors for changed documents, old vectors remain, causing retrieval of outdated code. Option B (chunking at function boundaries) may cause missing imports but not specifically deprecation. Option C (embedding model not fine-tuned on code) might affect quality but not freshness. Option D (low top-k) would affect recall, not freshness.

Answer

The retrieval top-k is set too low, missing some relevant snippets.

Answer

The chunking strategy splits code at function boundaries, losing import statements.

Answer

The embedding model is not fine-tuned on code; it was trained on natural language.

Question 9

Which TWO of the following are advantages of using Byte-Pair Encoding (BPE) tokenization compared to word-level tokenization?

Accepted Answer

Smaller vocabulary size. BPE reduces vocabulary size by representing words as subword units, and it can handle out-of-vocabulary words by breaking them into known subwords. Fixed-length tokens and losslessness are not advantages of BPE.

Answer

Guaranteed lossless encoding of all Unicode characters

Answer

Fixed token length for every input

Answer

Faster inference due to reduced sequence length

Question 10

An organization needs to deploy a custom fine-tuned model for real-time inference with consistent low latency, and they must keep the model isolated from other tenants. Which deployment option should they choose?

Accepted Answer

Provision a dedicated AI cluster with model units. A dedicated AI cluster provides exclusive, low-latency inference for custom models.

Answer

Use the shared infrastructure endpoint with an on-demand serving

Answer

Deploy the model on OCI Data Science using a custom container

Answer

Use the OCI Generative AI Agents service

Question 11

A large enterprise is deploying a generative AI model for internal document summarization. The model is deployed on OCI Data Science using a custom container. The inference endpoint is behind a public load balancer. The security team requires that all traffic between the client and the endpoint be encrypted in transit and that the endpoint not be accessible from the public internet. The current setup uses a public load balancer with an SSL certificate. The VCN has a public subnet for the load balancer and a private subnet for the model deployment. The security team is concerned that the load balancer is publicly accessible. The enterprise wants to maintain high availability and low latency. What should the architect do to meet the security requirements?

Accepted Answer

Replace the public load balancer with a private load balancer in a private subnet, and attach an SSL certificate for encryption.. Replacing the public load balancer with a private load balancer in a private subnet ensures the endpoint is not accessible from the public internet, while attaching an SSL certificate maintains encryption in transit. This satisfies both security requirements without sacrificing high availability or low latency, as the private load balancer still provides load balancing and TLS termination within the VCN.

Answer

Use a site-to-site VPN to connect clients to the VCN and access the endpoint via private IP.

Answer

Remove the load balancer and use a service gateway to access the model deployment directly from the VCN.

Answer

Keep the public load balancer but add a Web Application Firewall (WAF) to block unauthorized IPs.

Question 12

A team has fine-tuned a Cohere Command R model using T-Few on a dataset of 5,000 prompt/completion pairs. After deployment, they notice the model sometimes generates off-topic responses. Which action is most likely to improve response relevance without requiring new training data?

Accepted Answer

Set a preamble override with instructions to stay on topic. Preamble override allows setting a system message that guides the model's behavior, helping to keep responses on topic. Adjusting it is a low-cost intervention.

Answer

Decrease the max_tokens

Answer

Increase the temperature to 1.5

Answer

Increase the frequency_penalty

Question 13

A machine learning engineer is fine-tuning a Cohere Command R model using T-Few. They have prepared a JSONL dataset with 500 prompt-completion pairs. After submitting the fine-tuning job, they notice the model's performance on validation data is poor. Which action is MOST likely to improve performance?

Accepted Answer

Adding more high-quality training examples to the dataset. T-Few is parameter-efficient and may require more data. Increasing dataset size or using data augmentation is a likely improvement.

Answer

Increasing the number of training epochs

Answer

Setting the temperature to 0 in the fine-tuning configuration

Answer

Switching to a larger base model like Llama 3 70B

Question 14

A data scientist is using the OCI Generative AI Playground to test a model for a text generation task. They want to control the output to be more focused and avoid repeating the same phrases. Which THREE parameter adjustments should they consider?

Accepted Answer

Increase the presence penalty. Temperature controls randomness; frequency penalty reduces repetition of words/phrases; presence penalty encourages new topics.

Answer

Increase the temperature

Answer

Increase the max tokens

Question 15

Which API should a developer use to send a multi-turn conversation history to an LLM, including a system message and previous user/assistant exchanges, using OCI Generative AI?

Accepted Answer

Chat API. The Chat API is designed for multi-turn conversations with system prompts and history management.

Answer

Summarisation API

Answer

Generate API

Answer

Embedding API