1Z0-1127-25 Practice Test 19 — 15 Questions

Question 1

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use Retrieval-Augmented Generation (RAG) with the policy documents indexed in a vector store. RAG (Retrieval-Augmented Generation) allows the LLM to retrieve relevant document sections at inference time, so knowledge stays current without retraining. The other options either require expensive retraining for each update or lack document grounding.

Answer

Train a custom model from scratch on the policy documents each month

Answer

Use a larger foundation model with a longer context window and paste all documents into each prompt

Answer

Fine-tune a base LLM on the policy documents monthly

Question 2

An OCI user is comparing two embedding models: one with 768 dimensions and another with 1024 dimensions. Which of the following trade-offs is most relevant?

Accepted Answer

Higher dimensions may capture more fine-grained semantic information but require more storage and slower similarity search. Higher dimensions can capture more nuanced semantics but require more storage and may increase retrieval latency.

Answer

The 1024-dimensional model always yields better accuracy with no additional cost

Answer

The 768-dimensional model is always faster and more storage-efficient

Answer

Dimension size has no impact on performance or storage

Question 3

A company wants to build a sentiment analysis system for customer reviews. They have a labeled dataset of 10,000 reviews. Which approach is most cost-effective and likely to yield good performance?

Accepted Answer

Fine-tune a pre-trained BERT model on the labeled dataset. Fine-tuning a pre-trained encoder-only model like BERT on the labeled dataset is a standard approach for classification tasks, offering good performance with relatively modest data and compute.

Answer

Use GPT-4 with a prompt and no fine-tuning

Answer

Use a simple bag-of-words model with logistic regression

Answer

Train a Transformer model from scratch on the reviews

Question 4

Which of the following is a decoder-only model architecture?

Accepted Answer

GPT-3. GPT is a decoder-only model. BERT is encoder-only. T5 is encoder-decoder.

Answer

T5

Answer

BART

Answer

BERT

Question 5

A company's AI system uses RAG to answer customer questions. Users often get incomplete answers because the retrieved chunks do not contain all relevant information. Which step in the RAG pipeline is most likely the issue?

Accepted Answer

Chunking strategy (chunk size and overlap). Chunking determines how documents are split into pieces. If chunks are too small, key information may be split across chunks, causing incomplete retrieval. Adjusting chunk size and overlap can improve completeness.

Answer

Retrieval top-k setting

Answer

Generation model temperature

Answer

Embedding model selection

Question 6

A developer is using the Cohere Command model for text generation and wants to ensure the output is deterministic for testing purposes. Which sampling strategy should they use?

Accepted Answer

Greedy decoding. Greedy decoding always selects the token with highest probability, producing the same output for a given input. Temperature, top-k, and top-p introduce randomness.

Answer

Top-k sampling with k=50

Answer

Temperature sampling with temperature=0.7

Answer

Top-p (nucleus) sampling with p=0.9

Question 7

In the transformer architecture, what is the primary purpose of positional encoding?

Accepted Answer

To provide the model with information about the order of tokens. Since self-attention processes tokens in parallel without inherent order, positional encoding adds information about the position of each token in the sequence.

Answer

To normalize the input embeddings

Answer

To reduce the number of parameters in the model

Answer

To enable multi-head attention

Question 8

A research team is comparing two LLMs for a translation task. Model A uses greedy decoding, Model B uses beam search with width=5. Both models are otherwise identical. Which statement about their outputs is MOST likely true?

Accepted Answer

Model B will generally produce more fluent and accurate translations. Beam search considers multiple candidate sequences and selects the one with the highest overall probability, which often results in more fluent and accurate translations than greedy decoding, but at higher computational cost.

Answer

Model A will have higher BLEU scores than Model B

Answer

Model A will produce more diverse translations

Answer

Model B will have lower latency than Model A

Question 9

Which of the following best describes the difference between pre-training and fine-tuning?

Accepted Answer

Pre-training learns general language representations; fine-tuning adapts to a specific task. Pre-training is the initial phase where a model learns general language patterns from a large corpus. Fine-tuning adapts the pre-trained model to a specific task using a smaller labeled dataset.

Answer

Pre-training uses labeled data; fine-tuning uses unlabeled data

Answer

Fine-tuning requires more data than pre-training

Answer

Pre-training is done on a single task; fine-tuning is done on multiple tasks

Question 10

A team is evaluating two LLMs for a summarization task. Model X has a BERTScore of 0.85, Model Y has a BERTScore of 0.82. However, human evaluators prefer Model Y. Which TWO reasons could explain this discrepancy?

Accepted Answer

BERTScore is based on BERT embeddings, which may not fully capture summary-specific qualities like conciseness or readability. BERTScore correlates with human judgment but is not perfect; it may favor certain styles. Additionally, BERTScore may be inflated if the reference summaries are similar to the model's training data.

Answer

BERTScore uses precision only, so it misses recall aspects

Answer

Human evaluators were not given clear criteria for evaluation

Answer

Model Y was fine-tuned on a different dataset, causing distribution shift

Question 11

In the self-attention mechanism, what is the role of the 'scaling factor' (division by sqrt(d_k)) in the softmax computation?

Accepted Answer

To prevent the softmax from saturating and producing small gradients. Scaling prevents the dot products from growing too large in magnitude, which would push softmax into regions with extremely small gradients.

Answer

To make the attention mechanism translation invariant

Answer

To increase the variance of attention scores

Answer

To ensure the sum of attention weights equals 1

Question 12

What is the primary purpose of the self-attention mechanism in a Transformer model?

Accepted Answer

To compute a weighted sum of all token representations based on pairwise relevance. Self-attention allows each token to attend to every other token in the sequence, capturing contextual relationships regardless of distance.

Answer

To generate token embeddings in parallel

Answer

To reduce the dimensionality of token embeddings

Answer

To encode positional information of tokens

Question 13

An ML engineer is selecting a pre-trained model for a code generation task. The model must be able to generate syntactically correct code in multiple programming languages. Which model family is BEST suited for this task?

Accepted Answer

Meta Llama (Code Llama variant). Models like Code Llama (a variant of Llama) are specifically fine-tuned on code and are known for strong code generation capabilities. While other models can generate code, Code Llama is the best fit among the options.

Answer

BERT

Answer

Cohere Command

Answer

Mistral

Question 14

A researcher is evaluating two LLMs for a summarization task. Model A achieves a ROUGE-L score of 0.45 and a BERTScore of 0.92. Model B achieves a ROUGE-L score of 0.50 and a BERTScore of 0.88. Which model is likely better for producing summaries that are semantically faithful to the source, even if not using the exact same words?

Accepted Answer

Model A because BERTScore is higher. BERTScore measures semantic similarity using contextual embeddings, while ROUGE-L measures n-gram overlap. Higher BERTScore indicates better semantic faithfulness even without exact phrase matches.

Answer

Neither model is acceptable because ROUGE-L is below 0.6

Answer

Both are equally good because the scores are close

Answer

Model B because ROUGE-L is higher

Question 15

A data scientist wants to compare the semantic similarity between two sentences generated by an LLM. Which evaluation metric is most suitable for this purpose?

Accepted Answer

BERTScore. BERTScore computes cosine similarity between contextual embeddings, capturing semantic meaning better than surface-level n-gram metrics.

Answer

ROUGE-L

Answer

BLEU

Answer

Perplexity