1Z0-1127 LLM Fundamentals — Timed 25 Questions

Question 1

What is the primary purpose of the self-attention mechanism in a Transformer model?

Accepted Answer

To compute a weighted sum of all token representations based on pairwise relevance. Self-attention allows each token to attend to every other token in the sequence, capturing contextual relationships regardless of distance.

Answer

To generate token embeddings in parallel

Answer

To reduce the dimensionality of token embeddings

Answer

To encode positional information of tokens

Question 2

Which of the following best describes the difference between an encoder-only model (e.g., BERT) and a decoder-only model (e.g., GPT)?

Accepted Answer

Encoder-only uses bidirectional attention and is suited for classification or NER; decoder-only uses causal attention and is suited for text generation. Option A is correct because encoder-only models like BERT employ bidirectional attention, allowing each token to attend to all other tokens in both directions, which is ideal for tasks requiring full context understanding such as classification or named entity recognition (NER). In contrast, decoder-only models like GPT use causal (masked) attention, where each token can only attend to previous tokens, making them suitable for autoregressive text generation.

Answer

Encoder-only is trained for text generation; decoder-only is trained for classification

Answer

Both use the same attention pattern but differ in number of layers

Answer

Encoder-only uses causal attention; decoder-only uses bidirectional attention

Question 3

A practitioner wants to evaluate an LLM-generated summary against a human-written reference using a metric that focuses on recall of key information. Which metric is most appropriate?

Accepted Answer

ROUGE. ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is the most appropriate metric because it specifically measures recall of key information by comparing n-gram overlap between the generated summary and a reference summary. This aligns directly with the practitioner's goal of evaluating how well the LLM-generated summary captures the essential content from the human-written reference.

Answer

BLEU

Answer

Perplexity

Answer

Cosine similarity

Question 4

A company needs to generate embeddings for a large corpus of legal documents to enable semantic search. Which type of model should they use?

Accepted Answer

An encoder-only embedding model like Cohere Embed. An encoder-only embedding model like Cohere Embed is designed to convert text into dense vector representations (embeddings) that capture semantic meaning, which is exactly what is needed for semantic search over a large corpus of legal documents. These models use a bidirectional transformer architecture to encode context from both directions, producing fixed-size embeddings that can be efficiently compared using cosine similarity or other distance metrics.

Answer

A decoder-only generation model like GPT

Answer

A text-to-speech model

Answer

A machine translation model

Question 5

Which of the following sampling strategies selects tokens based on a cumulative probability threshold from the highest probability tokens?

Accepted Answer

Top-p (nucleus) sampling. Top-p (nucleus) sampling cuts off the tail of the probability distribution where cumulative probability exceeds p, allowing dynamic vocabulary size.

Answer

Top-k sampling

Answer

Greedy decoding

Answer

Temperature sampling

Question 6

An OCI Generative AI practitioner observes that a Cohere Command model generates responses with outdated information about a recent event. The model was fine-tuned six months ago. Which technique should be applied to incorporate new knowledge without retraining the model?

Accepted Answer

Implement a RAG pipeline that indexes the latest documents into a vector store and retrieves relevant passages at query time. RAG retrieves relevant, current documents at inference time, providing up-to-date context without modifying the model parameters.

Answer

Use a longer context window and include all new articles in the prompt

Answer

Fine-tune the model again with the new data

Answer

Increase the temperature parameter to encourage more creative outputs

Question 7

What is the main advantage of using Byte-Pair Encoding (BPE) over word-level tokenization?

Accepted Answer

It can represent any word as a sequence of subword tokens, including rare or unseen words. Byte-Pair Encoding (BPE) is a subword tokenization algorithm that iteratively merges the most frequent pairs of bytes or characters into new tokens. Its main advantage over word-level tokenization is that it can represent any word, including rare or unseen words, as a sequence of subword tokens, thereby eliminating the out-of-vocabulary (OOV) problem. This allows models like GPT and BERT to handle arbitrary input without requiring a fixed word vocabulary.

Answer

It produces fixed-length token sequences

Answer

It reduces the number of tokens by merging all letters into single tokens

Answer

It eliminates the need for a vocabulary altogether

Question 8

When using an LLM for code generation, a developer notices the model occasionally produces syntactically incorrect code. Which approach is most likely to reduce syntax errors while still allowing diverse output?

Accepted Answer

Set temperature to 0 and use greedy decoding. Lowering temperature reduces randomness, making outputs more deterministic and less prone to errors, while still allowing some variation.

Answer

Use top-k sampling with k=100

Answer

Increase the context window size

Answer

Increase the temperature to 1.5

Question 9

In a Transformer model, what is the role of positional encoding?

Accepted Answer

To provide information about the position of each token in the sequence. Positional encoding is essential in Transformer models because the self-attention mechanism processes all tokens in parallel and has no inherent notion of sequence order. By adding positional encodings (often sinusoidal or learned) to the input embeddings, the model can distinguish between tokens at different positions, enabling it to capture word order and relative positions. Without this, the model would treat the sequence as a bag of tokens, losing all sequential context.

Answer

To reduce the number of parameters in the model

Answer

To enable the model to process tokens in parallel

Answer

To encode the semantic meaning of each token

Question 10

An LLM is being used to answer customer queries about a product catalog. The answers are fluent but sometimes include plausible-sounding but incorrect product details. What is this phenomenon called, and which technique is most effective to mitigate it?

Accepted Answer

Hallucination; use Retrieval-Augmented Generation (RAG) with the catalog indexed. Hallucination is the generation of false information; RAG grounds responses in retrieved factual documents, reducing hallucinations.

Answer

Knowledge cutoff; fine-tune the model on the catalog

Answer

Bias amplification; increase temperature

Answer

Overfitting; reduce the model size

Question 11

Which of the following metrics is most suitable for evaluating a translation model's output against multiple reference translations?

Accepted Answer

BLEU. BLEU (Bilingual Evaluation Understudy) is the most suitable metric for evaluating a translation model's output against multiple reference translations because it measures n-gram precision between the candidate translation and one or more reference translations. It directly quantifies how many words and phrases in the candidate match those in the references, making it the standard metric for machine translation tasks.

Answer

ROUGE

Answer

Perplexity

Answer

BERTScore

Question 12

An OCI user is comparing two embedding models: one with 768 dimensions and another with 1024 dimensions. Which of the following trade-offs is most relevant?

Accepted Answer

Higher dimensions may capture more fine-grained semantic information but require more storage and slower similarity search. Higher dimensions can capture more nuanced semantics but require more storage and may increase retrieval latency.

Answer

The 1024-dimensional model always yields better accuracy with no additional cost

Answer

The 768-dimensional model is always faster and more storage-efficient

Answer

Dimension size has no impact on performance or storage

Question 13

A data scientist is building a RAG pipeline on OCI. Which TWO components are essential for the retrieval step?

Accepted Answer

Embedding model to convert chunks into vectors. Chunking splits documents into manageable pieces, and embedding converts them into vectors for similarity search.

Answer

Fine-tuned generation model

Answer

Beam search decoder

Answer

Human feedback loop

Question 14

A team wants to reduce hallucinations in their LLM-powered question-answering system. Which TWO techniques are most effective?

Accepted Answer

Implementing RAG to retrieve relevant documents. RAG provides factual grounding, and reducing temperature makes outputs more deterministic, reducing fabricated details.

Answer

Switching to a smaller model

Answer

Using a larger context window

Answer

Increasing the temperature to 1.5

Question 15

An OCI practitioner is comparing BERTScore with traditional n-gram metrics (ROUGE, BLEU) for evaluating summarization. Which THREE statements about BERTScore are true?

Accepted Answer

BERTScore uses pre-trained contextual embeddings from BERT. BERTScore uses contextual embeddings, captures semantic similarity, and correlates better with human judgment than n-gram metrics.

Answer

BERTScore does not require a reference text

Answer

BERTScore computes similarity based on exact n-gram overlap

Question 16

A data scientist wants to compare the semantic similarity between two sentences generated by an LLM. Which evaluation metric is most suitable for this purpose?

Accepted Answer

BERTScore. BERTScore computes cosine similarity between contextual embeddings, capturing semantic meaning better than surface-level n-gram metrics.

Answer

ROUGE-L

Answer

BLEU

Answer

Perplexity

Question 17

Which component of the Transformer architecture allows the model to focus on different parts of the input sequence when generating each output token?

Accepted Answer

Self-attention mechanism. Self-attention computes attention scores between all pairs of positions, enabling the model to weigh the importance of different input tokens.

Answer

Positional encoding

Answer

Feed-forward network

Answer

Layer normalization

Question 18

An OCI user notices that their Llama 3 model generates the same output sequence regardless of the input prompt when using default generation parameters. Which setting is most likely causing this lack of diversity?

Accepted Answer

Greedy decoding (temperature=0). Greedy decoding, which is equivalent to setting temperature=0, always selects the token with the highest probability at each step. This deterministic behavior causes the model to produce the exact same output sequence for any input prompt, as there is no randomness or variation in token selection. The lack of diversity is a direct consequence of eliminating all stochasticity from the generation process.

Answer

Top-k sampling with k=50

Answer

Temperature sampling with temperature=0.8

Answer

Top-p (nucleus) sampling with p=0.9

Question 19

A developer is building a code generation assistant and wants to minimize the number of API calls to the OCI Generative AI service. Which tokenization approach results in the lowest token count for a given code snippet?

Accepted Answer

BPE tokenizer trained on code corpora. Option C is correct because BPE (Byte Pair Encoding) tokenizers trained specifically on code corpora learn subword units that align closely with programming language syntax (e.g., common keywords, operators, and variable patterns), resulting in fewer tokens for a given code snippet compared to general-purpose tokenizers. This reduces API calls by encoding more semantic meaning per token, directly minimizing token count.

Answer

WordPiece tokenizer

Answer

SentencePiece tokenizer with unigram LM

Answer

Character-level tokenization

Question 20

An organization wants to deploy a model that can summarize long financial reports (5000+ tokens) without losing context. Which model architecture is best suited for this requirement?

Accepted Answer

Encoder-decoder model (e.g., T5). Encoder-decoder models like T5 or BART are designed for sequence-to-sequence tasks such as summarization, and can handle long inputs with their encoder.

Answer

Mixture-of-experts model

Answer

Decoder-only model (e.g., GPT)

Answer

Encoder-only model (e.g., BERT)