LLM Fundamentals

What is the primary purpose of the self-attention mechanism in a Transformer model?

Which of the following best describes the difference between an encoder-only model (e.g., BERT) and a decoder-only model (e.g., GPT)?

A practitioner wants to evaluate an LLM-generated summary against a human-written reference using a metric that focuses on recall of key information. Which metric is most appropriate?

A company needs to generate embeddings for a large corpus of legal documents to enable semantic search. Which type of model should they use?

Which of the following sampling strategies selects tokens based on a cumulative probability threshold from the highest probability tokens?

An OCI Generative AI practitioner observes that a Cohere Command model generates responses with outdated information about a recent event. The model was fine-tuned six months ago. Which technique should be applied to incorporate new knowledge without retraining the model?

What is the main advantage of using Byte-Pair Encoding (BPE) over word-level tokenization?

When using an LLM for code generation, a developer notices the model occasionally produces syntactically incorrect code. Which approach is most likely to reduce syntax errors while still allowing diverse output?

In a Transformer model, what is the role of positional encoding?

An LLM is being used to answer customer queries about a product catalog. The answers are fluent but sometimes include plausible-sounding but incorrect product details. What is this phenomenon called, and which technique is most effective to mitigate it?

Which of the following metrics is most suitable for evaluating a translation model's output against multiple reference translations?

An OCI user is comparing two embedding models: one with 768 dimensions and another with 1024 dimensions. Which of the following trade-offs is most relevant?

A data scientist is building a RAG pipeline on OCI. Which TWO components are essential for the retrieval step?

A team wants to reduce hallucinations in their LLM-powered question-answering system. Which TWO techniques are most effective?

An OCI practitioner is comparing BERTScore with traditional n-gram metrics (ROUGE, BLEU) for evaluating summarization. Which THREE statements about BERTScore are true?

A data scientist wants to compare the semantic similarity between two sentences generated by an LLM. Which evaluation metric is most suitable for this purpose?

Which component of the Transformer architecture allows the model to focus on different parts of the input sequence when generating each output token?

An OCI user notices that their Llama 3 model generates the same output sequence regardless of the input prompt when using default generation parameters. Which setting is most likely causing this lack of diversity?

A developer is building a code generation assistant and wants to minimize the number of API calls to the OCI Generative AI service. Which tokenization approach results in the lowest token count for a given code snippet?

An organization wants to deploy a model that can summarize long financial reports (5000+ tokens) without losing context. Which model architecture is best suited for this requirement?

Which of the following is a known limitation of large language models that Retrieval-Augmented Generation (RAG) aims to address?

A practitioner is using a Cohere Command model on OCI for a translation task. They notice that the output is often incomplete and cuts off mid-sentence. Which parameter should they adjust to address this?

In the self-attention mechanism, what is the role of the 'scaling factor' (division by sqrt(d_k)) in the softmax computation?

Which of the following best describes the difference between pre-training and fine-tuning?

A team wants to use an LLM to answer questions about a private codebase that is updated hourly. They cannot afford to fine-tune every hour. Which OCI feature or approach is most suitable?

An OCI user observes that their Mistral model produces very repetitive text when temperature is set to 0.9 and top-p to 1.0. Which adjustment is most likely to reduce repetition?

Which of the following sampling strategies is most likely to produce the most diverse and creative text?

A data scientist needs to evaluate the quality of a text summarization model. Which TWO metrics are appropriate for this task?

An OCI user wants to reduce the cost of running a generative AI model while maintaining output quality. Which THREE strategies can help achieve this?

Which TWO components are essential in a Retrieval-Augmented Generation (RAG) pipeline?

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Which of the following best describes the role of the self-attention mechanism in a Transformer model?

A data scientist is using OCI Generative AI to process a large batch of legal documents. The total cost is higher than expected. Which factor is most likely the primary driver of cost?

A researcher is evaluating two LLMs for a summarization task. Model A achieves a ROUGE-L score of 0.45 and a BERTScore of 0.92. Model B achieves a ROUGE-L score of 0.50 and a BERTScore of 0.88. Which model is likely better for producing summaries that are semantically faithful to the source, even if not using the exact same words?

A developer is implementing a text generation pipeline using OCI Generative AI and needs to produce diverse, creative outputs for a marketing campaign. Which sampling strategy should they choose?

Which of the following is a recognized limitation of large language models?

A team is building a code generation assistant using OCI Generative AI. They notice that the model occasionally produces code with subtle security vulnerabilities. Which approach would most effectively reduce this risk without compromising the assistant's usefulness?

In the context of LLMs, what is the primary function of tokenization?

A company wants to build a sentiment analysis system for customer reviews. They have a labeled dataset of 10,000 reviews. Which approach is most cost-effective and likely to yield good performance?

An OCI user observes that their embedding model returns vectors that are not normalized, and they want to compute cosine similarity between two text embeddings. What should they do?

A developer is using OCI Generative AI for a question-answering system. The model frequently provides outdated information because the training data cutoff is over a year old. Which approach would most effectively address this issue?

Which of the following best describes the difference between an embedding model and a generation model?

A machine learning engineer is designing a RAG pipeline in OCI to improve the accuracy of an LLM-based FAQ bot. Which TWO components are essential for the retrieval phase? (Select TWO.)

A data scientist is evaluating an LLM's performance on a summarization task. They observe that the model produces fluent summaries but often misses key information. Which TWO metrics would best capture this issue? (Select TWO.)

An organization wants to use OCI Generative AI for a multilingual translation task. They need high quality and must avoid biases present in the training data. Which THREE strategies should they consider? (Select THREE.)

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

What is the primary purpose of the self-attention mechanism in a transformer model?

A data scientist is fine-tuning a Llama 2 7B model on a custom dataset using OCI Data Science. After training, the model generates fluent but factually incorrect statements about the new domain. Which post-training technique would BEST address this issue without retraining?

A developer is implementing a text generation pipeline and wants to produce diverse, creative outputs. They set temperature=1.2, top_k=50, and top_p=1.0. What is the MOST likely effect of this combination?

Which tokenization algorithm is used by models like BERT and GPT-2?

A company needs to evaluate a text summarization model. They have reference summaries and want a metric that measures overlap of n-grams. Which metric is MOST appropriate?

An OCI practitioner observes that an LLM consistently generates incorrect answers for questions about recent events (last 6 months). The model was fine-tuned on company data but not retrained recently. What is the MOST likely root cause?

A practitioner wants to generate embeddings for a set of legal documents to enable semantic search. Which type of model should they use?

What is the key advantage of multi-head attention over single-head attention in transformer models?

A team is deploying an LLM-based application that must adhere to strict data residency requirements. All processing must occur within a specific OCI region. Which OCI service should they use to host and serve the LLM?

A researcher is comparing BLEU and ROUGE scores for a machine translation model. They notice that the BLEU score is high but the ROUGE score is low. Which scenario is MOST consistent with this observation?

Which of the following is a decoder-only model architecture?

An enterprise is building a document Q&A application with OCI Generative AI. They want to minimize hallucinations. Which TWO techniques should they implement? (Choose two.)

A data scientist is evaluating an LLM for a summarization task. They have a set of human-written reference summaries. Which THREE metrics are commonly used to evaluate summarization quality? (Choose three.)

An OCI customer is deploying a chatbot using a pre-trained LLM. They are concerned about the model generating biased or harmful content. Which TWO strategies should they implement as part of their responsible AI approach? (Choose two.)

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Which component of the Transformer architecture allows the model to weigh the importance of different words in a sequence when processing a given word?

A data scientist is evaluating two LLMs for a summarization task. Model X scores 45 on ROUGE-L, while Model Y scores 42. However, in human evaluation, Model Y is preferred 60% of the time. What is the most likely explanation?

An ML engineer notices that when using temperature sampling with temperature=0.8 for code generation, the model sometimes produces syntactically incorrect code. The engineer needs to ensure syntactically valid outputs while maintaining some creativity. Which combination of sampling parameters is MOST appropriate?

Which of the following is a primary limitation of large language models that can lead to generating factually incorrect information?

A team is implementing a RAG pipeline in OCI. They have a large collection of PDF documents. After chunking and embedding the documents, retrieval quality is poor. Which step is MOST likely the root cause?

A practitioner needs to choose a pre-trained model for a sentiment analysis task on customer reviews. The model must be efficient for inference and capable of handling multiple languages. Which architecture is MOST suitable?

An organization wants to deploy an LLM for legal document analysis where accuracy is critical, and the model must not reference any external data outside the provided legal corpus. Which approach BEST satisfies these requirements?

Which of the following best describes the role of positional encoding in the Transformer architecture?

A company has a large dataset of legal documents in multiple languages. They need to find documents semantically similar to a query. Which step is essential for this task?

A research team is comparing two LLMs for a translation task. Model A uses greedy decoding, Model B uses beam search with width=5. Both models are otherwise identical. Which statement about their outputs is MOST likely true?

What is the primary difference between pre-training and fine-tuning in the context of large language models?

A data scientist is debugging a RAG system where the generated answers are not relevant to the retrieved documents. Which TWO factors are MOST likely causing this issue?

An ML engineer is choosing an LLM for a code generation assistant. The model must generate syntactically correct code, handle multiple programming languages, and be cost-efficient. Which THREE characteristics should the engineer prioritize?

A team is evaluating two LLMs for a summarization task. Model X has a BERTScore of 0.85, Model Y has a BERTScore of 0.82. However, human evaluators prefer Model Y. Which TWO reasons could explain this discrepancy?

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Which component of the Transformer architecture allows the model to weigh the importance of different tokens in the input sequence when generating an output?

A data scientist is fine-tuning a Llama 2 model on a custom dataset for a summarization task. After fine-tuning, the model produces summaries that are too similar to the input text, often copying sentences verbatim. Which adjustment is MOST likely to reduce copying and improve abstractive summarization?

An LLM generates a response that contains a plausible-sounding but factually incorrect statement about a historical event. This is an example of which known limitation?

Which evaluation metric is designed to measure the overlap of n-grams between a generated summary and a reference summary, focusing on recall of content words?

A developer is building a code generation assistant and needs to ensure the LLM follows a specific output format (e.g., JSON). Which approach is MOST effective for achieving format adherence without retraining?

Which tokenization algorithm is commonly used by models like GPT and BERT, and works by merging frequently occurring character pairs iteratively?

A machine learning engineer needs to select an embedding model to compute semantic similarity between customer reviews. Which property is MOST important for the embedding model to produce useful similarity scores?

Which sampling strategy selects the token with the highest probability at each step, resulting in deterministic and often repetitive outputs?

An organization wants to use an LLM for translation tasks but is concerned about data privacy and wants to keep all data within OCI. Which model family is natively available in OCI Generative AI service?

A developer is using the OCI Generative AI service and notices that the cost per API call is higher than expected. Which factor contributes MOST to the cost of an LLM inference call?

When evaluating a summarization model, the team notices that the ROUGE-L score is high but human evaluators rate the summaries poorly for coherence. What does this discrepancy MOST likely indicate?

Which TWO of the following are characteristics of decoder-only models like GPT? (Select TWO)

Which THREE of the following are known limitations of large language models? (Select THREE)

A team is building a RAG pipeline on OCI. Which THREE steps are essential components of a standard RAG pipeline? (Select THREE)

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Which component of the transformer architecture allows the model to weigh the importance of different words in a sentence when processing input?

A data scientist wants to reduce the cost of token usage when summarizing large documents using an LLM on OCI. Which tokenization approach is MOST likely to lower token count for English text?

An organization is evaluating two LLMs for a code generation task. Model A has a perplexity of 1.5 on the validation set, and Model B has a perplexity of 3.0. However, Model A generates more syntactically incorrect code. Which conclusion is MOST valid?

A developer is using the Cohere Command model for text generation and wants to ensure the output is deterministic for testing purposes. Which sampling strategy should they use?

Which of the following is a known limitation of large language models?

A data scientist needs to compare two summaries generated by different models against a reference summary. Which metric focuses on recall of n-grams and is commonly used for summarization evaluation?

A team is building a system to detect duplicate customer support tickets. They have a dataset of 10,000 resolved tickets and want to find pairs with similar intent. Which approach would be MOST efficient and effective?

In the transformer architecture, what is the primary purpose of positional encoding?

A company wants to translate legal documents from English to Spanish. They have a small parallel corpus of 500 sentence pairs. Which approach is MOST likely to yield the best translation quality?

An AI engineer is deploying a RAG pipeline using OCI Generative AI. They notice the generated answers sometimes include information not present in the retrieved documents. What is the MOST likely cause?

Which of the following best describes the difference between pre-training and fine-tuning?

A data scientist is evaluating BERTScore to compare model-generated summaries with reference summaries. Which TWO statements about BERTScore are correct?

A team is designing a text classification system using OCI Generative AI. They have a small labeled dataset of 200 examples per class. Which THREE techniques can help improve model performance without requiring additional labeled data?

An organization is concerned about bias in their LLM-powered hiring assistant. Which TWO actions are MOST effective in mitigating bias?

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Which component of the Transformer architecture allows the model to weigh the importance of different tokens in the input sequence when generating each output token?

A data scientist is evaluating a summarization model on a news article dataset. They compute ROUGE-L and BLEU scores. The ROUGE-L score is high, but the BLEU score is low. Which of the following best explains this discrepancy?

An organization needs to select a tokenisation algorithm for a multilingual LLM that will process English, Chinese, and Korean text efficiently. Which tokenisation method is BEST suited for this requirement?

A developer is using OCI Generative AI with a Cohere Command model for text generation. They want the output to be more creative and diverse, but still relevant. Which sampling strategy should they use?

Which of the following is a key limitation of large language models that RAG (Retrieval-Augmented Generation) aims to address?

A researcher wants to compare two summarization models. Model A achieves a higher ROUGE-L score than Model B, but human evaluators prefer Model B's summaries. Which of the following is the MOST likely reason?

A team is building a code generation assistant and needs to choose between fine-tuning a base LLM or using in-context learning with a few examples. They have 500 high-quality code examples. The assistant must generate code for a wide variety of tasks. Which approach is BETTER and why?

Which model architecture is used by BERT for natural language understanding tasks?

An OCI user wants to generate embeddings for a large corpus of technical documents to enable semantic search. Which type of model should they use?

A developer notices that an LLM-based question-answering system sometimes provides answers that are correct but from an outdated version of the knowledge base. The system uses RAG with a vector database updated daily. What is the MOST likely root cause?

Which of the following is a distinguishing feature of in-context learning compared to fine-tuning?

An enterprise is deploying an LLM application on OCI and must minimize hallucinations. Which TWO strategies should they implement? (Choose two.)

A machine learning engineer is evaluating the performance of a translation model using BLEU score. Which THREE statements about BLEU are correct? (Choose three.)

A developer is building a text generation application using OCI Generative AI and wants to control the creativity of the output. Which THREE sampling parameters can they adjust? (Choose three.)

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Which component of the Transformer architecture allows each token to consider the relevance of every other token in the input sequence?

A data scientist is comparing BLEU, ROUGE, and BERTScore to evaluate a summarization model. The client cares most about whether the summary captures all key facts from the source document. Which metric is most aligned with this requirement?

A developer notices that a text generation model produces repetitive phrases when using greedy decoding. Which sampling strategy would best introduce controlled randomness to reduce repetition while maintaining coherence?

Which tokenization algorithm is commonly used in models like GPT and BERT and builds tokens by merging the most frequent pairs of characters or subwords iteratively?

An organization needs to deploy a model that can both understand and generate text, such as for a translation task where the input is in English and output is in French. Which model architecture is most suitable?

A team is deploying a chatbot that must never output harmful or biased statements. They plan to use a pre-trained LLM with in-context learning. Which additional measure is MOST effective at reducing harmful outputs without retraining?

A developer wants to compare two sentences for semantic similarity using embeddings. Which distance or similarity metric is most commonly used for dense vector representations?

Which of the following is a known limitation of large language models where the model generates plausible-sounding but factually incorrect information?

A team is evaluating two embedding models for a similarity search task. Model A has a higher BERTScore on a reference dataset. Model B has a lower perplexity on the same dataset. Which model is likely better for retrieval?

A company's AI system uses RAG to answer customer questions. Users often get incomplete answers because the retrieved chunks do not contain all relevant information. Which step in the RAG pipeline is most likely the issue?

An ML engineer is selecting a pre-trained model for a code generation task. The model must be able to generate syntactically correct code in multiple programming languages. Which model family is BEST suited for this task?

A data scientist is building a text summarization system using an LLM. They want to evaluate the model's output against human-written summaries. Which TWO metrics are most appropriate for this evaluation? (Choose two.)

An organization is deploying an LLM for document question answering. They want to reduce hallucinations and ensure answers are grounded in provided documents. Which THREE techniques should they implement? (Choose three.)

A developer is comparing different foundation models for a text completion API on OCI. Which TWO of the following are model families available through OCI Generative AI service? (Choose two.)

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Which TWO of the following are advantages of using Byte-Pair Encoding (BPE) tokenization compared to word-level tokenization?

A data scientist is evaluating an LLM's performance on a summarization task. Which TWO metrics are most suitable for this evaluation?

A developer is debugging a RAG pipeline where the LLM frequently ignores retrieved documents and produces hallucinations. Which THREE factors could contribute to this problem?

Which TWO of the following are characteristics of the transformer decoder-only architecture (e.g., GPT)?

Which TWO of the following sampling strategies introduce randomness into text generation?

A team wants to compare the semantic similarity between two sentences using embeddings. Which THREE steps are required?

A practitioner is choosing a model for a code generation assistant that must run on OCI with low latency. Which THREE considerations are most important?

Which THREE of the following are known limitations of LLMs that practitioners must account for?

Which TWO of the following are true about positional encoding in transformer models?