LLM Fundamentals

An enterprise is building a document Q&A application with OCI Generative AI. They want to minimize hallucinations. Which TWO techniques should they implement? (Choose two.)

A data scientist is comparing BLEU, ROUGE, and BERTScore to evaluate a summarization model. The client cares most about whether the summary captures all key facts from the source document. Which metric is most aligned with this requirement?

Which of the following is a known limitation of large language models where the model generates plausible-sounding but factually incorrect information?

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

When evaluating a summarization model, the team notices that the ROUGE-L score is high but human evaluators rate the summaries poorly for coherence. What does this discrepancy MOST likely indicate?

A developer is using the OCI Generative AI service and notices that the cost per API call is higher than expected. Which factor contributes MOST to the cost of an LLM inference call?

A developer is implementing a text generation pipeline using OCI Generative AI and needs to produce diverse, creative outputs for a marketing campaign. Which sampling strategy should they choose?

An OCI user wants to generate embeddings for a large corpus of technical documents to enable semantic search. Which type of model should they use?

A team is deploying a chatbot that must never output harmful or biased statements. They plan to use a pre-trained LLM with in-context learning. Which additional measure is MOST effective at reducing harmful outputs without retraining?

An OCI customer is deploying a chatbot using a pre-trained LLM. They are concerned about the model generating biased or harmful content. Which TWO strategies should they implement as part of their responsible AI approach? (Choose two.)

Which of the following is a key limitation of large language models that RAG (Retrieval-Augmented Generation) aims to address?

An OCI Generative AI practitioner observes that a Cohere Command model generates responses with outdated information about a recent event. The model was fine-tuned six months ago. Which technique should be applied to incorporate new knowledge without retraining the model?

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

A team is building a code generation assistant using OCI Generative AI. They notice that the model occasionally produces code with subtle security vulnerabilities. Which approach would most effectively reduce this risk without compromising the assistant's usefulness?

A researcher is comparing BLEU and ROUGE scores for a machine translation model. They notice that the BLEU score is high but the ROUGE score is low. Which scenario is MOST consistent with this observation?

Which THREE of the following are known limitations of LLMs that practitioners must account for?

Which of the following sampling strategies is most likely to produce the most diverse and creative text?

When using an LLM for code generation, a developer notices the model occasionally produces syntactically incorrect code. Which approach is most likely to reduce syntax errors while still allowing diverse output?

A data scientist is evaluating an LLM for a summarization task. They have a set of human-written reference summaries. Which THREE metrics are commonly used to evaluate summarization quality? (Choose three.)

A developer is building a text generation application using OCI Generative AI and wants to control the creativity of the output. Which THREE sampling parameters can they adjust? (Choose three.)

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

A data scientist is evaluating an LLM's performance on a summarization task. Which TWO metrics are most suitable for this evaluation?

A team is deploying an LLM-based application that must adhere to strict data residency requirements. All processing must occur within a specific OCI region. Which OCI service should they use to host and serve the LLM?

Which TWO of the following are true about positional encoding in transformer models?

An OCI user wants to reduce the cost of running a generative AI model while maintaining output quality. Which THREE strategies can help achieve this?

Which tokenization algorithm is commonly used by models like GPT and BERT, and works by merging frequently occurring character pairs iteratively?

A data scientist is evaluating a summarization model on a news article dataset. They compute ROUGE-L and BLEU scores. The ROUGE-L score is high, but the BLEU score is low. Which of the following best explains this discrepancy?

An enterprise is deploying an LLM application on OCI and must minimize hallucinations. Which TWO strategies should they implement? (Choose two.)

A team wants to compare the semantic similarity between two sentences using embeddings. Which THREE steps are required?

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

Which of the following is a recognized limitation of large language models?

A data scientist is fine-tuning a Llama 2 model on a custom dataset for a summarization task. After fine-tuning, the model produces summaries that are too similar to the input text, often copying sentences verbatim. Which adjustment is MOST likely to reduce copying and improve abstractive summarization?

A developer is debugging a RAG pipeline where the LLM frequently ignores retrieved documents and produces hallucinations. Which THREE factors could contribute to this problem?

A developer notices that a text generation model produces repetitive phrases when using greedy decoding. Which sampling strategy would best introduce controlled randomness to reduce repetition while maintaining coherence?

A company needs to generate embeddings for a large corpus of legal documents to enable semantic search. Which type of model should they use?

A data scientist is building a text summarization system using an LLM. They want to evaluate the model's output against human-written summaries. Which TWO metrics are most appropriate for this evaluation? (Choose two.)

An organization needs to deploy a model that can both understand and generate text, such as for a translation task where the input is in English and output is in French. Which model architecture is most suitable?

Which of the following metrics is most suitable for evaluating a translation model's output against multiple reference translations?

A data scientist is debugging a RAG system where the generated answers are not relevant to the retrieved documents. Which TWO factors are MOST likely causing this issue?

Which evaluation metric is designed to measure the overlap of n-grams between a generated summary and a reference summary, focusing on recall of content words?

A team wants to use an LLM to answer questions about a private codebase that is updated hourly. They cannot afford to fine-tune every hour. Which OCI feature or approach is most suitable?

A practitioner is using a Cohere Command model on OCI for a translation task. They notice that the output is often incomplete and cuts off mid-sentence. Which parameter should they adjust to address this?

Which TWO components are essential in a Retrieval-Augmented Generation (RAG) pipeline?

A practitioner needs to choose a pre-trained model for a sentiment analysis task on customer reviews. The model must be efficient for inference and capable of handling multiple languages. Which architecture is MOST suitable?

Which of the following is a distinguishing feature of in-context learning compared to fine-tuning?

What is the key advantage of multi-head attention over single-head attention in transformer models?

What is the primary difference between pre-training and fine-tuning in the context of large language models?

A developer is using OCI Generative AI with a Cohere Command model for text generation. They want the output to be more creative and diverse, but still relevant. Which sampling strategy should they use?

Which component of the Transformer architecture allows the model to focus on different parts of the input sequence when generating each output token?

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

An OCI practitioner is comparing BERTScore with traditional n-gram metrics (ROUGE, BLEU) for evaluating summarization. Which THREE statements about BERTScore are true?

An organization wants to deploy a model that can summarize long financial reports (5000+ tokens) without losing context. Which model architecture is best suited for this requirement?

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

An organization needs to select a tokenisation algorithm for a multilingual LLM that will process English, Chinese, and Korean text efficiently. Which tokenisation method is BEST suited for this requirement?

A data scientist is building a RAG pipeline on OCI. Which TWO components are essential for the retrieval step?

An OCI user observes that their Mistral model produces very repetitive text when temperature is set to 0.9 and top-p to 1.0. Which adjustment is most likely to reduce repetition?

An LLM generates a response that contains a plausible-sounding but factually incorrect statement about a historical event. This is an example of which known limitation?

A practitioner is choosing a model for a code generation assistant that must run on OCI with low latency. Which THREE considerations are most important?

An OCI user is comparing two embedding models: one with 768 dimensions and another with 1024 dimensions. Which of the following trade-offs is most relevant?

A data scientist is using OCI Generative AI to process a large batch of legal documents. The total cost is higher than expected. Which factor is most likely the primary driver of cost?

An organization wants to use OCI Generative AI for a multilingual translation task. They need high quality and must avoid biases present in the training data. Which THREE strategies should they consider? (Select THREE.)

Which model architecture is used by BERT for natural language understanding tasks?

A company has a large dataset of legal documents in multiple languages. They need to find documents semantically similar to a query. Which step is essential for this task?

A researcher wants to compare two summarization models. Model A achieves a higher ROUGE-L score than Model B, but human evaluators prefer Model B's summaries. Which of the following is the MOST likely reason?

A company wants to build a sentiment analysis system for customer reviews. They have a labeled dataset of 10,000 reviews. Which approach is most cost-effective and likely to yield good performance?

An LLM is being used to answer customer queries about a product catalog. The answers are fluent but sometimes include plausible-sounding but incorrect product details. What is this phenomenon called, and which technique is most effective to mitigate it?

An OCI user observes that their embedding model returns vectors that are not normalized, and they want to compute cosine similarity between two text embeddings. What should they do?

Which of the following is a decoder-only model architecture?

A company's AI system uses RAG to answer customer questions. Users often get incomplete answers because the retrieved chunks do not contain all relevant information. Which step in the RAG pipeline is most likely the issue?

A team is building a code generation assistant and needs to choose between fine-tuning a base LLM or using in-context learning with a few examples. They have 500 high-quality code examples. The assistant must generate code for a wide variety of tasks. Which approach is BETTER and why?

A developer is using the Cohere Command model for text generation and wants to ensure the output is deterministic for testing purposes. Which sampling strategy should they use?

Which of the following best describes the role of the self-attention mechanism in a Transformer model?

In the transformer architecture, what is the primary purpose of positional encoding?

In a Transformer model, what is the role of positional encoding?

An organization is deploying an LLM for document question answering. They want to reduce hallucinations and ensure answers are grounded in provided documents. Which THREE techniques should they implement? (Choose three.)

A team is implementing a RAG pipeline in OCI. They have a large collection of PDF documents. After chunking and embedding the documents, retrieval quality is poor. Which step is MOST likely the root cause?

Which TWO of the following are advantages of using Byte-Pair Encoding (BPE) tokenization compared to word-level tokenization?

A research team is comparing two LLMs for a translation task. Model A uses greedy decoding, Model B uses beam search with width=5. Both models are otherwise identical. Which statement about their outputs is MOST likely true?

Which of the following best describes the difference between pre-training and fine-tuning?

A practitioner wants to generate embeddings for a set of legal documents to enable semantic search. Which type of model should they use?

A team is evaluating two LLMs for a summarization task. Model X has a BERTScore of 0.85, Model Y has a BERTScore of 0.82. However, human evaluators prefer Model Y. Which TWO reasons could explain this discrepancy?

A company wants to build a customer service chatbot that answers questions about their internal policy documents. The documents are updated monthly, and the team cannot afford to retrain a model each time. Which approach is MOST appropriate?

A developer is using OCI Generative AI for a question-answering system. The model frequently provides outdated information because the training data cutoff is over a year old. Which approach would most effectively address this issue?

Which of the following is a primary limitation of large language models that can lead to generating factually incorrect information?

A data scientist is evaluating two LLMs for a summarization task. Model X scores 45 on ROUGE-L, while Model Y scores 42. However, in human evaluation, Model Y is preferred 60% of the time. What is the most likely explanation?

In the self-attention mechanism, what is the role of the 'scaling factor' (division by sqrt(d_k)) in the softmax computation?

What is the primary purpose of the self-attention mechanism in a Transformer model?

A developer is implementing a text generation pipeline and wants to produce diverse, creative outputs. They set temperature=1.2, top_k=50, and top_p=1.0. What is the MOST likely effect of this combination?

Which TWO of the following sampling strategies introduce randomness into text generation?

An ML engineer is selecting a pre-trained model for a code generation task. The model must be able to generate syntactically correct code in multiple programming languages. Which model family is BEST suited for this task?

A researcher is evaluating two LLMs for a summarization task. Model A achieves a ROUGE-L score of 0.45 and a BERTScore of 0.92. Model B achieves a ROUGE-L score of 0.50 and a BERTScore of 0.88. Which model is likely better for producing summaries that are semantically faithful to the source, even if not using the exact same words?

A data scientist wants to compare the semantic similarity between two sentences generated by an LLM. Which evaluation metric is most suitable for this purpose?

Which TWO of the following are characteristics of decoder-only models like GPT? (Select TWO)

Which tokenization algorithm is commonly used in models like GPT and BERT and builds tokens by merging the most frequent pairs of characters or subwords iteratively?

A machine learning engineer is designing a RAG pipeline in OCI to improve the accuracy of an LLM-based FAQ bot. Which TWO components are essential for the retrieval phase? (Select TWO.)

A team is evaluating two embedding models for a similarity search task. Model A has a higher BERTScore on a reference dataset. Model B has a lower perplexity on the same dataset. Which model is likely better for retrieval?

Which component of the Transformer architecture allows each token to consider the relevance of every other token in the input sequence?

A data scientist is evaluating an LLM's performance on a summarization task. They observe that the model produces fluent summaries but often misses key information. Which TWO metrics would best capture this issue? (Select TWO.)

A developer notices that an LLM-based question-answering system sometimes provides answers that are correct but from an outdated version of the knowledge base. The system uses RAG with a vector database updated daily. What is the MOST likely root cause?

A developer is building a code generation assistant and needs to ensure the LLM follows a specific output format (e.g., JSON). Which approach is MOST effective for achieving format adherence without retraining?

A developer wants to compare two sentences for semantic similarity using embeddings. Which distance or similarity metric is most commonly used for dense vector representations?

What is the primary purpose of the self-attention mechanism in a transformer model?

A developer is comparing different foundation models for a text completion API on OCI. Which TWO of the following are model families available through OCI Generative AI service? (Choose two.)

A team wants to reduce hallucinations in their LLM-powered question-answering system. Which TWO techniques are most effective?

An organization wants to deploy an LLM for legal document analysis where accuracy is critical, and the model must not reference any external data outside the provided legal corpus. Which approach BEST satisfies these requirements?

Which of the following sampling strategies selects tokens based on a cumulative probability threshold from the highest probability tokens?

A machine learning engineer is evaluating the performance of a translation model using BLEU score. Which THREE statements about BLEU are correct? (Choose three.)

A data scientist is fine-tuning a Llama 2 7B model on a custom dataset using OCI Data Science. After training, the model generates fluent but factually incorrect statements about the new domain. Which post-training technique would BEST address this issue without retraining?

Which component of the Transformer architecture allows the model to weigh the importance of different tokens in the input sequence when generating each output token?