How do you make a large language model answer questions about your company's internal policies without it making things up or needing to be completely retrained? This chapter explains the two main techniques that let you give an AI model new knowledge or new abilities without rebuilding it from scratch—Retrieval-Augmented Generation (RAG) and Fine-Tuning. If you are studying for the 1Z0-1127 exam, understanding these methods is critical because Oracle Cloud Infrastructure Generative AI services rely on them to make models useful in the real world.
Jump to a section
A simple way to picture RAG and Fine-Tuning Techniques
A family recipe book is the central object in a kitchen tasked with feeding a large and opinionated family. The book contains hundreds of tried-and-tested recipes, but it was written by a grandmother who lived in a different time. The family now has a son who is a vegan, a daughter with a nut allergy, and a father who is on a low-sodium diet. The exact recipes in the book do not work for these new needs.
Retrieval-Augmented Generation (RAG) is like the family using a stack of sticky notes to write down the latest dietary restrictions, favourite substitutions, and new cooking tips. When a new recipe is needed, they do not rewrite the whole book. Instead, they search the sticky notes for the relevant information—'Substitute coconut milk for dairy,' 'Use almond flour for wheat'—and read it alongside the original recipe. The result is a meal that follows the spirit of the original but is customised for the present moment. The book itself remains unchanged.
Fine-tuning is a different process. It is like the family deciding to write a new edition of the recipe book. They gather all the sticky notes, the family feedback, and the new cooking techniques, and they rewrite every recipe. The new book now inherently knows that every pasta dish can be made gluten-free and that every dessert should have a no-sugar option. The book has been permanently updated. This is more work up front, but once it is done, the book no longer needs the sticky notes. Every meal from the new book is already adapted to the family's current preferences.
Let us start by understanding the core problem. A large language model (LLM) is a type of artificial intelligence that has been trained on a massive amount of text data from the internet, books, and other sources. This training gives the model a general understanding of language, facts, and how to generate coherent text. However, the model only knows what it learned up to the point it was trained. It does not know your company's unique internal data, recent events, or specialised information that is not publicly available. RAG and fine-tuning are two very different ways to bridge this gap.
Retrieval-Augmented Generation (RAG) is a technique that combines a retrieval system with a generative model. Think of it as giving the model a search engine that it can consult before it answers. Here is how it works step by step:
First, you take your company's documents—policy manuals, product catalogues, support tickets—and you break them into small chunks of text. You then convert each chunk into a mathematical representation called a vector, which captures the meaning of the text. These vectors are stored in a special database called a vector database.
Second, when a user asks a question, the RAG system converts the user's question into a vector as well. It then searches the vector database for chunks whose vectors are most similar to the question vector. This is the retrieval step.
Third, the system takes the retrieved chunks of text and inserts them into the prompt that is sent to the LLM. The prompt now contains both the user's original question and the relevant context from your documents.
Fourth, the LLM generates an answer based on the question and the provided context. Because the context is real, current information, the answer is grounded in facts and less likely to be a hallucination—a made-up answer.
The key advantage of RAG is that you never change the model itself. You only change the database it can search. This means you can update your knowledge base anytime by adding or removing documents. If a new policy is released, you just add the new document to the vector database, and the model will start using it immediately. There is no expensive retraining required.
Fine-tuning is a completely different approach. Instead of giving the model information to look up when it needs to, you permanently change the model's internal weights—the mathematical parameters that determine how it behaves. Fine-tuning is like giving the model additional training on a specific dataset that represents the kind of task you want it to excel at.
There are two main types of fine-tuning: full fine-tuning and Low-Rank Adaptation (LoRA).
Full fine-tuning means you take the pre-trained model and update all of its billions of parameters using your custom dataset. The result is an entirely new model that is a version of the original but specialised for your domain. This is powerful but also very expensive. It requires significant computational resources—often multiple powerful graphics processing units (GPUs) running for days or weeks. It also requires a large, high-quality dataset to avoid overfitting, where the model learns the training data too well and cannot generalise to new inputs.
LoRA is a much more efficient alternative. Instead of updating every parameter in the model, LoRA adds small, trainable modules into the existing model's layers. During fine-tuning, only these small modules are updated. The original model weights remain frozen and unchanged. This dramatically reduces the number of parameters that need to be trained—often by a factor of thousands. As a result, LoRA fine-tuning can be done on a single consumer-grade GPU in a matter of hours. The trained LoRA modules are tiny files—often just a few megabytes—that can be attached to the original model at runtime.
Why would you choose one over the other? RAG is ideal when you have a large, frequently changing knowledge base. It is also simpler to set up and maintain. Fine-tuning is better when you need the model to learn a specific behaviour or style—for example, writing in a certain tone, converting natural language to SQL queries, or following a strict output format. Both techniques can also be used together. A common pattern is to fine-tune a model to improve its ability to reason or follow instructions, then use RAG to supply it with the latest factual data.
For the 1Z0-1127 exam, you need to understand these differences clearly. Oracle Cloud Infrastructure (OCI) offers services that support both RAG and fine-tuning. OCI Data Science and OCI Generative AI service provide tools to build RAG pipelines and to fine-tune models using both full and LoRA methods. Knowing when to use each technique and how they complement each other is exactly what the exam will test.
Prepare the Knowledge Base
Collect all the documents that the AI should know about—policy manuals, product descriptions, support articles. Clean the text and split it into smaller chunks (typically 500–1000 characters each). This chunking step is crucial because large documents are inefficient to search and may exceed the model's context window.
Generate Vector Embeddings
Use an embedding model (e.g., OCI's embedding model or a popular open-source one like text-embedding-ada-002) to convert each text chunk into a vector—a list of numbers that represents the meaning of the chunk. These vectors are stored in a vector database such as OCI PostgreSQL with pgvector or Pinecone. The database indexes these vectors so they can be searched quickly.
Retrieve Relevant Context
When a user submits a query, convert that query into a vector using the same embedding model. The retrieval system searches the vector database for the 'k' most similar vectors (usually 3–10). The corresponding text chunks are retrieved. This step defines how much relevant information the LLM will receive.
Construct the Augmented Prompt
Take the original user query and insert the retrieved text chunks into a specially designed prompt template. The template tells the LLM to use the provided context to answer the question. A well-designed template is critical to getting useful answers and avoiding confusion between the user's query and the inserted context.
Generate the Final Answer
Send the augmented prompt to the LLM (e.g., OCI Generative AI's language model). The LLM processes the prompt, reads the provided context, and generates a response that is grounded in the retrieved facts. The response is returned to the user. No model weights are changed during this step.
Monitor and Iterate
Log the queries, the retrieved contexts, and the generated answers. Analyse cases where the answers were poor—for example, the wrong context was retrieved or the model ignored the context. Use these insights to improve the chunking strategy, the prompt template, or the retrieval parameters. This step ensures the RAG system gets better over time.
An IT professional working for a large bank needs to build a system that allows customer service agents to ask questions about the bank's complex lending policies. These policies change every quarter and contain sensitive information that cannot be shared with a public AI service. The professional must decide whether to use RAG, fine-tuning, or a combination of both.
Here is the step-by-step scenario:
The professional first gathers all the policy documents—over 5,000 pages of text—and securely stores them in OCI Object Storage. The data is sensitive, so all file transfers use encryption.
The professional then builds a RAG pipeline using OCI Generative AI and OCI Data Science. They install an open-source tool called LangChain to orchestrate the pipeline. The documents are split into chunks of 500 characters each, and each chunk is converted into a vector using an OCI embedding model. The vectors are stored in an OCI PostgreSQL database with the pgvector extension.
When an agent asks 'What is the maximum loan-to-value ratio for a residential mortgage with a credit score below 650?', the system converts the question into a vector, searches the database for the five most similar policy chunks, and inserts them into a carefully designed prompt. The LLM then generates an answer that quotes the relevant policy section.
The professional notices that the model sometimes misunderstands the question and retrieves the wrong context. To fix this, the professional fine-tunes a smaller model using LoRA on a dataset of 1,000 example question-answer pairs based on past policy queries. The LoRA adapters are small—only 50MB—and can be loaded alongside the base model without slowing it down.
Now the system works in two stages. The fine-tuned model is better at understanding the intent of the question. The RAG pipeline ensures the model always has the latest policy text to ground its answer in facts. The combination is much more accurate than either method alone.
The IT professional also sets up monitoring using OCI Logging and OCI Monitoring to track how often the models retrieve outdated or irrelevant contexts. They create a feedback loop where agents can rate the quality of answers, and those ratings are used to improve the retrieval pipeline or to create additional training data for future fine-tuning rounds. This is a common real-world pattern: RAG handles the dynamic knowledge, fine-tuning handles the behavioural improvement, and both together deliver a production-ready generative AI system.
The 1Z0-1127 exam asks very specific questions about RAG and fine-tuning. You will not be asked to write code or design a full system. Instead, you will be tested on your conceptual understanding and ability to choose the right technique for a given scenario.
Here are the exact concepts and question patterns to focus on:
Distinguishing between 'what RAG solves' versus 'what fine-tuning solves'. The exam will give you a business scenario—for example, 'A company needs an AI to answer questions about a constantly updating product catalogue'—and ask whether RAG or fine-tuning is more appropriate. The trap is that some candidates think fine-tuning is the only way to add new knowledge. The correct answer is nearly always RAG when the information changes frequently.
Understanding that RAG does not change the model weights. A common question is: 'What happens to the base model after a RAG pipeline is deployed?' The correct answer: nothing. The model stays exactly the same. Only the retrieved context changes.
Knowing the difference between full fine-tuning and LoRA in terms of cost, speed, and output. The exam loves to ask: 'Which fine-tuning method requires the least computational resources?' The answer is LoRA because it only trains a tiny fraction of the parameters.
Recognising when to use fine-tuning. Questions will describe a use case like 'The model must always output JSON in a specific format' or 'The model must use a very formal tone for a legal chatbot.' In these cases, fine-tuning is better because it permanently changes the model's behaviour, whereas RAG only adds temporary context.
Understanding the components of a RAG system. You should be able to identify: the embedding model, vector database, retrieval mechanism, and the LLM itself. The exam might list components and ask you to pick the one that is NOT part of RAG. For example, 'Which of the following is not a component of a RAG pipeline?' The answer might be 'Fine-tuning dataset' if that is listed.
Knowing OCI-specific services. The exam tests which OCI services support RAG and fine-tuning. OCI Generative AI, OCI Data Science, and OCI Database with vector capabilities are the key ones. You do not need to know every configuration detail, but you must recognise their roles.
Traps to watch for:
Confusing 'retrieval' with 'training'. Retrieval searches an existing database; training updates the model. If a question says 'the model learns from the new data', it is describing fine-tuning, not RAG.
Assuming RAG is only for question-answering. The exam may present a scenario like 'Using RAG to generate marketing copy based on a product database'. That is a valid RAG use case. Do not assume RAG is limited to Q&A.
Forgetting that LoRA adapters are separate files that must be loaded with the base model. A question might imply that LoRA creates a standalone model. The correct understanding is that LoRA creates an adapter that attaches to the original model.
Finally, memorise these key definitions:
Token: A piece of text—a word or part of a word—that the model processes.
Parameter: A numerical value inside the model that is adjusted during training. Models have billions of these.
Vector Embedding: A mathematical representation of text as a list of numbers that captures its meaning.
Vector Database: A database designed to store and search vectors efficiently.
Context Window: The amount of text the model can consider at once when generating an answer. RAG helps fill this window with relevant information.
RAG retrieves relevant documents from a database and inserts them into the prompt the model sees, so the model's answers are grounded in real, up-to-date information without changing the model itself.
Fine-tuning permanently updates a model's internal parameters so that the model's behaviour changes even when no external context is provided.
LoRA is a form of fine-tuning that only trains a small set of additional parameters, making it far cheaper and faster than full fine-tuning while still achieving meaningful task specialisation.
Full fine-tuning updates every parameter in a model and requires massive computational resources, but it can produce the most dramatic changes in model behaviour.
RAG is the better choice when your knowledge base changes frequently or is very large and sensitive, because you can just update the database without retraining.
Fine-tuning is the better choice when you need the model to consistently follow a specific output format, tone, or reasoning pattern, and when you have a stable, high-quality training dataset.
RAG and fine-tuning are complementary—you can fine-tune a model to improve its instruction-following and then use RAG to supply the latest facts.
In OCI, RAG pipelines can be built using OCI Generative AI, OCI Data Science, and vector databases like OCI PostgreSQL with pgvector, while fine-tuning uses OCI Data Science with GPU compute.
The model in a RAG system never 'learns' the retrieved context; reading text in a prompt is temporary and has no lasting effect on the model's weights.
LoRA adapters are small files that must be loaded alongside the original base model at inference time; they are not standalone models.
A vector database stores mathematical representations of text chunks and enables efficient similarity search to find the most relevant context for a user query.
Hallucination reduction is a primary reason to use RAG—by providing real context, the model is less likely to invent facts out of thin air.
These come up on the exam all the time. Here's how to tell them apart.
RAG (Retrieval-Augmented Generation)
Does not change model weights; model remains exactly as pre-trained.
Requires a vector database and a retrieval mechanism to find relevant context.
Best for scenarios where information changes frequently or is very large.
Fine-Tuning
Permanently updates model weights; produces a new specialised model.
Requires a training dataset of examples and significant GPU compute resources.
Best for scenarios where you need the model to adopt a consistent behaviour or output format.
Full Fine-Tuning
Updates all parameters in the model; extremely computationally expensive.
Produces a single new model file that can be several gigabytes in size.
Can drastically change the model's behaviour across all tasks.
LoRA (Low-Rank Adaptation)
Only trains a small set of additional parameters (usually 0.1%–1% of total).
Produces a small adapter file (megabytes) that must be loaded with the original model.
Best for task-specific specialisation without changing overall model capabilities.
Vector Database
Optimised for storing and searching vector embeddings based on similarity.
Typically used for semantic search in RAG pipelines.
Does not enforce a fixed schema; vectors are stored as arrays of floats.
Relational Database
Optimised for exact matches, range queries, and joins on structured data.
Not suitable for semantic similarity search due to lack of vector indexes.
Enforces a rigid schema and ACID compliance.
Mistake
RAG and fine-tuning do the same thing: they both teach the model new facts.
Correct
RAG does not teach the model anything new; it provides temporary context that the model reads before answering. Fine-tuning permanently changes the model's weights so it behaves differently even without external context.
Both techniques make the model produce better answers, so beginners assume they are just two ways to do the same job. The 'how' is what matters for the exam.
Mistake
You need to fine-tune a model to use RAG, because the model must be trained to understand the retrieved documents.
Correct
Most modern LLMs are already capable of reading and using context provided in a prompt. No fine-tuning is required for a basic RAG pipeline. The model uses its pre-existing language understanding to incorporate the retrieved text.
People assume that because RAG involves adding new information, the model must be trained on it. But providing text in the prompt is not training; it is just context.
Mistake
Fine-tuning with LoRA produces a completely new and smaller model that can replace the original.
Correct
LoRA produces a small set of adapter weights that must be loaded onto the original base model. The adapter alone is not a functioning model. It is like a custom filter that modifies the base model's output.
Because the LoRA output files are tiny (megabytes versus gigabytes), beginners think they are standalone models. They are not—they are modifications to the original model's layers.
Mistake
RAG is only useful for answering questions based on documents, not for generating creative content.
Correct
RAG can be used to supply any kind of context to a generative model, including product descriptions, style guidelines, or brand voice examples. It is used for summarisation, translation, and content generation as well as Q&A.
The name 'Retrieval-Augmented Generation' and most tutorials focus on question-answering, so learners mentally box RAG into that single use case.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
RAG stands for Retrieval-Augmented Generation. It is a way to give a language model extra information to read before it answers your question, like giving a student a textbook to consult during a test.
No. Most modern large language models can already use context provided in a prompt without any additional training. RAG simply adds relevant text to the prompt, so no fine-tuning is needed unless you want to improve the model's general behaviour.
Full fine-tuning updates every one of the model's billions of internal parameters. LoRA only trains a tiny set of new parameters that are added to the existing model layers. LoRA is much cheaper and faster but is less flexible for drastically changing model behaviour.
Yes, absolutely. A common approach is to fine-tune a model to better understand your domain or follow specific instructions, and then use RAG to provide the latest factual data. The fine-tuned model is better at using the retrieved context effectively.
A vector database stores text as mathematical representations called vectors and can quickly find the vectors that are most similar to a given query. You need it because a regular database is too slow at searching for meaning, and RAG requires fast retrieval of relevant context.
The context window is the maximum amount of text (in tokens) that a model can process at once. RAG fills this window with the most relevant retrieved documents. If the retrieved text plus the query exceeds the window, you must truncate the context, which can lose important information.
You've finished RAG and Fine-Tuning Techniques. Continue through the 1Z0-1127 study guide to build a complete picture of the exam.
Done with this chapter?