How do you get a large language model to answer questions about your company's internal policies without retraining it from scratch? That's the core problem RAG (Retrieval-Augmented Generation) solves, and the 'Hands-On Lab: RAG Deployment' is where you learn to set it up on OCI. Mastering this is crucial for the 1Z0-1127 exam because it tests your practical ability to connect OCI Generative AI to OCI Search, creating a system that can fetch and summarise information in real time.
Jump to a section
A simple way to picture Hands-On Lab: RAG Deployment
A kitchen that has never been renovated, with old cabinets, a cracked tile backsplash, and a fridge that hums louder than it cools. You want to replace the old cabinets with custom-built ones, but you don't know the exact measurements, the style you want, or the type of wood that's available. Your friend, who has a catalog of thousands of cabinet designs, can help you choose, but they need to see your kitchen first.
In this scenario, your kitchen is the user's original question or prompt, and the friend with the catalog is the large language model (LLM) trained on a huge amount of general data. But the friend doesn't know about your specific kitchen measurements or the local hardware store's stock. So, before your friend can suggest a design, you take a few photos of your kitchen, note the measurements, and check what the local store has in stock. You then take those notes and use them to refine your request to your friend: 'Give me a cabinet design based on these exact dimensions, this style, and using wood from this specific store.' The friend then gives you a perfect, custom design.
This is exactly what a RAG (Retrieval-Augmented Generation) deployment does. The initial request (the prompt) is the raw question. The OCI Search service is the process of taking photos and checking the store's inventory – it searches your private data or a specific knowledge base for the most relevant pieces of information. The OCI Generative AI service is the friend, who uses that retrieved information as a guide to produce a final, accurate, and context-specific answer. Without the retrieval step (the photos and store check), the friend would just give you a generic design that doesn't fit your kitchen. RAG ensures the answer fits the user's specific 'kitchen'.
RAG stands for Retrieval-Augmented Generation. It’s a technique used to improve the answers given by a large language model (LLM). Think of an LLM as a very smart but closed book. It knows a lot about the world up to the date it was trained, but it doesn't know anything about your private documents, recent news, or your specific company database. If you ask it a question about your company's expense policy, it might guess or make something up (a hallucination) because it has no information about that policy.
RAG solves this by giving the LLM a specific set of notes to read before it answers. The process works in three main stages: Ingestion, Retrieval, and Generation.
First is Ingestion. This is where you prepare your data. You take all your documents - PDFs, Word files, Confluence pages, database records - and break them down into smaller chunks. These chunks are then converted into a special numeric format called a vector embedding. An embedding is like a unique fingerprint for the meaning of that text. These vector embeddings are stored in a vector database (part of OCI Search). This whole process happens once, or is updated periodically when the documents change.
Second is Retrieval. When a user types a query, like 'What is the expense policy for flights?', that query is also converted into a vector embedding. The system then performs a similarity search inside the vector database. It finds the top 3, 5, or 10 chunks of text whose meaning is closest to the user's question. This is much faster and more accurate than a keyword search (like searching for the exact words 'expense' and 'policy').
Third is Generation. The original user query is combined with the retrieved chunks of text into a single, structured prompt. This prompt is sent to the LLM (OCI Generative AI). The LLM is instructed: 'Based on the following context, answer the user's question.' Because the LLM now has the relevant, factual context, it can generate a precise, accurate answer without hallucinating.
Why does this matter for the exam? The 1Z0-1127 exam specifically tests your ability to configure and debug this pipeline on OCI. You need to understand which OCI services are involved, how the data flows between them, and the key parameters you can set.
The key components in an OCI RAG deployment include:
OCI Generative AI: The service that provides the LLM for generating the final answer and, sometimes, for creating the vector embeddings during ingestion.
OCI Search with OpenSearch: The service that acts as the vector database and performs the similarity search. It stores the document chunks and their embeddings.
An AI Agent (like OCI Generative AI Agent): An orchestration service that manages the overall flow. It takes the query, calls OCI Search, retrieves the context, and then calls OCI Generative AI.
Data Sources: Where your original documents live (e.g., Object Storage, a database).
This system replaces the older approach of fine-tuning, where you would retrain an LLM on your specific documents, which is expensive, time-consuming, and requires deep ML expertise. RAG is cheaper, faster to deploy, easier to update, and gives the LLM access to information that might not have existed during its original training. For the exam, remember that RAG is the practical, enterprise-ready way to ground an LLM in your proprietary data.
Prepare Your Data Source
You must first organise your documents in a location OCI can access, such as an Object Storage bucket. This step is critical because the quality of your final answers depends on the quality and relevance of the source documents. The system will read everything in the bucket, so ensure only useful, accurate documents are stored there.
Create an AI Agent in OCI Console
Navigate to the OCI Console and create a new Generative AI Agent. This agent acts as the brain of the operation. You will configure it with a specific LLM (like Cohere Command R) and define a welcome message and instruction prompt. This is the entry point that will receive user queries.
Create and Link a Data Source
Inside the AI Agent configuration, create a new Data Source. You will point it to your Object Storage bucket. The system will then automatically run the ingestion process. This step is where the magic of vectorisation happens—your documents are chunked, embedded, and indexed in OCI Search.
Test the Agent with Sample Queries
Use the built-in test console to ask questions related to your documents. Observe the generated answer and check if it references the correct information. This step is vital for debugging. If the answer is wrong, you might need to adjust the chunk size, the number of chunks retrieved, or the instruction prompt.
Deploy the Agent via an Endpoint
After testing, deploy the agent. OCI will generate a REST API endpoint. You can then integrate this endpoint into your front-end application (like a chat widget, a support ticketing system, or a web app) so real users can interact with the RAG-powered assistant.
A company called 'Acme Corp' has an internal knowledge base containing thousands of technical documents, onboarding guides, and product manuals. Their customer support team spends hours searching through this knowledge base to answer customer queries. They decide to implement a RAG solution using OCI Generative AI and OCI Search.
An IT professional, let's call her Priya, is responsible for deploying this solution. Here is exactly what she does, step by step:
Priya first identifies all the data sources. She has a bucket in OCI Object Storage containing the latest PDF versions of all product manuals. She also has a PostgreSQL database with frequently asked questions and their answers. She decides to start with the Object Storage bucket.
She navigates to the OCI Console and creates an AI Agent called 'Acme-Support-Agent'. This agent will orchestrate the RAG flow. She configures it to use a specific OCI Generative AI model (like 'cohere.command-r-16k').
Next, she creates a 'Data Source' within the AI Agent configuration. She points it to the Object Storage bucket. The ingestion process begins. OCI Search reads each PDF, breaks it into chunks of 500 tokens (a token is roughly 3/4 of a word), and creates vector embeddings for each chunk. These are stored in a vector index.
Priya tests the agent by asking a query in the console: 'How do I reset the password on the Model X1000 printer?' The AI Agent retrieves the top 3 relevant chunks from the vector index, combines them with the query, and sends them to the LLM. The LLM generates a step-by-step answer that correctly references the manual.
She then integrates this agent into the company's ticketing system (like ServiceNow) via REST APIs. When a support agent types a query, it is sent to the OCI AI Agent endpoint, and the generated answer is returned to the support agent's screen within 2-3 seconds.
The real-world benefit is massive. The support team's average resolution time drops from 15 minutes to 2 minutes. The system is also easy to update: when a new product manual is published, Priya simply uploads the PDF to the Object Storage bucket, and she can trigger a manual re-index or set up a schedule to automatically refresh the data source. The IT professional's job is not just to set it up, but to monitor the logs, fine-tune the chunk size (larger chunks for broader context, smaller chunks for precision), and adjust the prompt template to ensure the agent always cites its sources and says 'I don't know' when it doesn't have a relevant document.
The 1Z0-1127 exam focuses heavily on the practical implementation steps of a RAG solution. Do not just memorise the theory—you need to understand the exact sequence of operations and the configuration parameters.
Here are the specific topics and traps to watch out for:
The exam loves to test your understanding of the Ingestion vs. Retrieval phase. A common question will describe an action (e.g., 'converting a document into vector embeddings') and ask you whether this happens during ingestion or retrieval. The answer is almost always ingestion.
They will frequently ask about the role of OCI Search with OpenSearch. Remember: it is not the LLM itself, nor is it just a keyword search engine. Its specific role in RAG is to act as a vector database for storing and performing similarity searches on embeddings.
Expect scenario-based questions where you must choose the correct order of API calls or service configurations to implement a RAG workflow. The correct path is: Ingest Data into OCI Search -> Create AI Agent pointing to the Data Source -> User Query -> Retrieve from OCI Search -> Pass context to OCI Generative AI -> Return Response.
A major trap is confusing RAG with Fine-tuning. The exam will present a scenario where a user wants to incorporate frequently changing data (like stock prices). The correct answer is RAG, because fine-tuning is static and expensive to retrain. Fine-tuning is better for adapting the model's tone or style, not for giving it new facts.
Be aware of chunking parameters. The exam might not ask for a specific number, but it will test the concept. They might ask: 'What is the impact of using very large chunks?' The correct answer is that it provides more context but can include irrelevant information and slow down retrieval. Small chunks are more precise but might lack necessary context.
Token limits are another key area. They will test you on what happens if the retrieved context exceeds the LLM's maximum token input limit. The answer is that you must truncate the context or reduce the number of chunks retrieved.
Finally, expect questions about data connectivity. How does the AI Agent access the data? The answer involves IAM policies and resource principals, allowing the agent to securely read from Object Storage without storing a user's password.
The correct answer pattern for RAG questions is almost always the option that explicitly separates the retrieval step from the generation step. Any option that suggests the LLM directly queries the database (without a retrieval step) is wrong. Also, avoid any answer that implies the documents are used to retrain the model.
RAG combines a retrieval step (searching a knowledge base) with a generation step (using an LLM) to produce accurate, grounded answers.
The ingestion phase converts documents into vector embeddings and stores them in a vector database like OCI Search with OpenSearch.
The retrieval phase converts the user's query into an embedding and finds the most semantically similar chunks in the vector database.
The generation phase sends the user's query plus the retrieved chunks to an LLM, which then produces an answer based on that context.
RAG is preferable to fine-tuning when you need to incorporate frequently changing or proprietary information without retraining the model.
On OCI, the AI Agent service orchestrates the RAG workflow, connecting your data sources, the search index, and the generative AI model.
These come up on the exam all the time. Here's how to tell them apart.
RAG (Retrieval-Augmented Generation)
Retrieves information from an external database when a query is made.
The base LLM is not modified; it is used 'as is'.
Costs less for initial setup and is cheaper to update with new data.
Fine-tuning
Trains the LLM on new data to permanently change its parameters.
Creates a new version of the model with the data baked in.
Expensive to retrain and update; requires GPU compute and ML expertise.
OCI Search with OpenSearch (Vector Database Role)
Uses vector embeddings to find semantically similar content.
Requires an ingestion step to pre-compute embeddings.
Handles synonyms and context well ('car' matches 'automobile').
OCI Search with OpenSearch (Keyword Search Role)
Matches exact keywords and phrases in the document.
No upfront processing needed; searches raw text indices.
Fails to find 'automobile' if the search term is 'car'.
AI Agent (Orchestrator)
Manages the flow: receives query, calls search, returns result.
Does not generate text; it orchestrates other services.
Configures data sources, chunk size, and prompt templates.
LLM (Generative Model)
Generates the final human-readable answer based on context.
Is called by the agent; does not know about data sources or search.
Is selected by type (e.g., Command R, Llama) and token limit.
Mistake
RAG replaces the need for any large language model, so you just use a search engine.
Correct
RAG does not replace the LLM; it enhances it. The LLM still generates the final answer, but it does so based on the specific context retrieved by the search component. Without the LLM, you just have a search result, not a synthesised answer.
Beginners often think RAG is just 'search but better,' not understanding the generative AI component is essential for synthesising the retrieved information into a human-readable answer.
Mistake
The vector database stores the original documents.
Correct
The vector database stores the mathematical vector embeddings and a pointer (reference) to the original document chunk. It does not store the full text of the document; the embeddings are just numbers that represent the meaning of the text.
This mistake is common because the term 'database' implies storing data. It's easier to imagine storing the whole document, but the efficiency of vector search depends on these compact numeric representations.
Mistake
You must fine-tune the LLM on your data before you can use RAG.
Correct
You do not need to fine-tune the LLM at all for RAG. The base LLM is used 'as is.' The retrieval step provides the necessary context, so the model does not need to have the information baked into its parameters.
Many beginners come from a machine learning background where fine-tuning is the standard way to incorporate new data. RAG is a completely different paradigm that avoids the cost and complexity of fine-tuning.
Mistake
RAG only works with text documents and cannot handle images or tables.
Correct
Modern RAG systems can handle images and tables by using multimodal embeddings or by converting the content (e.g., extracting text from a table using OCR) before ingestion. The OCI Search service can be configured to extract text from PDFs and other formatted files.
This misconception stems from the fact that many early RAG demos only used plain text. Beginners don't realise that document parsing and preprocessing are powerful steps in the ingestion pipeline.
Mistake
If the LLM gives a wrong answer, the problem is always with the LLM.
Correct
The problem is often with the retrieval step. The wrong chunks may have been retrieved, or the chunks might be irrelevant, too short, or too long. The LLM is highly dependent on the quality of the context it receives.
It's a natural instinct to blame the 'AI' when it fails. But in a RAG system, the retrieval pipeline (chunking, embedding, indexing) is a much larger source of errors than the LLM generation itself.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
A regular database (like MySQL) stores exact data and finds matches based on precise values. A vector database stores mathematical representations of data (embeddings) and finds matches based on semantic similarity, making it perfect for finding relevant information even if the exact words don't match.
Not necessarily. The OCI Console provides a graphical interface to create an AI Agent, configure data sources, and test queries. However, for production deployment and API integration, some basic scripting knowledge (like Python or cURL) is helpful.
During ingestion, the document is automatically split into smaller, manageable chunks (usually 500-1000 tokens). This prevents one document from overwhelming the context window and allows the system to retrieve only the most relevant section of a long document.
Yes. OCI Search supports incremental updates. You can either trigger a manual refresh of the data source or set up a schedule to automatically re-index the bucket, ensuring the agent always uses the latest documents.
Yes, because you are paying for both the LLM calls and the compute and storage costs of OCI Search. However, it is significantly cheaper and faster than fine-tuning, and it produces far more accurate results in scenarios requiring private knowledge.
An ingestion job is the automated process that reads your data sources and creates the vector index. You can monitor it in the OCI Console under the AI Agent's Data Source section, where you can see the status (succeeded, failed, in progress) and the number of documents processed.
You've finished Hands-On Lab: RAG Deployment. Continue through the 1Z0-1127 study guide to build a complete picture of the exam.
Done with this chapter?