Fine-tuning and inference are the two core actions that turn a generic, pre-trained AI model into a useful, specialised tool for a specific business task. For the 1Z0-1127 exam, understanding this process is critical because Oracle Cloud Infrastructure (OCI) Generative AI service provides the tools to build and deploy these custom models without needing a PhD in machine learning.
Jump to a section
A simple way to picture Hands-On Lab: Fine-Tuning and Inference
A base model of a large language model is like a new personal trainer who has read every textbook about exercise science. That base trainer knows the theory of thousands of exercises, nutrition plans, and recovery protocols. They can explain the theory behind a proper deadlift or a balanced meal plan. However, this trainer has never worked at your specific gym. They do not know your personal fitness goals, your past injuries, or your available equipment.
Fine-tuning this model is like putting that trainer through a specialised apprenticeship at your local gym. You give the trainer your past workout logs, your medical history, your goals, and the specific list of equipment your gym actually has. During this apprenticeship, the trainer adjusts their general knowledge to your specific reality. They learn that you hate running but love swimming. They learn that your left knee has a weakness. The trainer’s broad fitness knowledge gets shaped into a tailored coaching plan. This is the fine-tuning process.
Inference is what happens when you then ask that personalised trainer, "What should I do today?" The trainer does not go back to re-read all their textbooks. Instead, they instantly apply their newly specialised knowledge to give you a precise, relevant answer: a warm-up for your knee, three sets of seated rows using the cable machine, and a post-workout stretch for your hamstrings. The trainer does not generate new general principles; they apply the fine-tuned principles to your specific query in real-time.
To understand fine-tuning and inference, you first need to understand what a 'base model' or a 'pre-trained model' is. A base model, like Cohere Command or Llama on OCI, has been trained on an enormous amount of publicly available text from the internet—books, articles, code, and websites. This training process takes months and costs millions of dollars. The result is a model that understands grammar, facts, reasoning patterns, and many languages. It is incredibly broad but not specialised.
Now imagine you are a bank that wants a model to summarise loan application documents. If you use the base model as-is, it might summarise a loan document in the style of a Wikipedia article or a Shakespearean play. It lacks the specific knowledge of your bank's terminology, processes, and formatting requirements. This is where fine-tuning comes in.
Fine-tuning is a supervised learning process. You take the pre-trained base model and continue training it, but this time you use a much smaller, highly curated dataset that is specific to your task. For the bank, this dataset would be hundreds or thousands of pairs of (raw loan document, ideal summary). During fine-tuning, the model's internal weights—the mathematical parameters that represent its knowledge—are slightly adjusted. The model learns to pay attention to the specific patterns in your data. It learns that 'DTI' means 'debt-to-income ratio' in your context, not 'Department of Transportation'. It learns that your summaries must always start with a risk rating. This process is much cheaper and faster than pre-training from scratch.
On OCI Generative AI, you can perform fine-tuning through the console, the CLI (Command Line Interface), or the API (Application Programming Interface). You prepare your data, typically in JSONL format (a series of JSON objects, one per line), upload it to OCI Object Storage, and then create a 'dedicated AI cluster'—a group of computing resources (GPUs) reserved just for your job. The service then runs the fine-tuning job. You can monitor its progress. Once complete, you get a 'fine-tuned model endpoint'—a custom URL where your refined model lives, ready to be used.
Inference is the step after fine-tuning. It is the act of sending a prompt—a query or input—to your fine-tuned model and receiving a response. The model does not learn during inference; it applies what it learned during training and fine-tuning. In our bank example, you send a long loan document as the prompt, and the model returns a structured summary. The key performance metrics for inference are latency (how fast the response comes back) and throughput (how many requests you can handle per minute). OCI offers different serving shapes for inference, allowing you to choose a configuration that balances speed and cost.
Why does this matter for 1Z0-1127? The exam tests your understanding of the entire lifecycle: preparing data for fine-tuning, choosing the right base model, launching a fine-tuning job, and then deploying the model for inference. You do not need to write the code from scratch, but you must understand the sequence of steps, the required resources (like a dedicated AI cluster), and the output you get (a model endpoint).
Prepare the Training Data
You must gather and format your labelled data. Each example consists of a prompt and a completion. The data is saved as a JSONL file (one JSON object per line). This data is stored in OCI Object Storage. This step is critical because the model learns directly from this data; poor quality data leads to a poor fine-tuned model.
Create a Dedicated AI Cluster
Fine-tuning requires compute power, specifically GPUs. In OCI, you must create or select an existing 'dedicated AI cluster'. This is a reserved pool of compute resources that will be dedicated solely to your fine-tuning job. You cannot run fine-tuning on the default serverless infrastructure meant for inference.
Configure and Launch the Fine-tuning Job
In the OCI console, you navigate to the Generative AI service, choose 'Fine-tuning', and fill in the job details. You select the base model (e.g., cohere.command), point to your dataset in Object Storage, and select your dedicated AI cluster. You then launch the job. OCI handles the orchestration of the training process.
Monitor the Training Job
While the job is running, you can monitor metrics like 'training loss' in the console. A decreasing loss is a good sign that the model is learning. If the loss plateaus or increases, it indicates a problem with the data or configuration. You may choose to stop the job and correct the data.
Deploy the Fine-Tuned Model to an Endpoint
Once the fine-tuning job succeeds, OCI creates a new model resource. To use this model for inference, you must create an 'endpoint' from it. An endpoint is essentially the model plus the compute capacity to serve it. You configure the endpoint's scaling and access settings. This creates a URL you can send requests to.
Perform Inference via the Endpoint
With the endpoint active, you can now send prompts to it using an API call or the OCI console's playground. The model processes each prompt and returns a response. This is the inference step. The model does not learn from these new inputs; it only applies what it learned during fine-tuning.
Consider a legal firm, 'LexPredict', that uses OCI Generative AI to automate the review of non-disclosure agreements (NDAs). The base model is good at language but cannot tell a favourable clause from an unfavourable one in a legal context. An IT professional at LexPredict needs to fix this.
Here is the exact step-by-step scenario an IT professional would perform:
The professional defines the task. They want the model to read the text of an NDA and output a JSON object with three fields: 'risk_score' (high, medium, low), 'key_issue' (a one-sentence summary), and 'suggested_revision'.
They prepare the dataset. They gather 500 historical NDAs from their company's database. For each NDA, they manually—or with a small script—create the correct JSON output. This becomes their training data. They split this data into two parts: a training set and a validation set (to check the model's progress). The data is saved as a JSONL file and uploaded to an OCI Object Storage bucket.
They log into the OCI Console. They navigate to the Generative AI service, click on 'Fine-tuning', and then 'Create fine-tuning job'. They choose a base model (for example, cohere.command). They point the job to their training data in Object Storage. They also specify a 'dedicated AI cluster' of a certain size—say, one GPU shape for a small job.
They start the job. OCI provisions the cluster, loads the base model, and begins the training. The professional can monitor the loss metric—a number that shows how well the model is learning. A decreasing loss is good. If the loss plateaus or grows, something is wrong with the data.
Once the job finishes, OCI creates a new model resource: the fine-tuned model. The professional now creates an 'endpoint' from this model. They configure it for production use, ensuring the right amount of compute power is attached for inference.
The professional integrates this endpoint into their existing application via an API call. Now, when a paralegal uploads a new NDA into their document management system, it instantly gets analysed by the fine-tuned model. The output appears in the paralegal's dashboard within seconds.
The biggest real-world challenge is data quality. An IT professional spends more time cleaning and labelling data than running the actual fine-tuning job. The exam expects you to know that garbage in equals garbage out for fine-tuning. Another challenge is cost. Leaving an inference endpoint running 24/7 for no traffic is wasteful. Professionals learn to manage these resources, sometimes using auto-scaling configurations.
The 1Z0-1127 exam focuses on high-level concepts and the OCI-specific workflow, not on deep algorithm details. You will not be asked to calculate gradients or loss functions. You will be asked to identify the correct step in the process, the right data format, or the correct resource required.
Here are the exact concepts the exam loves to test:
Data format for fine-tuning: The exam will ask about the structure of the training data. The correct answer is always JSONL (JSON Lines). Trap options might include CSV, plain text files, or a single JSON object. Memorise: fine-tuning data must be one JSON object per line, with 'prompt' and 'completion' or 'messages' fields depending on the model.
The role of the dedicated AI cluster: The exam tests whether you know that a dedicated AI cluster is required for fine-tuning. You cannot fine-tune on a 'serverless' or 'shared' cluster. A trap question might describe fine-tuning as a process that happens instantly or without dedicated resources. The correct answer always involves reserving compute.
Difference between fine-tuning and pre-training: Pre-training happens only by the model provider (like Cohere or Meta). It is massive and expensive. Fine-tuning happens on OCI by the customer, using the customer's data. The exam will have questions mixing these two up.
Inference endpoint types: The exam tests the difference between a 'dedicated' endpoint (always on, higher cost, consistent performance) and a 'serverless' or 'on-demand' endpoint (pay per invocation, lower cost, may have cold start delays). You must know which is appropriate for which use case.
Monitoring fine-tuning jobs: You need to know that you monitor the 'training loss' to see if the model is learning. A decreasing loss is good. A non-decreasing loss means a problem. The exam may ask what to do if training loss is flat: the answer is to check the quality of your training data or increase the number of training epochs.
Specific trap patterns:
A question that says 'Which step is NOT required for fine-tuning?' and lists 'prepare data', 'create cluster', 'build a new base model from scratch'. The trap is 'build a new base model from scratch'—this is pre-training, not fine-tuning.
A question that asks about inference and lists 'updating model weights' as an option. This is wrong. Inference never updates weights. Only training updates weights.
A question that confuses 'endpoint' with 'model'. An endpoint is the URL and compute behind a model. You can have multiple endpoints for the same model.
Key definitions to memorise:
Fine-tuning: Supervised learning on a pre-trained model using your own labelled dataset.
Inference: Using a trained model to generate a response from a new prompt, without learning.
Dedicated AI cluster: Reserved GPU compute required to run a fine-tuning job.
JSONL: The required data format for fine-tuning on OCI Generative AI.
Fine-tuning adjusts a pre-trained base model using your own labelled data to make it specialised for a specific task.
Inference is the act of sending a prompt to a trained or fine-tuned model and receiving a generated response, without any learning occurring.
On OCI, fine-tuning requires a dedicated AI cluster (reserved GPU compute) and training data in JSONL format.
A fine-tuned model must be deployed to an endpoint before it can be used for inference.
Monitoring the training loss metric during fine-tuning tells you whether the model is learning effectively from your data.
Fine-tuning is much faster and cheaper than pre-training a model from scratch, but it still requires careful data preparation and management of compute resources.
The quality of your fine-tuning dataset directly determines the quality of your fine-tuned model: garbage in equals garbage out.
An inference endpoint can be configured as dedicated (always on) or serverless (pay-per-use), each suited for different traffic patterns.
These come up on the exam all the time. Here's how to tell them apart.
Fine-tuning
Permanent adjustment of model weights using training data
Requires a dedicated AI cluster with GPUs
Generates a new model resource on OCI
Inference
No change to model weights; only uses existing weights
Runs on an endpoint which can be serverless or dedicated
Returns a response to a given prompt without creating a new model
Dedicated AI Cluster
Used exclusively for training (fine-tuning) jobs
Reserved compute resources billed for the duration of the job
Cannot serve inference requests while training is active
Inference Endpoint
Used exclusively for serving inference requests
Compute resources can be dedicated or serverless; billed per invocation or per hour
Does not support training or fine-tuning tasks
Pre-training
Builds a model from scratch on massive public datasets
Takes months and costs millions of dollars
Performed by the model provider (e.g., Cohere, Meta)
Fine-tuning
Adjusts an existing pre-trained model on a small, specific dataset
Takes hours or days and costs far less
Performed by the customer on OCI using their own data
JSONL Data Format
One JSON object per line, each representing a prompt-completion pair
Standard format for fine-tuning on OCI Generative AI
Supports nested structures and variable-length fields natively
CSV Data Format
Rows and columns of data, best for tabular data
Not supported for fine-tuning on OCI Generative AI
Cannot represent nested prompt-completion structures easily
Mistake
Fine-tuning a model creates a completely new and original AI from nothing.
Correct
Fine-tuning starts with a pre-trained base model and makes small adjustments to its weights using your specific data. It does not build a model from scratch.
The name 'fine-tuning' sounds like you are building something from the ground up, but in machine learning it specifically means adjusting an existing, trained model.
Mistake
You need to be an expert programmer to fine-tune a model on OCI.
Correct
Oracle provides a user-friendly console where you can upload data, configure a job, and launch it with clicks or simple API calls. Deep coding knowledge is not required for the basic workflow tested in the exam.
Many beginners equate AI with complex programming, but cloud providers like OCI abstract the complexity. The exam tests your understanding of the process, not your Python skills.
Mistake
Fine-tuning and inference use the same computational resources at the same time.
Correct
Fine-tuning is a resource-intensive training process that uses a dedicated AI cluster for a period of time. Inference is the ongoing usage of the resulting model, which uses a different endpoint configuration, often with different scaling requirements.
People new to cloud services often think of 'running the model' as a single step, not realising that training and serving are separate phases with different costs and infrastructure needs.
Mistake
If a base model works well for some tasks, you can fine-tune it with just one or two examples.
Correct
Fine-tuning typically requires hundreds to thousands of high-quality labelled examples to be effective. One or two examples is a technique called 'few-shot prompting', which is not fine-tuning.
The word 'fine' suggests a small adjustment, leading beginners to think a tiny amount of data will suffice. In reality, the adjustment requires a statistically significant dataset to avoid overfitting.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
Fine-tuning permanently changes the model's weights by training it on many examples, making the behaviour consistent and baked-in. Prompting with examples (a technique called few-shot learning) only guides the model temporarily and uses up prompt context space; it does not change the model itself.
There is no strict minimum, but best practice suggests at least a few hundred high-quality examples. For the exam, remember that more diverse and relevant data leads to better results. Using only 5-10 examples is generally insufficient for meaningful fine-tuning.
You can perform the entire fine-tuning process through the OCI web console using point-and-click steps. You do not need to write Python code for the basic workflow. OCI also provides APIs and SDKs for automation, but the console is the primary tested method for the exam.
OCI will report an error message with details. Common failure reasons include incorrect data formatting (not valid JSONL), insufficient compute resources, or corrupted data files. You must fix the issue and resubmit the job. The dedicated AI cluster resources you reserved are still billed for the time used.
Yes. The fine-tuned model and its endpoint are created within your OCI tenancy and are not shared with other customers. Your data used for fine-tuning remains in your tenancy as well. This is a key security and compliance feature for enterprises.
Not necessarily. You can periodically retrain the model when you have a significant new batch of labelled data. For continuous learning from live data, you would typically use a process called 'online learning' or set up a schedule for periodic fine-tuning jobs, which is more advanced than the basic workflow tested in the exam.
You've finished Hands-On Lab: Fine-Tuning and Inference. Continue through the 1Z0-1127 study guide to build a complete picture of the exam.
Done with this chapter?