Generative AI: Model Selection and Deployment on Azure

Let's start with the basics. Generative AI is a type of artificial intelligence that creates new content — text, images, code, music — rather than just classifying or predicting things. Think of it as an extremely fast, pattern-matching mimic. It learns from millions of examples (called training data) and then uses that knowledge to generate brand-new outputs that look like the originals.

A model is the mathematical brain that does this work. It's a huge file containing billions of numbers (parameters) that represent the patterns learned during training. The most famous generative AI models today are large language models (LLMs) like GPT-4, Claude, or Llama. They are called 'language models' because they are trained on text, but they can be fine-tuned for images, code, or audio.

Now, why can't you just download any model and plug it into your business? Because models come in different sizes, speeds, costs, and specialties. Selecting the right one is like choosing between a bicycle, a family car, and a cement truck for your commute. Each does a different job. Azure hosts dozens of models through a service called Azure AI Studio — it's a central marketplace where you can browse, test, and deploy models.

What are the key factors when selecting a model? - Task type: What do you want the model to do? Summarise documents? Generate images? Answer customer questions? Write code? Different models are optimised for different tasks. - Performance: How accurate and coherent must the output be? A model that writes marketing copy might need high creativity; a model that answers medical questions must be extremely factual and safe. - Cost: Models are often billed per token. A token is a chunk of text (roughly 3/4 of a word). More capable models cost more per token. Running a GPT-4 level model for a small internal tool might bankrupt your project. - Latency: How fast do you need the answer? Real-time customer chat needs sub-second responses, which smaller models provide. A background document analysis can wait 10 seconds. - Data privacy: Do you need the model to run inside your own virtual network, or is it acceptable to send data to a shared Azure endpoint? Some industries (finance, healthcare) require models deployed within a private environment. - Customisation: Can you fine-tune the model on your own data? Foundation models are general-purpose; you may need to train them on your company's documents to make them useful.

Deployment is the process of making your selected model available for use by applications. In Azure, you have three main deployment options.

First, Model-as-a-Service (MaaS) through a serverless API endpoint. This is the simplest option: you call a REST API, send your input, and get back an output. Azure handles all the infrastructure. It's perfect for prototyping and low-volume use. You pay per token used, with no upfront cost.

Second, managed compute deployments. Here you reserve dedicated virtual machines (VMs) or GPU clusters to run the model. You pay for the compute even when idle, but you get predictable performance, low latency, and full control over the model's runtime. This is for high-volume, production-critical workloads.

Third, you can deploy your own containerised model using Azure Kubernetes Service (AKS) or Azure Container Instances. This is for scenarios where you have built a custom model or need extreme customisation.

Once deployed, you must manage the model's lifecycle. That means monitoring performance, updating the model when a new version is released, rolling back if it degrades, and scaling up or down based on demand. Azure AI Studio provides tools for all of this: you can track metrics, set up alerts, and create deployment slots for safe updates.

The key principle is this: selection and deployment are two halves of a single decision. Picking a model that is too large for your latency budget is as bad as deploying a perfect model on infrastructure that costs more than your entire IT budget. You must balance both together.

Imagine you work at a mid-sized UK insurance company called 'SafeGuard Insurance Ltd'. The CEO wants a chatbot that can answer customer questions about policy documents in real-time, across web and mobile. The chatbot must be accurate to avoid legal liability, respond in under 3 seconds, and process 10,000 conversations per day. Your IT team knows nothing about running AI infrastructure.

Your first task is model selection. You go to Azure AI Studio and browse the model catalogue. You see small models like Phi-3 (fast, cheap, but less accurate) and large models like GPT-4o (extremely accurate but slower and costly per token). Given the legal risk, you cannot afford hallucination. You choose GPT-4o, but you are worried about latency.

You then test the model with sample insurance questions. Using the playground in AI Studio, you send 50 different policy queries. The model answers well, but responses take 4 seconds each — too slow for the 3-second target. You switch to GPT-4 Turbo, a slightly smaller model, and latency drops to 2.5 seconds with minimal accuracy loss. That is your selection.

Now deployment. You have three options on Azure. Given your team is not operations-savvy, you choose Model-as-a-Service (serverless). You create an API endpoint in AI Studio, configure it with a content filter to block toxic or off-topic questions, set up a rate limit of 100 requests per minute per user, and enable logging to capture all responses for compliance. The whole setup takes two hours.

You then build the chatbot. Your front-end developer writes a simple function that calls the GPT-4 Turbo API with the user's question and the insurance policy context. You test it. Everything works.

But after one week, usage spikes because a competitor announced a price rise and customers flood your chatbot. The serverless endpoint handles it automatically — Azure scales up. However, your monthly bill shoots from £200 to £2,800. You see the cost in Azure Cost Management and decide to switch to a provisioned throughput unit model. You purchase 10 PTUs (provisioned throughput units) for GPT-4 Turbo, paying a flat monthly fee of £1,500. This caps your maximum cost and provides reserved capacity. You redeploy, and now your chatbot runs predictably and economically.

What did you do as an IT professional? You evaluated models, tested latency, chose an appropriate deployment model, configured content safety, monitored cost, and optimised pricing. That is the everyday reality of model selection and deployment on Azure.