Azure OpenAI Service for Developers

Azure OpenAI Service is a cloud-based platform that gives developers access to OpenAI's state-of-the-art generative AI models through a REST API, with the added benefits of Azure's enterprise-grade security, compliance, and scalability. It is not a separate set of models; rather, it is OpenAI's models hosted on Azure infrastructure, providing the same capabilities as OpenAI's API but with Azure's networking, monitoring, and identity integration.

The service supports several model families:

Azure OpenAI Service is deployed within your Azure subscription as a resource. When you create an Azure OpenAI resource, you choose a region (e.g., East US, West Europe) and a pricing tier (Standard or Provisioned). The resource gets an endpoint URL and two API keys (key1 and key2) for authentication, similar to other Azure services.

Under the hood, the service consists of: - Model Deployments: You must deploy a specific model version to an endpoint before you can use it. Each deployment has a name you choose, and you reference it in API calls. Deployments are regional and can be scaled independently. - Content Filtering: Azure applies content filters to all model inputs and outputs to detect harmful content (hate, violence, sexual, self-harm). These filters run before and after the model call. You can configure filter severity levels (low, medium, high) and even turn off filtering for some categories (with approval). - Tokenization: Models process text in tokens — chunks of characters (~4 characters per token for English). API calls have token limits per request (e.g., 4096 tokens for GPT-3.5 Turbo, 8192 or 32768 for GPT-4). You pay per token used. - Rate Limiting: Each deployment has a rate limit measured in tokens per minute (TPM) and requests per minute (RPM). The default for Standard tier is 1M TPM and 1000 RPM, but you can request increases.

Using Azure CLI to create an Azure OpenAI resource:

az cognitiveservices account create \
    --name myOpenAIResource \
    --resource-group myResourceGroup \
    --kind OpenAI \
    --sku S0 \
    --location eastus \
    --yes

To deploy a model:

az cognitiveservices account deployment create \
    --resource-group myResourceGroup \
    --name myOpenAIResource \
    --deployment-name myGpt35Turbo \
    --model-name gpt-35-turbo \
    --model-version 0613 \
    --model-format OpenAI \
    --sku-capacity 1 \
    --sku-name Standard

The --model-name uses a specific format (e.g., gpt-35-turbo not gpt-3.5-turbo). Note the hyphen difference.

To test the endpoint with curl:

curl $AZURE_OPENAI_ENDPOINT/openai/deployments/myGpt35Turbo/chat/completions?api-version=2023-12-01-preview \
  -H "Content-Type: application/json" \
  -H "api-key: $AZURE_OPENAI_KEY" \
  -d '{"messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"Hello"}],"max_tokens":100}'

Azure OpenAI supports fine-tuning on GPT-3.5 Turbo (and some older models). Fine-tuning requires a training dataset in JSONL format with prompts and completions. The process: 1. Upload training and validation files to Azure Blob Storage. 2. Create a fine-tuning job via the API or CLI. 3. The service creates a new model deployment.

Fine-tuning is not available for GPT-4 in all regions and may have additional costs.

Azure OpenAI applies content filters to all inputs and outputs. The filters are configurable per deployment. You can set severity levels (safe, low, medium, high) for categories: hate, sexual, violence, self-harm. Filters can be turned off for some categories with Microsoft approval. If a request or response is filtered, the API returns a 400 status with an error message. The exam may test that filtered requests still incur token charges (the input is processed before filtering).

Common HTTP status codes: - 400: Bad request (e.g., invalid parameters, content filtered). - 401: Unauthorized (invalid or missing API key). - 429: Rate limit exceeded. Retry after Retry-After header. - 500: Internal server error.

The SDKs provide automatic retries for 429 and 500 errors.

For predictable performance, you can purchase Provisioned Throughput Units (PTUs). This reserves model capacity for your deployment, ensuring low latency even under high load. It is billed hourly regardless of usage. The exam may contrast this with Standard (pay-per-token) tier.

Azure OpenAI Service can be secured using: - Private Endpoints: Access the service from a virtual network without exposing it to the internet. - IP Firewall: Restrict access to specific IP addresses. - Disable Local Authentication: Require Azure AD tokens instead of API keys.

Use Azure Monitor to track: - Azure OpenAI Requests: Count, latency, token usage. - Content Filtering: Number of blocked requests. - Fine-tuning: Status of jobs.

You can also enable diagnostic settings to send logs to Log Analytics, Storage, or Event Hubs.