Platform Engineering on Azure

Platform engineering is the practice of building and maintaining an internal developer platform (IDP) that provides a curated set of tools, services, and workflows for application teams. It emerged to solve the friction between development speed and operational stability. In traditional setups, developers waste time on infrastructure configuration, security compliance, and deployment pipelines—tasks that are repetitive and error-prone. Platform engineering centralizes these concerns into a shared platform, enabling self-service while enforcing organizational standards.

On Azure, platform engineering involves creating a thin layer of abstraction over Azure's native services, often using Infrastructure as Code (IaC), CI/CD pipelines, and policy enforcement. The platform team (sometimes called Cloud Center of Excellence) builds and maintains this layer, while application teams consume it through simple interfaces like pull requests or CLI commands.

At its core, platform engineering on Azure follows a three-tier architecture:

The platform tier is the key differentiator. It includes: - Source Control: Azure Repos or GitHub for storing code and IaC templates. - CI/CD: Azure Pipelines or GitHub Actions for building, testing, and deploying. - Policy Enforcement: Azure Policy to ensure compliance (e.g., only certain VM SKUs allowed). - Secret Management: Azure Key Vault for storing secrets, with access policies. - Monitoring: Azure Monitor, Application Insights, and Log Analytics for observability. - Service Catalog: A curated list of approved services and templates, often implemented via Azure Blueprints or Bicep modules.

When a developer wants to deploy a new application, they interact with the platform through a self-service portal (e.g., Azure DevOps Service Hooks or a custom portal). The platform validates the request against policies, provisions infrastructure, deploys code, and configures monitoring—all without manual intervention.

Azure DevOps is the most common platform engineering tool on Azure. Key components: - Azure Repos: Git repositories for source code. Default branch protection rules can enforce pull request reviews. - Azure Pipelines: YAML-based CI/CD. Default agent pool: ubuntu-latest. Timeout: 60 minutes per job (can be increased to 360 minutes). - Azure Artifacts: Package management. Default retention: 30 days for packages. - Azure Test Plans: Manual and exploratory testing.

Azure Policy enforces governance. Key elements: - Policy Definition: JSON that specifies conditions and effects (e.g., deny, audit, append). - Initiative: A group of policy definitions. - Assignment: Scope (management group, subscription, resource group). - Evaluation: Triggered on resource creation/update or every 24 hours (compliance scan). Default compliance scan interval: 24 hours.

Azure Blueprints (legacy, being replaced by Deployment Stacks) package artifacts like policies, role assignments, and ARM templates. Each blueprint has a version and can be assigned to subscriptions.

Bicep is a domain-specific language for Azure IaC. Key features: - Modules: Reusable templates. Default parameters can be overridden. - Deployment: Uses Azure Resource Manager (ARM) under the hood. Default deployment mode: incremental. - Linting: Built-in linter catches errors before deployment.

Azure Key Vault stores secrets, keys, and certificates. Default access policies: no one has access until granted. Soft-delete retention: 90 days (default, configurable from 7 to 90 days).

Azure Monitor collects metrics and logs. Default data retention: 30 days for metrics, 31 days for logs (can be extended to 730 days for logs with additional cost).

Deploy a Bicep template:

az deployment group create --resource-group myRG --template-file main.bicep

Create a policy assignment:

az policy assignment create --name 'allowed-locations' --policy 'e56962a6-4747-49cd-b67b-bf8b01975c4c' --params '{"listOfAllowedLocations":{"value":["eastus","westus"]}}'

List policy compliance states:

az policy state list --resource-group myRG

Create an Azure DevOps pipeline from YAML:

# azure-pipelines.yml
trigger:
- main
pool:
  vmImage: 'ubuntu-latest'
steps:
- script: echo Hello, world!
  displayName: 'Run a one-line script'

Verify pipeline run:

az pipelines run --name 'myPipeline' --org 'https://dev.azure.com/myorg' --project 'myProject'

Platform engineering integrates with: - Azure Active Directory (Azure AD): Authentication and authorization for the platform. Managed identities allow pipelines to access resources securely. - Azure Container Registry (ACR): Stores container images built by CI/CD pipelines. Platform policies can enforce image scanning. - Azure Kubernetes Service (AKS): Common deployment target. Platform provides Helm charts or Kubernetes manifests as part of the service catalog. - GitHub Actions: Alternative to Azure Pipelines. Integration via GitHub Apps and Azure AD. - Terraform: Alternative IaC tool. Azure supports Terraform via Azure Backend (state storage in Azure Storage). - Service Now: Often used for change management integration, triggering approvals before deployment.

The platform team must design for these integrations, ensuring consistent identity management (managed identities, service principals) and policy enforcement across the stack.

A large bank needed to deploy hundreds of microservices across multiple Azure regions while adhering to PCI DSS and SOC 2. The platform team built an internal developer platform using Azure DevOps and Bicep. They created a Bicep module registry with pre-approved configurations for VNets, NSGs, and AKS clusters. Azure Policy enforced that all resources had mandatory tags (cost center, environment) and that only certain VM SKUs (e.g., D-series for production) were allowed. The CI/CD pipeline included automated security scanning (SonarQube, Trivy) and required two approvals for production deployments. Key Vault stored database credentials and API keys, with access policies tied to managed identities. The platform reduced new service deployment time from weeks to hours. Common misconfiguration: developers tried to bypass the platform by creating resources directly via portal, leading to policy violation alerts and manual remediation. The platform team added deny effects for critical policies to prevent this.

An e-commerce company experienced rapid growth and needed to standardize deployment across multiple product teams. They adopted GitHub Actions as their CI/CD tool and Terraform for IaC. The platform team created a set of Terraform modules (stored in a private module registry) for App Service, Cosmos DB, and Redis Cache. GitHub Actions workflows included a step that called the Azure REST API to create a deployment stack (preview) for each environment. Azure Policy enforced that all App Service plans had minimum of two instances for production. The platform used Azure Front Door for global load balancing and Azure Monitor for unified logging. During Black Friday, the platform auto-scaled based on custom metrics. A common issue was that teams would modify the Terraform state manually, causing drift. The platform team implemented state locking and periodic drift detection using Azure Policy's audit effect.

Platform engineering must handle thousands of deployments per day. Key considerations: - Pipeline agents: Use Microsoft-hosted agents for standard builds, but self-hosted agents for compliance-sensitive workloads. Scale set agents can handle burst loads. - Policy evaluation: Azure Policy evaluation adds latency (a few seconds per resource). For high-throughput scenarios, use audit effect instead of deny to avoid blocking deployments. - Secret management: Key Vault has throttling limits (e.g., 10,000 operations per second per vault). Use caching or multiple vaults for high-volume scenarios. - IaC state: For Terraform, use Azure Storage with blob leasing for state locking. For Bicep, ARM handles state implicitly.

When misconfigured, platforms can become bottlenecks. Overly restrictive policies frustrate developers and lead to shadow IT. Insufficient testing of templates causes deployment failures. Lack of feedback loops results in a platform that doesn't meet user needs. Successful platform engineering requires continuous iteration and a strong partnership between platform and application teams.