IAM Audit Logging and Data Access Logs

Cloud Audit Logs are Google Cloud's built-in, immutable record of administrative and data operations within your GCP environment. They serve three primary purposes: security monitoring, compliance auditing, and operational troubleshooting. The ACE exam tests your understanding of the three audit log types: Admin Activity, Data Access, and System Event. Each type captures different categories of operations, with different retention periods and enablement defaults.

When an API call is made to a GCP service, the service's backend generates audit log entries based on the type of operation. The log entries are structured as LogEntry objects in Cloud Logging. Each entry contains:

The audit log entry is written after the operation completes (or fails). For Admin Activity, the entry is written regardless of success or failure. For Data Access, the entry is written only if the operation is successful (some services record failures as well).

To enable Data Access audit logs for a specific service (e.g., Cloud Storage), use the gcloud command:

gcloud logging settings update --enable-data-access --service=storage.googleapis.com

To view the current audit log configuration for a project:

gcloud logging settings describe

To list audit log entries:

gcloud logging read "logName=projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity" --limit=10

To create a log sink to Cloud Storage:

gcloud logging sinks create my-sink storage.googleapis.com/my-bucket --log-filter='logName="projects/PROJECT_ID/logs/cloudaudit.googleapis.com%2Factivity"'

#### Admin Activity Audit Logs

Admin Activity logs record API calls that modify resource configuration. They are always enabled. Examples:

These logs are essential for tracking who changed what and when. They are retained for 400 days. You cannot disable them.

#### Data Access Audit Logs

Data Access logs record API calls that read resource configuration or data, or that modify user data. They are disabled by default (except BigQuery). You must enable them per service. Examples:

Because these logs can be high-volume, enabling them for all services can significantly increase logging costs. Best practice: enable only for services that require compliance auditing.

#### System Event Audit Logs

System Event logs record actions taken by Google Cloud systems, not users. They are always enabled. Example:

These logs are useful for understanding automated system behavior. They are retained for 400 days.

{
  "logName": "projects/my-project/logs/cloudaudit.googleapis.com%2Factivity",
  "protoPayload": {
    "@type": "type.googleapis.com/google.cloud.audit.AuditLog",
    "serviceName": "compute.googleapis.com",
    "methodName": "v1.compute.instances.insert",
    "resourceName": "projects/my-project/zones/us-central1-a/instances/my-vm",
    "authenticationInfo": {
      "principalEmail": "admin@example.com"
    },
    "request": {
      "name": "my-vm",
      "zone": "us-central1-a"
    },
    "response": {
      "id": "123456"
    }
  },
  "resource": {
    "type": "gce_instance",
    "labels": {
      "project_id": "my-project",
      "zone": "us-central1-a",
      "instance_id": "123456"
    }
  },
  "timestamp": "2023-01-01T00:00:00Z",
  "severity": "NOTICE"
}

A healthcare provider stores patient health information (PHI) in Cloud Storage buckets. To comply with HIPAA, they must log every access to the data (read and write) for at least 6 years. They enable Data Access logs for storage.googleapis.com and create a log sink to a Cloud Storage bucket with a lifecycle rule to delete objects after 7 years. They also create a BigQuery sink for analysis. They assign roles/logging.privateLogViewer only to the compliance team. They use log exclusions to filter out internal health check requests to reduce volume by 30%. They set up alerts in Security Command Center for any access from unexpected IPs. The challenge: log volume is high (millions of entries per day), so they partition BigQuery tables by date and use clustering on principalEmail to speed up queries.

A fintech company processes credit card transactions. PCI DSS requires logging all access to cardholder data. They enable Data Access logs for BigQuery (already on by default) and Cloud Storage. They also enable for Compute Engine to track VM access. They create a log sink to a centralized project's BigQuery dataset for cross-project analysis. They use VPC Service Controls to prevent exfiltration of logs. The security team uses custom dashboards in Looker to monitor for suspicious patterns. A common misconfiguration: they initially forgot to enable Data Access logs for Cloud Storage, missing compliance requirements. After a breach simulation, they realized the gap and enabled it. They also learned that setting an exclusion filter incorrectly can exclude critical events, so they now review exclusions with the compliance officer.

A SaaS company hosts multiple customer environments in separate GCP projects. They enable Admin Activity logs by default for all projects. For customers with compliance requirements, they enable Data Access logs per project. They use log sinks to aggregate all audit logs into a single BigQuery dataset in a central logging project. They use IAM conditions to restrict log access to specific project prefixes. They also use the gcloud logging settings update command via Infrastructure as Code (Terraform) to enable Data Access logs for new projects automatically. Performance consideration: with 100+ projects, the aggregated BigQuery dataset grows by 500 GB per month. They use partition expiration and clustering to manage costs. A common failure: when a new service is added (e.g., Cloud SQL), they forget to enable Data Access logs, leading to a compliance gap. They now have a CI/CD pipeline that checks for enabled audit logs on all services.