Troubleshooting Scenario Questions

Increase the number of App Engine instances.

Why wrong: This may not address the root cause.

Enable Cloud CDN to cache responses.

Why wrong: Not suitable for dynamic content.

Disable caching to ensure fresh data.

Why wrong: Disabling caching worsens performance.

Increase the number of vCPUs of the Cloud SQL instance

Why wrong: Scaling up is a mitigation but should be done after understanding the cause.

Restart the Cloud SQL instance to clear the cache

Why wrong: Restarting causes downtime and does not fix the root cause.

Migrate the database to Cloud Spanner

Why wrong: Migration is a long-term project, not an immediate investigation step.

Analyze metrics, then review logs, then check health checks, then identify the problematic backend, then take corrective action.

Why wrong: This is incorrect because health checks should be performed first to rule out backend health issues before diving into metrics and logs.

Check health checks, then identify the problematic backend from logs, then take corrective action, then analyze metrics, then verify.

Why wrong: This is incorrect because taking action before analyzing metrics may not address the root cause; metrics are needed to quantify and confirm the issue after identification.

Review logs, then analyze metrics, then identify the problematic backend, then take corrective action.

Why wrong: This is incorrect because health checks are a crucial first step; skipping them may miss backend health issues that could cause latency.

Verify that the Cloud Build service account has the dns.networks.bindPrivateZone permission.

Why wrong: DNS permissions are needed for private zones, but the error is connection refused, not DNS resolution.

Check that the Cloud Build service account has the storage.objectViewer role on the Artifactory bucket.

Why wrong: Artifactory is not a GCS bucket; the error is network, not permissions.

Ensure that Cloud NAT is configured in the private pool's VPC.

Why wrong: Cloud NAT is for internet egress; on-premises is accessed via VPN/Interconnect.

Check if the nodes in that zone have been cordoned.

Why wrong: Cordoning would prevent pod scheduling, but app is still serving, so unlikely.

Check if the application's resource requests and limits have changed.

Why wrong: Resource requests affect scheduling, not metric collection.

Check if the Cloud Monitoring agent has been updated incorrectly.

Why wrong: GKE uses Metrics Server, not an agent, for resource metrics.

Increase the max instances per revision in Cloud Run to handle more concurrent requests

Why wrong: Increasing Cloud Run instances could increase load on the already stressed database, worsening the issue.

Reduce the min instances of Cloud Run to decrease load on the database

Why wrong: Reducing instances may cause cold starts and does not address the root cause of database overload.

Add a Cloud SQL read replica and route read queries to it

Why wrong: The endpoint causing 503 errors likely involves writes; read replicas won't reduce write load.

Increase the pod's CPU request

Why wrong: CPU increase won't resolve missing file error.

Increase the maximum number of pods per node

Why wrong: Affects scheduling, not missing file.

Enable pod anti-affinity

Why wrong: Anti-affinity affects pod placement, not missing file.

Request latency histogram.

Why wrong: Latency is not error cause.

CPU utilization of the service instances.

Why wrong: CPU may correlate but is not a root cause of errors.

Network bytes sent per instance.

Why wrong: Network traffic does not indicate error sources.

Security Command Center and Cloud Logging

Why wrong: Incorrect. Security Command Center is for security posture management, not for resource usage monitoring or log correlation.

Cloud Trace and Cloud Monitoring

Why wrong: Incorrect. Cloud Trace provides distributed tracing, not logs. Without Cloud Logging, logs cannot be correlated with resource usage.

Cloud Error Reporting and Cloud Logging

Why wrong: Incorrect. Cloud Error Reporting surfaces application exceptions, but does not provide resource usage metrics or logs for correlation.

Cloud Monitoring

Why wrong: Cloud Monitoring provides metrics but not detailed trace of individual API calls.

Cloud Profiler

Why wrong: Cloud Profiler focuses on CPU/memory consumption, not request latency.

Cloud Logging

Why wrong: Cloud Logging shows log entries but does not trace request paths across services.

The backend pods are out of memory and crashing.

Why wrong: Pods would be unhealthy, contradicting the scenario.

The IAM permissions for the load balancer service account are misconfigured.

Why wrong: Would cause 403 errors, not 502.

The backend service has been accidentally deleted by another engineer.

Why wrong: Would cause 'connection refused' errors, not 502.

The scan tool requires a specific dependency; add an installation step before scanning.

Why wrong: Missing dependency would cause command not found or similar errors.

There is network latency between Cloud Build and the container registry; use VPC Service Controls.

Why wrong: Network latency would cause connection errors, not a deadline exceeded within the build.

The build step is running out of memory; increase the machine type to e2-highcpu-8.

Why wrong: Memory issues would produce out-of-memory errors, not timeout.

The instances are in a different region and Cloud Monitoring doesn't support cross-region.

Why wrong: Cloud Monitoring supports all GCP regions globally.

The instances are preemptible and automatically stop reporting after 24 hours.

Why wrong: Preemptible instances report metrics while running, with no 24-hour limit.

The instances are in a different project and not peered.

Why wrong: Cross-project monitoring is supported through metrics scopes without peering.

Scale down the number of replicas to reduce memory pressure

Why wrong: Scaling down reduces total memory but each container still leaks, causing crashes.

Roll back the deployment immediately without further investigation

Why wrong: Rollback is mitigation, but the question asks for investigation and mitigation steps.

The GKE cluster has reached its maximum node quota.

Why wrong: Quota issues would produce resource exhaustion errors, not authorization.

The Cloud Deploy pipeline is missing the required IAM role for the Spinnaker service account.

Why wrong: The error is from Spinnaker directly, not Cloud Deploy.

The Kayenta canary analysis service is not configured correctly.

Why wrong: Kayenta handles metric analysis, not disabling server groups.