Troubleshooting practice questions

You are troubleshooting a health check that seems to be hanging. Which flag, if available, would provide more information about the check's execution within the agent logs?

Trap 1: -verbose

-verbose is not a Consul agent flag; verbosity is set through log_level, so it cannot reveal a hanging check's execution. It is tempting because verbose output feels diagnostic, and it would be correct for CLI commands that accept it, such as some consul CLI subcommands, not the agent.

Trap 2: -monitor-checks

-monitor-checks is not a Consul agent flag; check monitoring is exposed through the /v1/agent/checks HTTP endpoint, so it cannot surface execution detail in agent logs. It is tempting because it names checks directly, and it would suit querying check status via the API, not log-level debugging.

Trap 3: -trace

Consul has no -trace flag for agent logging; log levels stop at trace only via log_level configuration, not a CLI switch, so it cannot expose check execution. It is tempting because tracing sounds like the deepest diagnostic, and it would be right for instrumenting code paths, not for a hanging check's agent logs.

A
-verbose
Why it fails: -verbose is not a Consul agent flag; verbosity is set through log_level, so it cannot reveal a hanging check's execution. It is tempting because verbose output feels diagnostic, and it would be correct for CLI commands that accept it, such as some consul CLI subcommands, not the agent.
B
-log-level=debug
Raising the agent's log verbosity to debug makes Consul emit detailed health-check execution output, including script invocations and timing, which reveals where a hanging check stalls. The -log-level=debug flag therefore exposes the check's internal behaviour in the agent logs.
C
-monitor-checks
Why it fails: -monitor-checks is not a Consul agent flag; check monitoring is exposed through the /v1/agent/checks HTTP endpoint, so it cannot surface execution detail in agent logs. It is tempting because it names checks directly, and it would suit querying check status via the API, not log-level debugging.
D
-trace
Why it fails: Consul has no -trace flag for agent logging; log levels stop at trace only via log_level configuration, not a CLI switch, so it cannot expose check execution. It is tempting because tracing sounds like the deepest diagnostic, and it would be right for instrumenting code paths, not for a hanging check's agent logs.

Question 2hardmultiple choice

When troubleshooting a service that fails to start its Envoy proxy, which service definition field should you check for incorrect configuration?

Trap 1: checks

Health checks govern service registration and instance health, not Envoy sidecar startup; a malformed check fails the service, not the proxy process. It is tempting because checks are the usual cause of services being marked unhealthy, and it would be correct when instances are deregistered despite the proxy running.

Trap 2: meta

Meta holds arbitrary key-value metadata for filtering and UI display, and Consul never parses it to configure Envoy. It is tempting because meta appears in service definitions alongside proxy settings, and it would be correct when you need custom attributes for intentions or API filtering.

Trap 3: tags

Tags are opaque labels used for filtering and versioning, and they do not feed Envoy's bootstrap configuration. It is tempting because tags sit in the same service block as proxy settings, and it would be correct when restricting which instances a upstream selects.

A
proxy
The `proxy` block configures the sidecar's Envoy settings, including upstreams, listeners and bootstrap options. Incorrect values here directly prevent the proxy from starting, so it is the field to inspect first when diagnosing startup failure.
B
checks
Why it fails: Health checks govern service registration and instance health, not Envoy sidecar startup; a malformed check fails the service, not the proxy process. It is tempting because checks are the usual cause of services being marked unhealthy, and it would be correct when instances are deregistered despite the proxy running.
C
meta
Why it fails: Meta holds arbitrary key-value metadata for filtering and UI display, and Consul never parses it to configure Envoy. It is tempting because meta appears in service definitions alongside proxy settings, and it would be correct when you need custom attributes for intentions or API filtering.
D
tags
Why it fails: Tags are opaque labels used for filtering and versioning, and they do not feed Envoy's bootstrap configuration. It is tempting because tags sit in the same service block as proxy settings, and it would be correct when restricting which instances a upstream selects.

Study the full ACL explanation →

Question 3hardmultiple choice

An operator is troubleshooting a Consul datacenter that was recently federated with another datacenter. They notice that cross-datacenter service discovery works, but ACL tokens created in the primary datacenter are not being recognized in the secondary datacenter. What is the most likely cause?

Trap 1: The primary datacenter's servers are not configured with…

retry_join_wan is used for WAN gossip federation, not ACL replication. While WAN federation is necessary for cross-datacenter communication, ACL replication specifically requires a replication token. The primary datacenter does not need retry_join_wan to the secondary for ACL replication to work; the secondary initiates the replication connection.

Trap 2: The secondary datacenter's servers have a different gossip…

A mismatched gossip encryption key would prevent WAN gossip federation entirely, which would also break cross-datacenter service discovery. Since the operator reports that service discovery works, the gossip key is likely correct. ACL replication uses RPC over the WAN and is not directly affected by the gossip encryption key.

Trap 3: The secondary datacenter is configured with primary_datacenter set…

If primary_datacenter is set incorrectly, the secondary may not know which datacenter to replicate from, but this would typically cause broader issues with ACL replication and CA management. However, the most direct cause of tokens not being replicated is a missing or invalid acl_replication_token. Setting primary_datacenter incorrectly is a configuration error, but the replication token is the specific mechanism that enables ACL data transfer.

A
The primary datacenter's servers are not configured with retry_join_wan pointing to the secondary.
Why it fails: retry_join_wan is used for WAN gossip federation, not ACL replication. While WAN federation is necessary for cross-datacenter communication, ACL replication specifically requires a replication token. The primary datacenter does not need retry_join_wan to the secondary for ACL replication to work; the secondary initiates the replication connection.
B
The secondary datacenter's servers have a different gossip encryption key than the primary.
Why it fails: A mismatched gossip encryption key would prevent WAN gossip federation entirely, which would also break cross-datacenter service discovery. Since the operator reports that service discovery works, the gossip key is likely correct. ACL replication uses RPC over the WAN and is not directly affected by the gossip encryption key.
C
The secondary datacenter is configured with primary_datacenter set to its own name instead of the primary's name.
Why it fails: If primary_datacenter is set incorrectly, the secondary may not know which datacenter to replicate from, but this would typically cause broader issues with ACL replication and CA management. However, the most direct cause of tokens not being replicated is a missing or invalid acl_replication_token. Setting primary_datacenter incorrectly is a configuration error, but the replication token is the specific mechanism that enables ACL data transfer.
D
The secondary datacenter's servers are not configured with a valid acl_replication_token.
ACL replication requires the secondary datacenter to authenticate to the primary using a token with replication permissions. If acl_replication_token is missing, expired, or lacks the necessary policies, the secondary cannot pull token and policy updates. As a result, tokens created in the primary will not be replicated, and the secondary will not recognize them for authorization.

Question 4easymultiple choice

You are troubleshooting a Consul Connect service mesh and need to verify which intentions are currently configured. Which Consul CLI command lists all intentions?

Trap 1: consul connect intentions

There is no 'consul connect intentions' subcommand. The 'consul connect' command has subcommands like 'envoy', 'proxy', and 'ca', but not 'intentions'. Intentions are managed under the top-level 'consul intention' command. Therefore, this command does not exist and would produce an error.

Trap 2: consul intention get -all

The 'consul intention get' command is used to retrieve a specific intention by its ID or source/destination pair. It does not support an '-all' flag to list all intentions. To list all, you use the 'list' subcommand. Thus, this command is incorrect for listing all intentions.

Trap 3: consul catalog intentions

The 'consul catalog' command is used to interact with the catalog of registered services and nodes. It does not have an 'intentions' subcommand. Intentions are separate from the catalog and are managed via the 'consul intention' command. Therefore, this command does not exist and is not correct.

A
consul connect intentions
Why it fails: There is no 'consul connect intentions' subcommand. The 'consul connect' command has subcommands like 'envoy', 'proxy', and 'ca', but not 'intentions'. Intentions are managed under the top-level 'consul intention' command. Therefore, this command does not exist and would produce an error.
B
consul intention get -all
Why it fails: The 'consul intention get' command is used to retrieve a specific intention by its ID or source/destination pair. It does not support an '-all' flag to list all intentions. To list all, you use the 'list' subcommand. Thus, this command is incorrect for listing all intentions.
C
consul intention list
The 'consul intention list' command queries Consul and displays all intentions that have been created. It shows the source and destination services, the action (allow or deny), and other metadata. This is the correct command to verify the current intentions in the mesh. It is a straightforward and commonly used command for auditing Connect authorization.
D
consul catalog intentions
Why it fails: The 'consul catalog' command is used to interact with the catalog of registered services and nodes. It does not have an 'intentions' subcommand. Intentions are separate from the catalog and are managed via the 'consul intention' command. Therefore, this command does not exist and is not correct.

Question 5hardmultiple choice

A service mesh operator is troubleshooting a connectivity issue between two services in a Consul 1.15 datacenter. The sidecar proxy for the 'frontend' service cannot reach the 'backend' upstream. The operator has confirmed that intentions allow the traffic. Which log should they inspect first to diagnose the issue on the sidecar proxy?

Trap 1: The systemd journal for the 'backend' service.

The backend service's systemd journal would show logs from the application itself, not from the frontend sidecar proxy. The issue is that the frontend proxy cannot reach the backend, so the problem could be in the frontend proxy's configuration or network path. Backend application logs might show incoming requests if they arrive, but they won't explain why the frontend proxy fails to connect. Therefore, this is not the most relevant log.

Trap 2: The Consul agent logs on the node running the frontend service.

Consul agent logs primarily show agent-level events such as membership, anti-entropy, and intention updates. While they may contain some proxy-related messages, they do not provide detailed connection-level debugging for the Envoy sidecar proxy. The sidecar proxy's own logs are more granular for network issues. Therefore, agent logs are not the first place to look for proxy connectivity failures.

Trap 3: The Consul server logs on the leader.

Consul server logs track Raft, catalog, and intention operations, but they do not record individual service-to-service connection attempts. They might show intention evaluations, but if intentions are confirmed to allow traffic, the issue is likely in the data plane. Server logs would not provide the granular proxy-level errors needed to diagnose a sidecar connectivity problem. Thus, they are not the best starting point.

A
The systemd journal for the 'backend' service.
Why it fails: The backend service's systemd journal would show logs from the application itself, not from the frontend sidecar proxy. The issue is that the frontend proxy cannot reach the backend, so the problem could be in the frontend proxy's configuration or network path. Backend application logs might show incoming requests if they arrive, but they won't explain why the frontend proxy fails to connect. Therefore, this is not the most relevant log.
B
The Consul agent logs on the node running the frontend service.
Why it fails: Consul agent logs primarily show agent-level events such as membership, anti-entropy, and intention updates. While they may contain some proxy-related messages, they do not provide detailed connection-level debugging for the Envoy sidecar proxy. The sidecar proxy's own logs are more granular for network issues. Therefore, agent logs are not the first place to look for proxy connectivity failures.
C
The Envoy proxy logs for the frontend sidecar.
Envoy proxy logs contain detailed information about connection attempts, including upstream cluster selection, TLS handshakes, and errors such as connection refused or timeout. Since the sidecar proxy is Envoy, its logs are the most direct source for diagnosing why it cannot reach the 'backend' upstream. These logs are typically written to a file or stdout depending on the proxy configuration. Inspecting them first will reveal specific failure reasons.
D
The Consul server logs on the leader.
Why it fails: Consul server logs track Raft, catalog, and intention operations, but they do not record individual service-to-service connection attempts. They might show intention evaluations, but if intentions are confirmed to allow traffic, the issue is likely in the data plane. Server logs would not provide the granular proxy-level errors needed to diagnose a sidecar connectivity problem. Thus, they are not the best starting point.

Question 6mediummulti select

When troubleshooting a failing health check, which tools are recommended? (Select TWO)

Trap 1: DNS query

A DNS query tests name resolution, not the HTTP or TCP health check Consul performs against the service, so it cannot show why the check returns a critical status. It is tempting because service discovery relies on DNS, making resolution failures a plausible suspicion worth ruling out.

Trap 2: Consul key-value store

The key-value store holds configuration and coordination data; it does not execute or report the health check, so it cannot show why the check is failing. It is tempting because service registration data lives there, and inspecting it would be correct when verifying whether a service registered at all.

Trap 3: System metrics

System metrics report host CPU, memory and disk usage, which do not reveal whether Consul's health check request itself reached the service or what status it returned. They are tempting because resource exhaustion is a common cause of failing services, and metrics would be the right tool for diagnosing that specific symptom.

A
Consul UI
The Consul UI displays each service's health check status, output and last-updated timestamp, letting you identify which check is failing and why. This directly satisfies the troubleshooting scenario by exposing the check's current state without command-line inspection.
B
Agent logs
Agent logs record health check execution results, including failures and their error output, on the node running the check. Reviewing them reveals the specific cause of a failing check, satisfying the troubleshooting requirement for diagnostic evidence.
C
DNS query
Why it fails: A DNS query tests name resolution, not the HTTP or TCP health check Consul performs against the service, so it cannot show why the check returns a critical status. It is tempting because service discovery relies on DNS, making resolution failures a plausible suspicion worth ruling out.
D
Consul key-value store
Why it fails: The key-value store holds configuration and coordination data; it does not execute or report the health check, so it cannot show why the check is failing. It is tempting because service registration data lives there, and inspecting it would be correct when verifying whether a service registered at all.
E
System metrics
Why it fails: System metrics report host CPU, memory and disk usage, which do not reveal whether Consul's health check request itself reached the service or what status it returned. They are tempting because resource exhaustion is a common cause of failing services, and metrics would be the right tool for diagnosing that specific symptom.

Question 7hardmultiple choice

You are troubleshooting a partition between two datacenters. The WAN gossip shows high latency. Which Consul CLI tool provides the best insight into the WAN gossip state?

Trap 1: consul operator raft list-peers

consul operator raft list-peers reports the Raft consensus peer set and leadership within a single datacenter, not WAN gossip membership or latency between datacenters. It is the right tool for diagnosing Raft quorum, leader elections and peer health inside one cluster.

Trap 2: consul monitor

consul monitor streams the agent's log output, which records gossip-related messages but gives no direct view of WAN member health or latency between datacenters. It suits watching live agent logs during general troubleshooting rather than inspecting gossip membership state.

Trap 3: consul kv get

consul kv get reads a key-value entry from the KV store and exposes nothing about gossip membership, datacenter reachability or latency. It is the correct choice when retrieving stored configuration or application data from Consul's KV store.

A
consul operator raft list-peers
Why it fails: consul operator raft list-peers reports the Raft consensus peer set and leadership within a single datacenter, not WAN gossip membership or latency between datacenters. It is the right tool for diagnosing Raft quorum, leader elections and peer health inside one cluster.
B
consul members -wan
`consul members -wan` queries the WAN gossip pool directly, listing every datacenter's server agents with their gossip status and round-trip latency. That satisfies the stem's need for insight into WAN gossip state under high latency, unlike LAN-scoped commands or RPC-based tooling that bypass the gossip layer entirely.
C
consul monitor
Why it fails: consul monitor streams the agent's log output, which records gossip-related messages but gives no direct view of WAN member health or latency between datacenters. It suits watching live agent logs during general troubleshooting rather than inspecting gossip membership state.
D
consul kv get
Why it fails: consul kv get reads a key-value entry from the KV store and exposes nothing about gossip membership, datacenter reachability or latency. It is the correct choice when retrieving stored configuration or application data from Consul's KV store.

Question 8hardmultiple choice

A Consul operator is troubleshooting a service that intermittently disappears from the catalog. The service is registered with a TTL health check that is updated every 30 seconds by a script. The operator notices that the service is marked as critical and then deregistered. Which configuration change should the operator make to prevent the service from being deregistered?

Trap 1: Set the `status` field of the service to `passing` in the service…

Manually setting the status to `passing` would override the health check results and could mask real issues. It is not a recommended practice and does not prevent deregistration if the check later fails. It also does not solve the intermittent failure problem.

Trap 2: Disable the health check by removing the `check` block from the…

Removing the health check would prevent the service from being marked critical, but it also removes health monitoring, which is not desirable. The service would always be considered healthy, but the operator would lose visibility into its actual health. This is not a proper solution for intermittent failures.

Trap 3: Change the health check to a script check that runs every 30 seconds

Switching to a script check would not prevent deregistration if the check fails. The issue is that the service is marked critical, and the deregistration is based on the critical timeout. A script check might still fail, and the service would still be deregistered after the timeout. It does not address the root cause.

A
Increase the `deregister_critical_service_after` value in the agent configuration
The `deregister_critical_service_after` setting controls how long a service can be critical before being automatically deregistered. By default, it is 30 minutes. If the service is intermittently failing health checks, increasing this value gives more time for the service to recover before being removed. This is the correct approach to prevent deregistration due to temporary failures.
B
Set the `status` field of the service to `passing` in the service definition
Why it fails: Manually setting the status to `passing` would override the health check results and could mask real issues. It is not a recommended practice and does not prevent deregistration if the check later fails. It also does not solve the intermittent failure problem.
C
Disable the health check by removing the `check` block from the service definition
Why it fails: Removing the health check would prevent the service from being marked critical, but it also removes health monitoring, which is not desirable. The service would always be considered healthy, but the operator would lose visibility into its actual health. This is not a proper solution for intermittent failures.
D
Change the health check to a script check that runs every 30 seconds
Why it fails: Switching to a script check would not prevent deregistration if the check fails. The issue is that the service is marked critical, and the deregistration is based on the critical timeout. A script check might still fail, and the service would still be deregistered after the timeout. It does not address the root cause.

Question 9easymultiple choice

What is the function of the 'consul intention check' command?

Trap 1: To list all intentions.

Listing intentions is performed by 'consul intention list', which outputs the current intention set. The check command instead tests whether a specific source service is authorised to connect to a destination service, returning allow or deny.

Trap 2: To create a new intention.

Creating intentions is done with 'consul intention create', which writes a new source-to-destination rule. The check command performs no write; it queries the existing intention set to report whether a given connection would be permitted.

Trap 3: To delete an intention.

The command inspects whether traffic between two services is permitted by current intentions, returning the matching decision; it performs no deletion. Deleting an intention requires `consul intention delete`. The option tempts because both commands share the `intention` prefix, so candidates may assume any intention-related verb modifies intentions rather than merely querying the authorisation outcome.

A
To list all intentions.
Why it fails: Listing intentions is performed by 'consul intention list', which outputs the current intention set. The check command instead tests whether a specific source service is authorised to connect to a destination service, returning allow or deny.
B
To verify if traffic is allowed between two services.
The `consul intention check` command evaluates the intention rules governing a connection between a specified source and destination service, returning whether that traffic is permitted or denied. It directly satisfies the stem's requirement to verify authorisation between two services, without modifying any configuration.
C
To create a new intention.
Why it fails: Creating intentions is done with 'consul intention create', which writes a new source-to-destination rule. The check command performs no write; it queries the existing intention set to report whether a given connection would be permitted.
D
To delete an intention.
Why it fails: The command inspects whether traffic between two services is permitted by current intentions, returning the matching decision; it performs no deletion. Deleting an intention requires `consul intention delete`. The option tempts because both commands share the `intention` prefix, so candidates may assume any intention-related verb modifies intentions rather than merely querying the authorisation outcome.

Question 10hardmulti select

Which TWO of the following are valid ways to monitor Consul Connect performance?

Trap 1: Modifying Consul log levels to 'debug'.

Raising log verbosity to debug produces diagnostic detail for troubleshooting a specific fault, but it does not expose the proxy-level metrics that measure Connect performance, and it adds overhead. Debug logging is the right choice when diagnosing a reproducible failure, not for ongoing monitoring.

Trap 2: Checking the system RAM.

System RAM reflects host resource consumption, not Consul Connect's service-mesh behaviour; Connect performance is observed through Envoy proxy metrics and the /agent/metrics endpoint. RAM monitoring is genuinely useful for sizing Consul servers, which is why it appears plausible here.

Trap 3: Directly querying the database.

Consul Connect's performance is measured from its own agent telemetry and Envoy sidecar metrics; querying an application database returns application data, not mesh statistics. Direct database queries are correct when validating data integrity or replication, which is why the option looks plausible.

A
Envoy stats endpoint.
Envoy's `/stats` endpoint exposes per-sidecar metrics for every proxied connection, including request rates, latency histograms and upstream health. Since Consul Connect injects Envoy as the data-plane proxy, scraping these endpoints directly satisfies the requirement to monitor service-mesh performance at the connection level.
B
Modifying Consul log levels to 'debug'.
Why it fails: Raising log verbosity to debug produces diagnostic detail for troubleshooting a specific fault, but it does not expose the proxy-level metrics that measure Connect performance, and it adds overhead. Debug logging is the right choice when diagnosing a reproducible failure, not for ongoing monitoring.
C
Prometheus metrics.
Prometheus metrics expose Consul Connect's service mesh telemetry, including sidecar proxy latency, connection counts and upstream health, via the `/v1/agent/metrics` endpoint. This satisfies the stem's requirement for monitoring Connect performance by scraping Envoy and Consul's native metric sinks, enabling alerting on mesh throughput and failures.
D
Checking the system RAM.
Why it fails: System RAM reflects host resource consumption, not Consul Connect's service-mesh behaviour; Connect performance is observed through Envoy proxy metrics and the /agent/metrics endpoint. RAM monitoring is genuinely useful for sizing Consul servers, which is why it appears plausible here.
E
Directly querying the database.
Why it fails: Consul Connect's performance is measured from its own agent telemetry and Envoy sidecar metrics; querying an application database returns application data, not mesh statistics. Direct database queries are correct when validating data integrity or replication, which is why the option looks plausible.

Question 11mediummulti select

Which TWO of the following are valid ways to troubleshoot inter-datacenter communication issues?

Trap 1: Change the datacenter name in the config.

Changing the datacenter name alters the local agent's identity and breaks its existing federation membership, worsening the issue rather than diagnosing it. It is tempting because datacenter naming appears in configuration, but inter-datacenter troubleshooting relies on inspecting WAN federation and server connectivity instead.

Trap 2: Restart all clients in the datacenter.

Restarting all clients disrupts the datacenter without isolating the cause of the communication failure. It is tempting because restarts often clear transient faults, but effective troubleshooting examines WAN federation, ports, and server health rather than blindly cycling agents.

Trap 3: Use 'consul kv export' to sync data.

Key-value export merely serialises the KV store for backup or migration; it neither queries datacentre peer health nor surfaces WAN, gossip or RPC failures. It tempts because KV data is replicated across datacentres, so operators assume exporting it verifies that replication, but the correct tools are 'consul members -wan' and 'consul operator raft list-peers'.

A
Change the datacenter name in the config.
Why it fails: Changing the datacenter name alters the local agent's identity and breaks its existing federation membership, worsening the issue rather than diagnosing it. It is tempting because datacenter naming appears in configuration, but inter-datacenter troubleshooting relies on inspecting WAN federation and server connectivity instead.
B
Inspect server logs for RPC errors between datacenters.
Inspecting server logs surfaces RPC errors that reveal failed cross-datacenter RPCs, directly satisfying the need to diagnose inter-datacenter communication faults. Consul's server-to-server RPC traffic carries replication and forwarding requests, so logged RPC failures pinpoint connectivity or TLS problems between datacenters.
C
Restart all clients in the datacenter.
Why it fails: Restarting all clients disrupts the datacenter without isolating the cause of the communication failure. It is tempting because restarts often clear transient faults, but effective troubleshooting examines WAN federation, ports, and server health rather than blindly cycling agents.
D
Use 'consul members -wan' to verify remote server visibility.
The 'consul members -wan' command lists all WAN gossip pool members and their status, exposing whether remote datacenter servers are visible and healthy. This directly verifies inter-datacenter gossip connectivity, making it a valid troubleshooting method.
E
Use 'consul kv export' to sync data.
Why it fails: Key-value export merely serialises the KV store for backup or migration; it neither queries datacentre peer health nor surfaces WAN, gossip or RPC failures. It tempts because KV data is replicated across datacentres, so operators assume exporting it verifies that replication, but the correct tools are 'consul members -wan' and 'consul operator raft list-peers'.