HashiCorp Certified: Consul Associate (Consul 1.15) (Consul 1.15) (Consul 1.15) — Questions 76–150

For script health checks, Consul interprets the exit code: 0 means passing, 1 means warning, and any other non-zero code means critical. However, in Consul 1.15, the behavior is that exit code 1 is actually treated as critical for script checks. This is because the script check does not differentiate between warning and critical; any non-zero exit code results in a critical status. Therefore, the service will be marked critical.

Consul KV values are stored as raw byte arrays rather than text-only strings, so arbitrary binary payloads survive intact. This satisfies the stem's requirement, letting operators store certificates, images or serialised blobs directly without encoding them first.

The catalog is the source of truth.

The -recurse flag makes the delete operation walk the key's entire subtree, removing the key and every child beneath it in one command. Without it, Consul deletes only the single key, leaving children orphaned, so this satisfies the requirement to remove the key and all its children.

Mesh gateways provide a single entry point that routes Consul Connect traffic between datacenters, letting services in separate datacenters reach each other without direct network peering or sidecar-to-sidecar links across WAN boundaries. This satisfies the stem's cross-datacenter communication requirement, unlike ingress gateways, which only expose services within one datacenter.

Consul creates lock sessions through the session endpoint, which returns a session ID used as the lock-delimiter when acquiring KV locks. The /v1/session/create path is the documented API for this, unlike destroy or renew operations.

Secondary datacentres keep serving their local workloads because each maintains its own replicated state, but ACL token and policy updates originate from the primary and cannot propagate. The outage therefore blocks only ACL changes, not ongoing operations.

The check-and-set (`cas`) parameter makes the write succeed only when the key's current `ModifyIndex` equals the supplied value. Reading the key first to capture that index, then writing with `cas`, gives the pipeline an atomic compare-and-swap that fails if another pipeline already changed the value, exactly the guard the blue/green switch requires.

A TTL check is marked critical when the TTL expires without an update. Setting ttl = "30s" means the service remains passing as long as the application sends a heartbeat at least every 30 seconds. This matches the requirement to mark the service unhealthy after 30 seconds without a heartbeat, and no external script or HTTP endpoint is needed.

Consul's DNS interface only returns instances whose health checks are passing. A check transitioning to critical marks the instance unhealthy, so it is omitted from the service's DNS response, preventing clients from resolving to the failed node.

Consul's KV store caps each individual value at 512 KB, a hard limit imposed by the Raft replication and storage layer. Larger payloads must be split across multiple keys or stored externally, so 512 KB is the precise maximum for a single key's value.

By default, an HTTP check in Consul considers any response code in the 2xx range as passing. If no explicit status codes are configured, the check uses this default behavior. Therefore, simply defining an HTTP check with a URL and interval will treat 2xx responses as healthy. This directly satisfies the requirement without additional configuration.

An upstream entry causes the web sidecar to open a local port that the web application connects to. The sidecar then forwards that traffic through the mesh to a healthy api proxy instance, using the port specified in the upstream. This decouples the application from service discovery and lets Connect handle mTLS and load balancing.

The `/v1/catalog/service/:service` endpoint returns the nodes and service instances for a given service. Adding `?dc=us-east` specifies the datacenter. The response includes node names, addresses, and service details. This is the correct endpoint to retrieve all nodes providing the `api` service in the specified datacenter.

The local_bind_port field fixes the port on which the sidecar listens for that upstream. Assigning 8080 makes the application's hard-coded address valid. After updating the registration and restarting the sidecar, the listener binds to the chosen port and forwards to the upstream over the mesh.

Consul's DNS interface allows querying services in other datacenters by appending the datacenter name to the query, such as service.service.dc2.consul. This works out of the box once WAN federation is established. No additional configuration is needed beyond the initial federation. This is the standard way to perform cross-datacenter service discovery.

A session TTL of 10 seconds is the minimum Consul permits; any lower value is rejected. This satisfies the stem's constraint by matching Consul's documented lower bound for session TTLs, ensuring the lock-delay and health-check timing mechanisms retain meaningful granularity rather than collapsing into effectively immediate expiry.

Consul sessions provide a mechanism for liveness checking. When you create a session with a TTL and use the acquire parameter on a KV PUT, the key is locked to that session. If the session expires due to missed TTL heartbeats, Consul automatically releases the lock, allowing another instance to acquire it. This is the standard way to implement leader election.

A TCP check opens a connection to the target host and port and reports success only if the socket is accepted. Pointing it at each server's Serf LAN port (8301) from the client agent directly tests the gossip path that matters, so a server that answers HTTP on 8500 but has a blocked or broken 8301 port is correctly marked critical.

The ttl field is mandatory for a TTL check. It tells Consul how long to wait for the next report before marking the check critical. Without a TTL value, Consul cannot determine when the check has expired, so the check cannot function as intended for the batch worker.

`consul members -wan` queries the WAN gossip pool, listing servers from every datacenter joined through the WAN federation. This directly satisfies the requirement to see the remote datacenter's server nodes, unlike the LAN pool, which only returns members of the local datacenter.

Consul's DNS interface filters out service instances whose health checks are not passing. Since the `web` service check is critical, no healthy instances exist, so the SRV query returns no records. This is expected behavior and explains why the service appears in the catalog but not in DNS results.

SRV records expose both host and port for a service, plus priority and weight for selection. Consul returns them so clients can discover not just where a service runs but on which port, satisfying the stem's service discovery requirement.

Service splitting uses a service resolver with a splitter configuration to route a defined percentage of traffic to the canary subset while the remainder goes to the stable version, satisfying the gradual, controlled traffic-shifting constraint of a canary deployment.

The Consul health check HTTP API returns an Output field containing the textual result produced by the check script or HTTP probe. This satisfies the stem's requirement by surfacing diagnostic detail alongside status, letting operators see why a check reported passing, warning, or critical.

For script health checks, Consul interprets exit code 0 as passing and any non-zero exit code as critical. This binary behavior is standard for script checks in Consul 1.15. The engineer's script returning 1 correctly results in a critical status when unhealthy.

Health checks execute on the Consul agent running on each node, not centrally on the servers. This satisfies the stem's requirement that checks are agent-local: the agent owns the check definition, performs the probe, and reports status to the servers via the catalog. Server-side execution would not scale to thousands of nodes.

Consul accepts JSON service definition files as a first-class configuration method, loaded via the agent's config directory or the /agent/service/register endpoint. This satisfies the stem's requirement for a valid Connect configuration mechanism alongside HCL definitions.

The `deregister_critical_service_after` field specifies a duration after which a service that remains in a critical state is automatically deregistered from the Consul catalog. After 10 minutes of continuous critical status, Consul will remove the service registration. This helps prevent stale entries for failed services. The service must be re-registered to become available again.

Catalog changes are replicated through the Raft consensus protocol, which elects a leader and commits entries only once a quorum of servers acknowledges them. This guarantees all nodes converge on identical service discovery data despite concurrent writes or failures.

Using a wildcard source intention for the 'frontend' namespace allows all services within that namespace to access 'payments', while setting the default intention behavior to deny ensures that any other source is blocked. This is efficient because it requires only one intention and a global default, rather than creating individual intentions for every service in the frontend namespace. It leverages Consul Enterprise namespaces to group services logically.

Creating a wildcard deny intention for all sources to 'payments' blocks any service by default. Then a specific allow intention for 'checkout' to 'payments' overrides the deny for that source. Consul evaluates intentions with specificity, so the more specific allow takes precedence. This achieves the desired least-privilege access control.

In a federated setup, the secondary datacenter maintains its own state and can continue to serve existing services using local data. Replicated ACLs and intentions are enforced based on the last successful replication. However, any changes to ACLs or intentions must be made in the primary, so new changes cannot be applied until the primary is back online.

The deregister_critical_service_after timer is reset when the check leaves the critical state. Since the check returned to passing after 20 minutes, well before the 30-minute threshold, the service is not deregistered. The timer only triggers deregistration if the check stays critical continuously for the configured duration.

The -raw flag prints the value exactly as stored, with no key name, no metadata, and no JSON wrapper. That makes it the correct choice when a script must capture the value verbatim, for example with command substitution into an environment variable.

Ingress gateway listeners that terminate TLS require a server certificate and key. Without them the listener cannot complete the handshake, which matches the observed failures. Consul can source these from inline configuration, a file path, or an external SDS provider such as Vault, and the gateway then presents the certificate to clients.

For HTTP checks, Consul marks the check critical when the monitored endpoint returns any HTTP status other than 200, or when the connection fails or times out. A 500 response therefore immediately puts the check into the critical state, which removes the service from healthy DNS results and from the passing portion of the catalog.

Mesh gateways route service traffic through a shared gateway rather than exposing pod or node addresses, so datacentres with identical CIDR ranges never need direct IP reachability. This satisfies the stem's overlapping-IP constraint without renumbering either network.

Consul's HTTP health check treats any response in the 2xx range as passing, marking the service healthy. Other ranges (3xx redirects, 4xx client errors, 5xx server errors) are interpreted as failing or warning states, so 2xx is the only range that satisfies the passing-state requirement.

The Consul UI displays each service's health check status, output and last-updated timestamp, letting you identify which check is failing and why. This directly satisfies the troubleshooting scenario by exposing the check's current state without command-line inspection.

Consul's client agent obtains the leaf certificate and key from the server via the Connect CA, then serves them to the local Envoy sidecar through the xDS API. This satisfies the requirement that certificates reach proxies without manual distribution.

Consistent forces a leader check for strong consistency.

The /v1/agent/service/register endpoint accepts a JSON body where Name sets the catalog service name, Tags is an array of strings, and Address/Port define the reachable endpoint. Consul uses Name to build billing.service.consul and exposes the tags for filtering. This body registers the service on the local agent, making it discoverable via DNS and catalog queries immediately.

Consul intentions define L4 authorisation rules that permit or deny connections between specific services, enforced via sidecar proxies or Connect. They restrict which local services may reach the remote datacenter services, satisfying the access-control requirement.

The tcp check type opens a TCP connection to the specified address and port and considers the check passing if the connection is established. It sends no application-level data, which matches the requirement exactly. The interval schedules repeated connection attempts, and the check fails when the socket cannot be opened.

Cross-datacenter service-to-service traffic traverses the WAN through mesh gateways, which proxy Consul's mTLS connections between datacenters. Deploying a mesh gateway in each datacenter is required so the local sidecar can reach the remote service without direct network peering.

The agent service deregister endpoint accepts the service ID in the path and removes only that instance from the local agent and catalog. Because web-01 is the unique service ID, other instances remain registered. This is the precise, targeted way to remove a single instance without impacting the rest of the service's endpoints.

Consul Connect's CA interface is pluggable, so it can delegate certificate signing and issuance to an external provider such as SPIRE rather than using the built-in CA. This lets SPIRE-issued identities secure service mesh mTLS without replacing Connect.

A secondary keeps its replicated ACL state and continues authorizing requests and answering catalog queries locally. What stops is the replication stream: any new token or policy must be created in the primary and cannot be propagated while it is offline, so ACL changes made in the secondary would not be authoritative or replicated.

ACLs with a policy specifying rules for the KV path allow fine-grained control over read and write access. By attaching a token with that policy, you restrict who can read or write the key. This directly satisfies the requirement that only authorized tokens can access the key.

By specifying a trusted CA certificate with ca_file or ca_path, Consul can verify the server's certificate against that CA. This resolves the certificate error while maintaining proper TLS verification. This is the secure and correct approach when the service uses a certificate signed by a private or internal CA.

When WAN traffic cannot route directly, Consul 1.15 supports WAN federation via mesh gateways. You set primary_gateways on the secondary servers to the primary's mesh gateway addresses and enable mesh gateways in both datacenters, allowing the secondary servers to establish the WAN gossip and RPC connections through the gateways.

TTL health checks require the service to periodically report its status using the /v1/agent/check/pass endpoint or equivalent. If no update is received within the configured TTL, Consul marks the check critical automatically. This is the intended behavior to detect unresponsive services, and it explains the observed transition in the scenario.

Service tags are used to attach metadata to a service instance for filtering during discovery. By registering the service with the tag 'v2', consumers can query for 'v2.api.service.consul' to retrieve only instances with that tag. This allows for version-based routing. Tags are not automatically applied; they must be specified in the service definition and can be used in DNS queries or API calls.

The passing parameter filters DNS results to endpoints whose Consul health checks currently report passing status, excluding any critical or warning instances. This satisfies the stem's requirement for returning only services with passing health checks, rather than all registered instances regardless of health.

Sidecar proxies can be configured to emit access logs that record each incoming connection, including source service identity and timestamp. This provides the necessary audit trail for connections to 'api'. Configuring access logs on the proxy is the standard way to achieve connection logging in Consul Connect.

Consul's gossip protocol uses the encryption key to authenticate and encrypt Serf messages; if the WAN key differs between datacenters, members cannot decrypt each other's traffic, so the join attempt fails rather than silently forming a split cluster.

Envoy's `/stats` endpoint exposes per-sidecar metrics for every proxied connection, including request rates, latency histograms and upstream health. Since Consul Connect injects Envoy as the data-plane proxy, scraping these endpoints directly satisfies the requirement to monitor service-mesh performance at the connection level.

Consul's gossip protocol and RPC compatibility depend on matching versions; mixed server versions across WAN-federated datacenters can break Serf membership and cause failed joins or inconsistent catalog replication. Uniform versions across every datacenter keep the WAN pool stable during and after the join.

'retry_join_wan' provides a list of WAN addresses that the server agent attempts to join, retrying indefinitely with backoff. It is the standard way to make WAN federation resilient across restarts, because the agent re-establishes the WAN gossip link automatically rather than requiring an operator to run 'consul join -wan' again.

Consul DNS supports datacenter-specific queries by inserting the datacenter name as a subdomain before 'consul'. The format 'api.service.dc2.consul' resolves to instances of 'api' in 'dc2'. This is the correct way to query a remote datacenter via DNS.

ACL replication in Consul is asynchronous. Changes made in the primary datacenter are replicated to secondary datacenters on a periodic basis, so there can be a delay before the updated policy is visible. This scenario matches that behavior: the security engineer checked immediately, before the next replication cycle completed.

When a script check exceeds its `timeout`, the agent reports the check as critical and moves on, running the script again when the next `interval` elapses. This gives operators the unhealthy signal they asked for while keeping the check loop on schedule, though the still-running process from the previous attempt is not forcibly stopped.

Consul's DNS interface publishes SRV records that carry both the target hostname and the service port, letting clients discover where a service listens. A records return only addresses, so SRV is the record type that answers service-address queries.

A TTL check expects a heartbeat within its TTL window. Once 30 seconds elapse without one, Consul marks the check critical, so the service status changes immediately at that expiry rather than after a further grace period.

Inspecting server logs surfaces RPC errors that reveal failed cross-datacenter RPCs, directly satisfying the need to diagnose inter-datacenter communication faults. Consul's server-to-server RPC traffic carries replication and forwarding requests, so logged RPC failures pinpoint connectivity or TLS problems between datacenters.

Service tags supply arbitrary metadata labels that Consul exposes through the catalog and DNS interface, enabling categorisation, versioning and filtered queries. This satisfies the stem's requirement to describe the tag's purpose: attaching non-functional identifiers to a service registration for discovery and grouping, rather than configuring health checks or routing behaviour.

Consul KV watches rely on blocking queries, where the client sends a request with an index and the server holds it open until data changes or the wait timeout expires. This satisfies the stem's requirement for a true statement about watch internals, avoiding polling overhead through long-poll semantics.

A successful TCP handshake confirms the target port is open and accepting connections, which is the sole pass criterion for Consul's TCP health check. Unlike HTTP checks, no status code or payload is evaluated, so the SYN-ACK response itself satisfies the check's passing condition.

The handler field names the external executable Consul invokes when watched KV data changes, satisfying the requirement to trigger a program on updates. Consul runs this process directly, passing change details via environment variables and stdin, rather than performing any in-process callback or HTTP notification.

The datacenter field labels which Consul datacenter an agent belongs to, letting servers, services and queries be scoped to that logical segment. Agents in different datacenters form separate clusters joined only through WAN gossip, enabling multi-region segmentation.

The deregister_critical_service_after field specifies a duration after which a service in critical state is automatically deregistered. Since the check has been critical for 15 minutes, which exceeds the 10-minute threshold, Consul will remove the service from the catalog. This prevents stale unhealthy services from lingering.

The /v1/health/checks/<service> endpoint returns each check's status plus its Output field, which carries the exact error text the check script produced. That message pinpoints why the check is failing, unlike the aggregated node or service health endpoints.

A TTL check uses the ttl field to define how long Consul waits for a check-in before marking the check critical, and the status field seeds the initial state so the new registration begins as passing. This matches the requirement of a 30-minute window and a non-critical initial state for the batch job.

In Consul, a lock is acquired by associating a session with a key. If the key already has a `Session` field set to an active session, the lock is held. The `-acquire` flag will fail if the lock is already held by a different active session. This is the most likely cause of the error, as it directly reflects the lock semantics.