SPLK-2002 Troubleshooting And Operations Practice Test 3 — 15 Questions

Question 1

You are monitoring indexer performance. Which metric in the Monitoring Console's 'Indexing Performance' dashboard best indicates a bottleneck in the ingestion pipeline?

Accepted Answer

Average Queue Fill Percentage. The Average Queue Fill Percentage metric in the Indexing Performance dashboard directly measures how full the indexing pipeline queues are. A persistently high or rising queue fill percentage indicates that events are arriving faster than the indexers can process them, which is the clearest signal of an ingestion bottleneck. Disk, memory, and CPU metrics can be symptoms, but queue fill is the direct indicator of pipeline backpressure.

Answer

Disk Read IOPS

Answer

Memory Usage

Answer

CPU Usage

Question 2

When upgrading a cluster, you have successfully upgraded the Master. Now you are moving to the Indexer Peers. What is the recommended order for upgrading the peers?

Accepted Answer

One by one, ensuring the cluster returns to a healthy state before moving to the next. Splunk's documented rolling upgrade procedure for indexer clusters requires upgrading peers one at a time (or in small batches), waiting for the cluster to return to a healthy state (all buckets searchable, replication and search factors met) before proceeding to the next peer. This preserves data availability and prevents search failures during the upgrade window. The Master must be upgraded first, then peers, then search heads.

Answer

The peers must be upgraded before the Master

Answer

Half the cluster, then the other half

Answer

All peers at once

Question 3

You notice that your indexers are using different versions of a lookup file. Where should you look to find the authoritative version of the lookup file if it is being managed by the Cluster Master?

Accepted Answer

$SPLUNK_HOME/etc/master-apps. When the Cluster Master manages indexer configuration via a cluster bundle, the authoritative source is $SPLUNK_HOME/etc/master-apps on the Cluster Master. Apps and lookups placed here are packaged into the cluster bundle and pushed to all peers. The master-apps directory is the staging area for the master's bundle.

Answer

$SPLUNK_HOME/etc/slave-apps

Answer

$SPLUNK_HOME/etc/system/local

Answer

$SPLUNK_HOME/etc/apps

Question 4

A user claims that a search is not returning data from a specific time range. You suspect the data has been rolled to cold storage and the cold storage is not being searched. Which configuration file controls which cold paths are included in the search?

Accepted Answer

indexes.conf. The indexes.conf file defines index properties, including the paths for hot, warm, cold, and frozen buckets. The 'coldPath' setting specifies where cold buckets are stored, and Splunk searches these paths by default unless configured otherwise. Therefore, indexes.conf controls which cold paths are included in searches.

Answer

distsearch.conf

Answer

server.conf

Answer

limits.conf

Question 5

You are using the Monitoring Console to view the 'Search Head Clustering: Status' dashboard. You see that one of the nodes is showing as 'Missing'. What does this status imply?

Accepted Answer

The captain has not received a heartbeat from the node. In the Search Head Clustering: Status dashboard, a node showing 'Missing' means the cluster captain has not received a heartbeat from that node within the expected interval. The captain tracks member health via heartbeats, and a missing heartbeat indicates the node is unreachable, down, or partitioned from the cluster. This is a connectivity/health signal, not a performance or licensing state.

Answer

The node is overloaded

Answer

The node is in maintenance mode

Answer

The node has an invalid license

Question 6

You are troubleshooting a slow indexer cluster. You identify that the 'replication_port' is experiencing high latency. Where is the replication port defined?

Accepted Answer

server.conf. The replication_port setting is defined in server.conf under the [replication_port://<port>] stanza. It specifies the port on which an indexer listens for replication data from peer indexers in an indexer cluster. Since server.conf holds cluster-wide and node-level server settings, this is the correct file for replication port configuration.

Answer

indexes.conf

Answer

inputs.conf

Answer

cluster.conf

Question 7

You are troubleshooting a configuration bundle replication issue in an Indexer Cluster. The peers are not receiving the latest configuration from the Master. What is the first thing you should check on the Cluster Master?

Accepted Answer

splunk validate cluster-bundle. When peers are not receiving the latest configuration bundle, the first diagnostic step on the Cluster Master is to validate the bundle with 'splunk validate cluster-bundle'. This command checks the bundle for syntax errors, invalid configurations, or missing files before it is pushed to peers, and it reports exactly what is wrong so you can fix the source rather than guessing.

Answer

Check the replication_port

Answer

Restart the Master

Answer

Check /etc/apps for errors

Question 8

You are troubleshooting slow search performance across the indexer cluster. Which THREE of the following could be potential causes?

Accepted Answer

The indexers have high CPU usage. Option A is correct because high CPU usage on the indexers directly degrades search performance, as indexers must parse, search, and return results from buckets, and CPU saturation slows query processing and result aggregation. Option C is correct because when search concurrency limits are reached on the search head, additional searches are queued or throttled, which manifests as slow search performance for users. Option D is correct because bucket re-balancing in an indexer cluster involves copying and moving buckets between peers, consuming disk I/O, network bandwidth, and CPU, which competes with search workloads and slows them down. Option B is not correct because the KV store on the search head supports features like saved searches, lookups, and knowledge objects, but disabling it does not directly cause slow search performance across the indexer cluster. Option E is not correct because approaching the daily license limit triggers license violations and search warnings or blocking, but it is not a direct cause of slow search performance.

Answer

The KV store on the search head is disabled

Answer

The Splunk license is close to the daily limit

Question 9

When planning a Splunk upgrade, which THREE of the following actions are considered best practices for maintaining system health?

Accepted Answer

Perform a mock upgrade in a staging environment. Option B is correct because performing a mock upgrade in a staging environment lets you validate the upgrade path, catch configuration or app issues, and rehearse rollback procedures before touching production. Option C is correct because verifying compatibility with all installed apps and add-ons ensures that third-party and custom components support the target Splunk version, preventing broken searches, dashboards, or data ingestion after the upgrade. Option E is correct because a full backup of indexer buckets and configuration files (such as $SPLUNK_HOME/etc and the index directories) provides a recovery point if the upgrade fails or corrupts data, which is essential for maintaining system health. Option A is not a best practice because the license master should generally be upgraded first (or in the correct order relative to other components) so it can properly serve licenses to upgraded peers. Option D is not a best practice because indexers should typically be upgraded before search heads, since search heads depend on indexers being available and compatible for search functionality.

Answer

Upgrade the license master last

Answer

Run the upgrade on the search heads before the indexers

Question 10

You are investigating why an indexer node is not reporting to the Cluster Master. Which THREE of the following are the most likely causes?

Accepted Answer

The security key ('pass4SymmKey') is mismatched. Option B is correct because the pass4SymmKey in the [clustering] stanza of server.conf must match across all cluster members and the master; a mismatch prevents the indexer from authenticating and registering with the Cluster Master. Option C is correct because indexers communicate with the master over the management port (default 8089), so a firewall blocking that port stops the indexer from reporting its status. Option E is correct because an indexer must be configured with the cluster-member role (mode = slave in [clustering]) to join and report to the master; without it, the instance never registers. Option A is not a cause because warm bucket count affects disk usage and search performance, not cluster registration. Option D is not a cause because the replication port (default 9887) is used for peer-to-peer bucket replication between indexers, not for reporting to the Cluster Master.

Answer

The indexer has too many warm buckets

Answer

The replication port is blocked by a firewall

Question 11

You have determined that a specific app deployed via the Deployment Server is causing indexer errors. Which THREE of the following actions can you take to rectify this?

Accepted Answer

Disable the server class associated with that app. Option A is correct because disabling the server class in the Deployment Server stops the deployment of the problematic app to the indexers, preventing further errors while you investigate. Option C is correct because removing the app folder from $SPLUNK_HOME/etc/deployment-apps on the Deployment Server removes it from the deployment payload, so it will no longer be pushed to clients. Option E is correct because replacing the app in the deployment-apps directory with a known stable version lets the Deployment Server distribute a corrected app, resolving the errors. Option B is not correct because restarting the master node (cluster manager) does not address the app causing indexer errors and is unrelated to deployment remediation. Option D is not correct because manually deleting the app folder on indexer peers is not a supported or persistent fix; the Deployment Server will simply redeploy the app on the next phone-home unless the server class or app is changed.

Answer

Restart the master node

Answer

Manually delete the app folder on all indexer peers

Question 12

An Indexer Cluster is in 'Maintenance Mode'. Which THREE of the following implications must be considered during this time?

Accepted Answer

The peers will continue to index incoming data normally. In Splunk maintenance mode, option A is correct because the indexer peers keep indexing incoming data normally; maintenance mode only suppresses certain master-level activities, not data ingestion. Option B is correct because the cluster master stops alerting on missing buckets, so replication/primacy violations are not reported during this window. Option E is correct because the master also halts bucket fix-up activities, meaning it will not attempt to search for and replicate or restore missing bucket copies. Option C is wrong because the master does not force an immediate bucket re-balance in maintenance mode; rebalancing is a separate, manual operation. Option D is wrong because searches are not automatically disabled by maintenance mode; search functionality continues unless separately restricted.

Answer

The Master will force a bucket re-balance immediately

Answer

All searches will be automatically disabled

Question 13

A Splunk architect notices that an indexer cluster's search head is returning incomplete results for a search spanning the last 7 days. The Monitoring Console shows that several buckets are missing their searchable copies, and the cluster master reports a search factor (SF) of 2 but replication factor (RF) of 3. Which of the following best explains why the search results are incomplete and what should be done to remediate the issue?

Accepted Answer

The indexer cluster is under-replicated; wait for automatic fix-up or manually trigger fix-up to create additional searchable copies to meet the search factor.. In an indexer cluster, the search factor determines how many searchable copies of each bucket are maintained. When searchable copies are missing, searches may not see all data because the search head can only search buckets that are searchable on the indexers. The cluster master orchestrates fix-up to create additional copies, but if fix-up is not progressing, manual intervention may be required. Addressing the searchable copy deficit directly resolves the incomplete search results.

Answer

The indexer cluster's replication factor is set higher than the number of indexers; reduce RF to match the number of indexers and restart the cluster.

Answer

The search head is not configured to use the cluster master for search; configure the search head to point to the cluster master and restart Splunk.

Answer

The search head is experiencing a network partition; verify connectivity between the search head and all indexers, and restart the search head.

Question 14

An architect is troubleshooting a Splunk deployment where search head cluster members are unable to replicate the search head captain's knowledge bundle. The captain's log shows repeated errors about failing to push the bundle to a member. Which of the following is the most likely cause and the appropriate resolution?

Accepted Answer

The captain's management port (default 8089) is blocked by a firewall between the captain and the members; open the port to allow bundle replication.. In a search head cluster, the captain is responsible for replicating the knowledge bundle to all members. This replication occurs over the management port (8089 by default). If a firewall blocks this port between the captain and a member, the captain cannot push the bundle, leading to replication errors. Ensuring the management port is open and accessible on all members resolves the issue. This is a common configuration oversight in environments with strict network policies.

Answer

The search head cluster members are using a different Splunk version than the captain; upgrade all members to match the captain's version.

Answer

The search head cluster members have insufficient disk space to store the knowledge bundle; free up disk space on the members.

Answer

The captain's replication factor for the knowledge bundle is set to 0; increase the replication factor to at least 1.

Question 15

An indexer cluster has a replication factor of 3 and a search factor of 2. The Monitoring Console shows several buckets in the cluster with a search factor of 1. You need to confirm whether the cluster manager is actively fixing these buckets. Which command should you run on the cluster manager to view the status of bucket fixup tasks?

Accepted Answer

`splunk list cluster-fixup`. Bucket fixup is orchestrated by the cluster manager, and the `splunk list cluster-fixup` command on the manager reports active fixup tasks. When buckets fall below the search factor, the manager schedules searchable copies on other peers; this command shows those tasks so you can confirm that repair is in progress and not stalled.

Answer

`splunk show cluster-status`

Answer

`splunk show cluster-bundle-status`

Answer

`splunk show cluster-peers`