SPLK-2002 High Availability And Distributed Search — 20 Questions

Question 1

A customer is experiencing 'Search peer [peer-name] is not responding' errors during indexer maintenance. Which configuration parameter in server.conf should be tuned to prevent search failures during rolling restarts?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

restart_timeout. The 'restart_timeout' parameter in the [clustering] stanza controls how long a search head waits for a peer to return before marking it as down during a rolling restart.

Answer

peer_down_timeout

Answer

max_search_time

Answer

heartbeat_timeout

Answer

search_retry_timeout

Question 2

Which component in a distributed search environment is responsible for the 'Search Affinity' feature?

Accepted Answer

Search Head. Search Affinity is a Search Head feature that ensures a search dispatched to an indexer cluster is routed to the same peer that holds the replicated bucket, minimizing cross-network data transfer. The Search Head (specifically the search head's cluster master integration) manages this routing decision.

Answer

Deployment Server

Answer

License Master

Answer

Indexer Peer

Answer

Cluster Master

Question 3

You need to migrate a single-site Indexer Cluster to a multi-site configuration. What is the first step you must perform before modifying the server.conf on the Cluster Master?

Accepted Answer

Set the cluster to maintenance_mode=true.. Before changing the cluster master's server.conf to convert a single-site cluster to multisite, you must set the cluster to maintenance mode (maintenance_mode=true) so the master does not attempt fixup or rebalance while the topology is being reconfigured. This prevents the master from reacting to the site changes mid-edit and creating unnecessary bucket replication churn. After the change and restart, you exit maintenance mode and the master applies the new multisite factors.

Answer

Delete all existing buckets.

Answer

Configure the site-specific replication factors.

Answer

Upgrade the license server.

Answer

Disable all forwarder data ingestion.

Question 4

You are designing a multi-site Indexer Cluster with two sites. To ensure search availability during a site failure, which configuration is mandatory in the site_replication_factor stanza of server.conf?

Accepted Answer

Define explicit replication counts for each site using 'siteN' syntax.. In a multi-site indexer cluster, the site_replication_factor attribute in server.conf must use explicit per-site syntax (e.g., 'origin:2, total:3' or 'site1:2, site2:2, total:4') to define how many copies of each bucket exist at each site. This explicit siteN syntax is mandatory because it ensures that search availability survives a full site failure by guaranteeing replicas exist at each site. Without per-site counts, Splunk cannot enforce site-aware replication.

Answer

Enable search_factor to 2 for all sites.

Answer

Set replication_factor to 1 on each site.

Answer

Set cluster_mode to 'multisite_master'.

Answer

Configure the indexer as a site-local peer.

Question 5

When configuring a Search Head Cluster, which of the following is true regarding the 'shcluster_replication_port'?

Accepted Answer

It must be configured in server.conf.. The `shcluster_replication_port` is a setting in the `[shclustering]` stanza of server.conf that specifies the port used for replication traffic between search head cluster members. It must be configured in server.conf on each member, typically with the same value across the cluster (default 9887).

Answer

It must be different for every member.

Answer

It is only used by the captain.

Answer

It is used for forwarding data.

Answer

It is automatically detected by the Cluster Master.

Question 6

In a disaster recovery scenario, which file must be restored to a new Cluster Master to ensure it recognizes the existing indexer peers?

Accepted Answer

server.conf. The `server.conf` file contains the `[clustering]` stanza with the cluster master's configuration, including the `pass4SymmKey` (Cluster Secret) and the list of peer nodes. Restoring this file to a new Cluster Master ensures it can authenticate and recognize existing indexer peers, allowing the cluster to resume operations.

Answer

inputs.conf

Answer

distsearch.conf

Answer

props.conf

Answer

indexes.conf

Question 7

Which feature ensures that Search Head Cluster members share knowledge objects like saved searches and reports?

Accepted Answer

Search Head Cluster replication. Search Head Cluster replication is the mechanism that synchronizes knowledge objects (saved searches, reports, dashboards, etc.) across all members of a Search Head Cluster. It uses a replication factor to ensure that each artifact is copied to multiple members, providing both redundancy and consistency. This allows any search head in the cluster to serve the same user-created content, which is essential for a unified search experience.

Answer

Cluster Master

Answer

Index-time extraction

Answer

Bucket replication

Answer

Deployment Server

Question 8

You want to perform a rolling restart of an Indexer Cluster without stopping ingestion. What configuration ensures that indexers remain available?

Accepted Answer

Set 'restart_mode=rolling'.. A rolling restart in an Indexer Cluster is initiated by setting the restart_mode to rolling (e.g., via the CLI 'splunk rolling-restart cluster-peers' or by setting restart_mode=rolling in server.conf on the cluster master). This ensures that indexers are restarted one at a time (or in batches) while maintaining replication and searchability, so ingestion continues without interruption. The cluster master orchestrates the restart, waiting for buckets to be replicated before proceeding.

Answer

Disable the indexer peers.

Answer

Stop the Cluster Master first.

Answer

Restart all indexers at once.

Answer

Set 'replication_factor=1'.

Question 9

In a multisite cluster, how does the 'site_replication_factor' override the global 'replication_factor'?

Accepted Answer

It defines the total number of copies per site.. In a Splunk multisite cluster, the global replication_factor sets the total number of copies across all sites, but site_replication_factor overrides this by specifying exactly how many copies must exist per site. For example, site_replication_factor = origin:2, total:3 means each site must have at least 2 copies (origin site) and the cluster must maintain 3 total copies. This ensures data redundancy and availability even if an entire site fails, as each site independently satisfies its replication requirement.

Answer

It only applies to the primary site.

Answer

It ignores the search factor.

Answer

It is used for load balancing.

Answer

It is only used when the CM is down.

Question 10

When migrating to a Search Head Cluster, what happens to existing saved searches that were defined in local/savedsearches.conf on a standalone search head?

Accepted Answer

They are ignored by the cluster.. These need to be manually migrated into the 'shcluster' application directory to be recognized by the new cluster.

Answer

They are automatically migrated by the cluster.

Answer

They are deleted by the captain.

Answer

They are replicated to all members automatically.

Answer

They cause a cluster startup error.

Question 11

If an Indexer Cluster is configured with site-based replication, where should the 'site' attribute be defined?

Accepted Answer

In the indexer's server.conf.. In a site-based indexer cluster, the 'site' attribute is defined in the indexer's server.conf under the [general] stanza (site = site1 or site2). This tells the Cluster Master which site each peer belongs to, enabling site-aware replication and search affinity. It is configured on each indexer, not on the master or forwarders.

Answer

On the license master.

Answer

In the inputs.conf on the forwarders.

Answer

On the Cluster Master only.

Answer

In the search head cluster settings.

Question 12

Which THREE of the following are valid states for a bucket in an Indexer Cluster?

Accepted Answer

Cold. In a Splunk Indexer Cluster, buckets progress through lifecycle states that reflect their age and storage role, and the three valid states among the options are Cold (A), Hot (B), and Warm (C). Hot buckets are the actively written-to buckets on an indexer, Warm buckets are rolled from hot and are still searchable and may be replicated, and Cold buckets are older rolled buckets that are searchable but no longer actively written. These three states are the standard bucket states managed by the cluster master for replication and searchability. The unmarked options do not belong: Active (D) is not a defined bucket state in the indexer cluster lifecycle, and Pending (E) is not a bucket state either (pending is not part of the hot/warm/cold/roll-to-frozen progression).

Answer

Active

Answer

Pending

Question 13

When a Search Head Cluster is running in a multi-site environment, what configuration ensures search results are optimized?

Accepted Answer

Search affinity settings in server.conf.. In a multi-site Search Head Cluster, search affinity settings in server.conf (specifically, search_affinity under the [shclustering] stanza) ensure that searches are directed to search heads within the same site as the indexers holding the data. This reduces cross-site traffic and latency, optimizing search performance. The setting allows the search head to prefer local indexers, improving efficiency in distributed environments.

Answer

Index-level bucket settings.

Answer

Disabling search head replication.

Answer

Site-specific replication factor.

Answer

Increasing the search timeout.

Question 14

Which TWO of the following are prerequisites for a functional Search Head Cluster?

Accepted Answer

Open replication port between all members.. A functional Splunk Search Head Cluster requires that every member can replicate configuration and artifacts to the other members, which is why option D is correct: the replication port (default 8080, configurable via replication_port in server.conf) must be open between all members so the captain and members can push knowledge objects and run-time artifacts. Option E is also correct because all members must share identical security keys (the pass4SymmKey in the [general] stanza of server.conf) so they can authenticate to each other and form the cluster; mismatched keys prevent members from joining. Option A is not required because a search head cluster uses its own replication mechanism rather than a shared network file system, and option B is not required because a dedicated license master is not a prerequisite (a license master may be shared or the cluster can use a license manager, but it is not mandatory for cluster functionality). Option C is not required because a search head cluster uses a captain elected from among its members, not a Cluster Master node, which is used for indexer clusters.

Answer

A shared network file system.

Answer

A dedicated license master.

Answer

A Cluster Master node.

Question 15

You are investigating a search issue where results are inconsistent across SHC members. What is a possible cause?

Accepted Answer

Different indexes.conf versions on SHC members.. In a Search Head Cluster (SHC), all members must have identical configuration, including indexes.conf, to ensure consistent search results. If indexes.conf differs across members, some search heads may reference indexes or retention settings that others do not, producing inconsistent results. This is a classic SHC configuration drift issue.

Answer

Indexer peers are in maintenance mode.

Answer

The forwarder is sending bad data.

Answer

The license master is unreachable.

Answer

The Cluster Master is too busy.

Question 16

Which THREE of the following are components involved in a distributed search architecture?

Accepted Answer

Deployment Server. In a distributed search architecture, the Search Head (B) is the component that receives and dispatches search requests, coordinates the search across peers, and consolidates the results returned to the user. The Indexer (D) is the component that stores the indexed data and performs the actual search processing on its local buckets, returning partial results to the search head. The Deployment Server (A) is also part of the distributed search ecosystem because it distributes configuration and app bundles to search heads, indexers, and forwarders, ensuring consistent knowledge objects and settings across the distributed deployment. An Application Server (C) and a Database Server (E) are generic enterprise roles not specific to distributed search; they are not search-head, indexer, or deployment-management components in this architecture.

Answer

Application Server

Answer

Database Server

Question 17

Which TWO of the following are true about the 'splunk apply shcluster-bundle' command?

Accepted Answer

It distributes configuration bundles to all members.. Option A is correct because 'splunk apply shcluster-bundle' is the CLI command used to push the current configuration bundle from the deployer (or captain) out to all members of the search head cluster, ensuring every member receives the same updated configuration. Option D is correct because after the bundle is distributed, the search head cluster members must be restarted (or the bundle must be applied with the appropriate restart behavior) for the new configuration to take effect, since bundle application on an SHC requires a restart of the members. Option B is incorrect because indexer cluster management uses different commands such as 'splunk apply cluster-bundle' on the master node, not the shcluster-bundle command. Option C is incorrect because 'splunk apply shcluster-bundle' must be run from the deployer (or the captain in certain configurations), not from any arbitrary search head cluster member. Option E is incorrect because the '--skip-version-check' flag is not a required flag for this command; it is optional and only used to bypass version compatibility checks in specific situations.

Answer

It is used for indexer cluster management.

Answer

It can be run from any SHC member.

Answer

It requires the '--skip-version-check' flag.

Question 18

Which TWO of the following are true about Indexer Discovery?

Accepted Answer

It dynamically provides a list of indexers to forwarders.. Indexer Discovery is a Splunk mechanism that dynamically provides forwarders with the current list of peer indexers in an indexer cluster, so option C is correct because the master node pushes the updated peer list to forwarders via the indexer discovery endpoint. Option E is correct because Indexer Discovery relies on the indexer cluster's master node to supply that peer list, meaning the indexers must belong to an Indexer Cluster. Option A is incorrect because Indexer Discovery is a forwarder-to-indexer mechanism, not a way for SHC members to locate indexers; SHC members use their own search head clustering configuration. Option B is incorrect because Indexer Discovery is configured on the forwarders (typically via outputs.conf with indexerDiscovery), not on every forwarder as a universal requirement in the sense implied, and it is not a mandatory setting for all forwarders. Option D is incorrect because Indexer Discovery does not replace a load balancer; it is an alternative distribution method, and load balancers can still be used in other architectures.

Answer

It is used for SHC members to find indexers.

Answer

It must be configured on every forwarder.

Answer

It replaces the load balancer.

Question 19

Which THREE of the following are reasons to use Maintenance Mode in an Indexer Cluster?

Accepted Answer

To modify cluster configuration settings.. Maintenance Mode in an Indexer Cluster is designed for controlled administrative operations that would otherwise trigger unnecessary bucket replication or search degradation. Option A is correct because modifying cluster configuration settings (such as replication_factor or search_factor) requires Maintenance Mode so the cluster does not immediately re-replicate or rebalance buckets while the change is being applied. Option B is correct because a rolling upgrade of the indexers is performed in Maintenance Mode to prevent the cluster from initiating re-replication each time a peer is taken down and brought back up during the upgrade. Option D is correct because Maintenance Mode suppresses the automatic re-replication of buckets when a peer restarts, avoiding wasted replication work and preserving cluster stability. Option C is not a valid reason, as clearing the cache of the Cluster Manager is not a Maintenance Mode function and is not a supported operational task in this context. Option E is incorrect because Maintenance Mode does not improve search performance; in fact, it can temporarily reduce search capabilities since primaries may be offline and searchable copies may be affected.

Answer

To clear the cache of the CM.

Answer

To speed up search performance.

Question 20

Which TWO of the following are necessary to configure a multi-site Indexer Cluster?

Accepted Answer

Define sites in server.conf.. Option A is correct because multi-site indexer clusters require site definitions in server.conf: each indexer and the cluster manager must declare their site (site0, site1, etc.) via the [general] stanza's site attribute, and the manager must also set multisite = true so the cluster operates in multi-site mode. Option D is correct because a multi-site cluster needs site-specific replication and search factors (e.g., site_replication_factor and site_search_factor in server.conf) so that copies of each bucket are distributed across sites according to the desired availability policy. Option B is not required, since a load balancer is optional and only used for distributing search or forwarding traffic, not a prerequisite for multi-site configuration. Option C is incorrect because the search factor is a cluster-wide setting configured on the manager, not something enabled individually on each indexer. Option E is incorrect because the license manager is a separate licensing concern and is not a requirement specific to configuring a multi-site indexer cluster.

Answer

Use a load balancer for all traffic.

Answer

Enable search factor on each indexer.

Answer

Configure the license master.