SPLK-2002 Deployment Architecture And Clustering — Timed 20 Questions

Question 1

A Search Head Cluster (SHC) is experiencing frequent 'captain election' issues during high-load periods. Upon reviewing the logs, you notice network latency spikes between nodes. Which parameter in the SHC configuration should be adjusted to allow for more tolerance in node communication?

20 minute time limit — choose an answer to begin.

Accepted Answer

heartbeat_timeout. In a Search Head Cluster, the captain is elected and maintained via heartbeat messages between members. The 'heartbeat_timeout' setting defines how long a member waits before considering another member unreachable. Raising it tolerates longer network latency spikes without triggering unnecessary captain elections.

Answer

election_timeout

Answer

replication_factor

Answer

search_timeout

Question 2

In an Indexer Cluster, you want to perform a rolling restart to update the configuration. Which CLI command ensures that the rolling restart happens only after the indexer cluster has reached a state of 'Complete' synchronization?

Accepted Answer

splunk rolling-restart cluster-peers. The 'splunk rolling-restart cluster-peers' command allows for wait-related flags, but specifically, one must ensure the cluster is ready before initiating the rolling process to prevent data loss or search degradation.

Answer

splunk apply cluster-bundle --force

Answer

splunk rolling-restart cluster-peers --check-status

Answer

splunk restart indexer

Question 3

You are managing a large Indexer Cluster and need to reduce the time it takes for a new node to catch up. Which setting allows you to limit the amount of bandwidth used during bucket replication?

Accepted Answer

replication_throttle. The 'replication_throttling' setting in server.conf allows for control over the network bandwidth used for bucket replication, preventing saturation.

Answer

cluster_bandwidth_control

Answer

bandwidth_limit

Answer

max_replication_speed

Question 4

You are designing an Indexer Cluster with three sites. To ensure data availability during a site failure, you have configured multisite replication. Which setting must be explicitly defined in server.conf on the cluster master to ensure that buckets are replicated across sites according to your policy?

Accepted Answer

multisite_replication_factor. In a multisite indexer cluster, the cluster master uses multisite_replication_factor in server.conf to define how many copies of each bucket must exist across sites (expressed as site:count pairs, e.g., site1:2,site2:1). This is distinct from the standard replication_factor, which governs total copies within a site. Without explicitly setting multisite_replication_factor, the cluster cannot enforce cross-site replication policy and site-failure availability is not guaranteed.

Answer

replication_factor

Answer

site_replication_policy

Answer

cluster_replication_mode

Question 5

Which component is responsible for communicating with the License Master to verify indexer entitlements in an Indexer Cluster?

Accepted Answer

Indexer Peer. Every peer node in an indexer cluster must communicate directly with the License Master.

Answer

Deployment Server

Answer

Search Head

Answer

Cluster Master

Question 6

When configuring a Search Head Cluster, what is the purpose of the 'deployer'?

Accepted Answer

To manage the replication of search configurations. The deployer is a standalone Splunk instance that distributes apps, configurations, and knowledge objects (like saved searches, lookups, and field extractions) to all members of a Search Head Cluster. It pushes the replicated configuration bundle to the cluster, and the captain then handles distributing it to members. The deployer does not participate in search, captain election, or user load balancing.

Answer

To coordinate the election of the captain

Answer

To act as a load balancer for users

Answer

To store the search results for the cluster

Question 7

In a multisite indexer cluster, what does the 'site' designation in server.conf specifically control?

Accepted Answer

The replication policy and bucket placement. In a Splunk multisite indexer cluster, the site designation in server.conf (under the [general] stanza) defines which site an indexer belongs to. This directly controls the cluster's replication and search factor policies, as well as bucket placement and replication across sites. The site value is used by the cluster master to enforce multisite replication rules, such as origin site replication and total site replication, ensuring data redundancy and search affinity.

Answer

The load balancing algorithm

Answer

The user access rights

Answer

The search head affinity

Question 8

When configuring a Search Head Cluster, what is the best practice for managing the 'shcluster-bundle' directory?

Accepted Answer

Version control the bundle in a separate directory. The `shcluster-bundle` directory contains the configuration bundle that the captain pushes to members. Best practice is to version control this bundle in a separate directory (e.g., using Git) to track changes, enable rollback, and maintain an audit trail. This ensures that configuration changes are managed systematically and can be reverted if needed.

Answer

Delete it after every deployment

Answer

Keep it in the /opt/splunk/etc/system/local directory

Answer

Automatically sync it to the Cluster Master

Question 9

A customer needs to decommission a single indexer peer from an Indexer Cluster without losing data. Which step is mandatory to ensure all data is safely migrated to other peers?

Accepted Answer

Run 'splunk remove cluster-peers' on the CM. To safely decommission an indexer peer from a Splunk indexer cluster, you must run 'splunk remove cluster-peers' on the cluster master (or use the equivalent offline command) so the master can reassign the peer's buckets to other peers and maintain replication and search factors. Simply shutting down the peer risks under-replication and data loss if other copies fail.

Answer

Force a cluster bundle update

Answer

Delete the indexer directory on the CM

Answer

Simply shut down the indexer service

Question 10

You notice that your indexer cluster has buckets in a 'Searchable' state but some are marked as 'Streaming'. What does this indicate?

Accepted Answer

Bucket replication is in progress to other peers. In an indexer cluster, a bucket marked 'Streaming' alongside 'Searchable' indicates that the bucket is currently being replicated to other peer nodes to satisfy the replication_factor. The 'Searchable' state means the bucket is available for search on at least one peer, while 'Streaming' means replication is actively in progress. Once replication completes, the Streaming flag clears.

Answer

The bucket is corrupt

Answer

The indexer is performing a hot-to-warm rollover

Answer

The data is being archived to cold storage

Question 11

Which setting in server.conf on the Cluster Master prevents buckets from being replicated to peers that have been offline for a specific amount of time?

Accepted Answer

rep_max_offline_time. The 'stale_bucket_threshold' or similar replication settings help ensure that the cluster does not attempt to replicate data to long-dead nodes.

Answer

max_peer_offline_time

Answer

bucket_expiry_time

Answer

cluster_peer_timeout

Question 12

Which TWO of the following are necessary prerequisites for configuring an Indexer Cluster?

Accepted Answer

Unique server names for every peer. All indexer peers must have the same server name if they are to be uniquely identified, and they must communicate with the CM using the same security key.

Answer

A dedicated license master

Answer

A dedicated load balancer for ingestion

Answer

A shared storage array for all peers

Question 13

Which TWO of the following are true regarding the Splunk 'Deployment Server' (DS) in a clustered environment?

Accepted Answer

The DS is used to distribute apps to indexers. Option C is correct because a Deployment Server is the standard Splunk mechanism for pushing apps and configuration bundles (via deployment apps and server classes) out to indexers, including those in an indexer cluster. Option E is correct because the same Deployment Server can distribute apps to search heads, including members of a search head cluster, using server classes targeted at those hosts. Option A is wrong because the DS is not mandatory for an indexer cluster; the cluster requires a cluster manager, and the DS is an optional management/distribution tool. Option B is wrong because indexer cluster configurations (replication factor, search factor, etc.) are managed through the cluster manager, not the Deployment Server. Option D is wrong because search head cluster captain election is handled by the search head cluster members themselves via the captaincy protocol, not by the Deployment Server.

Answer

The DS is a mandatory component for indexer clusters

Answer

The DS should be used to update indexer cluster configurations

Answer

The DS manages the search head cluster captain election

Question 14

Which THREE conditions must be met for an indexer to be successfully added to an Indexer Cluster?

Accepted Answer

The indexer must have network connectivity to the CM. Option A is correct because a peer indexer must be able to reach the Cluster Manager (CM) over the network on the management port (default 8089) so it can register, receive the cluster configuration bundle, and participate in replication and search factor enforcement. Option B is correct because the indexer and the CM must share the same pass4SymmKey in the [clustering] stanza of server.conf; without a matching security key, the peer cannot authenticate to the cluster and will be rejected. Option E is correct because a peer indexer must be added with no existing buckets (an empty indexer), since the CM replicates and distributes bucket copies across peers and pre-existing data would violate cluster bucket management. Option C is not required, as peers may run different supported OS versions as long as they run compatible Splunk Enterprise versions. Option D is not required, since a search head is a separate cluster role and an indexer joining as a peer is not designated as a search head.

Answer

The indexer must be running on the same OS version as the CM

Answer

The indexer must be designated as a search head

Question 15

Which THREE features are provided by Splunk's multisite Indexer Clustering?

Accepted Answer

Site-specific bucket replication policies. Splunk multisite Indexer Clustering lets you configure site-specific replication and search factor policies (option A), so each site can maintain its own replication factor for buckets, which is essential for meeting local data-residency or redundancy requirements. It also provides site-aware search affinity (option C), where the search head prefers to dispatch searches to indexers in its own site to reduce cross-site latency and bandwidth usage. Additionally, multisite clustering ensures data availability in the event of a site failure (option E), because buckets replicated to other sites allow the cluster to maintain searchable copies and continue serving data when an entire site goes down. Options B and D are not features of multisite Indexer Clustering: the cluster does not automatically load-balance across sites (search heads use site affinity, not automatic balancing), and it does not migrate user roles between sites, since user authentication and role management are handled by the search head or external authentication, not by the indexer cluster.

Answer

Automatic load balancing across sites

Answer

Automatic user role migration between sites

Question 16

If you need to move a bucket from one peer to another in an indexer cluster manually, how should you proceed?

Accepted Answer

Use the Cluster Master to trigger a rebalance. To move a bucket between peers in an indexer cluster, the correct approach is to use the Cluster Master to trigger a rebalance (or a fixup). The CM orchestrates bucket placement according to replication and search factors, and a rebalance redistributes buckets across peers to balance load. Manually moving files bypasses the CM's metadata and corrupts cluster state.

Answer

Edit the bucket.json file on the peer

Answer

Delete the bucket directory from the peer

Answer

Use the 'mv' command on the filesystem

Question 17

A Splunk indexer cluster has a replication_factor of 3 and a search_factor of 2. The cluster manager is configured with site_replication_factor = origin:2, total:3 and site_search_factor = origin:1, total:2 for a multisite cluster with two sites. A peer in site1 fails and is removed from the cluster. What happens to the buckets that were replicated on that failed peer?

Accepted Answer

The cluster manager immediately creates new replicas on other peers in site1 to satisfy origin:2, and if site1 lacks capacity, it creates replicas in site2 to satisfy total:3.. When a peer fails in a multisite cluster, the cluster manager initiates bucket fixup to restore the configured site_replication_factor and site_search_factor. It prioritizes creating copies within the same site to meet the origin requirement, but will use the other site if necessary to achieve the total count. This ensures that data remains highly available and searchable even after a peer loss.

Answer

The cluster manager promotes the remaining searchable copies in site1 to primary buckets and stops replicating to site2 to avoid cross-site traffic.

Answer

The cluster manager waits for the failed peer to come back online before taking any action, because multisite replication requires all original peers to be available.

Answer

The buckets on the failed peer are marked as searchable only in site2, and the cluster manager reduces the replication_factor to 2 to compensate.

Question 18

A Splunk administrator is deploying a multisite indexer cluster with two sites: site1 and site2. The cluster master is configured with site1 as the primary site. The administrator wants to ensure that search affinity is honored and that the cluster can tolerate the loss of an entire site. Which two configurations are required to achieve these goals? (Choose two.)

Accepted Answer

Set site_search_factor to origin:1, total:2.. To tolerate the loss of an entire site in a multisite cluster, you must configure site_replication_factor and site_search_factor so that each site holds a complete set of copies. With two sites, site_replication_factor should be set to origin:2, total:4 to ensure each site has two copies, and site_search_factor to origin:1, total:2 to ensure each site has a searchable copy. These settings guarantee that if one site fails, the other site can continue to serve searches without data loss.

Answer

Set replication_factor to at least 2 and search_factor to at least 1.

Answer

Set search_affinity to true on all search heads.

Answer

Set multisite to true in the cluster master's server.conf.

Question 19

When planning a deployment, which component is required to act as the primary communication hub for managing configuration bundles in an Indexer Cluster?

Accepted Answer

Cluster Master. The Cluster Master (now called Cluster Manager in newer versions) is the central coordination point for an indexer cluster. It manages the replication and search factors, distributes configuration bundles to peer nodes, and coordinates bucket replication and fixup. It is the required primary communication hub for configuration bundle management in an indexer cluster.

Answer

License Master

Answer

Deployment Server

Answer

Search Head

Question 20

You are configuring a Search Head Cluster and need to ensure that local user configurations are not overwritten by cluster-wide bundles. What is the correct way to handle these local configurations?

Accepted Answer

Use the local/ directory for configuration files that should not be replicated. In a Search Head Cluster, configurations are replicated from the deployer or captain via bundles, but files in the local/ directory of an app or system are not replicated and are preserved on each member. To keep local user configurations from being overwritten by cluster-wide bundles, you place them in the local/ directory, which takes precedence and is not overwritten by replicated configurations.

Answer

Move files to the var/run directory

Answer

Set the configuration to immutable in server.conf

Answer

Use the shcluster/ directory