Splunk Enterprise Certified Admin (SPLK-1003) (SPLK-1003) (SPLK-1003) — Questions 76–150

The License Master's usage report attributes indexed volume to each peer by host and source, exposing the specific indexer breaching the daily licence quota. This directly satisfies the requirement to identify the responsible indexer, since the report aggregates per-indexer usage rather than cluster-wide totals.

indexes.conf defines index-specific settings. frozenTimePeriodInSecs specifies the age in seconds after which data is frozen (deleted or archived), and maxTotalDataSizeMB sets the maximum size of the index before freezing occurs. Setting frozenTimePeriodInSecs to 7776000 (90 days) and maxTotalDataSizeMB to 512000 (500 GB) achieves the desired retention and size limits.

The Universal Forwarder's own splunkd.log records connection failures, parsing errors and output pipeline issues before any data leaves the host. Since the indexer shows no incoming traffic, the fault lies on the forwarder side, so checking $SPLUNK_HOME/var/log/splunk/splunkd.log first isolates whether the forwarder is failing to connect or failing to read its inputs.

The 'admin' role is a predefined, built-in Splunk role, so it cannot be deleted, satisfying the stem's requirement for a true description. Unlike custom roles, which administrators can remove, built-in roles such as admin, power and user are permanent fixtures of the authentication system, ensuring at least one privileged account always exists.

frozenTimePeriodInSecs defines the age in seconds after which buckets roll to frozen. Seven days equals 604800 seconds, so this value causes data older than 7 days to be frozen and, absent a frozen archive path, deleted automatically. This directly satisfies the temporary retention requirement without manual steps.

outputs.conf is where forwarders define target indexers. The [tcpout] stanza sets global defaults such as defaultGroup and ssl settings. To configure a specific port and enable encryption, you edit the [tcpout] stanza and define a target group with server and ssl settings. This file and stanza are correct for this purpose.

The edit_user capability allows a user to create, edit, and delete users in Splunk Web. Without it, the Users page is inaccessible and the Settings menu may be hidden. Adding this capability to a custom role grants the necessary permission to manage users.

Port 9997 is the default port that a Splunk indexer listens on to receive data from Splunk forwarders using the Splunk-to-Splunk protocol. When you enable receiving via the Splunk Web UI or CLI, the indexer starts listening on 9997. Therefore, this port must be open on the firewall to allow forwarders to send data to the indexer.

The forwarder's deploymentclient.conf holds the targetUri setting, which specifies the Deployment Server's address and management port. Without this stanza, the forwarder never phones home, so the administrator's server-class and app deployment policies cannot reach it. This file satisfies the stem's requirement for establishing Deployment Server communication.

A Universal Forwarder contacts the Deployment Server over TCP port 8089; if a firewall blocks that path, the forwarder never checks in and appears as 'missing'. Ping success does not prove the management port is open, so connectivity is the most common cause.

System-level configuration in `$SPLUNK_HOME/etc/system/local/` takes precedence over an app's `default` directory, because Splunk's precedence order ranks system local above app default. This satisfies the stem's conflict: the same stanza and key in both files resolves to the system/local value.

Search factor determines how many searchable copies of each bucket exist in the cluster. To ensure that a bucket remains searchable after one indexer fails, the search factor must be at least 2. This means that even if the indexer holding the primary copy fails, another indexer with a searchable replica can serve search requests.

The rtsearch capability grants a role the ability to run real-time searches, a privilege Splunk reserves for Power Users alongside scheduling and accelerated searching. It satisfies the stem's requirement for a capability typically assigned to that role, since ordinary users cannot invoke real-time search.

License pools allow you to partition the total license volume among groups of indexers. By creating a pool with a 200 GB quota and assigning the sensitive indexer to it, you cap its usage at 200 GB. The remaining 800 GB stays in the default pool for other indexers. This enforces the desired limit without affecting the overall license.

The 'reload deploy-server' command forces the Deployment Server to push new configurations to clients.

Scheduled searches run under the context of their owner, and if that account's credentials become invalid the scheduler cannot authenticate and the search silently stops firing. Updating the stored credentials or reassigning ownership to a valid, enabled account restores execution. This matches the scenario where a password change broke the schedule without any alert configuration being involved.

TIME_PREFIX tells the Splunk parsing pipeline where the timestamp begins within an event, so it can extract the correct time value when the timestamp does not start the line. This satisfies the stem's requirement by anchoring timestamp recognition to a specific preceding string, enabling accurate event time assignment.

outputs.conf defines multiple target groups, each pointing to a separate indexer cluster, while transforms.conf uses a routing rule to select the target group per sourcetype. This combination satisfies the requirement to route data to two clusters based on sourcetype.

The User Filter is an LDAP search filter that narrows down the search results to the specific user attempting to log in. If the filter is too restrictive or incorrectly formatted, Splunk may fail to locate the user's DN, resulting in a failed login. It should be tested to ensure it matches the user objects in the directory.

maxDataSize specifies the maximum size, in megabytes, that a hot bucket can reach before it rolls to warm. Adjusting this value allows the administrator to control how much data is stored in a single bucket, directly impacting search efficiency and bucket management. Setting a lower value causes more frequent rolling, which can improve performance for high-volume indexes.

If an indexer fails to report to the license master, the master may not have accurate usage data and could generate a warning about missing or incomplete data. This can happen due to network issues, incorrect license master URI, or the indexer being down. The warning might indicate a violation because the master cannot verify total usage.

The Deployment Server centrally distributes configuration files and apps to forwarders through server classes, satisfying the requirement for managing many distributed instances from one point. It pushes updated configurations to clients via phone-home, rather than relying on manual edits per host, which is precisely the centralised management role described.

License pools allow separate tracking of license usage for a group of indexers. Each pool has a defined quota, and if the combined indexing volume of its members exceeds that quota, a violation is recorded for the pool. This helps isolate license consumption and manage compliance across different data types or business units.

The script stanza uses the script:// scheme with the script path, and interval = 60 schedules execution every 60 seconds. This satisfies the requirement to poll the script output once per minute, with Splunk indexing each run's stdout.

The _internal index is defined in indexes.conf. To change its maximum size, the administrator should edit the [_internal] stanza in indexes.conf. Placing it in system/local ensures it overrides any default settings. The setting 'maxTotalDataSizeMB' controls the maximum size. This is the correct and targeted approach to modify the internal index's size limit without affecting other indexes.

`$SPLUNK_HOME/etc/system/local` holds locally authored configuration that overrides defaults, satisfying the need to locate admin-created settings on a Splunk instance. Splunk merges this directory at highest precedence within the system layer, so custom `inputs.conf`, `outputs.conf` and `server.conf` changes made directly on the host reside here.

Heavy Forwarders run the full Splunk parsing pipeline, so they can inspect event content and apply transforms.conf routing rules to direct data by field or sourcetype. This content-based routing capability distinguishes them from Universal Forwarders, which forward raw data only.

A malformed user search filter prevents Splunk from locating the authenticating account in the directory, so the bind succeeds but no matching entry returns. This is a genuine LDAP troubleshooting issue because the filter directly governs which directory objects Splunk can resolve during authentication.

The 'splunk list forward-server' command displays the indexers that the forwarder is configured to send data to. If it shows no active forwarders, it indicates that the forwarder has no indexer targets defined, which is typically due to a missing or incorrect outputs.conf. This file specifies the indexer addresses and settings.

splunkd is the core Splunk Enterprise daemon that runs on the indexer and performs the actual indexing of incoming data, writing events to disk and building the index buckets. Other components such as the search head or forwarder handle parsing, searching or forwarding rather than indexing.

Editing props.conf in the app's local directory with the specific sourcetype stanza ensures the timestamp format and timezone settings apply only to this app. The local directory takes precedence over default, and configurations in an app's local context are scoped to that app, preventing impact on other apps.

When two different inputs read the same file, each input processes the data independently and sends it to the indexer, resulting in duplicate events. In this scenario, the file monitor input and the scripted input are both configured to read the same file, so every event is ingested twice. This is a common misconfiguration when administrators add a scripted input without disabling the existing file monitor. The solution is to remove one of the inputs or ensure they target different data sources.

frozenTimePeriodInSecs defines, in seconds, how long a bucket remains in the index before rolling to frozen and being archived or deleted. Setting it to 31536000 seconds enforces the 365-day retention requirement, which is exactly the parameter the stem asks for.

Splunk Enterprise blocks search after seven consecutive days of license violations. This means that if the indexed volume exceeds the license quota for seven days in a row, search is disabled for all users except those with the 'can_override_license' capability. The administrator must resolve the violation within this window to avoid search blocking.

The power role includes the 'schedule_search' and 'edit_search_schedule' capabilities, allowing users to create and schedule saved searches and alerts. This directly supports the analyst's need to save reports and set up alerts without requiring administrative privileges. It is a core part of the power role's design for advanced search users who are not administrators.

In a search head cluster, scheduled searches are automatically distributed among members, and search artifacts are replicated. The scheduler runs on each member for its assigned searches, and if a member fails, its searches are reassigned. This provides both load distribution and high availability without manual intervention.

The power role includes capabilities such as schedule_search, edit_search_schedule, and run saved searches, which allow the analyst to create and schedule searches and alerts. It does not grant app installation or user management capabilities, so it adheres to the principle of least privilege for this scenario.

When a license master is unreachable, indexers continue to index data during a grace period. Warnings are generated to alert administrators, and enforcement such as blocking searches or indexing occurs only after the grace period expires. This design prevents minor network interruptions from causing immediate data loss or service outages in a distributed deployment.

Local directories sit outside the app's default tree, so Splunk's upgrade process leaves them untouched while it replaces default files. This satisfies the stem's preservation constraint: custom configurations survive version upgrades without manual backup, unlike default directory content, which is overwritten by the installer.

In a distributed deployment, configuration files on the search head do not automatically propagate to indexers. The indexers need their own copy of props.conf, typically deployed via a deployment server or manually placed in the appropriate app. Since the sourcetype definition lives only on the search head, indexers cannot recognize it, which directly explains the observed failure in this scenario.

When creating license pools, the total quota allocated to all pools must not exceed the total available license volume. If you attempt to allocate more than the total, the configuration will be rejected. This ensures that the license master does not over-allocate license capacity, which could lead to violations across the deployment.

Splunk Web uses TCP port 8000 by default for the browser-based user interface. This is the management and search UI port, distinct from the management port 8089 and the receiving port 9997. An administrator can confirm the setting in web.conf under the [settings] stanza with httpport. Changing it requires a restart of Splunk Web or the splunkd process depending on the deployment.

Settings > Access Controls > Users lets an administrator create an account and assign roles such as 'power' directly, satisfying the requirement for the most direct interface. Splunk Web exposes user creation and role assignment together in this location.

This file on the Deployment Server defines server classes and maps them to deployment clients using filters such as host, IP, or DNS name. It is the correct location to specify which forwarders receive a particular app. The administrator would create a server class, add the app to it, and set the appropriate whitelist or blacklist to target the subset of forwarders.

The splunk install app command installs an app package from a local file path or URL, using the -auth argument to authenticate against the management port. It unpacks the app into the appropriate apps directory and registers it. This is the supported CLI method for installing an app on a single instance without a browser, exactly as the scenario requires.

Editing `log-local.cfg` on the Universal Forwarder sets per-component logging levels, satisfying the requirement to change one component's verbosity without affecting others. Splunk merges this file over `log.cfg`, so your stanza, such as `[component_name]` with `loglevel = DEBUG`, overrides defaults and persists across upgrades.

The Deployment Server pushes configuration bundles to Universal Forwarders via the deployment client protocol, satisfying the need to distribute files across a large fleet. Forwarders poll the server and download server classes matching their hostname or IP, enabling centralised management without manual intervention on each instance.

In Splunk, role capabilities are additive: a user inherits the union of all assigned roles' permissions. Because RoleA grants the rtsearch capability, the user retains real-time search access even though RoleB disables it. Disabled capabilities in one role cannot override enabled ones in another, so the effective capability is granted.

srchFilter prepends a specified search string to every search a role's users run, automatically restricting results to permitted data. This enforces role-based search filtering, matching the stem's stated purpose of prepending a filter to all searches.

License pools allow you to allocate a portion of the total license volume to specific indexers. By creating a pool with a 100 GB quota and assigning the indexer to it, that indexer's indexing volume is counted against the pool's quota. The rest of the license volume (400 GB) remains available to other indexers in the default pool. This effectively caps the indexer's usage.

The transforms.conf file defines field transformations, including regex-based sed and mask operations that redact or anonymise sensitive values during indexing. This directly satisfies the scenario's requirement for data masking, since masking is configured as a transform stanza referenced by props.conf, not handled elsewhere in the configuration set.

Heavy Forwarders run the full Splunk Enterprise instance, enabling index-time parsing, field extraction and routing before data leaves the host. Universal Forwarders only forward raw data, leaving parsing to the indexers. This satisfies the requirement for parsing at the collection point, reducing indexer load and enabling selective forwarding.

frozenTimePeriodInSecs defines the number of seconds after which buckets are frozen. Setting this to 63072000 seconds (2 years) will cause buckets older than that to be frozen. When combined with a frozen archive path, this automates the archiving and deletion process. This is the correct setting to enforce the retention policy.

Managing user roles sits under Settings > Users and Authentication in Splunk Web, where administrators create roles, assign capabilities and map users to them. This satisfies the stem's requirement for actions reachable from the Settings menu, since role administration is performed there rather than through configuration files.

Placing `inputs.conf` in `$SPLUNK_HOME/etc/apps/my_app/local/` is valid because app-level local directories hold user-defined configuration that overrides defaults without being overwritten during upgrades. This satisfies the stem's requirement for a legitimate custom location, since Splunk merges configuration by precedence, and the app's local folder ranks above its default folder.

The power role allows users to run searches, create reports, and use other knowledge objects, but it does not grant administrative privileges such as editing configurations or managing users. This matches the security analyst's needs exactly. It is a built-in role designed for advanced users who are not administrators.

Indexers are associated with pools through their licensing configuration, and the pool quota applies to the aggregate usage of all indexers in that pool. This aggregate enforcement is why splitting teams into separate pools provides independent accounting, and it accurately reflects how pool quotas constrain combined member usage in a distributed deployment.

The Monitoring Console License Usage view tracks daily indexing volume against the license quota, so an alert conditioned on reaching 90 percent of that quota fires before the 500 GB cap is breached. This gives lead time to throttle inputs or add license capacity instead of waiting for a violation and the resulting search restrictions.

License pools let administrators partition the license stack so a subset of indexers consumes only its allocated quota. Creating the pool, assigning stack members, and adding the indexers as pool members is the supported workflow, and it enforces the separate cap the scenario requires without affecting the rest of the deployment.

This stanza configures the forwarder to monitor the Windows Security event log. It is one of the required inputs to collect Event Log data. The stanza specifies the event log channel to monitor, and additional settings like disabled = 0 enable it. This satisfies the requirement to monitor Windows Event Logs.

Setting followTail=1 makes the monitor input skip existing file content and read only data appended after the input starts. This satisfies the requirement to ingest only new data, unlike default behaviour which reads from the beginning.

The [license] stanza in server.conf holds the manager_uri setting that tells this instance which license manager to contact on port 8089. Inspecting this file confirms the exact URI the search head is using, so the administrator can validate connectivity against the correct host and port before troubleshooting firewall rules or license warnings.

Settings > Licensing > Usage Report provides the daily licence consumption breakdown across a 30-day window, satisfying the stem's requirement for per-day visibility over that period. Unlike the Licence Usage dashboard, which shows only current totals, this report retains historical daily ingestion data, letting the administrator trend usage without building a custom search.

Splunk merges configuration by precedence, and app/local sits above system/local in the layer order. Because the app context is more specific than the system context, its inputs.conf stanza overrides the identical setting defined under system/local, so the app value wins.

The whitelist ensures only .log files are monitored. The ignoreOlderThan setting tells the forwarder to skip files whose modification time is older than 7 days. This directly meets both requirements: filtering by extension and age. This is the correct and efficient configuration.

The LDAP group mapping filter defines which directory groups Splunk ingests and maps to roles, satisfying the requirement to translate LDAP group membership into Splunk authorisation. Without it, group-based role assignment cannot occur, so it is commonly required alongside the user filter when configuring an LDAP strategy.

The User Filter is an LDAP search filter that is applied when searching for a user during authentication. By including a condition such as 'memberOf=CN=SplunkAdmins,OU=Groups,DC=example,DC=com', you ensure that only users who are members of that group are found and authenticated. This effectively restricts login to members of the specified group, meeting the requirement.

maxHotSpanSecs sets the maximum age a hot bucket may reach before Splunk rolls it to warm, directly satisfying the rotation trigger. Combined with maxHotBuckets, it bounds hot bucket lifetime by time rather than size, so the indexer rotates predictably even when ingest volume stays low.

The command 'splunk edit licenser-localslave -master_uri https://<license_master>:<mgmt_port>' configures the indexer to use the specified license master. The default management port is 8089. This command is run on the indexer and updates its license configuration to report to the license master.

The Universal Forwarder is a lightweight agent that consumes minimal resources while forwarding data to indexers, satisfying the requirement for a distributed Splunk component. It performs no parsing or indexing, unlike heavy forwarders, making it the standard mechanism for collecting and transmitting data across a Splunk deployment's tiers.

`maxTotalDataSizeMB` caps an index's on-disk footprint by triggering deletion of the oldest buckets once the threshold is crossed. Setting it to 512000 enforces the 500GB ceiling, satisfying the stem's disk-space constraint. Other settings govern retention by age or per-bucket sizing, not total index size.

Timestamp extraction and recognition rules are defined in props.conf, using settings such as TIME_PREFIX, TIME_FORMAT and MAX_TIMESTAMP_LOOKAHEAD within a sourcetype or host stanza. This file governs how Splunk parses event timestamps at index time, correcting the wrong-timestamp problem.

Duplicate serverName values cause Splunk to treat both Universal Forwarders as one logical source, so their internal metrics are aggregated under a single identity. This corrupts per-forwarder reporting and monitoring, since the licence and health data merge rather than remaining distinct.

Capability checks gate menu rendering, so a role lacking `license_tab` never receives the Licensing entry in Settings. Splunk evaluates capabilities before building the navigation, hiding rather than disabling the item. This satisfies the stem's constraint: the user sees no Licensing menu item at all.

Reassigning indexers to pools with more appropriate quotas can balance the load and prevent individual pools from being overwhelmed. This action helps distribute indexing volume across the available license capacity, reducing the risk of violations in any single pool.