# Search-Time Transformations and the eval Command

> Chapter 10 of the Courseiva SPLUNK-CORE-CERTIFIED-USER curriculum — https://courseiva.com/learn/splunk-core-certified-user/search-time-transformations-and-eval

**Official objective:** 3.3 — Use eval and other search-time transformations to manipulate data

## Introduction

If you cannot manipulate data the moment you search it, you will be stuck reading raw log lines like a foreign language, unable to count, filter, or compare anything useful. This chapter teaches you how to use the eval command and other search-time transformations to parse, calculate, and reshape your data on the fly. For the SPLK-1002 exam, understanding these transformations is essential because nearly every search you write will rely on them to extract meaning from machine data.

## The Recipe Ingredients Analogy

75 grams of unsalted butter. That is the first thing you learn when making a lemon drizzle cake from a recipe book. The recipe is your search command. The ingredients in your kitchen are your raw log data. Search-time transformations are the modifications you make to those ingredients before you bake the cake, not before you bought them at the supermarket. The eval command is your chef's knife, mixer, and grater all in one.

Imagine you have a list of all the ingredients you bought that week: "butter, sugar, lemons, flour, eggs, milk." That list is your raw data. But your recipe for lemon drizzle cake needs the butter softened, the lemons zested and juiced, and the flour sifted. You cannot change what you bought at the supermarket — that is index-time data that is already stored. Instead, at the moment you start baking (search time), you use your tools (eval) to transform the raw ingredients: you convert the butter from a solid block to soft cubes, you extract the lemon zest from the whole lemons, and you measure the sugar into a separate bowl. These transformations happen in the moment, while you are cooking, and they do not change the original shopping list.

In Splunk, the raw log entries stay unchanged in the index. But with eval, you create new calculated fields right then, during the search. The cake you end up with is your search results — made from transformed ingredients, but the shopping list still shows only the original items.

## Core explanation

Search-time transformations are modifications you apply to your data while your search is running, not before it is stored. When Splunk ingests data at index time, it packages the raw event with some default fields such as host, source, and sourcetype. After that, the raw event is frozen in the index. You cannot go back and change what was written. But at search time — the moment you type your search command — you can create new fields, change values, compute calculations, and extract pieces of text from the raw event. This is exactly like getting a printed letter and using a highlighter, a calculator, and a pair of scissors on a copy of that letter, while the original stays untouched in the filing cabinet.

The eval command is the Swiss Army knife of search-time transformations. It lets you create new fields using expressions, functions, and operators. For example, suppose you have a field called `duration` that is stored as a string like "500ms". You cannot perform arithmetic on a string. But with eval, you can write `eval duration_seconds = tonumber(substr(duration, 1, -3)) / 1000` to strip the "ms" suffix, convert the remaining characters to a number, and divide by 1000 to get seconds. The new field `duration_seconds` appears only in your search results. The original `duration` field with "500ms" remains untouched in the index.

The syntax of eval is: `... | eval new_field = expression`. The expression can include:

- Arithmetic operators: `+`, `-`, `*`, `/`, `%` (modulo, the remainder after division)
- Concatenation operators: `.` (dot) to join strings together, like `eval full_name = first_name . " " . last_name`
- Comparison operators: `==`, `!=`, `<`, `>`, `<=`, `>=` which return 1 for true and 0 for false
- Logical operators: `AND`, `OR`, `NOT`, `XOR`
- Conditional functions: `if(condition, true_value, false_value)`, `case(condition1, value1, condition2, value2, ...)`, `coalesce(value1, value2, ...)` which returns the first non-null value
- Mathematical functions: `round(x)`, `ceil(x)`, `floor(x)`, `abs(x)`, `random()`
- String functions: `lower(string)`, `upper(string)`, `len(string)`, `substr(string, start, length)`, `trim(string)`, `ltrim(string)`, `rtrim(string)`
- Date and time functions: `strftime(time, format)`, `strptime(string, format)`, `now()`, `relative_time(time, offset)`
- Statistical functions: `sum(x)`, `avg(x)`, `count(x)`, `max(x)`, `min(x)` but these require the stats command, not eval alone

The power of eval is that it creates a new field for every single event in the result set. If you have 10,000 events, eval will compute the expression 10,000 times, once per event. This is fundamentally different from commands like `stats` which aggregate many events into one summary row. Eval keeps the event structure intact but adds or modifies fields on each row.

There are other search-time transformations too. The `rex` command extracts fields using regular expressions (a pattern-matching language). The `convert` command changes the data type of a field, like converting a string to a number. The `tonumber` and `tostring` functions inside eval do the same thing. The `fillnull` command replaces null (empty) values with a default. But eval is the most flexible because it combines mathematics, logic, and text manupulation in one place.

Why does Splunk separate index-time and search-time? Performance. If Splunk had to parse every possible field at index time, search would be faster but data ingestion would be unbearably slow, and you would use vast amounts of storage for fields you never use. By leaving transformation to search time, Splunk stores data efficiently and lets you decide what to extract only when you need it. This is why the SPLK-1002 exam emphasises search-time transformations: they are a core design principle of Splunk.

## Real-world context

A real IT professional at a mid-sized e-commerce company named Elena manages the company's Splunk deployment. The company runs a website selling handmade furniture. Every time a customer visits the site, the web server logs an event that includes the URL, the HTTP status code, the response time in milliseconds, and the user's IP address. The raw log looks something like: `192.168.1.1 - - [20/May/2025:14:23:11 +0000] "GET /product/12345?color=walnut HTTP/1.1" 200 4532 "-" "Mozilla/5.0" 342`. The number 342 at the end is the response time in milliseconds.

Elena's manager wants to know: which product pages took longer than 500 milliseconds to load last month? The raw log does not have a field called `response_time_seconds`. The URL contains the product ID inside a path like `/product/12345`, but there is no `product_id` field. Elena needs to create these fields at search time.

Here is what Elena does step by step:

- She uses the `rex` command to extract the product ID from the URL. She writes: `... | rex field=_raw "/product/(?<product_id>\d+)"`. This uses a regular expression to find the digits after `/product/` and creates a new field called `product_id`.
- She uses the `rex` command again to extract the response time from the end of the log line: `... | rex field=_raw "(?<response_time_ms>\d+)$"`. The dollar sign anchors the pattern to the end of the string, so it captures the final number.
- She uses `eval` to convert milliseconds to seconds: `... | eval response_time_seconds = response_time_ms / 1000`.
- She uses `eval` again to create a new field called `slow_page` that flags which events have a response time over 0.5 seconds: `eval slow_page = if(response_time_seconds > 0.5, "yes", "no")`.
- Finally, she searches for only the slow pages: `search slow_page="yes"` and then uses `stats count by product_id` to count how many times each product loaded slowly.

Elena's search in total looks like this:
```
index=web sourcetype=access_combined
| rex field=_raw "/product/(?<product_id>\d+)"
| rex field=_raw "(?<response_time_ms>\d+)$"
| eval response_time_seconds = response_time_ms / 1000
| eval slow_page = if(response_time_seconds > 0.5, "yes", "no")
| where slow_page="yes"
| stats count by product_id
| sort - count
```

The result is a table showing the product IDs that had the most slow loads. Elena's manager can then ask the development team to optimise those product pages. Without search-time transformations, Elena would have to manually scroll through thousands of log lines, calculate response times on a calculator, and jot down product IDs by hand. With eval and rex, she finishes the analysis in under five minutes.

Another common scenario: a help desk manager wants to see the average ticket resolution time. The tickets have a field `_time` for when the ticket was created and a field `closed_time` for when it was closed, but both are in epoch time (seconds since 1970). Elena uses `eval resolution_time_hours = (closed_time - _time) / 3600` and then `stats avg(resolution_time_hours) by department`. This immediate calculation lets the business spot which teams resolve tickets fastest.

## Exam focus

The SPLK-1002 exam tests your ability to use eval and other search-time transformations accurately and efficiently. You will see approximately 4-6 questions on this topic across the exam. The questions fall into several distinct categories.

Question type 1: Identify the correct eval syntax.
- The exam will give you a line of eval with deliberate errors, such as using `=` instead of `==` for comparison, or using single quotes instead of double quotes for strings. The correct pattern: `... | eval new_field = expression` with double quotes around literal strings.
- Trap: They might show `eval new_field = if condition then value1 else value2`. This is incorrect. The correct syntax is `if(condition, value1, value2)`. Memorise the comma-separated argument format.
- Trap: They might show `eval new_field = "string" + "another"` using plus sign for concatenation. In Splunk eval, plus is for addition, not string joining. You must use the period `.` operator: `"string" . "another"`.

Question type 2: Which command performs a search-time transformation?
- The exam lists commands like `eval`, `rex`, `convert`, `fields`, `rename`. All four are search-time transformations, but `fields` and `rename` are not typically classified as "transformations" in the same sense. The exam expects you to know that `eval`, `rex`, `convert`, and `fillnull` are the primary transformation commands. They may try to trick you by including `stats` or `timechart` which are aggregations, not row-by-row transformations.

Question type 3: Choose the correct function or operator for a given task.
- Example: "You need to join two string fields with a space between them. Which operator do you use?" Answer: the dot operator `.`.
- Example: "You need to return the first non-null value from a list of fields. Which function?" Answer: `coalesce`.
- Example: "You need to round a value to one decimal place. Which function?" Answer: `round(x, 1)`.

Question type 4: Understand the difference between index-time and search-time.
- The exam asks: "A field is missing from your search results. Should you edit the props.conf file to extract it at index time, or use eval at search time?" The trap is that many beginners think you must always extract at index time. The correct answer is usually search-time via eval or rex, because it is more flexible and does not require re-indexing.
- Key definition to memorise: "Index-time field extraction happens when data is ingested. Search-time field extraction happens when a search is run."

Question type 5: Predict the output of an eval expression.
- They give you a mock event with existing fields and an eval expression, and ask what the new field value will be. For example: event has `bytes=2048` and `status=404`. Eval: `eval category = if(status==200, "success", if(status==404, "not found", "other"))`. The answer: "not found".
- Trap: They may use case-sensitivity. `if(status==200)` with lowercase `status` will not match a field named `Status` with capital S. Eval is case-sensitive with field names.
- Trap: They may include a function that requires a numeric argument, like `abs(x)`, but pass a string. The result will be null. The exam expects you to know that data types matter.

Key definitions to memorise for the exam:
- `eval`: creates or modifies a field using an expression.
- `rex`: extracts fields using regular expressions.
- `convert`: changes the data type of a field.
- `fillnull`: replaces null values with a default value.
- `if(condition, true, false)`: conditional function.
- `case(cond1, val1, cond2, val2, ...)`: multi-condition function.
- `coalesce(val1, val2, ...)`: returns first non-null value.
- `tonumber()`: converts a string to a number.
- `tostring()`: converts a number to a string.
- `round(x, p)`: rounds x to p decimal places.
- `substr(string, start, length)`: extracts a substring (start index begins at 1).
- `len(string)`: returns the character length of a string.
- `lower(string)` and `upper(string)`: changes case.
- `.`: concatenation operator for strings.

## Step by step

1. **Identify the Field You Need to Create** — Before writing any eval command, determine what new field you want to create and what data you will base it on. For example, if you have a field called 'response_time_ms' and you want it in seconds, your new field could be 'response_time_seconds'. This step prevents you from writing eval expressions that produce meaningless fields.
2. **Write the Basic Eval Command Structure** — Type the pipe character followed by the word eval, then the name of your new field, an equals sign, and the expression. For example: | eval response_time_seconds = response_time_ms / 1000. The new field name must start with a letter or underscore and contain only letters, numbers, or underscores.
3. **Choose the Correct Function or Operator** — Pick the appropriate Splunk eval function for your transformation. If you need to join two strings, use the dot operator. If you need a conditional value, use if(). If you need to extract part of a string, use substr(). Using the wrong function will either produce null values or cause your search to fail.
4. **Test the Eval Expression with Sample Data** — Run your search with a small time range and add the fields command after eval to see the newly created field values. For example: your_base_search | eval response_time_seconds = response_time_ms / 1000 | fields response_time_ms, response_time_seconds. This allows you to verify the calculation is correct before proceeding.
5. **Handle Data Type Mismatches** — If your eval expression returns null unexpectedly, check whether the source field is a string or a number. Use tonumber() on string fields before performing arithmetic, and use tostring() on numbers before concatenating with other strings. For example: eval total = tonumber(price) * quantity.
6. **Chain Multiple Eval Commands** — You can use multiple eval commands in a single search by separating them with pipe symbols, or you can combine multiple field creations into one eval command by separating them with commas. Example: | eval response_time_seconds = response_time_ms / 1000, slow = if(response_time_seconds > 0.5, 'yes', 'no'). Combining them is faster and keeps your search concise.
7. **Document Your Transformation** — Add a comment to your saved search explaining what the eval transformation does. In Splunk, comments start with a backtick (`) and end with a backtick. For example: `convert ms to seconds`. This helps other team members understand your logic and makes your searches maintainable.

## Comparisons

### eval vs rex

**eval:**
- Creates or modifies fields using expressions, functions, and operators
- Operates on existing field values without pattern matching
- Can perform arithmetic, string concatenation, and conditional logic

**rex:**
- Extracts fields using regular expressions (regex)
- Operates on raw text or existing field values by matching patterns
- Cannot perform arithmetic or conditional logic; only extraction

### if() function in eval vs case() function in eval

**if() function in eval:**
- Handles one condition with two outcomes: true and false
- Syntax: if(condition, true_value, false_value)
- Best for simple binary decisions (e.g., over/under threshold)

**case() function in eval:**
- Handles multiple conditions in order, returns first matching value
- Syntax: case(condition1, value1, condition2, value2, ...)
- Best for multi-category classifications (e.g., HTTP status code to status text)

### tonumber() vs tostring()

**tonumber():**
- Converts a string value to a numeric value
- Allows arithmetic operations on the result
- Returns null if the string cannot be parsed as a number

**tostring():**
- Converts a numeric value to a string value
- Allows string concatenation with other strings
- Often used before concatenation with the dot operator

### Index-time extraction (props.conf) vs Search-time extraction (eval/rex)

**Index-time extraction (props.conf):**
- Happens when data is ingested
- Fields are permanently available in every search
- Requires configuration file changes and potentially re-indexing

**Search-time extraction (eval/rex):**
- Happens when a search is run
- Fields are available only in that search
- No configuration changes needed, fully flexible

### eval with stats vs eval without stats

**eval with stats:**
- Uses eval to create per-event fields, then stats to aggregate across events
- Example: eval total = price * quantity then stats sum(total) by product
- Eval prepares data for aggregation

**eval without stats:**
- Uses eval only for per-event calculations
- No aggregation happens; each event retains its own calculated field
- Example: eval discounted_price = price * 0.9

## Diagram

_Flowchart showing the main search-time transformation commands and how they feed transformed data into aggregations._

```mermaid
flowchart TD
    A[Raw Log Events in Index] --> B[Search Command Starts]
    B --> C{Which transformation?}
    C -->|eval| D[Create new field using expression]
    C -->|rex| E[Extract field using regex]
    C -->|convert| F[Change data type of field]
    C -->|fillnull| G[Replace null with default]
    D --> H[Search results with transformed fields]
    E --> H
    F --> H
    G --> H
    H --> I[stats, timechart, etc.]
```

## Common misconceptions

- **Misconception:** Eval can change the original data in the index. **Reality:** Eval only affects the search results, not the underlying data stored in the index. The raw event remains unchanged. (This mistake is common because beginners see the new fields appearing in results and assume the data was permanently modified. They do not realise that changes are purely ephemeral for the current search.)
- **Misconception:** You can use the plus sign (+) to concatenate strings in eval. **Reality:** In Splunk eval, the plus sign (+) is used for numeric addition only. String concatenation uses the dot operator (.). (This mistake is common because many programming languages (like JavaScript, Python, Java) use plus for both addition and concatenation. Splunk's eval language is deliberately different to avoid ambiguity.)
- **Misconception:** The if function in eval uses the syntax if(condition) then value1 else value2. **Reality:** The correct syntax is if(condition, value1, value2) with commas separating the three arguments and no 'then' or 'else' keywords. (This mistake is common because 'if-then-else' is a natural language pattern that appears in many programming languages and SQL. Beginners write what feels intuitive rather than memorising the exact Splunk syntax.)
- **Misconception:** The eval command can replace the stats command for aggregations like sum or average. **Reality:** Eval works on individual events. To calculate aggregates across multiple events, you need stats or timechart. Eval can compute per-event mathematical fields, but it cannot sum a field across all events without a stats command. (This mistake is common because both commands involve calculations. Beginners try to write eval sum(bytes) as if eval were a spreadsheet cell that automatically aggregates, not realising that sum() inside eval requires a prior stats command.)
- **Misconception:** You must extract a field at index time using props.conf if you want to use it in a search at all. **Reality:** You can extract fields at search time using eval, rex, or other transformations. Index-time extraction is optional and mostly used for fields you need in every single search for performance reasons. (This mistake is common because beginners learn about props.conf early and assume it is the only way to create fields. They do not understand that search-time extraction is more flexible and often easier.)
- **Misconception:** Eval fields are available for use in subsearches or saved searches immediately after creation without any special handling. **Reality:** Eval fields are available in the same search pipeline after the eval command, but if you want to use them in a subsearch, you must pass them as arguments. Also, eval fields cannot be used in the same search before the eval command. (This mistake is common because beginners think all fields exist everywhere at all times. They do not grasp that the search pipeline is sequential: fields created by eval at step 2 are not available in step 1 before the eval command.)

## Key takeaways

- The eval command creates or modifies a field for every single event in the search result set using an expression.
- Search-time transformations like eval do not change the raw data in the index; they only affect the current search results.
- In eval, arithmetic uses standard operators (+, -, *, /, %) but string concatenation uses the dot operator (.), not the plus sign.
- The if function in eval requires three comma-separated arguments: if(condition, value_if_true, value_if_false).
- The case function allows multiple conditions and returns the value for the first condition that evaluates to true.
- The coalesce function returns the first non-null value from a list of fields or expressions.
- The rex command extracts fields using regular expressions and is a separate search-time transformation from eval.
- Data types matter in eval: functions like abs() require numeric values, and tonumber() converts strings to numbers before arithmetic.
- Search-time field extraction is more flexible than index-time extraction because it does not require configuration file changes or re-indexing.
- The order of commands in a search pipeline matters: eval fields are only available after the eval command, not before.

## FAQ

**Can I use eval to modify an existing field instead of creating a new one?**

Yes. If you use an existing field name on the left side of the equals sign, eval overwrites the value of that field in the search results. For example, `eval status = if(status=="200", "OK", "Error")` replaces the original status field values. The original data in the index is not affected.

**What is the difference between eval and rex?**

Eval creates or modifies fields using expressions, functions, and operators. Rex extracts fields using regular expressions (regex). Use rex when you need to pull a piece of text out of a larger string based on a pattern. Use eval when you need to calculate, concatenate, or conditionally assign values.

**Why does my eval expression return nothing when I know the source field has data?**

This usually happens because of a data type mismatch. If you try to perform arithmetic on a string field, eval returns null. Use tonumber() to convert the string to a number first. Also, check for leading/trailing spaces using trim(). Alternatively, the field name might be case-sensitive; verify you are using the exact case of the field.

**Can I use if statements inside if statements in eval?**

Yes, you can nest if functions. For example: `eval category = if(status==200, "success", if(status==404, "not found", "other"))`. This is equivalent to a case function. Each nested if must have commas between its three arguments.

**How do I concatenate a string and a number in eval?**

Use the dot operator after converting the number to a string with tostring(). Example: `eval result = "Order number " . tostring(order_id)` produces a string like 'Order number 12345'. If you forget to use tostring(), eval will convert the number to a string automatically in many cases, but it is safer to be explicit.

**What is the maximum length of an eval expression?**

There is no hard-coded maximum, but for readability and performance, keep expressions under a few hundred characters. If your expression is very long, consider breaking it into multiple eval commands or using a subsearch. The exam does not test a specific length limit but expects you to write clean, efficient expressions.

## Check your understanding

1. **What operator do you use in eval to join two strings together?**

   Answer: The dot operator (.) is used to concatenate strings in eval. For example, `eval full_name = first_name . " " . last_name`.

2. **Write the correct eval syntax to create a new field called 'slow' that has the value 'yes' if the field 'response_time' is greater than 500, and 'no' otherwise.**

   Answer: The correct syntax is: `eval slow = if(response_time > 500, "yes", "no")`. Note the commas separating the three arguments and the double quotes around the string values.

3. **What does the coalesce function do in eval?**

   Answer: Coalesce returns the first non-null value from a list of fields or expressions. For example, `eval priority = coalesce(priority_override, priority_default, 5)` uses priority_override if it is not null, otherwise priority_default, and finally 5 if both are null.

4. **Can you use the plus sign (+) to add two string fields together in eval?**

   Answer: No. The plus sign in eval is used only for numeric addition. To join strings, you must use the dot operator (.). Using plus on strings will treat them as numbers if possible, or return null if they cannot be converted.

5. **What is the difference between how eval and stats handle fields in a search?**

   Answer: Eval creates or modifies fields on a per-event basis, keeping each event as a separate row. Stats aggregates values across multiple events, producing a single summary row (or grouped rows). Eval preserves the original event count; stats reduces it.

---

Interactive version with quiz and diagrams: https://courseiva.com/learn/splunk-core-certified-user/search-time-transformations-and-eval