Streams and Lambda Pipelines

Streams and Lambda Pipelines are a way to process collections of data (like ArrayLists or arrays) without writing complicated for-loops or while-loops. Instead of telling the computer step-by-step 'take this element, do X, then take the next element, do X', you describe the overall processing logic in one clear chain of operations. The computer handles the iteration internally.

Let's start with the absolute basics. A lambda expression (or just 'lambda') is an anonymous function — a short block of code that can be treated like data. You don't give it a name, you just write the parameter and the operation. For example, (String s) -> s.length() is a lambda that takes a string and returns its length. The arrow (->) separates the parameter list from the body. The lambda replaces what used to require a full anonymous inner class. It is the building block of the pipeline.

A Stream is a sequence of elements that supports sequential and parallel aggregate operations. It is not a collection itself — it is a view, or a conveyor belt, over a data source (list, array, or Set). Crucially, a stream does not modify the underlying data source. It creates a new stream with the result of each operation. This is called a 'stream pipeline'.

The three parts of a stream pipeline are:

Why does this matter? Before Java 8, you would write something like:

List<String> names = new ArrayList<>(); for (Person p : people) { if (p.getAge() > 18) { names.add(p.getName().toUpperCase()); } }

This works, but it is verbose, it mutates an external list (bad for concurrency), and if you make a mistake, it is hard to pinpoint. With streams and lambdas, you write:

List<String> names = people.stream() .filter(p -> p.getAge() > 18) .map(p -> p.getName().toUpperCase()) .collect(Collectors.toList());

This code is declarative — you state what you want (filter, map, collect), not how to iterate. The pipeline is lazy: filter and map do nothing until collect is called. Then they process each element in one pass. This is more efficient and easier to parallelise.

Now, let us drill into the key operations. The filter() method takes a Predicate — a lambda that returns a boolean. It keeps only elements for which the predicate returns true. The map() method takes a Function — a lambda that transforms each element into something else (possibly a different type). The collect() method is a terminal operation that accumulates elements into a mutable container, like a List, Set, or Map. The Collectors utility class provides ready-made collectors: toList(), toSet(), joining(), groupingBy(), and many more.

The reduce() method is another terminal operation that performs a reduction on the elements. It takes a binary operator (a lambda with two parameters that returns a single value) and combines elements one by one. For example, numbers.stream().reduce(0, (a, b) -> a + b) sums all numbers. The first argument (0) is the identity value — the starting point.

A common point of confusion: streams can be used only once. After calling a terminal operation, the stream is consumed. If you try to call another operation on it, you get an IllegalStateException.

Finally, parallel streams (people.parallelStream()) can split the data across multiple threads, but they require that the lambda operations be stateless and non-interfering — they must not modify shared state. The exam tests whether you know that parallel streams can give different order results, and that you must not rely on ordering unless you use forEachOrdered() or collect().

Imagine you are a developer for a large online retailer. The system holds a daily log of 10 million customer orders. Your manager asks you to produce a report: a list of the top 10 most popular product categories today, sorted by number of orders, but only for customers who spent more than £100 and are not flagged as internal test accounts.

Without streams, you would write nested loops, an if-statement for each condition, a custom comparator for sorting, and a manual count using a HashMap. You would inevitably introduce a bug — perhaps forgetting to skip test accounts, or mutating the map while iterating, causing a ConcurrentModificationException.

With streams and lambda pipelines, you write:

Map<String, Long> categoryCounts = orders.stream() .filter(o -> o.customer().getTotalSpent() > 100) .filter(o -> !o.customer().isTestAccount()) .collect(Collectors.groupingBy( o -> o.product().getCategory(), Collectors.counting() ));

List<String> topCategories = categoryCounts.entrySet().stream() .sorted(Map.Entry.<String, Long>comparingByValue().reversed()) .limit(10) .map(Map.Entry::getKey) .collect(Collectors.toList());

In the first pipeline, you filter out irrelevant orders and then group the remaining ones by category, counting each group. This replaces a loop with a conditional and a HashMap update. In the second pipeline, you take the map entries, sort by count descending, keep only the first 10, extract only the category names, and collect them into a list.

Behind the scenes, the JVM can optimise this pipeline — for example, it may fuse the two filter operations into one pass, or automatically parallelise the grouping if the data set is large. You do not need to manage threads.

A real IT professional also uses streams for file processing. For example, reading lines from a large log file:

Files.lines(Paths.get("server.log")) .filter(line -> line.contains("ERROR")) .map(line -> line.substring(0, 19)) .distinct() .forEach(System.out::println);

This reads the file lazily (each line is read only when needed), filters errors, extracts the timestamp, deduplicates timestamps, and prints them. The file is automatically closed when the stream is closed (using try-with-resources). Managing this with a BufferedReader and manual loops would require careful resource cleanup and more error-prone code.

Key actions an IT professional uses daily with streams: