A data engineer is designing a data pipeline for a real-time recommendation system. The pipeline must handle high velocity streams and ensure data quality. Which three components should be included in the pipeline? (Select THREE).
Stream processing engines process data in real-time with low latency.
Why this answer
Apache Kafka Streams is a correct choice because it is a stream processing library specifically designed for building real-time applications and microservices that process data in motion. For a high-velocity recommendation pipeline, it provides exactly-once semantics, stateful processing (e.g., windowed joins, aggregations), and seamless integration with Kafka topics, enabling low-latency transformations without requiring an external cluster.
Exam trap
CompTIA often tests the distinction between stream processing and batch processing, and the trap here is that candidates mistakenly select a batch framework like Apache Spark or a data warehouse because they associate 'data pipeline' with traditional ETL, overlooking the strict real-time and low-latency requirements of the scenario.