AI0-001 AI Infrastructure and Technologies Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✓

  Apache Spark with Structured Streaming

  Why this is correct

  Structured Streaming runs the feature engineering as continuous incremental queries over Kafka topics, writing curated results to the data lake, and reuses Spark's ML libraries for inference. It handles both the streaming transformation and lake writes in one engine, unlike batch-only or queue-only alternatives.

• ✗

  Apache Airflow

  Why it's wrong here

  Airflow orchestrates scheduled batch workflows; its DAG runs are time-triggered rather than continuous, so it cannot process Kafka events as they arrive. It is tempting because Airflow coordinates data pipelines and dependencies, but that fits periodic batch ETL, not real-time streaming feature engineering.

• ✗

  TensorFlow Data Validation

  Why it's wrong here

  TensorFlow Data Validation analyses and validates datasets for training-serving skew; it neither consumes Kafka streams nor writes engineered features to a data lake. It is tempting because it performs feature statistics and schema checks, but that suits offline training data validation, not continuous stream processing.

• ✗

  SageMaker Processing jobs

  Why it's wrong here

  SageMaker Processing jobs run batch, non-streaming workloads that terminate after completion, so they cannot continuously consume Kafka topics or emit low-latency features. It is tempting because Processing jobs handle feature engineering and data transformation, but that fits scheduled batch pipelines, not an always-on streaming inference path.