A data engineering team wants to ingest streaming data from Pub/Sub, transform it using Apache Beam, and load it into BigQuery for real-time analytics. They need a fully managed solution that handles autoscaling and does not require managing servers. Which TWO Google Cloud services should they use?
Dataflow runs Beam pipelines with autoscaling and serverless processing.
Why this answer
Dataflow is the fully managed service for executing Apache Beam pipelines, with autoscaling and serverless execution. Pub/Sub is the ingestion service. Cloud Composer is Airflow, not streaming; Dataproc is for Spark/Hadoop; Dataprep is for data preparation.