1
medium
Your company runs a data processing pipeline on Cloud Dataproc. The pipeline reads data from Cloud Storage, processes it with Spark, and writes results to BigQuery. Recently, the pipeline has been failing with errors indicating insufficient disk space on the worker nodes. The cluster is configured with standard worker nodes with 100 GB of standard persistent disk. The data size being processed has grown from 50 GB to 150 GB. What is the most cost-effective way to resolve the disk space issue?