MLA-C01 Data Preparation for Machine Learning Practice Question

Exhibit

Refer to the exhibit. A data scientist runs the following AWS Glue ETL job script (Spark) to prepare data for ML:

```python
import sys
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job

args = getResolvedOptions(sys.argv, ['JOB_NAME'])
sc = SparkContext()
glueContext = GlueContext(sc)
spark = glueContext.spark_session
job = Job(glueContext)
job.init(args['JOB_NAME'], args)

datasource = glueContext.create_dynamic_frame.from_options(
    connection_type = "s3",
    connection_options = {"paths": ["s3://bucket/input/"]},
    format = "csv",
    format_options = {"withHeader": True}
)

applymapping = ApplyMapping.apply(frame = datasource, mappings = [("id", "int", "id", "int"), ("value", "string", "value", "double")])
...
```

The job fails with an error: "Job run failed: org.apache.spark.SparkException: Job aborted due to stage failure: Task failed while writing rows." What is the most likely cause of this error?

A data scientist runs the exhibit AWS Glue ETL job. The job fails with a Spark stage failure error. What is the most likely cause?

⚠ Common exam trap

The trap here is that candidates often attribute Spark stage failures to resource issues (memory or missing paths) rather than recognizing that data type casting errors during transformations are a primary cause of stage-level failures in Glue ETL jobs.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

The data type mapping in ApplyMapping is incorrect; "value" column contains non-numeric strings that cannot be cast to double.

The Spark stage failure error in an AWS Glue ETL job is most likely caused by a data type mismatch during the ApplyMapping transformation. When the 'value' column contains non-numeric strings that cannot be cast to double, Spark throws a stage failure because it cannot complete the required type conversion, leading to task failures and job termination.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
The output path is missing.
Why it's wrong here
Missing output path error would occur before the job runs.
✗
The S3 bucket does not exist.
Why it's wrong here
Missing bucket error would occur at job initialization, not during task execution.
✗
The job does not have enough memory.
Why it's wrong here
Insufficient memory usually results in OutOfMemoryError, not a task writing failure.
✓
The data type mapping in ApplyMapping is incorrect; "value" column contains non-numeric strings that cannot be cast to double.
Why this is correct
Casting string to double fails on non-numeric data, causing task failure.

Go deeper

MLA-C01 Data Preparation for Machine Learning Practice Question

Exhibit

Why each option matters

Option-by-option breakdown

Related to this question