DEA-C01 Data Ingestion and Transformation Practice Question

A company uses AWS Glue crawlers to populate the Data Catalog from data in Amazon S3. The crawler fails to update the schema when new columns are added to the CSV files. What is the most likely cause?

⚠ Common exam trap

DEA-C01 often tests the misconception that IAM permissions or S3 versioning affect schema detection, when the real issue is the use of a custom classifier with a fixed schema that prevents dynamic schema inference.

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

The crawler uses a custom classifier that defines a fixed schema.

AWS Glue crawlers use classifiers to infer the schema of data in Amazon S3. If a custom classifier is used and it defines a fixed schema, the crawler will not detect new columns because the classifier's schema is static. The crawler relies on the classifier to parse the data and determine the schema; if the classifier does not include the new columns, they will be ignored. Therefore, the most likely cause is that the crawler uses a custom classifier with a fixed schema.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
The S3 bucket has versioning enabled.
Why it's wrong here
S3 versioning preserves object versions; it does not prevent the crawler from reading current objects or detecting added columns. Versioning is relevant when recovering overwritten or deleted data, and it is a plausible-sounding distractor because it touches S3 configuration. Schema staleness stems from crawler scope, not bucket versioning.
✗
The crawler is configured to only crawl new partitions.
Why it's wrong here
Crawling only new partitions means the crawler never re-reads existing partitions where the added columns appear, so the schema stays stale. Incremental crawling suits append-only data with stable schemas. Detecting new columns requires re-crawling the affected partitions or the whole table, which is the actual fix.
✗
The IAM role for the crawler lacks permissions to read the new columns.
Why it's wrong here
Missing read permissions would cause the crawler run to fail outright with access errors, not silently produce an outdated schema. IAM gaps are a common suspect when crawlers misbehave, and they are the right diagnosis when crawling returns nothing. Here the crawler succeeds but does not see new columns, pointing elsewhere.
✓
The crawler uses a custom classifier that defines a fixed schema.
Why this is correct
A custom classifier overrides the crawler's built-in schema inference, so it applies the fixed column definitions you supplied rather than re-reading the CSV headers. New columns in the S3 files are therefore ignored, which is exactly the failure described. Removing the classifier or updating its schema restores automatic detection.

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

Go deeper

DEA-C01 Data Ingestion and Transformation Practice Question

Why each option matters

Option-by-option breakdown

AWS S3 Storage Class Comparison

Related to this question