← MLS-C01·Amazon Web Services

Question 1,712 of 1,755

Data Engineering →mediumMultiple ChoiceObjective-mapped

AWS Glue Crawler Configuration: Multiple Formats and Partition Updates

This MLS-C01 practice question tests your understanding of data engineering. This is a configuration task: choose the command set that satisfies every stated requirement. Small differences — like 'secret' vs 'password' or 'transport input ssh' vs 'all' — change whether the answer is correct. After answering, compare your reasoning against the explanation and wrong-answer breakdown below. Once you have made your selection, read the full explanation to reinforce the concept and understand why each distractor is designed to mislead on exam day.

A company is building a data lake on Amazon S3 and wants to use AWS Glue to catalog the data. The data includes CSV, Parquet, and JSON files. The team wants to ensure that the Glue crawler can infer the schema correctly and update the Data Catalog when new partitions are added. Which crawler configuration should be used?

A
Create separate crawlers for each file format and schedule them at different times.
Why wrong: Incorrect: Not necessary; one crawler can handle multiple formats.
B
Use a crawler that only catalogs Parquet files because they are more efficient.
Why wrong: Incorrect: All formats need to be cataloged.
C
Use a crawler with 'Update all new and existing partitions' disabled to avoid schema conflicts.
Why wrong: Incorrect: Disabling update would cause new partitions to be missed.
D
Create a single crawler that includes all file extensions and set the 'Update all new and existing partitions' option.
Correct: Single crawler with partition updates ensures comprehensive cataloging.

Full breakdown with real-world context →

Answer choices

Why each option matters

Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.

Correct answer & explanation

✓

Create a single crawler that includes all file extensions and set the 'Update all new and existing partitions' option.

Option D is correct because a single AWS Glue crawler can handle multiple file formats (CSV, Parquet, JSON) in a data lake on Amazon S3, and enabling 'Update all new and existing partitions' ensures the Data Catalog is refreshed with both new partitions and any schema changes in existing partitions. This configuration maintains a consistent and up-to-date catalog without manual intervention, which is essential for downstream analytics and machine learning workloads.

Key principle: Answer the scenario, not the keyword: identify the specific constraint before choosing the most familiar-sounding option.

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

✗
Create separate crawlers for each file format and schedule them at different times.
Why it's wrong here
Incorrect: Not necessary; one crawler can handle multiple formats.
✗
Use a crawler that only catalogs Parquet files because they are more efficient.
Why it's wrong here
Incorrect: All formats need to be cataloged.
✗
Use a crawler with 'Update all new and existing partitions' disabled to avoid schema conflicts.
Why it's wrong here
Incorrect: Disabling update would cause new partitions to be missed.
✓
Create a single crawler that includes all file extensions and set the 'Update all new and existing partitions' option.
Why this is correct
Correct: Single crawler with partition updates ensures comprehensive cataloging.
Related concept
Read the scenario before looking for a memorised answer.

Common exam traps

Common exam trap: answer the scenario, not the keyword

The trap here is that candidates mistakenly think disabling partition updates prevents schema conflicts, but in reality, it causes stale metadata for existing partitions, while a single crawler with updates enabled correctly handles schema evolution across all file formats.

Detailed technical explanation

How to think about this question

AWS Glue crawlers use classifiers to infer schema from different file formats, and when 'Update all new and existing partitions' is enabled, the crawler compares the current schema with the existing catalog entry and applies changes (e.g., adding new columns) to both new and existing partitions. This is critical in data lakes where schema evolution occurs over time, such as when new fields are added to JSON files or Parquet schemas change; without this setting, the crawler only updates new partitions, leaving old partitions with outdated schemas that can break queries.

KKey Concepts to Remember

Read the scenario before looking for a memorised answer.
Find the constraint that changes the correct option.
Eliminate answers that are true in general but not in this case.

TExam Day Tips

Watch for words such as best, first, most likely and least administrative effort.
Review why wrong options are wrong, not only why the correct option is correct.

Key takeaway

Answer the scenario, not the keyword: identify the specific constraint before choosing the most familiar-sounding option.

Real-world example

How this comes up in practice

A media company stores terabytes of video archives that are accessed once a year for audit purposes. Moving these objects to a cold storage tier (Azure Archive, S3 Glacier, or Google Nearline) costs a fraction of hot storage. Questions like this test whether you understand storage tiers, access frequency tradeoffs, and retrieval latency requirements.

Quick reference

AWS S3 Storage Class Comparison

Storage Class	Min Duration	Retrieval	Use Case
S3 Standard	None	Immediate	Frequently accessed data
S3 Standard-IA	30 days	Immediate	Infrequent access, rapid retrieval
S3 One Zone-IA	30 days	Immediate	Non-critical infrequent data
S3 Intelligent-Tiering	None	Immediate–hours	Unknown or changing access patterns
S3 Glacier Instant	90 days	Milliseconds	Archive with instant retrieval
S3 Glacier Flexible	90 days	Minutes–hours	Archive, flexible retrieval
S3 Glacier Deep Archive	180 days	Hours	Long-term compliance archive

What to study next

Got this wrong? Here's your next step.

Identify which exam domain this question belongs to, review the core concept, then practise similar questions from the same domain.

Related MLS-C01 practice-question pages

Use these pages to review the topic behind this question. This is how one missed question becomes focused revision.

Data Engineering practice questions

Practise MLS-C01 questions linked to Data Engineering.

Machine Learning Implementation and Operations practice questions

Practise MLS-C01 questions linked to Machine Learning Implementation and Operations.

Modeling practice questions

Practise MLS-C01 questions linked to Modeling.

Exploratory Data Analysis practice questions

Practise MLS-C01 questions linked to Exploratory Data Analysis.

MLS-C01 fundamentals practice questions

Practise MLS-C01 questions linked to MLS-C01 fundamentals.

MLS-C01 scenario practice questions

Practise MLS-C01 questions linked to MLS-C01 scenario.

MLS-C01 troubleshooting practice questions

Practise MLS-C01 questions linked to MLS-C01 troubleshooting.

Practice this exam

Start a free MLS-C01 practice session

Short sessions build daily habit. Longer sessions build exam-day stamina. Try a timed session to simulate real conditions.

10 questions 20 questions 30 questions 50 questions Timed 30

MLS-C01 practice-test guide →Study guide →Browse all practice tests

FAQ

Questions learners often ask

What does this MLS-C01 question test?

Data Engineering — This question tests Data Engineering — Read the scenario before looking for a memorised answer..

What is the correct answer to this question?

The correct answer is: Create a single crawler that includes all file extensions and set the 'Update all new and existing partitions' option. — Option D is correct because a single AWS Glue crawler can handle multiple file formats (CSV, Parquet, JSON) in a data lake on Amazon S3, and enabling 'Update all new and existing partitions' ensures the Data Catalog is refreshed with both new partitions and any schema changes in existing partitions. This configuration maintains a consistent and up-to-date catalog without manual intervention, which is essential for downstream analytics and machine learning workloads.

What should I do if I get this MLS-C01 question wrong?

Identify which exam domain this question belongs to, review the core concept, then practise similar questions from the same domain.

What is the key concept behind this question?

Read the scenario before looking for a memorised answer.

About these practice questions

Courseiva creates original exam-style practice questions with explanations and wrong-answer analysis. It does not publish real exam questions, exam dumps, or protected exam content. Learn why practice questions differ from exam dumps →

How Courseiva writes practice questions · Editorial policy

Same concept, more angles

2 more ways this is tested on MLS-C01

These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.

Variation 1. A company wants to build a data lake on Amazon S3. The data lake will store raw data in its original format and also transformed data in Parquet. The data is generated by various sources and must be cataloged for discovery. Which service should the company use to automatically discover, catalog, and make the data searchable?

easy

✓ A.AWS Glue Data Catalog
B.Amazon S3
C.Amazon Athena
D.Amazon EMR

Why A: AWS Glue Data Catalog is a central metadata repository that can automatically crawl S3 data sources to discover, catalog, and make data searchable. It is the correct choice for building a data lake with raw and transformed data in Parquet. Option B (Amazon S3) is storage, not a catalog. Option C (Amazon Athena) is a query engine, not a catalog. Option D (Amazon EMR) is a big data processing framework, not a catalog.

Variation 2. A team is building a data lake on Amazon S3 and using AWS Glue to catalog data. They notice that Glue crawlers are taking too long to update the catalog for a large dataset with millions of small files. Which approach will MOST improve crawler performance?

hard

A.Increase the frequency of the crawler runs.
✓ B.Consolidate the small files into larger files (e.g., 100 MB each).
C.Partition the data by date in S3.
D.Use a custom classifier to parse the data.

Why B: AWS Glue crawlers incur significant overhead when processing millions of small files because each file requires a separate read, schema inference, and metadata write operation. Consolidating small files into larger files (e.g., 100 MB each) reduces the total number of objects that the crawler must scan, dramatically decreasing the time spent on file-level operations and improving overall throughput.

Keep practising

Question Discussion

Share a tip, memory trick, or ask about the reasoning behind this question. Do not post real exam questions, leaked content, braindumps, or copyrighted exam material. Comments are moderated and may be removed without notice.

Loading comments…

This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.