Optimizing Amazon Athena Query Performance with Partition Layout
A data engineer is setting up a data lake on Amazon S3 for a large retail company. The data includes customer transactions, inventory, and web logs. The company wants to use AWS Glue for ETL and Amazon Athena for ad-hoc queries. The data is partitioned by year, month, day, and hour. The engineer notices that Athena queries are slow and often scan large amounts of data even when only a specific hour is needed. The engineer has already enabled partitioning and used columnar formats like Parquet. What additional step should the engineer take to optimize query performance and reduce data scanned?
Quick Answer
The correct answer is to use a coarser partition layout, such as partitioning only by date, and leverage Hive-style partitioning with AWS Glue Crawlers to avoid excessive small files. This is because partitioning by hour creates an extremely high number of small partitions, each with its own metadata overhead, which forces Athena to perform more file listing and metadata operations than actual data scanning. By consolidating partitions to a coarser granularity like day, you reduce the partition count, minimize metadata overhead, and allow Athena to efficiently prune large data scans even when querying a specific hour. On the AWS Certified Machine Learning Specialty MLS-C01 exam, this concept tests your understanding of how partition design directly impacts query performance and cost, often appearing as a trap where candidates mistakenly think more partitions always improve performance. A common memory tip is “coarser is faster for Athena”—think of partitions like chapters in a book; too many tiny chapters make finding a page slower than having fewer, thicker chapters.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use a coarser partition layout, such as partitioning only by date, and leverage Hive-style partitioning with AWS Glue Crawlers to avoid excessive small files.
Partitioning at a granularity of hour can result in a large number of small files, increasing metadata overhead and slowing query planning. By using a coarser partition layout (e.g., by date) and leveraging Hive-style partitioning with AWS Glue Crawlers, the number of partitions is reduced, which improves query performance and reduces the amount of data scanned. Option B is incorrect because converting Parquet to CSV would increase storage and scan costs due to lack of columnar compression and predicate pushdown. Option C is incorrect because S3 Select operates on a single object, not across partitions; it is not designed for optimizing Athena queries over many files. Option D is incorrect because increasing partition granularity (e.g., minute-level) would create even more small files, worsening the issue.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✓
Use a coarser partition layout, such as partitioning only by date, and leverage Hive-style partitioning with AWS Glue Crawlers to avoid excessive small files.
Why this is correct
Coarser partitions reduce the number of partitions and improve query planning.
- ✗
Convert the Parquet files to CSV format to reduce the overhead of columnar storage and improve compression.
Why it's wrong here
CSV is larger and slower to query than columnar formats like Parquet.
- ✗
Use S3 Select to push down filters to S3, reducing the amount of data scanned by Athena.
Why it's wrong here
S3 Select works at the object level, not across partitions, and Athena already push down filters.
- ✗
Increase the granularity of partitioning to include minute-level partitions to further limit data scanned.
Why it's wrong here
More partitions increase metadata overhead and can slow queries due to many small files.
Quick reference
AWS S3 Storage Class Comparison
| Storage Class | Min Duration | Retrieval | Use Case |
|---|---|---|---|
| S3 Standard | None | Immediate | Frequently accessed data |
| S3 Standard-IA | 30 days | Immediate | Infrequent access, rapid retrieval |
| S3 One Zone-IA | 30 days | Immediate | Non-critical infrequent data |
| S3 Intelligent-Tiering | None | Immediate–hours | Unknown or changing access patterns |
| S3 Glacier Instant | 90 days | Milliseconds | Archive with instant retrieval |
| S3 Glacier Flexible | 90 days | Minutes–hours | Archive, flexible retrieval |
| S3 Glacier Deep Archive | 180 days | Hours | Long-term compliance archive |
Go deeper
Related to this question
About these practice questions
This MLS-C01 question is part of Courseiva's 1,672-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on MLS-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A data scientist needs to query a 2 TB dataset stored in Amazon S3 using Amazon Athena. The data is in CSV format and is used for exploratory analysis. Queries are currently slow and expensive. Which action will improve query performance and reduce cost?
easy- A.Convert the data to JSON format to improve compression.
- B.Increase the number of workers in the Athena query engine.
- ✓ C.Convert the data to Parquet format and partition by a commonly filtered column.
- D.Create a composite index on the data using Athena's index feature.
Why C: Converting CSV data to Parquet (a columnar storage format) significantly reduces the amount of data scanned by Athena, as only the columns needed for the query are read. Partitioning by a commonly filtered column (e.g., date or region) further limits the data scanned to only relevant partitions, directly reducing both query cost (Athena charges per TB scanned) and query execution time.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.