DEA-C01 Data Store Management Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  Read the entire `sales_data` table into a DynamicFrame and then apply a `Filter` transform with the condition `year=='2023' and month=='10'`.

  Why it's wrong here

  Reading the entire table first loads all partitions into memory, which scans all data in S3, increasing cost and runtime. The Filter transform is applied after the data is read, so it does not reduce the amount of data scanned. For large datasets, this approach is inefficient and defeats the purpose of partitioning. It also may cause out-of-memory errors if the dataset is large.

• ✗

  Use the Glue `create_dynamic_frame.from_options` with the S3 path `s3://bucket/sales_data/year=2023/month=10/` and format `parquet`.

  Why it's wrong here

  While this reads only the specified prefix, it bypasses the Glue Data Catalog and does not automatically handle partition columns or schema. It also requires hardcoding the path, which is less flexible. Additionally, if the table has other partition columns like day, this method would read all days under the month, but it might not correctly infer partition values. Using the catalog with pushdown is more maintainable and efficient.

• ✓

  Use the Glue `create_dynamic_frame.from_catalog` with `push_down_predicate` set to `year=='2023' and month=='10'`.

  Why this is correct

  Using `push_down_predicate` with `create_dynamic_frame.from_catalog` allows Glue to filter partitions at the catalog level, so only the specified partitions are read from S3. This reduces data scanned and improves performance. The predicate syntax uses SQL-like expressions on partition columns, and Glue leverages the partition metadata to avoid listing and reading unnecessary partitions. This is the recommended approach for partitioned data in Glue ETL jobs.

• ✗

  Create a new Glue Data Catalog table that points only to the `year=2023/month=10` prefix, and then read from that table.

  Why it's wrong here

  Creating a new table for a specific partition is not scalable and duplicates metadata. It also requires manual updates when new partitions are added. This approach does not leverage the existing partition structure and can lead to inconsistencies. The original table already contains the partition metadata, so using pushdown predicates is simpler and more efficient. This method is not recommended for ad-hoc or recurring ETL jobs.

Go deeper

Related to this question