DEA-C01 Data Ingestion and Transformation Practice Question
A company uses AWS Glue to process data from multiple sources. The data is stored in an Amazon S3 data lake. The company needs to transform the data using a custom Python library that is not available in the default Glue environment. What is the MOST efficient way to make this library available to the Glue jobs?
⚠ Common exam trap
Many candidates think running 'pip install' directly in the script (Option D) is acceptable, but AWS explicitly recommends using the `--additional-python-modules` parameter for efficiency and reliability, as runtime pip installs can fail due to network timeouts or missing build dependencies.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Upload the library as a .whl file to Amazon S3 and reference it in the Glue job's --additional-python-modules parameter.
AWS Glue supports adding custom Python libraries by uploading a .whl file to Amazon S3 and referencing it via the `--additional-python-modules` job parameter. This method is the most efficient as it requires no manual node configuration, no custom Docker images, and no runtime pip installs, ensuring the library is automatically distributed to all worker nodes before the job executes.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Manually install the library on each node in the Glue cluster by editing the bootstrap script.
Why it's wrong here
Glue Spark clusters are ephemeral and managed, so bootstrap-script edits do not persist across job runs and must be reapplied per cluster. Bootstrap scripts are tempting for installing OS-level packages at cluster start, but they are not the supported mechanism for shipping a Python library to each job.
- ✓
Upload the library as a .whl file to Amazon S3 and reference it in the Glue job's --additional-python-modules parameter.
Why this is correct
Glue's --additional-python-modules parameter installs wheels from S3 into the job's Python environment at runtime, satisfying the need for a custom library absent from the default Glue image without building a custom connector or repackaging the job.
- ✗
Create a custom Docker image with the library and use it in AWS Glue for Ray.
Why it's wrong here
Glue for Ray is a separate engine for distributed Python workloads; it does not run the Spark ETL jobs described, so the library never reaches them. Custom Docker images are tempting because they bundle arbitrary dependencies, but that mechanism applies to Glue for Ray, not standard Spark jobs.
- ✗
Use a shell command in the Glue job script to run 'pip install <library>' before the job runs.
Why it's wrong here
Running pip install inside the job script executes at runtime on the driver only, adds startup latency, and fails on worker nodes that need the library. Inline pip is tempting for quick one-off experiments, but it is not a repeatable packaging mechanism for distributed Glue ETL jobs.
Quick reference
AWS S3 Storage Class Comparison
| Storage Class | Min Duration | Retrieval | Use Case |
|---|---|---|---|
| S3 Standard | None | Immediate | Frequently accessed data |
| S3 Standard-IA | 30 days | Immediate | Infrequent access, rapid retrieval |
| S3 One Zone-IA | 30 days | Immediate | Non-critical infrequent data |
| S3 Intelligent-Tiering | None | Immediate–hours | Unknown or changing access patterns |
| S3 Glacier Instant | 90 days | Milliseconds | Archive with instant retrieval |
| S3 Glacier Flexible | 90 days | Minutes–hours | Archive, flexible retrieval |
| S3 Glacier Deep Archive | 180 days | Hours | Long-term compliance archive |
Go deeper
Related to this question
About these practice questions
This DEA-C01 question is part of Courseiva's 1,321-question bank — original exam-style content with full explanations and wrong-answer analysis, never real exam questions or exam dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This DEA-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the DEA-C01 exam.