Transferring 50 TB of Data from On-Premises HDFS to S3 Using AWS DataSync
A data engineer needs to transfer 50 TB of data from an on-premises HDFS cluster to Amazon S3. The data must be encrypted in transit and at rest. The on-premises network has a 1 Gbps connection to AWS. The transfer must complete within 5 days. Which solution is MOST cost-effective and meets the requirements?
Quick Answer
The answer is AWS DataSync with an on-premises agent, because it can transfer 50 TB over a 1 Gbps link within 5 days—theoretical maximum is 54 TB—while encrypting data in transit via TLS and at rest with S3 server-side encryption. This solution is the most cost-effective for large data transfer on-prem to S3 AWS DataSync, as it avoids the expense of physical appliances like Snowball Edge when the bandwidth window is sufficient. On the MLS-C01 exam, this question tests your ability to calculate transfer feasibility and distinguish between data movement services: a common trap is choosing Snowball Edge for any large volume, but here the 1 Gbps link makes DataSync cheaper and faster to deploy. Another trap is S3 Transfer Acceleration, which optimizes public internet uploads but cannot read from HDFS directly. Memory tip: “DataSync for data that fits the pipe; Snowball for data that doesn’t.”
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
Use AWS DataSync with a DataSync agent installed on-premises to transfer the data to S3.
(AWS DataSync). With a 1 Gbps connection, the maximum theoretical transfer in 5 days is about 54 TB (1 Gbps = 0.125 GB/s, 0.125 * 86400 * 5 = 54000 GB = 54 TB), so network transfer is feasible within the time limit. AWS DataSync uses a DataSync agent installed on-premises to transfer data from HDFS to S3, encrypting data in transit (TLS) and at rest (S3 server-side encryption). This is the most cost-effective solution because it avoids the hardware and shipping costs of Snowball Edge (option C). Option A (S3 Transfer Acceleration) does not directly integrate with HDFS and is designed for speeding up uploads over public internet, not for encrypting data from HDFS. Option D (AWS Glue) is an ETL service, not a data transfer solution, and would require additional infrastructure and complexity.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
Use S3 Transfer Acceleration to upload the data directly from HDFS to S3.
Why it's wrong here
Transfer Acceleration is for public internet, not for HDFS.
- ✓
Use AWS DataSync with a DataSync agent installed on-premises to transfer the data to S3.
Why this is correct
DataSync can transfer over network with encryption and is optimized for speed.
- ✗
Order an AWS Snowball Edge device and copy the data to it, then ship it back.
Why it's wrong here
Snowball is cost-effective for large volumes but not necessary when network can handle it in 5 days.
- ✗
Use AWS Glue to read from HDFS and write to S3 in a continuous ETL job.
Why it's wrong here
Glue does not directly read from HDFS without a connector and is not optimized for bulk transfer.
Quick reference
AWS S3 Storage Class Comparison
| Storage Class | Min Duration | Retrieval | Use Case |
|---|---|---|---|
| S3 Standard | None | Immediate | Frequently accessed data |
| S3 Standard-IA | 30 days | Immediate | Infrequent access, rapid retrieval |
| S3 One Zone-IA | 30 days | Immediate | Non-critical infrequent data |
| S3 Intelligent-Tiering | None | Immediate–hours | Unknown or changing access patterns |
| S3 Glacier Instant | 90 days | Milliseconds | Archive with instant retrieval |
| S3 Glacier Flexible | 90 days | Minutes–hours | Archive, flexible retrieval |
| S3 Glacier Deep Archive | 180 days | Hours | Long-term compliance archive |
Go deeper
Related to this question
About these practice questions
One of 1,672 original MLS-C01 practice questions on Courseiva, each with a full explanation and wrong-answer analysis — not exam dumps or protected exam content. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on MLS-C01
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. A data engineer needs to move 50 TB of historical data from an on-premises Hadoop cluster to Amazon S3. The network bandwidth is limited to 100 Mbps. Which AWS service should be used to transfer the data most efficiently?
easy- A.Amazon S3 Transfer Acceleration to speed up the transfer.
- ✓ B.AWS Snowball Edge device to physically ship the data.
- C.AWS Direct Connect to establish a dedicated network connection.
- D.AWS Site-to-Site VPN to connect and copy data.
Why B: Given 50 TB of data and a 100 Mbps network link, the theoretical minimum transfer time over the network is over 46 days (50 TB * 8 / 100 Mbps ≈ 4,000,000 seconds ≈ 46.3 days), not accounting for protocol overhead, retransmissions, or contention. AWS Snowball Edge is a physical appliance that bypasses the network bottleneck entirely, allowing you to copy data locally and ship it to AWS, making it the most efficient option for this volume over a constrained link.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This MLS-C01 practice question is part of Courseiva's free Amazon Web Services certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the MLS-C01 exam.