How do you turn a chaotic pile of raw data into something a machine learning model can actually learn from? This is the central problem of data preparation, and for the MLA-C01 exam, it is the most critical skill you must master. Without clean, well-organised data, even the most sophisticated machine learning model is useless, like a sports car with no fuel.
Jump to a section
A simple way to picture Data Preparation Foundations: S3, Glue, and Data Wrangling
Ever tried to cook a complex meal in a kitchen where the ingredients are scattered in random boxes, some have no labels, and the recipe book is written in a language you don't fully understand? That is the nightmare of data preparation without the right tools.
Imagine you are renovating a restaurant kitchen. You start with a massive, disorganised pantry (your raw data). This is Amazon S3 (Simple Storage Service) it is the giant, secure, and almost infinitely large room where you store all your ingredients, from fresh produce to frozen stocks, in their original packaging. You can put anything in there, but finding a single spice jar among thousands of unmarked boxes is a nightmare.
Next, you need to actually prepare the ingredients for cooking the machine learning model. You cannot just throw a whole salmon into the oven. You need to descale it, remove the bones, and portion it into fillets. This is AWS Glue. It is your automated kitchen assistant that discovers what is in your pantry (data cataloguing), cleans the dirt off the vegetables (data cleansing), and slices the carrots into uniform sticks (data transformation). It moves the prepared ingredients from the messy pantry to a clean, organised prep station (the transformed data location).
Finally, you need to quickly grab those specific carrot sticks for the recipe you are cooking. This is Data Wrangling. It is the nimble chef who uses a small, sharp knife (a library like Pandas or AWS SDK for Pandas) to quickly chop, slice, and season the already-cleaned ingredients exactly as the machine learning recipe demands, without needing to turn on the entire industrial kitchen equipment. It is fast, flexible, and perfect for the final tweaks.
Data preparation is the process of taking raw data from its source, cleaning it, transforming it into a useful format, and making it available for machine learning training. For the MLA-C01 exam, you need to understand three core AWS services that work together to achieve this: Amazon S3, AWS Glue, and Data Wrangling libraries.
Let us start with Amazon S3 (Simple Storage Service). Think of S3 as a giant, secure, internet-accessible hard drive. It is an object storage service, meaning it stores your data as objects (like files) inside containers called buckets. You can store anything here: CSV files, images, log files, or Parquet files. For machine learning, S3 is where you will put your raw training data and where your final, processed training data will live. It is durable, meaning your data does not get lost, and it is scalable, meaning you can store terabytes without worrying about running out of space. A key concept for the exam is that S3 does not organise data for you by default it is just a flat storage space. You have to structure it yourself using folders (which are actually prefixes in the object key name) and naming conventions.
Next is AWS Glue. Glue is a fully managed extract, transform, and load (ETL) service. This is the heavy lifter for data preparation. Glue has several components you need to know for the exam:
AWS Glue Data Catalog: This is a central metadata repository. It does not store your data, but it stores information about your data, like the schema (column names, data types) and location (which S3 bucket). It is like a library card catalogue for your data, making it discoverable.
AWS Glue Crawlers: These are automated agents that scan your data sources (like S3 buckets or databases) and automatically populate the Glue Data Catalog with table definitions and schemas. You can run a crawler, and it will figure out that your CSV has columns called 'user_id' and 'age'.
AWS Glue ETL Jobs: These are scripts (usually written in Python or Scala on Apache Spark) that run on a serverless Spark environment. You use these to perform transformations like filtering out null values, joining two datasets, converting file formats from CSV to Parquet, and partitioning data for better query performance. Glue handles the heavy computation in the background, so you do not have to manage servers.
Finally, we have Data Wrangling. In the AWS ecosystem, this specifically refers to the AWS SDK for Pandas (previously called AWS Data Wrangle). This is a Python library that extends the popular Pandas library to integrate seamlessly with AWS services like S3, Glue, and Athena. It is a lighter-weight tool compared to Glue ETL jobs. Data Wrangling is for interactive data exploration and smaller-scale transformations. You might use it in a Jupyter notebook on SageMaker or on your local machine to quickly clean a dataset before uploading it. It is particularly good for reading and writing data directly from and to S3 with just a few lines of code, without needing to set up a full Glue job.
Why do these concepts exist? Before AWS, data engineers had to manually set up server infrastructure (hardware) for storage and processing. S3 replaced the need for physical hard drives and servers for storage. Glue replaced the need to manually install and configure Apache Hadoop or Spark clusters on EC2 instances. Data Wrangling replaced the tedious process of writing custom code to copy files from and to S3 and other AWS services. They all exist to make data preparation faster, cheaper, and more reliable, which is essential for the iterative nature of machine learning.
Step 1: Ingest Raw Data into S3 Landing Zone
Create an S3 bucket and set up folders for each data source (e.g., 'landing/customers/'). Load raw data from databases or logs into these folders. This is where the data sits in its original, unprocessed form, preserving the source format. This step ensures you have a durable, scalable backup of raw data before any transformations.
Step 2: Run Glue Crawler to Populate Data Catalog
Configure and run a Glue Crawler pointed at your S3 landing zone. The crawler scans the raw files, infers the schema (column names and types), and creates table metadata in the Glue Data Catalog. This makes your data discoverable and queryable by other services like Athena, without needing to know the schema beforehand.
Step 3: Write and Execute Glue ETL Job for Transformation
Write a PySpark script (or use the Glue Studio visual editor) that reads data from the Data Catalog tables. Perform necessary transformations: filtering nulls, joining tables, calculating new features, and converting file formats. The job runs on a serverless Spark cluster and writes the cleaned data to a new S3 location (e.g., 'curated/'). This is the core preparation step for ML.
Step 4: Validate and Perform Final Wrangling in Notebook
Use a SageMaker notebook or your local Jupyter environment with the AWS SDK for Pandas (Data Wrangling) to load the transformed data. Do a final check for anomalies, run statistics, and maybe drop one more column. This interactive step is for last-mile tweaks before feeding data to the model, and it is fast because you are working on a single node.
Step 5: Schedule the Pipeline with Glue Workflow or Step Functions
Create a Glue Workflow or an AWS Step Functions state machine to automate the entire process. Set triggers to run the crawler first, then the ETL job periodically (e.g., nightly). This ensures your training data is always fresh without manual intervention, which is critical for production ML systems.
Step 6: Store Final Prepared Data in S3 for Model Training
This step finalises the data preparation pipeline, ensuring the data is in the optimal format and location for machine learning training jobs in SageMaker.
Imagine you work for a retail company called 'ShopFast' that sells products online. Your task is to build a machine learning model that predicts which customers are likely to churn (stop buying) next month. Your raw data lives in several different places: a customer database (like PostgreSQL) containing user sign-up dates, a web server log on S3 containing page clicks, and a transactional database containing purchase history.
Here is how an IT professional would actually prepare this data using S3, Glue, and Data Wrangling:
Data Ingestion and Landing Zone: First, you create an S3 bucket called 'shopfast-ml-data'. Inside it, you create folders for each source: 'landing/customers/', 'landing/weblogs/', 'landing/transactions/'. You then set up automatic exports or AWS DMS (Database Migration Service) tasks to dump nightly snapshots of the customer and transaction tables into their respective S3 folders as CSV files. The web logs are already being pushed from your web servers directly into 'landing/weblogs/'. This 'landing' area is your raw data repository, exactly as it comes from the source.
Discover Schema with Glue Crawlers: You cannot just start joining tables if you do not know what columns they have. You configure a Glue Crawler pointing to 's3://shopfast-ml-data/landing/'. The crawler scans the CSVs and creates three tables in the Glue Data Catalog: 'customers', 'weblogs', and 'transactions'. It automatically detects that the 'customers' table has columns like 'customer_id' (int), 'signup_date' (date), and 'last_login' (date). This makes your data discoverable for querying with Amazon Athena, which you will use later.
Transform with Glue ETL Jobs: Raw data is messy. The transaction table might have null values for 'purchase_amount'. The web logs might have user agent strings you do not need. You write a Glue ETL job (using PySpark) that reads from the catalog tables, joins 'transactions' and 'weblogs' on 'customer_id', filters out rows from the last 90 days, fills null purchase amounts with zero, and calculates a new feature: 'days_since_last_purchase'. You then write the cleaned, transformed dataset to a new S3 location: 's3://shopfast-ml-data/curated/churn_features/' in Parquet format (a columnar format ideal for machine learning). This 'curated' zone is your clean, analysis-ready data.
Final Wrangling for Exploration: Before feeding the data to the model, you want to do a quick check in a SageMaker notebook. You use the AWS SDK for Pandas (Data Wrangling) to load the Parquet file directly into a Pandas DataFrame: wr.s3.read_parquet('s3://shopfast-ml-data/curated/churn_features/'). You check for missing values, plot some distributions, and decide to drop a column that has too many nulls. You then save the final version using wr.s3.to_parquet().
Automation and Scheduling: You do not want to run steps 2 and 3 manually every night. You set up a Glue Workflow or an AWS Step Functions state machine to trigger the crawler first, then the ETL job automatically every 24 hours. This ensures your training data is always fresh.
The IT professional here never manually downloaded files, never set up a SQL server for storage, and never managed a Spark cluster. They used S3 for storage, Glue for automated discovery and heavy transformation, and Data Wrangling for quick, interactive last-mile data handling. This is the standard modern data pipeline for machine learning.
The MLA-C01 exam will test your understanding of how these three services interact in a data pipeline. You are not expected to write full scripts, but you must know the purpose of each component and the correct sequence of operations. Expect scenario-based questions where you must choose the best service for a given task.
Key topics the exam loves to test:
S3 Storage Classes and Lifecycle Policies: You will be asked about moving data from S3 Standard to S3 Glacier to save costs on infrequently accessed datasets. The exam expects you to know that S3 Intelligent-Tiering can automate this.
Glue Data Catalog vs. Glue ETL: A classic trap. The exam will give you a scenario where data needs to be discovered and catalogued but not transformed. The correct answer would be to run a Glue Crawler, not a Glue ETL job.
AWS Glue vs. AWS Lake Formation: Lake Formation is a higher-level service built on top of Glue for managing data lakes with fine-grained access control. The exam might ask about which service to use for setting up row-level security on data.
Data Wrangling (AWS SDK for Pandas) vs. Glue ETL (Apache Spark): The exam will test when to use each. Data Wrangling is for interactive, small-to-medium datasets running on a single machine (e.g., in a SageMaker notebook). Glue ETL is for production, large-scale data processing that requires distributed computing (Apache Spark).
File Formats: You must understand why Parquet is preferred over CSV for ML training. Parquet is columnar, compresses better, and when you only need a few columns, it reads only those columns (predicate pushdown), making it much faster and cheaper. CSV is row-oriented and slower for this use case.
Partitioning: The exam expects you to know how to structure data in S3 for efficient querying with Athena or Glue. For example, partitioning by year, month, and day (e.g., s3://bucket/data/year=2023/month=01/day=15/) allows Glue crawlers and Athena to scan only relevant partitions, drastically reducing costs.
Common traps set by the exam:
Confusing a Glue Crawler with a Glue ETL job. A crawler only discovers schema and populates the catalog it does not transform data.
Thinking Glue is just for data transformation. It also includes the Data Catalog which is a metadata service, not a transformation service.
Assuming S3 is a database. S3 is object storage. You cannot run SQL queries directly on S3 without a service like Athena or Redshift Spectrum.
Forgetting that Glue ETL jobs run on Apache Spark. If a question mentions a need for complex, distributed data processing, Glue ETL is the right answer, not Data Wrangling.
Correct answer pattern: When you see a question asking for 'serverless' data preparation for machine learning, your first thought should be Glue ETL. When you see 'interactive exploration in a notebook', think Data Wrangling. When you need to find out what a dataset looks like, think Glue Crawler + Data Catalog.
Amazon S3 is object storage for your raw and processed data, but you cannot query it directly without a service like Amazon Athena.
AWS Glue is a serverless ETL service that includes a Data Catalog for metadata, Crawlers for schema discovery, and ETL Jobs for transformation.
A Glue Crawler discovers schema and populates the Data Catalog it does not transform the underlying data.
Data Wrangling (AWS SDK for Pandas) is for interactive, single-node data manipulation in notebooks, not for production-scale distributed processing.
Parquet is the preferred file format for machine learning data on AWS because it is columnar and more efficient than CSV for analytical queries.
Partitioning your data in S3 (e.g., by year/month) is crucial for reducing query costs and improving performance with services like Glue and Athena.
AWS Glue ETL jobs run on Apache Spark, making them suitable for large-scale data transformations that require distributed compute.
Never use a Glue ETL job just to discover data schema that task belongs to a Glue Crawler.
These come up on the exam all the time. Here's how to tell them apart.
Glue Crawler
Discovers schema and populates the Data Catalog only.
Does not modify the underlying data files.
Runs a short scanning task, not a computation-heavy transformation.
Glue ETL Job
Executes custom PySpark code to transform data.
Modifies the data and writes output to a new location.
Runs a long-running job on a serverless Spark cluster.
Data Wrangling (AWS SDK for Pandas)
Designed for interactive, single-node processing (e.g., a notebook).
Uses the Pandas library which runs on a single machine's memory.
Best for datasets that fit in memory on one instance.
Glue ETL Job (PySpark)
Designed for production, large-scale distributed processing.
Uses Apache Spark which distributes work across multiple nodes.
Best for datasets that are terabytes in size.
CSV File Format
Row-oriented, meaning all columns for a row are stored together.
Usually uncompressed or plain text, leading to larger file sizes.
Inefficient for analytical queries that only need a few columns.
Parquet File Format
Column-oriented, meaning values for a single column are stored together.
Highly compresses (often 70-90% smaller than CSV).
Supports predicate pushdown: reads only the columns you need.
S3 (Storage Service)
Stores data objects (files) in buckets.
No querying capabilities on its own.
Charged for storage and API requests.
Amazon Athena (Query Service)
A serverless query engine that uses SQL to query data in S3.
Reads data from S3 and processes it.
Charged based on the amount of data scanned per query.
Glue Data Catalog
Stores metadata about the data (schema, location, partitions).
Does not store the actual data rows or files.
Is a logical reference, not a physical storage location.
Actual Data in S3
Stores the actual data files (e.g., CSV, Parquet).
Is the physical storage location in S3 buckets.
Is what the metadata in the Data Catalog points to.
Mistake
S3 is just like a folder on my laptop. I can just drag and drop files and query them directly with SQL.
Correct
S3 is object storage, not a file system. You cannot run SQL queries directly on S3 objects. You need a query engine like Amazon Athena or Redshift Spectrum to query data that is stored in S3.
People are used to local file systems or databases where you can directly interact with data. The boundary between storage and compute is not intuitive to beginners.
Mistake
AWS Glue only does data transformation (ETL) and has nothing to do with data discovery or cataloguing.
Correct
AWS Glue is a suite of services. It includes the Glue Data Catalog (metadata repository) and Glue Crawlers (for schema discovery) in addition to Glue ETL jobs for transformation.
The name 'Glue' sounds like it just sticks data together, leading people to focus only on the transformation part and forget the cataloguing components.
Mistake
Data Wrangling (AWS SDK for Pandas) is a replacement for Glue ETL jobs in all scenarios.
Correct
Data Wrangling is for interactive, single-node processing (like in notebooks). Glue ETL runs on a serverless Spark cluster for distributed processing of large datasets. They serve different purposes based on data size and job type.
Both tools can read and write data and perform transformations, making them seem interchangeable. The difference in scalability and execution context is subtle for a beginner.
Mistake
The Glue Data Catalog stores the actual data from my S3 buckets.
Correct
The Glue Data Catalog stores metadata (schema, location, data type) about the data, not the data itself. The actual data remains in S3.
The word 'catalog' might sound like a storage location. People confuse a catalogue of data (like a library catalogue) with the data itself (the books).
Mistake
Using CSV files is always fine for machine learning data on S3 because they are easy to read.
Correct
Parquet is almost always better for ML on AWS. It is columnar, compresses better, and enables predicate pushdown, leading to faster reads and lower costs. CSV is row-oriented and inefficient for analytical queries that only need a few columns.
CSV is universally understood and easy to view in a text editor. Beginners default to it because it is familiar, without understanding the performance implications at scale.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
Yes, SageMaker Data Wrangler is a visual tool within SageMaker that can perform data preparation and feature engineering without coding, but it is different. It is best for exploration and smaller datasets inside SageMaker. The MLA-C01 exam specifically tests Glue and the SDK for Pandas (Data Wrangling) for ETL pipelines, so you still need to know those.
You do not need to write full PySpark code, but you need to understand what Glue ETL jobs can do. The exam will ask you to select the right service or configuration for a given scenario, like 'which service would you use to join two large datasets?' knowing it is Glue ETL is enough.
A Glue Crawler is for discovery it scans your data, infers schema, and populates the Data Catalog. A Glue Job is for transformation it runs code to clean, join, and convert data. A crawler does not change the data itself; a job does.
AWS SDK for Pandas adds many convenience functions to Pandas that let you directly read from and write to S3, Glue Data Catalog, Athena, and other AWS services with a single function call. Regular Pandas requires you to manually handle AWS API calls, which is more complex.
No, Glue can read from many sources, including JDBC connections (like RDS or Redshift), DynamoDB, and more. But for the MLA-C01 exam, S3 is the most common source and destination for data in a machine learning pipeline.
Partitioning means organising your S3 objects into a hierarchical folder structure based on a column value, like 'year=2023/month=01/'. Glue and Athena can skip reading partition folders that are not needed for a query, making the process much faster and cheaper. The exam expects you to know this for performance optimisation.
You've finished Data Preparation Foundations: S3, Glue, and Data Wrangling. Continue through the MLA-C01 study guide to build a complete picture of the exam.
Done with this chapter?