Data preprocessing and feature engineering are the processes that take raw, messy data and transform it into a format a machine learning model can actually understand and learn from. If you feed a model dirty or poorly structured data, it will make bad predictions, no matter how clever the algorithm is. Mastering these techniques is essential for the PMLE exam because Google Cloud and TensorFlow offer specific tools to handle these tasks, and the exam will test your ability to choose the right approach for real-world data problems.
Jump to a section
A simple way to picture Data Preprocessing and Feature Engineering
A head chef's kitchen prep station is the nerve centre of every great meal. The whole process starts with raw ingredients straight from the supplier: a sack of potatoes, a whole chicken, a bundle of fresh herbs. Before any cooking can happen, you must preprocess these ingredients. You wash the potatoes to remove dirt (cleaning data), peel off the skins (removing irrelevant parts), and chop them into uniform cubes so they cook evenly (normalising values). The chicken gets deboned and sliced into consistent portions (feature extraction). The herbs are stripped from their stems and finely minced (transformation). This entire prep stage is data preprocessing.
Then comes feature engineering. Imagine the chef has a special recipe book that requires specific measurements for each dish. The chef doesn't just throw in random amounts of potato. Instead, he creates a 'mash-potatoability' score by combining the starch level, moisture content, and cube size into a single metric. He invents a 'roast-crust factor' from the chicken's skin thickness and fat distribution. These new scores are engineered features — they didn't exist in the raw ingredients but dramatically improve the final dish. Without this prep and engineering, the meal would be burnt, uneven, or flavourless, just as a machine learning model will fail without proper data preprocessing and feature engineering. The kitchen prep station directly maps to the data pipeline, transforming messy reality into something a machine can reliably use.
Data preprocessing and feature engineering are the two most critical steps in any machine learning project. They are the difference between a model that makes accurate, reliable predictions and one that is useless or even dangerous. Let’s break down exactly what each term means, why they exist, and how they work together.
What is Data Preprocessing?
Data preprocessing is the act of cleaning and preparing raw data so it can be used by a machine learning model. Raw data is almost never ready for training. It might have missing values (e.g., a customer’s age field is blank), inconsistent formats (e.g., dates written as '01/02/2023' and '2023-02-01'), or outliers (e.g., a temperature reading of 10,000 degrees Celsius). Preprocessing fixes these issues. Common steps include:
Handling missing values: You might remove rows with missing data, fill them with the average value, or use a model to predict the missing number. This is crucial because many machine learning algorithms cannot handle gaps in data.
Scaling and normalisation: Imagine you have a dataset with two features: 'house price' (ranging from £100,000 to £1,000,000) and 'number of bedrooms' (ranging from 1 to 5). The model would naturally treat the price as more important because the numbers are larger. Scaling techniques like min-max scaling (resizing all values to a 0-to-1 range) or standardisation (transforming data to have a mean of 0 and a standard deviation of 1) prevent this bias.
Encoding categorical data: Machine learning models work with numbers, not words. If you have a 'colour' feature with values like 'red', 'blue', and 'green', you must convert them to numbers. One-hot encoding creates separate binary columns for each category (e.g., 'is_red', 'is_blue', 'is_green'), while label encoding assigns a number to each category (e.g., red=1, blue=2).
Removing duplicates and errors: If the same customer appears twice, or a date says '32nd January 2023', those records must be cleaned or removed.
What is Feature Engineering?
Feature engineering is the creative step where you transform existing data or combine multiple data points to create new, more informative features that help the model learn better patterns. It is not about cleaning data; it is about designing the data to make the model’s job easier. For example:
Creating interaction features: If you know that 'number of bedrooms' and 'square footage' individually matter for house price, you might create a new feature called 'rooms_per_sqft' by dividing one by the other. This captures the density of rooms, which might be a stronger predictor than either feature alone.
Extracting date parts: From a timestamp like '2023-03-15 14:30:00', you can extract 'day_of_week', 'month', 'hour', or 'is_weekend'. A model might learn that sales are higher on weekends, but it will never find that pattern if you only give it the raw timestamp string.
Binning continuous data: Instead of using a person's exact age (24, 55, 72), you could create age groups like '0-18', '19-35', '36-60', '60+'. This can make the model more robust to small fluctuations and easier to interpret.
Why Do These Steps Exist?
Before modern machine learning, analysts often had to manually study data and make decisions. They would look for patterns by hand, which was slow and error-prone. Automation through preprocessing and feature engineering allows models to learn from much larger and messier datasets without human intervention. These steps replace the need for a human to manually 'eyeball' every data point. Instead, the computer gets clean, well-engineered data and can focus on finding the mathematical relationships.
How Do They Work Together?
Think of data preprocessing as the 'basic hygiene' step and feature engineering as the 'creative optimisation' step. First, you preprocess to ensure the data is clean and consistent. Only then do you engineer new features. If you engineered features on dirty data, you would amplify the errors. The entire pipeline—raw data, preprocessing, feature engineering, and then model training—is automated in tools like TensorFlow and Google Cloud’s Vertex AI. TensorFlow provides functions like tf.feature_column to define how features should be transformed, while Vertex AI offers managed services to run these transformations at scale.
A Concrete Example
Imagine you are building a model to predict whether a customer will buy a product. Your raw data has columns: 'age', 'income', 'last_purchase_date', and 'product_category'. Preprocessing would handle missing income values (fill with median), convert 'last_purchase_date' from a string to a date object, and one-hot encode 'product_category'. Feature engineering might create a new feature called 'days_since_last_purchase' (by subtracting the last purchase date from today), which captures customer recency. Another new feature could be 'income_per_age', dividing income by age. The model then trains on this enhanced dataset and learns patterns that would have been invisible with just raw data.
In the PMLE exam, you will be asked to identify which preprocessing or feature engineering step is appropriate for a given scenario, and how to implement it using TensorFlow or Google Cloud tools like Dataflow, Cloud Dataprep, or the tf.Transform library. The key is to understand the 'why' behind each technique, not just the syntax.
Finally, remember that there is no universal 'best' way to preprocess or engineer features. It requires experimentation and domain knowledge. The exam will test your ability to recognise common patterns and choose the most effective transformation for the problem described.
Data Ingestion and Initial Inspection
Load the raw data from a source (e.g., Cloud Storage, BigQuery, or a CSV file) into your data processing environment. Use tools like Cloud Dataprep or a Jupyter notebook to inspect the data's shape, column types, missing value counts, and basic statistics. This step identifies obvious problems like missing values, inconsistent formats, and outliers before any cleaning begins.
Data Cleaning (Preprocessing)
Fix the identified issues: handle missing values (impute or remove), correct data types (e.g., convert strings to datetime), remove duplicates, and deal with outliers (cap or transform). This step ensures the data is structurally sound and ready for transformation. In TensorFlow, you might use the `tf.data` API to do this within a pipeline.
Feature Transformation and Scaling
Apply scaling (min-max or standardisation) to numerical features so they have a consistent range. Encode categorical features using one-hot encoding, label encoding, or embeddings, depending on the cardinality. For text, tokenise and vectorise. Use TensorFlow feature columns to define these transformations declaratively, which can then be applied both in training and in production.
Feature Engineering (Creation of New Features)
Create new features that capture relationships or patterns in the data. This includes combining existing features (e.g., ratio of two columns), extracting date parts (day of week, hour), binning continuous values into categories, or aggregating data across a group (e.g., average purchase per user). This step requires domain knowledge and creativity — it is where the biggest performance gains often come from.
Splitting Data and Saving the Transformation Graph
Split the preprocessed and engineered dataset into training, validation, and test sets. Crucially, save the transformation logic (e.g., the mean and standard deviation used for scaling) so it can be reapplied to new data during inference without recomputation. In TensorFlow, this is done using `tf.Transform` which produces a saved model that includes the transformations.
What does an IT professional actually do with this? Let’s follow a real-world scenario at a fictional e-commerce company called 'ShopWave'. The company wants to build a machine learning model to predict which products a customer is likely to purchase next week, so they can send personalised email recommendations.
The data team receives a raw export from the company’s database. This export contains millions of rows of customer interactions: clicks, purchases, page views, and search queries. The raw data is a mess.
Step 1: Assess data quality
The first thing the IT professional does is run a quick profile on the data using a tool like Google Cloud Dataprep. They discover: - 15% of rows are missing a 'customer_id' (these are probably bot visits and must be removed). - The 'purchase_date' column has two different date formats: 'MM/DD/YYYY' and 'YYYY-MM-DD'. - The 'price' column has a few entries with text like 'out of stock' instead of a number. - There are duplicate rows where the same event was logged twice.
Step 2: Clean the data (preprocessing)
The professional writes a pipeline in TensorFlow Extended (TFX) or uses Vertex AI Pipelines. The pipeline does the following:
Drops rows with missing customer_id.
Standardises all dates to the ISO 8601 format (YYYY-MM-DD).
Replaces 'out of stock' entries in the price column with the median price of that product category.
Removes exact duplicate rows, keeping only the first occurrence.
Scales numerical features like 'price' and 'number_of_previous_purchases' to a range of 0 to 1 using min-max scaling. This ensures the model does not favour larger numbers.
Step 3: Engineer new features
Now the data is clean, but it is still not ready for a model. The team needs to create features that capture customer behaviour patterns. They write custom transformation functions (often using Apache Beam for scalability): - Recency: For each customer, they calculate the 'days_since_last_purchase' by subtracting the last purchase date from today’s date. - Frequency: They count how many times the customer has purchased in the last 30 days. - Monetary value: They calculate the total amount spent by the customer in the last 90 days. - Interaction features: They create 'average_order_value' by dividing total spend by number of orders. - Time-based features: They extract the hour of day and day of week from the timestamp so the model can learn that customers tend to browse at night and purchase on Saturdays.
Step 4: Split the data
The dataset is split into training (80%), validation (10%), and test (10%) sets. This is done carefully so that data from the same customer does not leak into both training and test sets (a technique called splitting by key). The professional ensures the split does not create data leakage — for example, if you split randomly, information about a customer’s future purchases could end up in the training set, making the model appear more accurate than it really is.
Step 5: Save and serve the transformation
The professional saves the preprocessing and feature engineering logic as a TensorFlow tf.Transform graph. This graph is then deployed alongside the model so that when the model is used to make predictions on new data (inference), the same transformations are applied automatically. This is critical: if the training data was scaled to 0-to-1, but the new data is not scaled the same way, the model will make wrong predictions.
The outcome
After deploying the pipeline, the model improves its accuracy from 65% (when trained on raw data) to 89% (with proper preprocessing and feature engineering). The IT professional’s work is not done — they continuously monitor for data drift (when the incoming data changes over time) and update the preprocessing rules as needed. This real-world process shows that data preprocessing and feature engineering are not one-time tasks but ongoing responsibilities. In the PMLE exam, you will be tested on understanding this entire workflow, especially the order of operations and the correct tool to use at each stage.
What exactly does PMLE test about 'Data Preprocessing and Feature Engineering'? The exam focuses on your ability to apply these concepts using Google Cloud and TensorFlow tools. You will not be asked to write full code from scratch, but you will need to read and debug snippets. Here is exactly what you should expect:
Key concepts Google tests heavily:
- Data leakage: This is the number one trap. The exam loves to describe a scenario where a feature inadvertently contains information from the future or from the test set. For example, scaling the entire dataset (including the test set) before splitting is data leakage. The correct answer will always advocate splitting first, then applying transformations only on the training set, and using the same transformation parameters (like mean and standard deviation from the training set) on the test set.
- TensorFlow feature columns: You must understand the difference between numeric_column, categorical_column_with_vocabulary_list, categorical_column_with_hash_bucket, indicator_column, and embedding_column. The exam will give you a data type and ask which column to use. For example, if you have a high-cardinality categorical feature (like a product ID with 10,000 values), embedding_column is usually preferred over one-hot encoding for memory efficiency.
- `tf.Transform` and `tfx`: Know that tf.Transform is used for preprocessing that must be computed over the full dataset (like scaling that uses the dataset’s mean). In contrast, standard TensorFlow preprocessing (within the model itself) operates on a single batch at a time. The exam will test when to use each.
- Google Cloud tools: You need to know the purpose of Cloud Dataprep (visual data cleaning), Dataflow (scalable batch and stream processing using Apache Beam), and Vertex AI Pipelines (orchestrating the ML workflow).
- Encoding categorical data: The exam will test when to use one-hot encoding (few categories) vs. label encoding (ordinal categories) vs. embeddings (many categories). A common trap is that label encoding implies an ordinal relationship, so it should not be used for nominal categories like 'colour'.
Question types that appear: - Case studies: A long paragraph describing a business problem and a dataset. You must choose the correct sequence of preprocessing steps. Often there are distractor options that are technically valid but introduce data leakage or are inefficient. - Multiple choice with 'select all that apply': For example, 'Which of the following are valid methods to handle missing data?' The correct answers will include 'remove rows', 'impute with median', and 'impute with model prediction'. A common distractor is 'impute with zero', which is rarely a good idea. - Debugging a diagram: You might see a pipeline diagram with a step missing or in the wrong order. For example, a diagram that shows feature engineering happening before handling missing values would be incorrect.
Trap patterns to watch for: - The exam often presents 'improve model accuracy' as the goal, but then the incorrect option improves training accuracy while destroying generalisation (e.g., by using the target variable to create a feature). Always check whether the feature could leak the answer. - Another trap is using an expensive transformation when a cheaper one works. For example, using a deep neural network to impute missing values when a simple median would suffice and be more stable. - They may test your understanding of 'bucketising' (binning) by offering a candidate that puts all data into one bucket — clearly wrong.
Key definitions to memorise: - Data preprocessing: Cleaning, transforming, and organising raw data to make it suitable for a model. - Feature engineering: Creating new input features from existing ones to help the model learn patterns. - Data leakage: A mistake where information from outside the training set is used to create the model, giving an unrealistic performance estimate. - Normalisation: Rescaling data to a 0-to-1 range (min-max scaling). - Standardisation: Rescaling data to have a mean of 0 and a standard deviation of 1 (Z-score).
What they do NOT test: You do not need to memorise TensorFlow function signatures. The exam provides enough context in the question to choose the correct approach. Focus on concepts, not syntax. Additionally, you will not be asked to manually compute scaling parameters; the exam relies on recognising the correct approach.
Data preprocessing and feature engineering are not optional — they are the most critical steps in any machine learning project, often determining success or failure more than the choice of algorithm.
Always split your dataset into training and test sets before performing any data scaling or transformation to avoid data leakage.
Use embeddings (via `embedding_column` in TensorFlow) for high-cardinality categorical features instead of one-hot encoding to reduce memory usage and improve performance.
Imputation (filling missing values) is often better than dropping rows because it preserves useful data; common methods include using the median, mean, or a model prediction.
Feature engineering creates new informative features from existing data, such as interaction features, date parts, or aggregations, which can unlock patterns the model would otherwise miss.
Google Cloud tools like Cloud Dataprep, Dataflow, and Vertex AI Pipelines manage preprocessing at scale, while `tf.Transform` handles full-dataset transformations in TensorFlow.
Data leakage is a silent killer — it makes a model look perfect in training but useless in production, and the exam will test your ability to spot it.
The PMLE exam tests your ability to choose the right preprocessing and feature engineering method for a given scenario, not to memorise syntax or write code from scratch.
These come up on the exam all the time. Here's how to tell them apart.
Min-Max Scaling (Normalisation)
Rescales data to a fixed range, typically 0 to 1.
Highly sensitive to outliers because the min and max can be extreme.
Best for data with a known bounded range, like pixel values in images.
Z-Score Standardisation
Rescales data to have a mean of 0 and a standard deviation of 1.
Less sensitive to outliers because it uses mean and standard deviation.
Best for data that follows a Gaussian (normal) distribution.
One-Hot Encoding
Creates one binary column per unique category, leading to high-dimensional and sparse data.
Works well for features with low cardinality (few unique values).
No learned relationships between categories; each is treated independently.
Embeddings
Maps each category to a dense, low-dimensional vector of learned weights.
Ideal for high cardinality features (e.g., thousands of product IDs) as it saves memory and captures semantic similarities.
Categories that appear together in similar contexts get similar embedding vectors, revealing latent relationships.
Imputation (Filling Missing Values)
Preserves the size of the dataset, which is important for small datasets.
Uses statistical measures (mean, median) or predictive models to fill gaps.
Can introduce bias if the imputed values are systematically wrong.
Removing Rows with Missing Values
Simplest approach — just delete rows with missing data.
Only safe when the proportion of missing data is very small (e.g., less than 5%).
Can discard valuable information and reduce the sample size, harming model performance.
tf.Transform (Full-Dataset Preprocessing)
Computes statistics (e.g., mean, max) over the entire training dataset before applying transformations.
Produces a SavedModel that contains the transformation graph, making it reproducible in production.
Required for operations like scaling that depend on global dataset statistics.
Standard TensorFlow Preprocessing (Batch Preprocessing)
Processes data one batch at a time within the model graph during training.
Cannot compute full-dataset statistics (e.g., min/max for scaling) because it only sees a batch.
Suitable for operations that are batch-independent, such as tokenising a single text string.
Mistake
You should normalise every numerical feature to a 0–1 range, even if the feature has a normal (bell-shaped) distribution.
Correct
For features with a normal distribution, standardisation (Z-score) is usually better because it preserves the shape of the distribution and is less sensitive to outliers. Min-max scaling is better for features with a known bounded range, like pixel values from 0 to 255.
Beginners often think all scaling is the same. They reach for 'normalise' as a one-size-fits-all solution because tutorials often only show one example. They do not realise that the choice depends on the data's distribution and the algorithm being used.
Mistake
If you have missing values, you should always remove the rows with missing data because it is the safest approach.
Correct
Removing rows should only be done when the amount of missing data is small (e.g., less than 5% of rows). If a large portion is missing, you lose too much information, and imputation (filling in the gaps) or modelling is preferred.
This misconception comes from a desire for 'clean' data. Beginners recall that models cannot handle missing values, so they choose the most drastic fix. They do not yet understand that data is expensive and losing it hurts model performance.
Mistake
Feature engineering is just an optional step that might improve accuracy slightly, but it is not necessary for a good model.
Correct
Feature engineering is often the single most impactful step in the entire machine learning pipeline. A simple model with well-engineered features can outperform a complex model with raw data. The PMLE exam treats feature engineering as essential, not optional.
Many beginners focus on the model algorithm (which is exciting) and underestimate the importance of data preparation. They think a clever algorithm will 'figure out' the patterns, but in reality, algorithms struggle with raw, messy, or weakly informative features.
Mistake
One-hot encoding is always the best way to handle categorical features because it is straightforward and does not imply order.
Correct
One-hot encoding is great for features with few categories (e.g., 5 or fewer unique values). For high-cardinality features (like user IDs with thousands of categories), one-hot encoding creates a huge, sparse matrix that wastes memory and can hurt model performance. Embeddings or hash buckets are preferred.
Beginners learn one-hot encoding first and it seems intuitive. They do not encounter its limitations until working with real-world large datasets. The misconception persists because many simple examples use low-cardinality features.
Mistake
It is fine to scale the entire dataset (including the test set) before splitting it into training and test sets. It simplifies the code.
Correct
Scaling before splitting causes data leakage: the training set gets information about the overall distribution that includes the test set. You must always fit scaling parameters on the training set only and then apply them to the test set.
This is a very common error because it feels efficient to process all data at once. Beginners do not understand the statistical concept of data leakage or why it inflates model performance unrealistically.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
Data preprocessing focuses on cleaning and organising raw data (handling missing values, scaling, encoding) to make it usable. Feature engineering creates new features from existing ones (like combining columns or extracting date parts) to help the model learn better patterns. Preprocessing is hygiene; engineering is creativity.
No, absolutely not. You must always fit the scaling parameters (e.g., min and max for min-max scaling) only on the training set, then apply the same transformation to the test set. Scaling before splitting causes data leakage and overestimates model performance.
One-hot encoding converts a categorical feature (like 'colour') into multiple binary columns (is_red, is_blue, is_green), each indicating the presence of that category. Use it when the feature has a small number of categories (e.g., fewer than 10) to avoid creating a huge, sparse dataset.
Data leakage happens when information from outside the training set (like the test set or future data) is used during training, making the model appear more accurate than it really is. Avoid it by splitting data before any transformations, not using the target variable to create features, and ensuring time-series data is split chronologically.
It depends. If only a small percentage of rows have missing values (e.g., less than 5%), you can safely drop them. If many rows are missing data, you should impute (fill) the missing values using the median, mean, or a model prediction to avoid losing too much information.
TensorFlow feature columns are a way to define how different input features should be preprocessed before being fed into a model. You specify transformations like scaling, one-hot encoding, or embeddings for each column. The feature column definition is portable and can be used consistently during both training and inference.
You've finished Data Preprocessing and Feature Engineering. Continue through the PMLE study guide to build a complete picture of the exam.
Done with this chapter?