What do you do when your data is incomplete, full of errors, or systematically unfair? This chapter tackles the foundational problems of data quality, missing values, and bias detection, which are the first hurdles every machine learning project must clear to avoid building a model that is inaccurate, unreliable, or even unethical. For the MLS-C01 exam, understanding how to identify and fix these issues is critical because exam questions frequently test your ability to spot the right tool for the job—whether it's imputing a missing number, detecting a skewed sample, or cleaning a messy dataset.
Jump to a section
A simple way to picture Data Quality, Missing Values, and Bias Detection
A family photo album is a record of shared history, meant to capture everyone and every special moment accurately.
When you gather the whole family for a holiday dinner photo, you check for data quality: is everyone in focus? Is the lighting good enough to see faces? Are there any thumbs covering the lens? These are obvious issues. Missing values are like when your cousin forgets to show up, leaving an empty space in the picture. You then have to decide: do you take the photo without them and accept the gap, or do you wait and hope they arrive, knowing you might miss the perfect lighting? Bias detection is the trickiest part. If you only ever photograph the same three people because they are the loudest or most photogenic, your album gives a skewed story of the family. It will look like the quiet aunt or the baby in the corner never existed, even though they were just as present. A data scientist faces the same problem: a dataset might be missing entries for certain groups of people, or it might have been collected in a way that over-represents one type of customer while ignoring others. If you build a machine learning model on that biased album, it will learn that only the loud, photogenic relatives matter, and it will fail when it encounters the quiet aunt in the real world. The goal is to create a complete, well-lit, and fair photo of reality before you let a model learn from it.
Before you can teach a machine learning model anything, you need to feed it data. But raw data from the real world is rarely perfect. It is often messy, incomplete, and sometimes unfairly biased. This chapter explains the three big problems you will face: data quality issues, missing values, and bias in the data.
Data quality refers to how accurate, complete, and consistent your data is. Imagine you are building a model to predict house prices. If your dataset has a column called 'number of bedrooms' and someone accidentally typed '100' for a small apartment, that is a data quality issue. A second issue is consistency: if one column uses 'Jan' and another uses 'January' for the same month, the model gets confused. A third issue is duplication: if the same house appears twice in your dataset, the model might think that house is twice as important. In machine learning, the phrase 'garbage in, garbage out' is a core truth: if your input data is low quality, your output predictions will be unreliable.
Missing values are exactly what they sound like: gaps in your dataset. A row might be missing the price of a house, or the number of bathrooms, or the year it was built. Models often cannot handle these gaps directly. You have three main options for dealing with them. The first is deletion: you can delete the entire row with the missing value. This is simple but risky because you might throw away valuable data. The second is imputation (filling in the missing value): you can fill the empty spot with a reasonable guess, like the average price of all other houses, or the most common value, or a prediction based on other columns. The third is to use a model that can handle missing values natively, like certain tree-based algorithms. The choice depends on why the data is missing. Is it missing completely at random (like a survey question someone accidentally skipped)? Or is it missing because of a pattern (like people with higher incomes being less likely to report their income)? The second case is called 'missing not at random' and is trickier to fix.
Bias in machine learning is when your data does not fairly represent the real world. This is not about the 'bias' term in a linear equation; it is about systematic errors that cause your model to treat certain groups of people or situations unfairly. There are several types of bias you need to recognise. - Sampling bias: This happens when your data is collected in a way that over-represents one group and under-represents another. For example, if you train a facial recognition model only on photos of people with light skin, it will perform poorly on people with dark skin. - Label bias: This is when the 'answers' (labels) in your data are inconsistent or unfair. Human labelers might assign different scores to the same image because of fatigue or prejudice. - Historical bias: This occurs when the data reflects past inequalities. If you train a hiring model on data from a company that historically hired mostly men, the model will learn to prefer male candidates.
Identifying bias requires careful analysis. You can look at the distribution of your data: does one demographic group make up 90% of your dataset when they are only 30% of the population? That is a red flag. You can also measure things like 'demographic parity' (does your model predict the same outcome for different groups?) or 'equal opportunity' (does your model have similar accuracy for different groups?). For the MLS-C01 exam, you need to know that Amazon SageMaker Clarify is a specific AWS service that helps detect bias in datasets and models. It can generate reports showing whether certain features (like 'gender' or 'postcode') are correlated with bias.
In summary, before training any model, you must inspect your data for quality issues (errors, inconsistencies, duplicates), handle missing values appropriately (delete or impute with care), and check for bias that could make your model unfair or useless in the real world.
Data Inspection
Load your dataset and visually inspect it using tools like pandas `df.head()` or AWS Glue DataBrew. Look for obvious errors: values outside normal ranges, mixed data types in the same column, and inconsistent formatting. Record any issues you find.
Fix Data Quality Issues
Address inconsistencies: standardise date formats, convert text to numbers where needed, and remove or correct clearly wrong values (e.g., negative age). Drop duplicate rows to avoid overcounting. This step ensures your data is clean enough for the next stages.
Handle Missing Values
Analyse the proportion of missing values per column. If a column is 80% empty, consider deleting it. For columns with a small percentage missing, choose an imputation method: mean/median for numerical data, mode for categorical data, or a predictive model if the missingness pattern is complex. Document why you chose each method.
Detect Bias in the Data
Run a bias analysis on your cleaned dataset, ideally using Amazon SageMaker Clarify. Generate a bias report that shows metrics like Disparate Impact across sensitive features (e.g., age, postcode, gender). Identify which features have a high bias score (typically above 0.8 or below 0.5 for Disparate Impact ratio).
Mitigate Detected Bias
If you find bias, choose a mitigation technique: reweighing the training samples to give equal importance to under-represented groups, resampling (oversampling minority classes or undersampling majority classes), or removing the biased feature entirely if it's not essential for prediction. Re-run the bias report to confirm improvement before training the model.
Imagine you are a data scientist at a bank, and your boss asks you to build a machine learning model that predicts whether a customer will default on a loan. The bank has a historical dataset of 100,000 loan applications with columns like 'annual income', 'credit score', 'loan amount', 'employment length', and 'defaulted (yes/no)'. Your first task is not to build the model; your first task is to clean the data and check for bias.
You start by inspecting the data for quality issues. You notice that the 'credit score' column has values above 850, which is impossible (credit scores in most systems max out at 850). These are data entry errors, so you either correct them or remove those rows. You also see that 'employment length' is stored as text ('2 years', '5 yrs', '2') instead of a uniform number. You standardise all entries to numeric years, turning '5 yrs' into 5. You then find that about 5% of the rows are duplicates, likely because a customer applied twice. You drop the duplicates to avoid overcounting.
Next, you handle missing values. You notice that 10% of the rows have no 'annual income' entry. You check if the missingness is random. It turns out that customers who later defaulted are slightly more likely to have missing income data, which is called 'missing not at random'. Simply filling with the average income would be risky because it might hide the relationship between missing income and default risk. Instead, you use a more sophisticated imputation method: you build a small regression model using 'credit score' and 'loan amount' to predict the missing incomes. For a few rows that are missing almost all fields, you simply delete them to avoid introducing noise.
Finally, you check for bias. You look at the distribution of applicants by postcode (which can be a proxy for race or income level in some countries). You find that 80% of your training data comes from wealthy postcodes, even though only 40% of the bank's actual customers live there. This is sampling bias. If you train a model on this data, it will be much better at predicting defaults for wealthy customers and very poor for lower-income areas. You use Amazon SageMaker Clarify to run a bias analysis on your dataset. The report shows that the feature 'postcode' has a high 'Disparate Impact' value, meaning the model would likely make unfair decisions against applicants from certain postcodes. You decide to either rebalance the dataset (collect more data from under-represented postcodes) or remove 'postcode' as a feature if it is not essential.
After all this cleaning and bias checking, you finally train your model. Without these steps, you would have built a flawed model that might deny loans unfairly, harm the bank's reputation, and even violate regulations. In practice, a data scientist spends 60-80% of their time on these quality and bias tasks, not on building the fancy model.
The MLS-C01 exam tests Data Quality, Missing Values, and Bias Detection thoroughly within Exam Objective 3.3. Expect multiple-choice questions that give you a scenario and ask you to identify the correct technique or tool. The questions are designed to be tricky, so you must understand the nuances.
Here are the exact concepts the exam loves to test:
The difference between data quality issues: They will give you a dataset description and ask you to classify the problem. Is it a missing value? A duplicate? An inconsistency? A label error? Memorise these categories: completeness (missing values), consistency (different formats), accuracy (wrong values), duplication (same row twice), and timeliness (old data).
Handling missing values: The exam presents scenarios like 'a column has 30% missing values, and the missingness is random'. The correct answer is typically imputation with the mean or median, but if the missingness is not random, they expect you to recognise that simple imputation can introduce bias. They also test the difference between deleting rows (listwise deletion) vs. deleting columns (if the column is mostly empty).
Types of bias: You need to know the definitions of sampling bias, label bias, historical bias, and measurement bias (when the measurement device is flawed). A common question gives a description of a dataset (e.g., 'a survey done online only collects responses from people with internet access') and asks which bias type applies. The answer is sampling bias.
AWS services for bias detection: Amazon SageMaker Clarify is the key service. You must know that it can compute bias metrics like 'Disparate Impact', 'Difference in Positive Proportions', and 'Equal Opportunity Difference'. A question might ask 'which tool can generate a bias report for training data?' and the answer is SageMaker Clarify, not SageMaker Data Wrangler (data preparation) or SageMaker Ground Truth (labelling).
Bias mitigation techniques: The exam tests techniques like reweighing (giving different weights to examples from under-represented groups), resampling (oversampling the minority group or undersampling the majority group), and post-processing (adjusting model thresholds per group to achieve fairness). They will give you a scenario and ask you to choose the best mitigation approach.
Common traps the exam sets include:
Confusing data cleaning with bias detection. A question might describe a messy dataset with errors and ask for 'the best tool to detect bias'. Novices pick data cleaning tools, but the correct answer is SageMaker Clarify.
Thinking that deleting all rows with missing values is always the best approach. The exam will give a scenario where 80% of rows have a missing value in one column, and they want you to choose to delete the entire column instead.
Assuming 'mean imputation' is always safe. The exam will test that mean imputation can reduce variance and may mask real patterns when data is 'missing not at random'.
Mixing up bias types. They will describe 'label bias' and 'sampling bias' in similar ways, and the trick is to notice whether the problem is in how data was collected (sampling) or how labels were assigned (label bias).
To summarise, focus on memorising the definitions of the three types of data quality issues, the three common missing value strategies, the four major bias types, and the specific capabilities of SageMaker Clarify. Practice with sample questions that ask you to pick the right technique for a missing value scenario or identify a bias type from a description.
Data quality encompasses accuracy, consistency, completeness, and duplication; always fix these before training a model.
Missing values can be handled by deletion, mean/median imputation, or predictive imputation, but the choice depends on why the data is missing.
Bias in datasets leads to unfair and inaccurate models; types include sampling bias, historical bias, label bias, and measurement bias.
Amazon SageMaker Clarify is the primary AWS service for detecting bias in datasets and model predictions, using metrics like Disparate Impact.
Never assume a large dataset is free from bias—always examine the distribution of sensitive features (e.g., postcode, gender) against the target population.
Imputing missing values with the mean is not always safe; it can reduce variance and mask patterns, especially when data is missing not at random.
The phrase 'garbage in, garbage out' applies directly: flawed input data produces a flawed machine learning model, no matter how sophisticated the algorithm.
These come up on the exam all the time. Here's how to tell them apart.
Missing at Random (MAR)
The missingness depends on other observed columns, not the missing value itself.
Can be safely imputed using models that use other columns as predictors.
Example: older people are less likely to report income, but you have age data.
Missing Not at Random (MNAR)
The missingness depends on the missing value itself, even after controlling for other columns.
Imputation is difficult and may introduce bias because the missing values are systematically different.
Example: people with very high incomes are less likely to report it.
Sampling Bias
Caused by how data is collected: certain groups are over- or under-represented.
Can be fixed by collecting more data from missing groups or reweighting.
Example: a survey done only online misses people without internet.
Historical Bias
Caused by past decisions or societal inequalities that are reflected in the data.
Fixing requires understanding that the data itself encodes past discrimination.
Example: a hiring dataset from a company that historically hired mostly men.
Mean Imputation
Fills missing values with the column's average, which is simple and fast.
Reduces the variance of the column, potentially hiding true relationships.
Works poorly when data is skewed or has outliers.
Predictive Imputation
Uses a regression or machine learning model to predict the missing value from other columns.
Preserves the relationships between features better than a simple average.
More computationally intensive, but more accurate when the missingness is MAR.
Mistake
If I have a large dataset, I can just delete all rows with missing values and the model will still be accurate.
Correct
Deleting rows with missing values can dramatically shrink your dataset, introduce bias (if missingness correlates with the outcome), and reduce the model's performance. Imputation or column deletion is often better.
It seems like the simplest solution, and beginners assume more data is always better—but they overlook the bias introduced by systematic missingness.
Mistake
Bias in machine learning is only about the model's predictions being wrong for a particular race or gender.
Correct
Bias can also be about geography (postcodes), age, income brackets, or even device type (mobile vs desktop users). Any systematic over- or under-representation of a group in the data constitutes bias.
Media coverage focuses on racial and gender bias, so learners assume those are the only types. The exam tests a broader understanding.
Mistake
If I fill missing values with the average of the column, I ensure the data is completely unbiased.
Correct
Mean imputation can actually introduce bias by reducing the natural variability in the data and potentially masking relationships between the missing value and other variables.
It sounds mathematically neutral, but the act of filling gaps with a single value distorts the distribution, which many novices do not consider.
Mistake
Data quality only matters for the training dataset, not for the test set.
Correct
Both training and test sets must be cleaned and checked for quality. If the test set has errors or missing values, the model's evaluation will be inaccurate, and you won't know how it truly performs.
People focus on training the model and forget that evaluation data also needs cleaning to get a reliable performance metric.
Reveal each answer, then mark whether you got it right. Score 60%+ to unlock the next chapter.
Missing at random means the probability of a value being missing depends on other observed data (e.g., older people are less likely to report income). Missing not at random means the missingness depends on the value itself (e.g., people with very high incomes avoid reporting it). The latter is harder to fix because you cannot predict the missing value from other columns.
No, outliers skew the mean, so the imputed values will be misleading. Use median imputation instead, as it is robust to extreme values.
SageMaker Clarify computes several fairness metrics by comparing the distribution of predictions or features across different groups. For example, it calculates Disparate Impact (ratio of positive outcomes for a minority group to the majority group) to flag potential unfairness.
No, it is acceptable if the missing values are a small proportion (e.g., under 1%) and are missing completely at random. But if a large proportion of rows are affected, deletion could bias your dataset, so imputation or column deletion is preferred.
Bias detection matters for both. Even for regression (predicting a continuous value like salary), you need to ensure the model is not systematically under-predicting or over-predicting for certain groups. You can use metrics like 'difference in mean predictions' across groups.
The most common trap is confusing 'data quality issues' with 'bias'. A question might describe duplicate rows and ask for a 'bias detection tool'. The correct answer is a data preparation tool like AWS Glue DataBrew, not SageMaker Clarify.
You've finished Data Quality, Missing Values, and Bias Detection. Continue through the MLS-C01 study guide to build a complete picture of the exam.
Done with this chapter?