Data Preparation for Machine Learning practice questions

Use AWS Glue ETL to write a custom Python script that imputes missing values with the mean.

Why wrong: Custom scripts require development effort and may not be the most efficient for large datasets.

Use pandas in a SageMaker notebook to impute missing values with the median.

Why wrong: pandas is not designed for large-scale data processing and may run out of memory.

Remove all rows with missing values from the dataset.

Why wrong: Removing rows can lead to data loss and biased models, and is not always efficient.

Use Amazon Athena to convert the data to JSON format and store it in S3.

Why wrong: Athena is a query service, not a data transformation service.

Use AWS Glue to convert the data to Apache Hive format.

Why wrong: Hive format is not a standard file format; Parquet is columnar and efficient.

Use Apache Spark DataFrame to write the data as CSV with Snappy compression.

Why wrong: CSV is not columnar and does not optimize query performance as well as Parquet.

Apply data augmentation to the majority class by adding noise.

Why wrong: Data augmentation is not standard for tabular data and may introduce noise.

Use a weighted loss function during training to penalize misclassifications of the minority class.

Why wrong: Weighted loss is a training technique, not a data preparation step.

Apply random under-sampling to reduce the majority class to match the minority class size.

Why wrong: Under-sampling discards data and may lose important patterns.

Keep the column as a feature because it uniquely identifies each customer.

Why wrong: Unique identifiers do not have predictive power and should be excluded.

Use the column as the target variable.

Why wrong: The target variable is churn, not CustomerID.

Encode the column using one-hot encoding.

Why wrong: One-hot encoding a unique identifier creates many sparse features with no value.

Use partition pruning on the source data to reduce the amount of data processed.

Why wrong: Partition pruning reduces data scanned but does not directly improve resource consumption efficiency.

Switch the output format from Parquet to CSV to reduce processing overhead.

Why wrong: CSV is not columnar and may increase I/O and storage costs.

Use a larger instance type for the Glue job by increasing the number of DPUs.

Why wrong: More DPUs increase parallelism but also cost; may not be cost-effective.

Apply hash encoding to map categories to a fixed number of buckets.

Why wrong: Hash encoding may cause collisions and lose information.

Apply one-hot encoding to all categorical features.

Why wrong: One-hot encoding high-cardinality features creates too many columns, increasing dimensionality.

Apply label encoding to assign integer values to each category.

Why wrong: Label encoding implies an order that may not exist, leading to misleading patterns.

Encode categorical transform.

Why wrong: Encode categorical is for categorical encoding, not date parsing.

Scale values transform.

Why wrong: Scale values is for numerical scaling, not date parsing.

Handle missing transform.

Why wrong: Handle missing is for missing values, not date parsing.

Manually remove outliers by inspecting the data in Amazon S3.

Why wrong: Manual removal is not scalable.

Train a neural network to identify anomalies and remove them.

Why wrong: Using neural networks for outlier detection is complex and not necessary for basic outlier handling.

Use pandas in a SageMaker notebook to calculate z-scores and filter outliers.

Why wrong: pandas is not scalable for large datasets.

Apply feature selection to reduce the number of features.

Why wrong: Feature selection is optional and not a mandatory step for all models.

Remove outliers from the dataset.

Why wrong: Outlier removal is not always required; it depends on the data and model.

Include the raw PII in the training dataset and rely on the model to not memorize it.

Why wrong: Including PII is a privacy risk and may violate regulations.

Use AWS Glue to copy the data to a separate bucket without any transformations.

Why wrong: Copying without transformation does not address PII concerns.

Random undersampling of the majority class

Why wrong: Removes potentially valuable data.

Random oversampling of the minority class

Why wrong: Duplicates existing samples, risk of overfitting.

Apply class weights during model training

Why wrong: Affects loss function, not data preparation.

Label encoding

Why wrong: Assumes ordinal relationship, not suitable for nominal categories.

One-hot encoding

Why wrong: Creates too many features, leading to high dimensionality.

Frequency encoding

Why wrong: Replaces with count/frequency, may lose information.

Set k=1 to minimize bias

Why wrong: k=1 is prone to overfitting and noise.

Use all 100,000 rows to find neighbors for each missing value

Why wrong: Computationally expensive; consider sampling or approximate methods.

Use only the feature with missing values to find neighbors

Why wrong: Does not use information from other features.

Write a custom Python script using pandas dt.day_name()

Why wrong: Unnecessarily complex when built-in transform exists.

Use one-hot encoding on the timestamp

Why wrong: One-hot encoding is for categorical features, not extraction.

Use the 'extract' transform with format '%A'

Why wrong: Not a supported transform in Data Wrangler.

Standardization (z-score)

Why wrong: Centers and scales but does not change distribution shape.

Min-max scaling to [0,1]

Why wrong: Only rescales, does not affect skewness.

Equal-width binning

Why wrong: Converts continuous to categorical, discards information.

Apply log transformation after adding a constant

Why wrong: Log transform is for positive data; adding constant is arbitrary.

Remove all rows with negative sales values

Why wrong: Loses data on returns, which may be informative.

Impute negative values with the mean

Why wrong: Incorrectly treats negative values as missing; mean may be positive.

Read the Parquet files as text using SparkContext.textFile

Why wrong: Loses the columnar benefits of Parquet.

Split the dataset into many small Parquet files (e.g., 1 MB each)

Why wrong: Too many small files cause I/O overhead.

Convert the Parquet files to CSV before processing

Why wrong: CSV is larger and slower to read than Parquet.

SMOTE to balance classes

Why wrong: Addresses class imbalance, not general overfitting.

One-hot encoding of all categorical features

Why wrong: Increases dimensionality, potentially worsening overfitting.

Log transformation of skewed features

Why wrong: Addresses skewness, not overfitting.

Apply one-hot encoding to each word

Why wrong: One-hot encoding is for categorical variables, not text preprocessing.

Compute TF-IDF vectors

Why wrong: TF-IDF is a feature extraction step, not preprocessing.

Use AWS Glue DataBrew to fill missing values with the median of total_purchases.

Why wrong: Median imputation, like mean imputation, does not preserve the distribution and can introduce bias.

Drop all records with missing total_purchases values.

Why wrong: Dropping 15% of records reduces sample size and may bias the model if missingness is not random.

Replace missing total_purchases with the mean of the non-missing values.

Why wrong: Mean imputation reduces variance and can distort relationships between variables.