Data Preparation for Machine Learning practice questions

The CSV file is missing a header row

Why wrong: Missing header affects column names, not number of columns per row.

The file uses a different delimiter like tab

Why wrong: A different delimiter would cause consistent column count errors, not inconsistent.

Some fields contain quoted commas

Why wrong: CSV parsers handle quoted commas correctly.

Use SageMaker Processing jobs to run custom scripts.

Why wrong: SageMaker Processing jobs require infrastructure provisioning and are more suited for ML-specific tasks, not general ETL.

Use Amazon Athena to query and transform the data.

Why wrong: Athena is for interactive querying, not complex ETL transformations with nested JSON parsing and joins.

Use Amazon EMR with Apache Spark.

Why wrong: EMR provides powerful Spark capabilities but requires cluster management, increasing infrastructure overhead.

Use Amazon Comprehend to detect PII and then a custom script to mask

Why wrong: Amazon Comprehend is designed for natural language processing and PII detection, but it is overkill and cost-prohibitive for simple regex-based masking. It does not scale as well as a distributed PySpark job for large CSV files.

Use a custom script in AWS Glue Python shell job with regex

Why wrong: AWS Glue Python shell jobs run on a single node and cannot scale to process 500 GB of data efficiently. They are suitable for small datasets or lightweight transformations, but not for large-scale data masking.

Modify the script to install scikit-learn using pip at runtime

Why wrong: Installing at runtime using pip is not recommended because it can lead to conflicts and is not the standard AWS Glue approach.

Switch to a Glue job using Spark instead of Python

Why wrong: Switching to a Spark job does not address the missing module; you would still need to add the library via --additional-python-modules or install it.

Use a Glue Python shell job instead

Why wrong: Incorrect. Python shell jobs do not come pre-installed with scikit-learn; you would still need to add the library.

Delete all records with missing values to ensure data integrity.

Why wrong: Deleting 30% of data causes significant information loss and potential bias.

Apply one-hot encoding to all categorical features regardless of cardinality.

Why wrong: One-hot encoding high-cardinality features creates many dummy variables, leading to high dimensionality and overfitting.

Randomly undersample the majority class to balance the dataset before training.

Why wrong: Random undersampling can discard useful majority class examples, reducing model performance; techniques like SMOTE or class weights are preferred.

Use AWS Glue ETL to write a custom Python script that imputes missing values with the mean.

Why wrong: Custom scripts require development effort and may not be the most efficient for large datasets.

Use pandas in a SageMaker notebook to impute missing values with the median.

Why wrong: pandas is not designed for large-scale data processing and may run out of memory.

Remove all rows with missing values from the dataset.

Why wrong: Removing rows can lead to data loss and biased models, and is not always efficient.

Use Amazon Athena to convert the data to JSON format and store it in S3.

Why wrong: Athena is a query service, not a data transformation service.

Use AWS Glue to convert the data to Apache Hive format.

Why wrong: Hive format is not a standard file format; Parquet is columnar and efficient.

Use Apache Spark DataFrame to write the data as CSV with Snappy compression.

Why wrong: CSV is not columnar and does not optimize query performance as well as Parquet.

Apply data augmentation to the majority class by adding noise.

Why wrong: Data augmentation is not standard for tabular data and may introduce noise.

Use a weighted loss function during training to penalize misclassifications of the minority class.

Why wrong: Weighted loss is a training technique, not a data preparation step.

Apply random under-sampling to reduce the majority class to match the minority class size.

Why wrong: Under-sampling discards data and may lose important patterns.

Keep the column as a feature because it uniquely identifies each customer.

Why wrong: Unique identifiers do not have predictive power and should be excluded.

Use the column as the target variable.

Why wrong: The target variable is churn, not CustomerID.

Encode the column using one-hot encoding.

Why wrong: One-hot encoding a unique identifier creates many sparse features with no value.

Use partition pruning on the source data to reduce the amount of data processed.

Why wrong: Partition pruning reduces data scanned but does not directly improve resource consumption efficiency.

Switch the output format from Parquet to CSV to reduce processing overhead.

Why wrong: CSV is not columnar and may increase I/O and storage costs.

Use a larger instance type for the Glue job by increasing the number of DPUs.

Why wrong: More DPUs increase parallelism but also cost; may not be cost-effective.

Apply hash encoding to map categories to a fixed number of buckets.

Why wrong: Hash encoding may cause collisions and lose information.

Apply one-hot encoding to all categorical features.

Why wrong: One-hot encoding high-cardinality features creates too many columns, increasing dimensionality.

Apply label encoding to assign integer values to each category.

Why wrong: Label encoding implies an order that may not exist, leading to misleading patterns.

Encode categorical transform.

Why wrong: Encode categorical is for categorical encoding, not date parsing.

Scale values transform.

Why wrong: Scale values is for numerical scaling, not date parsing.

Handle missing transform.

Why wrong: Handle missing is for missing values, not date parsing.

Manually remove outliers by inspecting the data in Amazon S3.

Why wrong: Manual removal is not scalable.

Train a neural network to identify anomalies and remove them.

Why wrong: Using neural networks for outlier detection is complex and not necessary for basic outlier handling.

Use pandas in a SageMaker notebook to calculate z-scores and filter outliers.

Why wrong: pandas is not scalable for large datasets.

Apply feature selection to reduce the number of features.

Why wrong: Feature selection is optional and not a mandatory step for all models.

Remove outliers from the dataset.

Why wrong: Outlier removal is not always required; it depends on the data and model.

Include the raw PII in the training dataset and rely on the model to not memorize it.

Why wrong: Including PII is a privacy risk and may violate regulations.

Use AWS Glue to copy the data to a separate bucket without any transformations.

Why wrong: Copying without transformation does not address PII concerns.

Random undersampling of the majority class

Why wrong: Removes potentially valuable data.

Random oversampling of the minority class

Why wrong: Duplicates existing samples, risk of overfitting.

Apply class weights during model training

Why wrong: Affects loss function, not data preparation.

Label encoding

Why wrong: Assumes ordinal relationship, not suitable for nominal categories.

One-hot encoding

Why wrong: Creates too many features, leading to high dimensionality.

Frequency encoding

Why wrong: Replaces with count/frequency, may lose information.

Set k=1 to minimize bias

Why wrong: k=1 is prone to overfitting and noise.

Use all 100,000 rows to find neighbors for each missing value

Why wrong: Computationally expensive; consider sampling or approximate methods.

Use only the feature with missing values to find neighbors

Why wrong: Does not use information from other features.

Write a custom Python script using pandas dt.day_name()

Why wrong: Unnecessarily complex when built-in transform exists.

Use one-hot encoding on the timestamp

Why wrong: One-hot encoding is for categorical features, not extraction.

Use the 'extract' transform with format '%A'

Why wrong: Not a supported transform in Data Wrangler.

Standardization (z-score)

Why wrong: Centers and scales but does not change distribution shape.

Min-max scaling to [0,1]

Why wrong: Only rescales, does not affect skewness.

Equal-width binning

Why wrong: Converts continuous to categorical, discards information.