AI0-001 AI Models and Data Engineering Practice Test 6 — 15 Questions

Question 1

Which THREE are common causes of data leakage in machine learning pipelines?

Accepted Answer

Using future information to predict the present. Using future information to predict the present is a classic form of data leakage. In time series or sequential data, if a model is trained on features that include values from a later time point, it gains access to information that would not be available at prediction time, leading to overly optimistic performance metrics and poor generalization.

Answer

Using time-based splitting for sequential data

Answer

Using cross-validation on the entire dataset

Question 2

Which TWO strategies are effective for handling missing values in a dataset when the missingness is not random (MNAR)?

Accepted Answer

Multiple imputation using chained equations. Multiple imputation using chained equations (MICE) is effective for MNAR because it models each variable with missing values as a function of other variables, iteratively generating plausible values that preserve the relationships and uncertainty in the data. This approach can account for the systematic pattern of missingness by incorporating auxiliary variables that are correlated with both the missing values and the missingness mechanism, making it robust even when missingness depends on unobserved data.

Answer

Listwise deletion

Answer

KNN imputation

Answer

Mean imputation

Question 3

A data scientist is building a regression model to predict house prices. The dataset contains features such as square footage, number of bedrooms, and year built. Initial model performance is poor, and the scientist suspects that feature engineering could help. Which approach is most likely to improve model accuracy?

Accepted Answer

Create interaction terms such as bedrooms times square footage. Creating interaction terms like bedrooms × square footage captures non-linear relationships and synergies between features that a linear model alone cannot represent. In real estate, the effect of square footage on price often depends on the number of bedrooms (e.g., a large house with few bedrooms may be less valuable), so interaction terms allow the model to learn these conditional patterns, directly improving predictive accuracy.

Answer

Use only linear features because polynomial terms overfit

Answer

Remove all features except square footage to reduce noise

Answer

Add random noise to the target variable to increase variance

Question 4

A credit risk model is being developed to predict loan defaults. The dataset has 95% non-default and 5% default instances. The data scientist trains a logistic regression model and obtains 95% accuracy, but the recall for defaults is only 10%. Which action is most appropriate to improve the model's ability to identify defaults?

Accepted Answer

Use SMOTE to oversample the default class. SMOTE (Synthetic Minority Oversampling Technique) is the most appropriate action because it generates synthetic samples for the minority class (defaults) rather than simply duplicating existing ones. This directly addresses the severe class imbalance (95% non-default vs. 5% default) that causes the logistic regression model to achieve high accuracy by predicting nearly all instances as non-default, while failing to identify actual defaults (recall of only 10%). By creating realistic synthetic default instances, SMOTE balances the training data and forces the model to learn decision boundaries that better capture the minority class.

Answer

Apply principal component analysis (PCA) to reduce dimensionality

Answer

Collect more data from loan applicants to increase dataset size

Answer

Undersample the non-default class to match the number of defaults

Question 5

A dataset used for training a classification model contains 10% missing values in a feature that is known to be important. The data scientist decides to impute the missing values. Which imputation method is most robust if the data is not missing completely at random?

Accepted Answer

Use multiple imputation to model missing values. Multiple imputation is the most robust method when data is not missing completely at random (NMAR) because it uses a statistical model to account for the relationships between the missing feature and other observed features, generating multiple plausible values and combining them to produce unbiased estimates and valid standard errors. This approach preserves the variability and structure of the data, unlike simpler methods that can introduce bias when missingness depends on unobserved data.

Answer

Delete all rows with missing values

Answer

Replace missing values with the mean of the feature

Answer

Fill missing values with 0

Question 6

A deep learning model for image classification achieves 99% training accuracy but only 85% validation accuracy. The model has millions of parameters. Which technique is most likely to reduce overfitting while maintaining high accuracy?

Accepted Answer

Add dropout layers with a rate of 0.5 after each convolutional block. Dropout is a regularization technique that randomly drops a fraction of neurons during training, which prevents co-adaptation of features and forces the network to learn more robust representations. With 99% training accuracy and 85% validation accuracy, the model is clearly overfitting, and adding dropout layers with a rate of 0.5 after each convolutional block directly addresses this by reducing the model's capacity to memorize the training data, while still allowing high accuracy on the validation set.

Answer

Reduce batch size from 32 to 8

Answer

Decrease the learning rate by a factor of 10

Answer

Increase the number of training epochs to 500

Question 7

A data engineer is splitting a dataset into training, validation, and test sets for a machine learning project. The dataset is large and representative of the population. Which split ratio is commonly recommended?

Accepted Answer

80% training, 10% validation, 10% test. (80% training, 10% validation, 10% test) is commonly recommended for large, representative datasets because it provides sufficient data for model training while retaining enough samples in the validation and test sets to reliably evaluate model performance and detect overfitting. This split balances the need for a robust training set with the requirement for statistically meaningful holdout sets, as recommended in standard machine learning practices for AI model development.

Answer

90% training, 5% validation, 5% test

Answer

70% training, 20% validation, 10% test

Answer

50% training, 25% validation, 25% test

Question 8

A natural language processing (NLP) team is building a sentiment analysis model. The raw text data contains punctuation, stop words, and URLs. Which TWO preprocessing steps are most appropriate to improve model performance? (Choose two.)

Accepted Answer

Remove all punctuation and URLs. Removing punctuation and URLs eliminates noise that does not contribute to sentiment (e.g., 'http://...' or '!!!'), allowing the model to focus on meaningful words. This step reduces vocabulary size and prevents the model from learning spurious correlations tied to formatting artifacts.

Answer

Apply stemming to reduce words to root forms

Answer

Convert all text to lowercase

Answer

Tokenize the text into individual words

Question 9

A data scientist is evaluating a binary classification model for fraud detection. The dataset is highly imbalanced (99% non-fraud, 1% fraud). Which TWO metrics are most appropriate for assessing model performance? (Choose two.)

Accepted Answer

Precision. Precision is appropriate because it measures the proportion of predicted fraud cases that are actually fraudulent, which is critical when false positives (flagging legitimate transactions as fraud) are costly. In a highly imbalanced dataset like this (99% non-fraud), precision directly evaluates the model's ability to avoid overwhelming fraud analysts with false alarms.

Answer

F1 score

Answer

Area under the ROC curve (AUC-ROC)

Answer

Accuracy

Question 10

A computer vision team is building an image classifier for rare wildlife species. The dataset has only 500 images per class, and the model overfits. Which THREE data augmentation techniques are most likely to reduce overfitting? (Choose three.)

Accepted Answer

Horizontal flip. Horizontal flip is a simple and effective data augmentation technique that doubles the training data by mirroring images, which helps the model generalize better to variations in orientation. This is particularly useful for wildlife images where the animal may appear facing left or right, reducing overfitting by exposing the model to more diverse examples without collecting new data.

Answer

Adding Gaussian noise

Answer

Random cropping

Question 11

A healthcare startup is deploying a machine learning model to predict patient readmission within 30 days using electronic health records (EHR). The data pipeline uses Apache Spark for preprocessing and training on an Amazon EMR cluster. The training dataset is 50 GB and composed of structured numeric and categorical features, along with unstructured clinical notes. The data scientist observes that training takes over 12 hours and frequently fails due to out-of-memory (OOM) errors, especially when processing the clinical notes via TF-IDF vectorization. The cluster has 10 nodes with 64 GB RAM each. The data engineer has already tried increasing spark.sql.shuffle.partitions to 400 and using Kryo serialization, but OOM persists. Which action should the data engineer take next to resolve the OOM errors?

Accepted Answer

Repartition the clinical notes data into 2000 partitions before TF-IDF. Repartitioning the clinical notes data into 2000 partitions before TF-IDF vectorization increases parallelism and reduces the memory pressure per partition. The default partition count (often based on spark.default.parallelism) is too low for 50 GB of data, causing individual partitions to exceed executor memory limits. By increasing partitions, each executor processes smaller chunks, preventing OOM errors during the memory-intensive TF-IDF stage.

Answer

Broadcast the TF-IDF model to all executors to avoid shuffling

Answer

Add 10 more nodes to the cluster to increase total memory

Answer

Use a single executor with 64 GB and increase driver memory to 128 GB

Question 12

A financial services company has a real-time fraud detection system that uses Apache Kafka to stream transaction events, a TensorFlow Serving model for scoring, and a Redis cache for lookup of historical fraud patterns. The system processes 10,000 transactions per second with an SLA of 100ms latency per transaction. Recently, after a model update, the latency for some transactions spiked to over 500ms, causing timeouts. The model uses a deep neural network with 10 million parameters. The engineering team suspects the issue is due to increased model inference time. Which action should be taken to reduce latency without significant loss in accuracy?

Accepted Answer

Quantize the model weights from FP32 to FP16. The latency spike is caused by increased model inference time after a model update. Quantizing model weights from FP32 to FP16 reduces memory bandwidth and computation requirements, directly speeding up inference on compatible hardware (e.g., GPUs with Tensor Cores) with minimal accuracy loss. This addresses the root cause—model inference latency—without changing the system architecture.

Answer

Add more Redis nodes to the cache cluster

Answer

Increase the number of Kafka partitions and consumer threads

Answer

Decrease the inference batch size from 32 to 1

Question 13

A medical imaging team is developing an AI model to detect tumors from CT scans. They have 10,000 labeled scans, but the labels were created by a semi-automated process with an estimated 20% error rate (mislabeled tumor vs. no tumor). The team trains a convolutional neural network (CNN) and achieves 90% accuracy on a held-out test set that was carefully validated by an expert radiologist. However, when deployed to a new hospital's patient population, the accuracy drops to 70%. The team suspects domain shift and label noise. Which strategy is most likely to improve model robustness for the new hospital?

Accepted Answer

Use active learning to select the most uncertain predictions from the new hospital's data, then have an expert radiologist correct those labels. Active learning selects the most uncertain predictions from the new hospital's data, allowing an expert radiologist to efficiently correct the most informative labels. This directly addresses both label noise (by correcting mislabeled examples) and domain shift (by focusing on samples where the model is uncertain in the new domain). Option B is wrong because random selection may not target the most impactful errors, wasting expert effort. Option C is wrong because adding more noisy labels from the same flawed process will amplify label noise without correcting the domain shift. Option D is wrong because reducing model complexity and dropout are regularization techniques that do not fix label noise or domain shift.

Answer

Randomly select 1,000 scans from the new hospital and have them re-labeled by the radiologist

Answer

Collect 20,000 more scans with the same semi-automated labeling process

Answer

Reduce the CNN's number of layers and apply dropout to combat overfitting

Question 14

An e-commerce company deploys a model to recommend products to users. The recommendation system uses collaborative filtering based on user-item interaction history. After deployment, the model shows decreasing click-through rates (CTR) over time. The data engineer notices that the model was trained on data from the past six months and is retrained daily. However, the trend suggests that user preferences are shifting more rapidly than expected. The engineer suspects that the model is suffering from distribution drift. Which approach should the engineer implement to adapt the model more quickly to changing user behavior?

Accepted Answer

Switch to an online learning algorithm that updates the model after each user click. Online learning algorithms update the model incrementally with each new user click, allowing it to rapidly adapt to shifting user preferences. This directly addresses distribution drift caused by fast-changing behaviors. Option A is wrong because increasing retraining to once per week reduces update frequency, making the model slower to adapt. Option C is wrong because adding complexity does not solve distribution drift and risks overfitting. Option D is wrong because training on only the last week may produce a noisy model and does not provide a mechanism for continuous adaptation.

Answer

Increase the retraining period to once per week to reduce computational cost

Answer

Increase the model complexity by adding more features and layers

Answer

Use only the last week of data for training to focus on recent trends

Question 15

A logistics company uses a machine learning model to predict delivery times based on historical data including distance, traffic, weather, and driver performance. The model is deployed as a REST API using Flask and run on a single server. Recently, the model has been returning predictions with high latency (over 2 seconds) during peak hours when the API receives 500 requests per second. The server has 8 CPU cores and 32 GB RAM. The model is a gradient boosting model (XGBoost) with 500 trees. The engineer wants to reduce inference latency to under 500ms without retraining the model. Which action is most effective?

Accepted Answer

Prune the model by reducing the number of trees to 100 and limit tree depth. Pruning the model by reducing the number of trees (e.g., from 500 to 100) directly reduces the number of decision paths evaluated during inference, which decreases latency. This does not require retraining; pruning can be applied post-training. Option B (linear regression) would require retraining and likely lose accuracy. Option C (horizontal scaling) improves throughput but does not reduce per-request latency; it may help if the server is overloaded, but pruning is more effective for latency. Option D (increasing RAM) does not address the CPU-bound tree traversal bottleneck.

Answer

Replace XGBoost with a linear regression model

Answer

Scale horizontally by deploying additional servers behind a load balancer

Answer

Increase server RAM to 128 GB