AI0-001 AI Models and Data Engineering Practice Test 7 — 15 Questions

Question 1

A model's training accuracy is 99% but validation accuracy drops to 60%. What is the most likely issue?

Accepted Answer

Overfitting. A training accuracy of 99% with a validation accuracy of only 60% is a classic symptom of overfitting. The model has memorized the training data, including noise and outliers, rather than learning generalizable patterns, causing it to perform poorly on unseen validation data.

Answer

Data leakage

Answer

Multicollinearity

Answer

Underfitting

Question 2

An organization uses a machine learning model to approve loans. The model shows higher false positive rates for a protected group. Which data engineering step should be taken to mitigate this?

Accepted Answer

Use adversarial debiasing technique. Adversarial debiasing is a technique that trains the model to minimize prediction error while simultaneously preventing an adversary from predicting the protected attribute from the model's outputs. This directly reduces disparate impact by forcing the model to learn representations that are uncorrelated with the protected group, thereby lowering false positive rates for that group without simply removing the attribute.

Answer

Remove the protected attribute from training data

Answer

Increase model complexity

Answer

Add synthetic data to balance groups

Question 3

A data pipeline ingests streaming data from IoT sensors. The current batch processing pipeline causes stale predictions. Which architecture change is most appropriate?

Accepted Answer

Implement Apache Kafka and stream processing. Apache Kafka plus a stream processing engine (e.g., Kafka Streams, Flink, or Spark Structured Streaming) processes events as they arrive, eliminating the staleness inherent in batch pipelines. This directly addresses the requirement to move from stale batch predictions to real-time ingestion and inference. It is the only option that changes the architecture from batch to true streaming.

Answer

Use a larger batch interval

Answer

Revert to micro-batch processing with Apache Spark

Answer

Store raw data in Hadoop HDFS

Question 4

A fraud detection model has high precision but low recall. The cost of false negatives is very high. Which threshold adjustment should be made?

Accepted Answer

Decrease classification threshold. Decreasing the classification threshold makes the model more sensitive, classifying more instances as positive. This increases recall by catching more true positives, directly addressing the high cost of false negatives, even though precision may drop.

Answer

Use class weights during training

Answer

Apply SMOTE to the training data

Answer

Increase classification threshold

Question 5

Which TWO data preprocessing techniques reduce the dimensionality of a dataset?

Accepted Answer

Principal Component Analysis (PCA). Principal Component Analysis (PCA) (D) is correct because it projects the original features onto a smaller set of orthogonal principal components that capture most of the variance, thereby reducing the number of dimensions while preserving as much information as possible. Feature selection (E) is also correct because it explicitly removes irrelevant or redundant features and keeps only a subset of the original variables, directly lowering the dataset's dimensionality. By contrast, one-hot encoding (A) increases dimensionality by creating a separate binary column per category, and imputation (B) merely fills in missing values without changing the number of features. Feature scaling (C) standardizes or normalizes feature values (e.g., via min-max or z-score) but leaves the feature count unchanged, so it does not reduce dimensionality.

Answer

One-hot encoding

Answer

Imputation

Answer

Feature scaling

Question 6

Which THREE are common causes of data leakage in machine learning pipelines?

Accepted Answer

Using future information to predict the present. Option B is correct because using future information to predict the present is the classic definition of temporal leakage: when training features contain values that would not have been available at prediction time (e.g., tomorrow's price used to predict today's), the model learns relationships that cannot generalize. Option D is correct because applying normalization (e.g., StandardScaler or MinMaxScaler) before splitting lets the scaler compute statistics such as the mean, variance, or min/max over the test data, so test-set information leaks into the training transformation; the scaler must be fit only on the training split. Option E is correct because features directly derived from the target variable (e.g., a 'remaining balance' column computed from the label, or target-encoded aggregates that include the current row's label) encode the answer into the inputs, producing artificially high validation scores that collapse in production. Option A is not a leakage cause but a mitigation: time-based splitting is the recommended approach for sequential data precisely to prevent temporal leakage. Option C is also not inherently leakage: cross-validation on the entire dataset is standard practice as long as the preprocessing is fit within each fold; leakage arises only if transformations are fit on the full dataset before cross-validation.

Answer

Using time-based splitting for sequential data

Answer

Using cross-validation on the entire dataset

Question 7

A deep learning model for image classification achieves 99% training accuracy but only 85% validation accuracy. The model has millions of parameters. Which technique is most likely to reduce overfitting while maintaining high accuracy?

Accepted Answer

Add dropout layers with a rate of 0.5 after each convolutional block. Dropout is a regularization technique that randomly drops a fraction of neurons during training, which prevents co-adaptation of features and forces the network to learn more robust representations. With 99% training accuracy and 85% validation accuracy, the model is clearly overfitting, and adding dropout layers with a rate of 0.5 after each convolutional block directly addresses this by reducing the model's capacity to memorize the training data, while still allowing high accuracy on the validation set.

Answer

Reduce batch size from 32 to 8

Answer

Decrease the learning rate by a factor of 10

Answer

Increase the number of training epochs to 500

Question 8

A machine learning engineer is training a Support Vector Machine (SVM) with an RBF kernel on a dataset with features on different scales (e.g., age 0-100, income 0-1,000,000). The model converges slowly and yields poor accuracy. What should the engineer do first?

Accepted Answer

Standardize the features to have zero mean and unit variance. Standardizing features to zero mean and unit variance is the correct first step because SVMs with RBF kernels are distance-based models. Features on vastly different scales (e.g., age 0-100 vs. income 0-1,000,000) cause the kernel to disproportionately weight larger-scale features, leading to slow convergence and poor accuracy. Standardization ensures each feature contributes equally to the distance calculations, improving both training speed and model performance.

Answer

Increase the regularization parameter C to penalize misclassifications more

Answer

Decrease the gamma parameter to reduce the influence of each data point

Answer

Switch to a linear kernel to avoid distance calculations

Question 9

A medical imaging team is developing an AI model to detect tumors from CT scans. They have 10,000 labeled scans, but the labels were created by a semi-automated process with an estimated 20% error rate (mislabeled tumor vs. no tumor). The team trains a convolutional neural network (CNN) and achieves 90% accuracy on a held-out test set that was carefully validated by an expert radiologist. However, when deployed to a new hospital's patient population, the accuracy drops to 70%. The team suspects domain shift and label noise. Which strategy is most likely to improve model robustness for the new hospital?

Accepted Answer

Use active learning to select the most uncertain predictions from the new hospital's data, then have an expert radiologist correct those labels. Active learning selects the most uncertain predictions from the new hospital's data, allowing an expert radiologist to efficiently correct the most informative labels. This directly addresses both label noise (by correcting mislabeled examples) and domain shift (by focusing on samples where the model is uncertain in the new domain). Option B is wrong because random selection may not target the most impactful errors, wasting expert effort. Option C is wrong because adding more noisy labels from the same flawed process will amplify label noise without correcting the domain shift. Option D is wrong because reducing model complexity and dropout are regularization techniques that do not fix label noise or domain shift.

Answer

Randomly select 1,000 scans from the new hospital and have them re-labeled by the radiologist

Answer

Collect 20,000 more scans with the same semi-automated labeling process

Answer

Reduce the CNN's number of layers and apply dropout to combat overfitting

Question 10

An e-commerce company deploys a model to recommend products to users. The recommendation system uses collaborative filtering based on user-item interaction history. After deployment, the model shows decreasing click-through rates (CTR) over time. The data engineer notices that the model was trained on data from the past six months and is retrained daily. However, the trend suggests that user preferences are shifting more rapidly than expected. The engineer suspects that the model is suffering from distribution drift. Which approach should the engineer implement to adapt the model more quickly to changing user behavior?

Accepted Answer

Switch to an online learning algorithm that updates the model after each user click. Switching to an online learning algorithm that updates the model after each user click allows the recommendation system to adapt in near real-time to shifting user preferences, directly addressing the rapid distribution drift. This is the most responsive approach when preferences change faster than daily retraining can capture.

Answer

Increase the retraining period to once per week to reduce computational cost

Answer

Increase the model complexity by adding more features and layers

Answer

Use only the last week of data for training to focus on recent trends

Question 11

A healthcare company is developing a predictive model to identify patients at risk of readmission within 30 days. The data engineering team has built a pipeline that collects data from multiple sources, including electronic health records (EHR), lab results, and wearable device data. During initial testing, the model's performance is poor, with high false positives. Upon investigation, the team discovers that the data contains significant temporal misalignment: lab results are timestamped when ordered, not when collected; wearable data is aggregated hourly; and EHR data has inconsistent update frequencies. The data pipeline currently joins all features on the patient ID without aligning timestamps. The data volume is large, and processing time is a concern. Which action should the data engineering team take to most effectively address the issue and improve model performance?

Accepted Answer

Implement a window-based feature aggregation (e.g., 6-hour windows) and align all features to the same time windows before joining.. Temporal misalignment causes features to be joined at incorrect times, leading to data leakage or irrelevant features that degrade model performance. Implementing window-based aggregation aligns all features to consistent time windows (e.g., 6-hour) before joining, ensuring that features reflect the patient's state at the same point in time. This addresses the root cause and improves model accuracy.

Answer

Discard all records where timestamps do not match exactly across sources, and only use records with perfect alignment.

Answer

Leave the pipeline unchanged and instead adjust the model's classification threshold to reduce false positives.

Answer

Use a data imputation algorithm to fill in missing timestamps and then join on the nearest timestamp.

Question 12

A retail company is building a recommendation system to suggest products to customers based on their purchase history. The data engineering team has collected data from point-of-sale systems, online browsing logs, and customer reviews. After cleaning the data, they notice that the feature set has over 500 dimensions, leading to high computational costs and potential overfitting. They need to reduce dimensionality while preserving as much variance as possible for the model. The team is considering various techniques. Which approach should they take to achieve this goal most effectively?

Accepted Answer

Use Principal Component Analysis (PCA) to reduce the feature space to the top 50 principal components that explain 95% of the variance.. Principal Component Analysis (PCA) is a linear dimensionality reduction technique that transforms the original high-dimensional feature space into a set of orthogonal principal components, ordered by the amount of variance they capture. By selecting the top 50 components that explain 95% of the variance, the team effectively reduces the feature set from over 500 dimensions while preserving the most informative structure in the data, directly addressing the goals of lowering computational cost and mitigating overfitting.

Answer

Keep all features but apply L1 regularization (Lasso) in the model to automatically reduce coefficients to zero.

Answer

Apply t-Distributed Stochastic Neighbor Embedding (t-SNE) to reduce the feature space to 50 dimensions.

Answer

Select only features that have a high correlation with the target variable, discarding all others.

Question 13

A logistics company uses a machine learning model to predict delivery times based on historical data. The model was performing well, but recently it started making inaccurate predictions, especially for routes that have experienced new traffic patterns and road closures. The data engineering team receives an alert that the model's accuracy has dropped by 15% over the last week. They suspect data drift. The team has access to the original training data and a continuous stream of new data. What is the most appropriate first step for the team to take?

Accepted Answer

Compare the distributions of key features between the training data and the recent data to quantify data drift.. The first step in diagnosing a suspected data drift is to statistically compare the distributions of key features between the training data and the recent streaming data. This quantifies whether the input data distribution has changed, which directly explains the accuracy drop. Without this analysis, any corrective action (like retraining or rollback) would be premature and could mask the root cause.

Answer

Roll back the model to the previous stable version and schedule a full audit of the data pipeline.

Answer

Immediately retrain the model using the most recent data to adapt to the new patterns.

Answer

Add more features to the model to capture the new traffic patterns and road closures.

Question 14

A media company is building a recommendation model from user clickstream logs. The raw data arrives as millions of small JSON files in an object store, and nightly training jobs currently take over ten hours because the training cluster reads thousands of tiny files per second. The team wants to reduce training time without changing the model or the underlying data values. Which data engineering approach is most appropriate?

Accepted Answer

Compact the small JSON files into larger columnar files such as Parquet and have the training pipeline read those instead.. The bottleneck is the small-file pattern in object storage, not compute capacity or parsing speed. Repacking the same records into fewer large columnar files cuts listing, open, and read overhead and enables column pruning and compression, so the training job moves the same data values with far less I/O. This addresses the root cause while preserving the model and data semantics.

Answer

Convert the JSON files to CSV because CSV parsing is faster than JSON parsing.

Answer

Increase the number of worker nodes in the training cluster so more files can be read in parallel.

Answer

Cache the JSON files on local SSD storage on each worker node before training begins.

Question 15

A fraud detection team trains a gradient boosted tree model on transaction data. During evaluation, the team notices the model performs extremely well on the training set but poorly on a holdout set drawn from the same time period. Investigation shows that a feature named 'chargeback_flag' is populated only after a dispute is resolved, sometimes weeks after the transaction. The team wants to deploy the model to score transactions in real time. Which action best addresses the problem?

Accepted Answer

Remove the 'chargeback_flag' feature and retrain the model using only features that are available at transaction scoring time.. The 'chargeback_flag' is populated only after a dispute is resolved, which is after the transaction outcome is known, so it leaks the label into training. The correct fix is to identify features that are actually available at real-time scoring and retrain without the leaky field. Hyperparameter tuning, larger validation sets, or partial replacements do not remove the leak and will keep offline metrics misleadingly high.

Answer

Increase the size of the holdout set so the evaluation becomes more statistically reliable.

Answer

Replace the flag with a rolling average of the customer's past chargebacks to preserve some of its signal.

Answer

Apply stronger regularization and reduce the model's maximum depth to prevent it from relying on the flag.