AI0-001 AI Models and Data Engineering Practice Test 3 — 15 Questions

Question 1

A large e-commerce company uses a recommendation system based on collaborative filtering. The system uses a matrix factorization model that is trained nightly on the entire user-item interaction history. Recently, the company launched a flash sale with thousands of new products. Users are reporting that the recommendations are not showing the new products, even for users who have purchased them during the sale. The data engineering team notices that the new products have very few interactions in the training data. The model's loss on the validation set has increased, and the recall@10 metric has dropped from 0.45 to 0.32. The team needs to improve the recommendation of new items without retraining the entire model from scratch every hour. Which approach should the team take?

Choose an answer to begin — your selection is scored in the full session.

Accepted Answer

Use a hybrid model that combines collaborative filtering with content-based features from product metadata. A hybrid model that combines collaborative filtering with content-based features (e.g., product metadata like category, price, or description) can recommend new products even with zero or very few user interactions. The content-based component leverages item attributes to compute similarity between new and existing items, enabling the system to surface new products without requiring extensive interaction history. This approach addresses the cold-start problem for new items while preserving the collaborative filtering signal for established items, and it does not require retraining the entire model from scratch every hour.

Answer

Retrain the model every hour to incorporate new interactions quickly

Answer

Remove the new products from the recommendation pool until they accumulate enough interactions

Answer

Increase the number of latent factors in the matrix factorization model

Question 2

A healthcare startup is developing a deep learning model to detect diabetic retinopathy from retinal images. The model is trained on a dataset of 10,000 labeled images. During initial testing, the model achieves 99% accuracy on the training set but only 85% on the test set. The startup wants to deploy the model in a clinical setting where false negatives (missing a disease) are critical. The team has access to additional unlabeled retinal images from multiple sources. Which strategy should the team use to improve the model's generalization and reduce false negatives?

Accepted Answer

Use semi-supervised learning with the unlabeled images to improve feature representations. Semi-supervised learning leverages the large pool of unlabeled retinal images to learn robust feature representations, which helps the model generalize better to unseen data. By reducing overfitting (the gap between 99% training and 85% test accuracy), this approach directly improves test-set performance. Additionally, semi-supervised methods can be tuned to emphasize recall, thereby reducing false negatives critical in clinical diabetic retinopathy screening.

Answer

Apply aggressive data augmentation to the training set

Answer

Increase the learning rate during training

Answer

Add more convolutional layers to the model

Question 3

A data scientist is preparing a dataset for a classification model. The dataset contains a column "Age" with 10% missing values and a column "Income" with 30% missing values. Which imputation strategy is MOST appropriate to minimize bias?

Accepted Answer

Replace missing Age with the mean and missing Income with the median.. Using mean imputation for Age (10% missing) and median imputation for Income (30% missing) minimizes bias. Mean is suitable for roughly symmetric distributions with low missingness, while median is robust to outliers and skewness, which is common in income data. This combination reduces distortion of central tendency and preserves data integrity better than uniform methods.

Answer

Delete all rows with missing values.

Answer

Replace missing Age with the mode and missing Income with a constant value.

Answer

Replace missing values with zeros.

Question 4

A team is building a regression model to predict house prices. The dataset includes numerical features (square footage, number of bedrooms) and categorical features (neighborhood, roof type). The categorical features have high cardinality (neighborhood has 200+ unique values). Which encoding strategy should the team use to avoid overfitting and maintain model interpretability?

Accepted Answer

Target encoding with regularization.. Target encoding with regularization is the best choice because it replaces each categorical value with the mean of the target variable for that category, which captures the relationship between the category and house prices. Regularization (e.g., adding a prior or using cross-validation) shrinks the encoded values toward the global mean, preventing overfitting on rare categories (e.g., neighborhoods with only a few houses). This maintains interpretability because each encoded value directly reflects the average price impact of that category, unlike black-box embeddings.

Answer

Label encoding.

Answer

Binary encoding.

Answer

One-hot encoding with feature selection.

Question 5

A data scientist trains a deep learning model on a large dataset. The training loss decreases steadily but the validation loss starts increasing after 20 epochs. The scientist uses early stopping with patience=5. Which of the following is the MOST likely cause and best corrective action?

Accepted Answer

Model is overfitting; add dropout regularization.. The training loss decreasing while validation loss increasing after 20 epochs is a classic sign of overfitting, where the model memorizes training data noise instead of generalizing. Early stopping with patience=5 would halt training after 5 epochs of no validation improvement, but the root cause is overfitting. Adding dropout regularization randomly drops neurons during training, forcing the network to learn more robust features and reducing overfitting.

Answer

Training data is not representative; collect more data.

Answer

Model is underfitting; increase model capacity.

Answer

Learning rate too high; reduce learning rate.

Question 6

A company streams sensor data from IoT devices. The data arrives as JSON messages at high velocity. Which data pipeline architecture is BEST suited to handle this streaming data for near-real-time analytics?

Accepted Answer

Stream processing using Apache Kafka and Spark Streaming.. Apache Kafka acts as a distributed, fault-tolerant ingestion layer that can handle high-velocity JSON messages, while Spark Streaming processes the data in micro-batches for near-real-time analytics. This combination provides the low-latency, scalable pipeline required for streaming IoT sensor data, unlike batch or single-node approaches.

Answer

Batch processing using Hadoop MapReduce every 24 hours.

Answer

Batch processing using nightly ETL jobs.

Answer

Single-node database with periodic inserts.

Question 7

A dataset for a binary classification problem has 95% of samples in class "0" and 5% in class "1". The data scientist trains a logistic regression model and achieves 95% accuracy. Which metric should the scientist primarily use to evaluate model performance?

Accepted Answer

Precision, recall, and F1-score.. In a highly imbalanced dataset (95% class 0, 5% class 1), accuracy is misleading because a model can achieve 95% accuracy by simply predicting the majority class for all samples. Precision, recall, and F1-score provide a more nuanced view of performance on the minority class, which is typically the class of interest in binary classification problems. The F1-score, in particular, balances precision and recall, making it the primary metric for evaluating model effectiveness on imbalanced data.

Answer

R-squared.

Answer

Accuracy.

Answer

Mean squared error.

Question 8

An e-commerce company needs to update its recommendation model continuously as user preferences change. The model currently retrains from scratch every night, but the training time is too long. Which approach would reduce training time while keeping the model up-to-date?

Accepted Answer

Implement incremental learning using online gradient descent.. Incremental learning using online gradient descent updates the model parameters with each new data point or mini-batch, avoiding the need to retrain from scratch. This approach significantly reduces training time while continuously adapting to changing user preferences, making it ideal for real-time recommendation systems.

Answer

Use dimensionality reduction on features.

Answer

Switch to a simpler model.

Answer

Increase the batch size for retraining.

Question 9

A data engineer discovers that a dataset contains duplicate rows. Which data cleaning step is MOST appropriate?

Accepted Answer

Remove duplicate rows.. Removing duplicate rows is the most appropriate data cleaning step because duplicate rows can bias statistical analyses and machine learning models by overrepresenting certain observations. In data engineering, deduplication is a standard preprocessing step to ensure data integrity and avoid skewed results. Option C directly addresses this by eliminating redundant entries without introducing artificial values or altering the data distribution.

Answer

Keep only the first occurrence.

Answer

Fill duplicates with the mean.

Answer

Convert duplicates to categorical.

Question 10

A machine learning model for credit card fraud detection is deployed. The model's precision is 0.95 and recall is 0.60. The business cost of missing a fraud is very high. Which of the following should the team prioritize to reduce the number of false negatives?

Accepted Answer

Decrease the classification threshold.. Decreasing the classification threshold makes the model more sensitive, classifying more transactions as fraudulent. This increases recall (reducing false negatives) at the cost of precision. Given the high cost of missing fraud, lowering the threshold is the direct way to capture more true positives, even if it increases false positives.

Answer

Use a different model algorithm.

Answer

Add more features.

Answer

Increase the classification threshold.

Question 11

A data scientist is cleaning a dataset. Which TWO actions are appropriate for handling missing data?

Accepted Answer

Use a predictive model to estimate missing values.. Using a predictive model to estimate missing values is a sophisticated imputation technique that leverages relationships between features to fill gaps, preserving data integrity and avoiding bias. This approach is particularly useful when data is not missing completely at random, as it can capture complex patterns that simpler methods miss.

Answer

Ignore missing values and train the model directly.

Answer

Impute missing values with the mean of the entire dataset.

Answer

Replace missing values with the most frequent value always.

Question 12

Which THREE practices are recommended for versioning machine learning models in a production environment?

Accepted Answer

Use a model registry like MLflow or DVC.. A model registry like MLflow or DVC provides a centralized repository for tracking model versions, metadata, and lineage. This enables reproducibility, rollback, and auditability in production, which is essential for managing the lifecycle of machine learning models.

Answer

Use Git to version model binaries.

Answer

Keep only the latest model to save storage.

Question 13

Which THREE data quality dimensions are critical for ensuring model reliability?

Accepted Answer

Timeliness.. Timeliness is critical because stale data can lead to incorrect model predictions, especially in dynamic environments like network traffic analysis or fraud detection. For AI models, data must reflect the current state of the system to ensure relevance and reliability. Without timely data, the model may act on outdated patterns, reducing its effectiveness.

Answer

Completeness.

Answer

Volume.

Question 14

Refer to the exhibit. A data scientist reviews the MLflow run for a Random Forest model on customer churn data. What is the most likely issue with this model?

Accepted Answer

The model is overfitting because there is a large gap between train and validation accuracy.. A large gap between training accuracy (e.g., 0.99) and validation accuracy (e.g., 0.82) indicates that the Random Forest model has memorized the training data but fails to generalize to unseen validation data. This is the classic symptom of overfitting, where the model captures noise rather than the underlying pattern. In MLflow, comparing train and validation metrics directly reveals this discrepancy.

Answer

The model is underfitting because training accuracy is too high.

Answer

The model is performing well because validation accuracy is above 0.8.

Answer

The model has a data leak because dataset version is v2.

Question 15

Refer to the exhibit. A stream processor ingests events. One event arrives with missing "user_id". What will happen?

Accepted Answer

The event will be rejected because user_id is required.. In stream processing systems like Apache Kafka or AWS Kinesis, if a required field such as 'user_id' is missing and the schema (e.g., Avro, JSON Schema) defines it as required, the event will be rejected at ingestion. The stream processor typically validates the event against the schema; if validation fails, the event is not accepted into the stream, preventing downstream processing errors.

Answer

The event will be stored in a dead-letter queue automatically.

Answer

The event will be accepted but user_id will be set to null.

Answer

The event will be accepted with a default user_id of 0.