AI0-001 Practice Test 36 — 15 Questions

Question 1

A medical imaging team is developing an AI model to detect tumors from CT scans. They have 10,000 labeled scans, but the labels were created by a semi-automated process with an estimated 20% error rate (mislabeled tumor vs. no tumor). The team trains a convolutional neural network (CNN) and achieves 90% accuracy on a held-out test set that was carefully validated by an expert radiologist. However, when deployed to a new hospital's patient population, the accuracy drops to 70%. The team suspects domain shift and label noise. Which strategy is most likely to improve model robustness for the new hospital?

Accepted Answer

Use active learning to select the most uncertain predictions from the new hospital's data, then have an expert radiologist correct those labels. Option C is correct. Active learning helps select the most informative samples (e.g., uncertain predictions) for expert review, efficiently improving the model with limited expert effort. Option A is wrong because simply adding more noisy labels will amplify errors. Option B is wrong because random sampling may not capture the most valuable corrections. Option D is wrong because reducing model complexity may underfit, and dropouts fine-tuning might not address label noise or domain shift.

Answer

Randomly select 1,000 scans from the new hospital and have them re-labeled by the radiologist

Answer

Collect 20,000 more scans with the same semi-automated labeling process

Answer

Reduce the CNN's number of layers and apply dropout to combat overfitting

Question 2

An e-commerce company deploys a model to recommend products to users. The recommendation system uses collaborative filtering based on user-item interaction history. After deployment, the model shows decreasing click-through rates (CTR) over time. The data engineer notices that the model was trained on data from the past six months and is retrained daily. However, the trend suggests that user preferences are shifting more rapidly than expected. The engineer suspects that the model is suffering from distribution drift. Which approach should the engineer implement to adapt the model more quickly to changing user behavior?

Accepted Answer

Switch to an online learning algorithm that updates the model after each user click. Option A is correct. Online learning allows the model to update incrementally with each new interaction, adapting quickly to changes. Option B is wrong because batch retraining weekly is slower than daily. Option C is wrong because using only last week's data may not provide enough data and could be noisy. Option D is wrong because increasing model complexity may cause overfitting and is not a direct solution to drift.

Answer

Increase the retraining period to once per week to reduce computational cost

Answer

Increase the model complexity by adding more features and layers

Answer

Use only the last week of data for training to focus on recent trends

Question 3

A healthcare company is developing a predictive model to identify patients at risk of readmission within 30 days. The data engineering team has built a pipeline that collects data from multiple sources, including electronic health records (EHR), lab results, and wearable device data. During initial testing, the model's performance is poor, with high false positives. Upon investigation, the team discovers that the data contains significant temporal misalignment: lab results are timestamped when ordered, not when collected; wearable data is aggregated hourly; and EHR data has inconsistent update frequencies. The data pipeline currently joins all features on the patient ID without aligning timestamps. The data volume is large, and processing time is a concern. Which action should the data engineering team take to most effectively address the issue and improve model performance?

Accepted Answer

Implement a window-based feature aggregation (e.g., 6-hour windows) and align all features to the same time windows before joining.. Implementing a window-based feature aggregation with consistent time windows (e.g., 6-hour or 12-hour) and aligning all data to those windows before joining ensures temporal consistency and reduces noise. This approach addresses the root cause of misalignment while managing data volume through aggregation. Simply discarding data or padding with zeros loses valuable information. Using an interpolation algorithm may introduce unrealistic values for irregularly sampled data. Leaving the pipeline as-is and tuning the model does not fix the data quality issue.

Answer

Discard all records where timestamps do not match exactly across sources, and only use records with perfect alignment.

Answer

Leave the pipeline unchanged and instead adjust the model's classification threshold to reduce false positives.

Answer

Use a data imputation algorithm to fill in missing timestamps and then join on the nearest timestamp.

Question 4

A logistics company uses a machine learning model to predict delivery times based on historical data including distance, traffic, weather, and driver performance. The model is deployed as a REST API using Flask and run on a single server. Recently, the model has been returning predictions with high latency (over 2 seconds) during peak hours when the API receives 500 requests per second. The server has 8 CPU cores and 32 GB RAM. The model is a gradient boosting model (XGBoost) with 500 trees. The engineer wants to reduce inference latency to under 500ms without retraining the model. Which action is most effective?

Accepted Answer

Prune the model by reducing the number of trees to 100 and limit tree depth. Option B is correct. Model pruning reduces the number of trees in the ensemble, directly lowering inference time. Option A is wrong because adding more servers (horizontal scaling) addresses throughput but not per-request latency; it may help if the bottleneck is CPU, but pruning is more efficient. Option C is wrong because using a simpler model (linear regression) would require retraining and likely lose accuracy. Option D is wrong because increasing server memory does not speed up CPU-bound tree inference.

Answer

Replace XGBoost with a linear regression model

Answer

Scale horizontally by deploying additional servers behind a load balancer

Answer

Increase server RAM to 128 GB

Question 5

An e-commerce company deploys a deep learning model for product recommendation. After a new data pipeline is implemented, the model's online performance degrades: recall drops by 20% and the click-through rate decreases. The data scientists suspect data drift. They compare the distribution of the input features between the training data and recent production data. The Kolmogorov-Smirnov test shows significant differences for two numerical features (price and rating). The team also notices that the frequency of categorical feature 'category' has changed. Which of the following is the MOST appropriate first step? A. Immediately retrain the model on all available data including new production data. B. Roll back to the previous data pipeline and investigate the root cause of drift. C. Use feature selection to remove the drifting features and retrain. D. Implement a monitoring dashboard to track drift over time and set up alerts.

Accepted Answer

Roll back to the previous data pipeline and investigate the root cause of drift.. Option B is correct. Since the drift occurred after a pipeline change, rolling back and investigating the root cause is the most prudent first step before making model changes. Retraining on drifted data (A) might incorporate a faulty distribution. Removing drifting features (C) could lose important information and may not fully address the issue. Implementing monitoring (D) is useful for long-term but does not address the immediate degradation.

Answer

Implement a monitoring dashboard to track drift over time and set up alerts.

Answer

Use feature selection to remove the drifting features and retrain.

Answer

Immediately retrain the model on all available data including new production data.

Question 6

A financial institution uses a deep learning model for fraud detection. The model is a feedforward neural network with three hidden layers. It was trained on a balanced dataset of 100,000 transactions. During deployment, the model achieves high accuracy on the test set but the fraud detection rate (true positive rate) is only 40% while the false positive rate is 0.1%. The business requires a true positive rate of at least 80%. Which of the following actions is most likely to achieve the required true positive rate while minimizing the increase in false positives?

Accepted Answer

Change the threshold for classifying a transaction as fraud from the default 0.5 to a lower value. Option A (more hidden layers) may not improve recall and could overfit. Option C (L2 regularization) would increase bias, likely lowering TPR. Option D (SMOTE) rebalances training but the model already trained on balanced data; threshold adjustment is more direct. Option B (lower decision threshold) directly increases TPR at the cost of FPR; threshold can be tuned to achieve 80% TPR with minimal FPR increase.

Answer

Increase the number of hidden layers to five to capture more complex patterns

Answer

Use synthetic minority oversampling (SMOTE) to rebalance the training set

Answer

Add L2 regularization to reduce overfitting

Question 7

A media company uses a natural language processing (NLP) model to classify news articles into topics. The model was trained on articles from 2015-2018. In 2023, the model's F1 score drops significantly. The data scientists find that the word embeddings no longer capture the meaning of some terms (e.g., 'covid', 'metaverse'). The model uses static word embeddings (Word2Vec) trained on the original corpus. Which solution BEST addresses the observed degradation? A. Replace static embeddings with contextual embeddings from a transformer model like BERT, then fine-tune the classifier. B. Retrain the static Word2Vec embeddings on a larger corpus from 2023. C. Apply data augmentation to the original training data by replacing words with synonyms. D. Increase the dimensionality of the static embeddings.

Accepted Answer

Replace static embeddings with contextual embeddings from a transformer model like BERT, then fine-tune the classifier.. Option A is correct. Contextual embeddings (e.g., BERT) capture meaning based on context, adapting to new uses of words like 'covid' meaning pandemic. Fine-tuning the classifier on new data would update the model. Option B (retraining static embeddings) might capture new word senses but still assigns a single vector per word, missing context. Option C (data augmentation) does not introduce new word meanings. Option D (increasing dimensionality) does not address the semantic shift.

Answer

Retrain the static Word2Vec embeddings on a larger corpus from 2023.

Answer

Increase the dimensionality of the static embeddings.

Answer

Apply data augmentation to the original training data by replacing words with synonyms.

Question 8

A hospital wants to deploy a machine learning model to predict patient readmission risk within 30 days. They have a dataset with 10,000 records, 70 features including demographics, lab results, and past admissions. The target variable is binary (readmitted or not). The data scientist trains a logistic regression model and achieves an AUC of 0.85 on the test set. However, the hospital's clinicians require interpretability of predictions to trust the model. Which action should the data scientist take to ensure the model meets the interpretability requirement while maintaining performance?

Accepted Answer

Use the logistic regression model as is, since it is inherently interpretable with coefficients. Option A (random forest) offers feature importance but is less interpretable. Option C (deep neural network with LIME/SHAP) adds complexity and may reduce transparency. Option D (PCA and retrain) loses information and may degrade performance. Option B (keep logistic regression) provides inherent interpretability through coefficients, meeting requirements without sacrificing performance.

Answer

Reduce the number of features to 10 using PCA and retrain the logistic regression

Answer

Replace logistic regression with a random forest model and use feature importance plots

Answer

Train a deep neural network and apply LIME or SHAP for explanations

Question 9

A financial institution uses a random forest model to approve loan applications. Recently, the model's false positive rate has increased, leading to more defaults. The data science team reviews the feature importance and finds that the model heavily relies on a feature 'zip code' which correlates with income. The company is concerned about fairness. The regulatory team requires that the model's predictions are not biased against protected groups. Which action BEST addresses the fairness concern while maintaining predictive performance? A. Remove the 'zip code' feature and retrain the model. B. Use adversarial debiasing to train a model that is invariant to protected attributes. C. Add more training data from underrepresented zip codes. D. Apply a post-processing technique that adjusts thresholds for different groups.

Accepted Answer

Use adversarial debiasing to train a model that is invariant to protected attributes.. Option B is correct. Adversarial debiasing directly forces the model to learn representations that are not predictive of protected attributes, thereby reducing bias while maintaining performance as much as possible. Option A (removing zip code) might lose important information, as zip code could be a proxy for other legitimate factors; also, other features may still correlate with protected attributes. Option C (adding data) does not directly address bias and may not remove the correlation. Option D (post-processing) can adjust thresholds but may not address the underlying model bias; it is a less robust solution.

Answer

Apply a post-processing technique that adjusts thresholds for different groups.

Answer

Remove the 'zip code' feature and retrain the model.

Answer

Add more training data from underrepresented zip codes.

Question 10

An e-commerce company uses a gradient boosting model to forecast daily sales. Recently, the model's predictions have become less accurate, showing a significant drop in R-squared on validation data. The data scientist checks for data drift but finds no significant changes in feature distributions. The model was trained on data from the past 24 months and is retrained monthly. Upon inspecting the feature importance, the data scientist notices that the top feature 'promotion_flag' has decreased in importance over time. What is the most likely cause of the performance degradation, and what should be done?

Accepted Answer

Concept drift has occurred; retrain the model more frequently with recent data only, or use an online learning approach. Option A (overfitting to promotions) does not explain the drop over time. Option C (hyperparameter tuning) is unlikely to fix the temporal change. Option D (leakage) would have caused issues from the start. Option B correctly identifies concept drift (changing relationship) and suggests retraining more frequently or using online learning to adapt.

Answer

The model is overfitting to historical promotions; apply more regularization

Answer

The model's hyperparameters need tuning; perform a grid search

Answer

The promotion_flag feature is leaking future information; remove it

Question 11

A healthcare startup is developing a diagnostic system using medical images. The team has collected 10,000 labeled images of skin lesions. They plan to train a convolutional neural network (CNN) from scratch. However, training converges slowly, and the validation accuracy plateaus at 70%. The data scientist suspects overfitting. The dataset contains 8,000 images of benign lesions and 2,000 of malignant. The team has limited GPU resources. Which of the following is the MOST effective course of action to improve validation accuracy? A. Reduce the number of convolutional layers. B. Apply transfer learning using a pre-trained model on ImageNet. C. Increase the learning rate by a factor of 10. D. Add more dropout after every convolutional layer.

Accepted Answer

Apply transfer learning using a pre-trained model on ImageNet.. Option B is correct. Transfer learning leverages a model pre-trained on a large dataset (e.g., ImageNet), which provides useful features for medical images and reduces the need for large amounts of data and computational resources. It is particularly effective when the dataset is small and imbalanced. Option A (reducing layers) may reduce capacity and underfit. Option C (increasing learning rate) might cause divergence or overshoot minima. Option D (adding dropout) can help with overfitting but is unlikely to jump from 70% to a significantly higher accuracy given limited data; transfer learning provides a stronger boost.

Answer

Increase the learning rate by a factor of 10.

Answer

Reduce the number of convolutional layers.

Answer

Add more dropout after every convolutional layer.

Question 12

A financial institution is developing a fraud detection model using historical transaction data. The dataset contains over 10 million records, but only 0.01% of transactions are fraudulent. The current model uses a neural network trained with standard cross-entropy loss, and the team applies random undersampling of the majority class to create a balanced training set. However, the model still produces a high number of false positives (legitimate transactions flagged as fraud) and misses approximately 30% of actual fraud cases. The business requires that at least 95% of frauds be caught, and the false positive rate must be below 1% to avoid overwhelming fraud analysts. The team has limited resources to collect additional data and cannot change the model architecture significantly. Which approach should the team take to best meet the business requirements?

Accepted Answer

Use cost-sensitive learning by assigning a higher misclassification cost to the fraud class.. Cost-sensitive learning adjusts the loss function to penalize false negatives more heavily, directly addressing the need to catch more frauds while controlling false positives. Collecting more data is impractical and may not resolve the imbalance. Anomaly detection models treat fraud as outliers but often have high false positive rates in this context. Feature selection does not inherently solve the imbalance or performance metric trade-off.

Answer

Apply feature selection to remove noisy predictors and then retrain the current model.

Answer

Switch to an anomaly detection algorithm such as Isolation Forest or One-Class SVM.

Answer

Collect more transaction data, especially fraudulent examples, to naturally balance the classes.

Question 13

A hospital's radiology department uses an AI model to detect lung nodules in CT scans. The model was trained on data from a specific brand of scanners and patient demographics common in Europe. Recently, the hospital acquired new scanners from a different manufacturer and started serving a more diverse patient population. Over the past month, the model's false-positive rate has increased by 15% and false-negative rate by 8%. The radiologists are losing confidence and are considering abandoning the AI tool altogether. The IT team has verified that the model inference is running correctly and the hardware is performing as expected. The data science team suspects the problem is related to the change in input data distribution. The hospital's AI operations policy requires that any model update must be validated on at least 500 recent cases before deployment. What is the BEST course of action for the AI operations team?

Accepted Answer

Collect 500 recent CT scans from the new scanners, retrain the model on a combined old and new dataset, and validate before deployment.. Option C is correct because collecting 500 recent CT scans that represent the new scanner and patient distribution allows for validation of a retrained model that adapts to the new data. Option A is wrong because adjusting thresholds only addresses false-positive rate but not false-negative rate, and does not tackle the root cause. Option B is wrong because rolling back to the previous model where performance was good on old data would not improve performance on the new data. Option D is wrong because retraining on the original training data would not include the new distribution and likely yield the same issues.

Answer

Roll back to the previous model version and restrict use of the AI tool to only European patients.

Answer

Retrain the model using the original training data but with increased regularization to avoid overfitting.

Answer

Adjust the model's decision threshold to reduce false positives and then monitor for two weeks.

Question 14

A retail company uses a machine learning model to predict daily sales. The model takes features like past sales, promotions, holidays, and weather data. Recently, the model's accuracy dropped significantly. The data engineer checks the data pipeline and finds that the weather data source changed from a free API to a new paid API that provides more detailed data. The new data includes additional attributes like humidity and wind speed, but the existing pipeline only ingests temperature and precipitation. Also, the time zone format changed from UTC to local time. The model was trained on the old format. Which action should the engineer take first to restore model performance?

Accepted Answer

Transform the new data to match the old format (time zone and selected features) and retrain the model.. The immediate problem is the time zone change causing misalignment between training and inference data. Transforming the new data to match the old format ensures consistency. Retraining with all new features may introduce drift; reverting may not be possible; merging data without alignment causes inconsistency.

Answer

Add a new step to merge old and new weather data before feeding to the model.

Answer

Revert to the old weather API.

Answer

Retrain the model with the new data including all new features.

Question 15

A financial services firm deploys an AI system to screen loan applications. The model was trained on historical data that reflected biased lending practices. After deployment, a regulatory body investigates and finds that the model denies loans at a disproportionately higher rate to a protected demographic group. The firm must address this issue while maintaining compliance with fair lending laws. The Chief AI Officer proposes four possible actions. Which action is the most appropriate first step?

Accepted Answer

Retrain the model using a debiased dataset and implement fairness-aware algorithms, then validate with fairness metrics. Option C is the correct first step because it directly addresses the root cause (biased training data) and aligns with regulatory expectations. Option A is insufficient as it only documents without fixing. Option B may help but is premature before understanding the bias source. Option D could expose the firm to further liability if the bias is systematic.

Answer

Document the model's predictions and submit a report to the regulator explaining the historical dataset bias

Answer

Immediately deploy a rule-based system to manually review all denial decisions from the AI system

Answer

Disclose the bias findings to all rejected applicants and offer them priority reconsideration