Data Drift Alert: Why Your Einstein Predictive Model May Be Unreliable
Exhibit
{
"predictionField": "Churn__c",
"recordCount": 1500,
"modelStatus": "Trained",
"accuracy": 0.85,
"auc": 0.72,
"featuresUsed": ["UsageRate", "SupportTickets", "ContractLength", "Region"],
"dataDrift": {
"detected": true,
"magnitude": 0.23
}
}Refer to the exhibit. A Salesforce admin evaluates an Einstein Prediction Builder model for customer churn. What should be the admin's primary concern based on the exhibit?
Quick Answer
The admin's primary concern should be the data drift alert because it signals something more fundamental than a model simply performing worse than expected; it means the statistical patterns in the current data no longer match what the model was originally trained on, so its predictions are no longer being made against a fair or realistic representation of today's customers. This matters even if the model's accuracy or AUC score looked strong when it was first built, because those metrics only reflect how well the model fit the data available at training time, not whether that data still resembles reality now. A churn model trained on outdated behavioral patterns can keep producing confident-looking predictions while those predictions steadily drift away from what's actually happening with real customers, which makes drift a more urgent problem than a merely mediocre performance metric, since it undermines the validity of every prediction the model produces going forward rather than just lowering overall accuracy. This is a useful distinction to carry into similar exhibit-based questions: strong historical performance metrics don't guarantee a model is still trustworthy, and when an exhibit specifically flags drift, that's the detail pointing to the real, active problem, since it means the model's picture of the business is going stale even if its original scorecard still looks fine.
⚠ Common exam trap
Salesforce often tests the distinction between model performance metrics (accuracy, AUC) and model health indicators (data drift), leading candidates to focus on missing or irrelevant metrics instead of the explicit alert shown.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
The model has detected data drift, indicating the training data may no longer represent current patterns.
The exhibit shows a data drift alert from Einstein Prediction Builder, which indicates that the statistical properties of the input data have changed over time. This is the admin's primary concern because a model trained on outdated patterns will produce unreliable predictions, even if its accuracy or AUC were initially high. Data drift directly undermines the model's validity in production.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
The model accuracy is too low for production use.
Why it's wrong here
0.85 accuracy is acceptable.
- ✓
The model has detected data drift, indicating the training data may no longer represent current patterns.
Why this is correct
Data drift makes predictions unreliable.
- ✗
The AUC is low, so the model is not better than random.
Why it's wrong here
AUC 0.72 is above random.
- ✗
The model uses too few features to be reliable.
Why it's wrong here
Four features may be sufficient.
Go deeper
Related to this question
About these practice questions
Courseiva writes every AI Associate question from scratch — 753 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
Same concept, more angles
1 more way this is tested on AI Associate
These questions test the same concept from different angles. Work through them to make sure you can recognise it however the exam phrases it.
Variation 1. An admin creates a predictive model in Einstein Prediction Builder to forecast customer churn. The model shows high accuracy on test data but poor performance in production. What is the most likely cause?
medium- A.Improper feature scaling in the training pipeline
- B.The model is overfitted to the training data
- C.Target leakage in the training dataset
- ✓ D.Data drift between training and production environments
Why D: The model's high accuracy on test data but poor performance in production is a classic symptom of data drift. In Einstein Prediction Builder, the model was trained on historical data that may not reflect current customer behavior patterns, leading to a mismatch between training and production distributions. This is not a model training issue but a data environment shift.
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI Associate practice question is part of Courseiva's free Salesforce certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI Associate exam.