AI Associate Data for AI Practice Question
A data scientist notices that an Einstein model for predicting customer churn has unusually high accuracy on training data but performs poorly on validation data. Which data issue is the most likely cause?
⚠ Common exam trap
Salesforce often tests the concept of data leakage by presenting it as a scenario where the model performs well on training data but poorly on validation data, and the trap is that candidates may confuse this with overfitting or class imbalance, rather than recognizing the inclusion of a future or target-related field as the root cause.
Answer choices
Why each option matters
Answer the question above first, then reveal the full breakdown to understand why each option is right or wrong.
Correct answer & explanation
✓
A field containing future information (e.g., 'churn_date') was included in features
Including a field like 'churn_date' in the feature set introduces target leakage, where the model has access to information that would not be available at prediction time. This causes the model to appear highly accurate on training data (since it can directly 'see' the outcome) but fails to generalize to validation data where such future information is absent. In Salesforce Einstein, features must be strictly historical or static to avoid this data leakage issue.
Answer analysis
Option-by-option breakdown
For each option: why learners choose it and why it is or isn't the right answer here.
- ✗
The dataset has an imbalanced class distribution
Why it's wrong here
Imbalance typically causes poor recall on minority class, not a huge train/validation gap.
- ✗
The dataset contains many missing values
Why it's wrong here
Missing values often reduce overall performance, not specifically a gap.
- ✗
The model was trained on stale data from a different season
Why it's wrong here
Stale data would affect both sets similarly if validation is also outdated.
- ✓
A field containing future information (e.g., 'churn_date') was included in features
Why this is correct
Data leakage from a field that reveals the outcome causes overfitting and high train accuracy.
Go deeper
Related to this question
About these practice questions
Courseiva writes every AI Associate question from scratch — 753 in total, each with an explanation and a wrong-answer breakdown. None are copied from real exams or dumps. Learn why practice questions differ from exam dumps →
JA
Written by Johnson Ajibi, MSc IT Security
Senior Network & Security Engineer · founder of Courseiva
This AI Associate practice question is part of Courseiva's free Salesforce certification practice question bank. Courseiva provides original exam-style practice questions with explanations, topic-based practice, mock exams, readiness tracking, and study analytics to help learners prepare for the AI Associate exam.