AI0-001 AI Concepts and Techniques Practice Question

Answer analysis

Option-by-option breakdown

For each option: why learners choose it and why it is or isn't the right answer here.

• ✗

  Standardize all numeric features using the mean and standard deviation computed over the full dataset

  Why it's wrong here

  Scaling with statistics from the entire dataset leaks information about the validation and test distributions into preprocessing, subtly inflating results. More importantly, standardization changes feature magnitudes but not when the values were recorded, so a transaction count that includes post-label activity remains contaminated. The team needs to fix feature timing and evaluation design, not the numerical scale of the inputs.

• ✓

  Evaluate the model with a time-based split that trains on earlier periods and validates on later periods

  Why this is correct

  A temporal split respects the chronological order of events, so the validation set consists of customers whose outcomes occur after the training period. This mimics the real deployment setting where the model predicts future churn from past behavior. Combined with point-in-time features, it exposes whether the model truly generalizes forward in time instead of exploiting patterns that only exist within a randomly shuffled dataset.

• ✓

  Construct features using only information that was available before each customer's label observation date

  Why this is correct

  Point-in-time correctness means every feature value reflects the state of the world as of the label's decision moment, not the extraction date. By rebuilding transaction counts and balances using snapshots captured before each churn label was set, the team removes future information that would otherwise leak into training. This is the core remedy for the inflated validation accuracy and produces a realistic estimate of how the model will behave in production.

• ✗

  Increase the number of trees in the gradient boosting ensemble until validation accuracy stops improving

  Why it's wrong here

  Adding trees can reduce training error but does nothing to remove leakage, because the leaked feature values remain in every row regardless of model complexity. The suspiciously high validation accuracy stems from future information in the features, not from underfitting. Tuning the ensemble size may even make the model fit the leaked signal more tightly, worsening the problem instead of producing a trustworthy performance estimate.

• ✗

  Apply SMOTE to oversample the minority churn class before splitting the data into train and test sets

  Why it's wrong here

  Synthetic minority oversampling interpolates new examples between existing minority points. If it is applied before the split, synthetic points derived from test-set neighbors appear in training, which is itself a form of leakage and inflates reported metrics. Even when applied correctly after splitting, SMOTE addresses class imbalance, not the temporal leakage caused by features captured after the label date, so it cannot fix the root problem.