XGBoost Overfitting: Adjust Max Depth and Min Child Weight

Quick Answer

The clue that should point you toward these two hyperparameters is the classic overfitting signature: a very strong training AUC of 0.95 paired with a much weaker test AUC of 0.72, meaning the model has memorized training-set noise instead of learning generalizable patterns. In XGBoost, tree depth is one of the most direct controls over how specific a model's decision boundaries can become: a large max_depth lets each tree carve out narrow, idiosyncratic regions that fit individual training rows rather than broad patterns, so decreasing it forces simpler, more general trees. min_child_weight works as a complementary regularizer: it sets a minimum threshold on the sum of instance weights (essentially the amount of evidence) required before a node is allowed to split further, so raising it prevents the algorithm from creating splits based on just a handful of noisy or sparse observations. Used together, these two settings attack overfitting from both directions, one limits how deep and specific any single tree can get, the other limits how eagerly the algorithm creates new splits, which is why they're a natural pair for closing a training-versus-test performance gap. When you see a large gap between training and test performance in a tree-based model like XGBoost, think first about the hyperparameters that control tree complexity and split eligibility, since those are usually the fastest way to rein in variance without redesigning the whole pipeline.