Core Concepts

Target Leakage Detection

Catch target correlations, timestamp anomalies, and outcome clones

Target leakage is one of the most dangerous bugs in applied machine learning. It occurs when features contain information from the target variable or future state that would not be available at inference time.

Why Target Leakage is Dangerous

Models trained on leaked features achieve deceptively high validation metrics (e.g. 99.9% ROC-AUC or near-zero loss) during development. However, when deployed to production where future outcome labels do not exist, the model fails completely.

Real-World Example

Suppose you are building a customer churn prediction model with target churn_label (1 = churned, 0 = active):

Leaked Feature: Including account_cancellation_date or refund_processed_amount.
The Bug: An account cancellation date is only recorded after a customer churns. In production at prediction time, cancellation dates are blank for active customers, causing the model to break.

The 6 Implemented Pattern Detectors

1. Target Correlation Detector: Flags features with Pearson correlation ≥ 0.99 with the target.
2. Identifier Shape Detector: Flags near-unique numeric ID columns correlated with the target outcome.
3. Timestamp Detector: Flags timestamp columns encoding post-outcome temporal data.
4. Future Information Detector: Flags columns named like outcome labels (e.g. refund_status).
5. Duplicate Target Detector: Detects near-identical transformed copies or encodings of the target.
6. Suspicious Correlation Detector: Flags unexpected strong feature correlations (≥ 0.95).