Core Concepts
Interpreting Review Findings
A beginner's guide to understanding findings and taking action
When Featuresmith reviews a dataset, it outputs structured RuleFinding objects. Featuresmith evaluates datasets deterministically — it detects and reports statistical issues, but does not automatically mutate or delete your raw data. This guide helps beginners understand what each finding means, why it matters, whether it requires remediation, and what to investigate next.
1. High Missing Values (Null Spikes)
- WHAT DOES THIS MEAN? A column contains a high percentage of missing or null entries exceeding the configured threshold (default 20%).
- WHY MIGHT IT MATTER? Many ML models (like linear regression, SVMs, or neural networks) fail when passed nulls, or require imputation strategies.
- IS IT ALWAYS BAD? Not necessarily. In tree-based models (like XGBoost/LightGBM) nulls are handled natively, or missingness itself may be an informative predictive signal.
- WHAT SHOULD I INVESTIGATE NEXT? Determine whether nulls stem from uncollected data, pipeline drops, or zero defaults, and apply domain-appropriate imputation (mean, median, mode, or indicator flag).
2. Duplicate Records
- WHAT DOES THIS MEAN? Identical rows exist in the dataset exceeding the duplicate threshold (default 10%).
- WHY MIGHT IT MATTER? Duplicate rows distort model loss functions, over-weight identical samples, and cause severe data leakage if duplicated across train/test splits.
- IS IT ALWAYS BAD? Almost always in supervised learning — identical rows distort validation metrics.
- WHAT SHOULD I INVESTIGATE NEXT? Inspect raw data collection logs and deduplicate rows prior to train/test splitting using
df.drop_duplicates().
3. Constant & Zero-Variance Columns
- WHAT DOES THIS MEAN? A column contains only one single unique non-null value, or is 100% empty.
- WHY MIGHT IT MATTER? A feature with zero variance carries zero statistical entropy and zero predictive information gain, inflating matrix dimensionality unnecessarily.
- IS IT ALWAYS BAD? Not an error, but useless for predictive modeling.
- WHAT SHOULD I INVESTIGATE NEXT? Safely drop zero-variance and fully empty columns prior to model feature selection.
4. High Cardinality Categorical Columns
- WHAT DOES THIS MEAN? A text or categorical column has an excessive number of unique category strings (e.g. >50% unique ratio).
- WHY MIGHT IT MATTER? One-hot encoding high-cardinality columns creates sparse, high-dimensional matrices that slow down training and cause severe overfitting.
- IS IT ALWAYS BAD? If the column is a unique identifier (e.g.
customer_id), it should be dropped. If it is raw text or zip codes, it requires specialized encoding. - WHAT SHOULD I INVESTIGATE NEXT? Drop raw identifier columns or apply target encoding, frequency encoding, or text embeddings.
5. Schema & Data Type Mismatches
- WHAT DOES THIS MEAN? Numeric values are stored as string objects (e.g.
"123"), or column names violate standard conventions. - WHY MIGHT IT MATTER? String-encoded numbers prevent mathematical transformations, cause silent type coercions, or crash downstream estimators.
- IS IT ALWAYS BAD? Yes for numerical features — estimators require clean float/int types.
- WHAT SHOULD I INVESTIGATE NEXT? Cast string columns to numeric types using
pd.to_numeric()or Polars type conversions during ingestion.
6. Statistical Anomalies (Skewness & Kurtosis)
- WHAT DOES THIS MEAN? A feature exhibits extreme distribution asymmetry (skewness > 2.0) or heavy-tailed outlier spikes (kurtosis > 10.0).
- Why might it matter? Highly skewed features destabilize gradient descent optimization and distort linear model coefficient estimation.
- IS IT ALWAYS BAD? No. Power-law distributions (like user spend or transaction amounts) are naturally skewed.
- WHAT SHOULD I INVESTIGATE NEXT? Apply log transformations (
np.log1p), Box-Cox, or quantile scaling to normalize distributions before training.
7. Numeric Outliers (IQR Method)
- WHAT DOES THIS MEAN? Feature values fall beyond Q3 + 1.5*IQR or below Q1 - 1.5*IQR.
- WHY MIGHT IT MATTER? Extreme outliers exert disproportionate influence on mean calculations, standard deviations, and mean-squared-error loss functions.
- IS IT ALWAYS BAD? No. Outliers may represent real, critical business events (e.g. fraud spikes or high-value sales).
- WHAT SHOULD I INVESTIGATE NEXT? Evaluate whether outliers represent data corruption or genuine tail events, and apply winsorization or robust scalers.
8. Target Leakage Findings (CRITICAL Severity)
- WHAT DOES THIS MEAN? A feature correlates near-perfectly (≥0.99) with the target, encodes future timestamp information, or is named like an outcome label.
- WHY MIGHT IT MATTER? The model will learn a trivial shortcut, achieving 100% validation metrics in development but failing completely in production.
- IS IT ALWAYS BAD? Almost always — genuine 0.99 feature-target correlations are extremely rare outside of leaked outcome copies or IDs.
- WHAT SHOULD I INVESTIGATE NEXT? Trace feature generation timestamps relative to outcome events, verify pipeline logic, and drop leaked features immediately.
9. ML Readiness Score Interpretation
- WHAT DOES THIS MEAN? Translates overall review health into a single 0–100 quality scorecard across 7 effective dimensions.
- HOW TO READ SCORES:
- 100.0 / 100: Clean baseline; zero rule findings triggered (does not guarantee predictive accuracy).
- 80.0 – 99.0 / 100: Minor warnings or info findings present; review suggested remediations.
- < 80.0 / 100: Critical findings or severe quality issues detected; gate pipeline before training.
- WHAT SHOULD I INVESTIGATE NEXT? Inspect individual dimension scores and suggested actions to remediate specific low-scoring areas.
10. Dataset Diff Verdicts (unchanged vs improved vs regressed)
- WHAT DOES THIS MEAN? Compares two dataset snapshot versions (old vs new) to evaluate snapshot health deltas.
- DIFFERENCE BETWEEN CHANGED AND REGRESSED: A changed dataset (e.g., added rows or new features) is normal. A regressed dataset indicates dropped columns, missingness spikes, or newly introduced leakage.
- WHAT SHOULD I INVESTIGATE NEXT? Inspect
diff.summary,diff.schema, andfs.diff_findings()to isolate specific snapshot deltas.