Core Concepts
Rule Engine
Deterministic quality and target leakage audits
Featuresmith separates profiling from auditing. The **Rule Engine** consumes a precomputed ProfileResult and evaluates a suite of deterministic, configurable rules. This allows rapid rules execution without re-scanning raw data.
Implemented Seed Rules
| Rule ID | Category | Severity | Description |
|---|---|---|---|
| quality.missing_value_threshold | quality | warning | Columns with > 20% missing values (configurable). |
| quality.duplicate_rows | quality | warning | Datasets with > 10% duplicate rows (configurable). |
| quality.constant_columns | quality | warning | Columns with exactly one unique non-null value. |
| quality.fully_empty_columns | quality | critical | Columns containing 100% null values. |
| statistical.high_cardinality | statistical | warning | Categorical columns with > 50% unique ratio. |
| statistical.outliers | statistical | warning | Numeric outliers detected via the IQR method (factor=1.5). |
| statistical.high_correlation | statistical | warning | Numeric pairs with Pearson correlation ≥ 0.90. |
| leakage.potential_leakage | leakage | critical | Features with Pearson correlation ≥ 0.99 to target. |
Rule Exception Isolation
A crash in a custom rule or internal rule evaluation must not block the rest of the profiling pipeline. Featuresmith handles exceptions internally per rule, listing rule failure stack traces in the final RuleResult.failed_rules mapping without terminating the run.