Core Concepts
Profiling Engine
High-speed deterministic data profiling
Featuresmith does not compute layout charts in core; it computes **deterministic statistical descriptors** returned as a typed, serializable ProfileResult.
Statistical Coverage
The profiling engine executes optimized, batched vectorized computations to compile:
- 23 Numeric Metrics: Mean, median, mode, min, max, standard deviation, variance, skewness, kurtosis, quantiles (IQR, Q1, Q2, Q3), sum, zero count, negative count, positive count, range, and counts.
- Categorical Profiles: Cardinality, unique count, entropy, top frequency tables (capped), least frequent tables, and missing rates.
- Datetime Profiles: Minimum (earliest), maximum (latest), range span in days, and missing rates.
- Text Profiles: Average/min/max string length, empty strings count, whitespace-only count, total character count, and word count.
Correlation Safeguard
On wide tables, Pearson correlation matrix computation scales quadratically, which can crash memory or hang the execution. Featuresmith enforces a config-controlled correlation cap (max_correlation_columns, default 100) to ensure predictable execution.
python
# Customize the correlation column cutoffprofile = fs.profile("wide_table.csv", max_correlation_columns=50)