Production-grade implementations demonstrating Featuresmith SDK pipelines, interactive Jupyter tutorials, automated CI/CD quality gates, and custom rule design.
Follow our step-by-step interactive learning path located in examples/notebooks/. Every notebook includes real code, problem statements, output interpretations, and best practices.
Learn how to load datasets (fs.load), run deterministic profiling (fs.profile), conduct automated reviews (fs.review), and extract ML readiness scores (fs.score).
01_getting_started.ipynbExplore the 10 automated reviewers evaluating schema health, data types, missingness spikes, duplicate records, constant columns, high cardinality, distributions, feature quality, leakage risk, and snapshot deltas.
02_dataset_review.ipynbDeep dive into the 0–100 quality scorecard, mathematical dimension weights, category breakdowns, and actionable remediation suggestions.
03_ml_readiness_score.ipynbMaster target correlation detectors, timestamp anomalies, identifier shapes, post-outcome names, and duplicate target copies using 6 specialized pattern detectors.
04_leakage_detection.ipynbCompare dataset snapshot versions (v1 vs v2) to detect schema drift, missingness spikes, distribution shifts, and receive an overall health verdict.
05_dataset_diff.ipynbBuild an automated Python pre-training quality gate function that validates datasets, enforces score thresholds, and halts pipelines on critical findings.
06_end_to_end_workflow.ipynbExtend BaseRule to write domain-specific quality checks, register custom rules in RuleRegistry, and evaluate them against ProfileResult.
07_custom_rules_and_extensions.ipynbThis pipeline script demonstrates loading data, executing automated dataset code reviews, extracting scorecards, and running snapshot comparisons via fs.diff().
import featuresmith as fsdef run_featuresmith_pipeline(data_path: str, target: str): # 1. Load data into normalized Dataset wrapper print(f"Loading dataset from {data_path}...") dataset = fs.load(data_path) print(f"Loaded {dataset.row_count} rows across {dataset.column_count} columns.") # 2. Perform automated dataset code review with 10 reviewers print("Running automated dataset review...") review_result = fs.review(dataset, target_column=target) # 3. Extract explainable 0–100 ML Readiness Scorecard scorecard = fs.score(review_result) if scorecard: print(f"ML Readiness Score: {scorecard.overall:.1f} / 100") print("Dimension Breakdown:") for dim in scorecard.dimensions: print(f" - {dim.label:<20}: {dim.score:5.1f}/100 ({len(dim.contributing_findings)} findings)") # 4. Compare with baseline dataset snapshot (Dataset Diff) diff_res = fs.diff(data_path, "baseline.csv", target_column=target) print(f"Dataset Health Verdict: {diff_res.summary.overall_health.upper()}")if __name__ == "__main__": run_featuresmith_pipeline("customer_churn.csv", target="churn_label")Integrate the Featuresmith CLI into your GitHub Actions workflow. Deterministic exit code gating ensures broken or leaked datasets never reach training.
# .github/workflows/data-quality-gate.ymlname: Data Quality Gateon: push: branches: [ main ] schedule: - cron: '0 0 * * *' # Daily pipeline auditsjobs: audit: runs-on: ubuntu-latest steps: - name: Checkout code uses: actions/checkout@v4 - name: Set up Python uses: actions/setup-python@v5 with: python-version: '3.11' - name: Install dependencies run: | pip install featuresmith-cli - name: Audit dataset quality & leakage run: | # Gate CI build on review findings featuresmith review data/train.csv --target churn_label --format json --output report.json - name: Upload audit report if: always() uses: actions/upload-artifact@v4 with: name: data-audit-report path: report.jsonExtend the BaseRule abstraction to create custom deterministic check rules. This example detects numeric columns with zero standard deviation.
from featuresmith.core.profile_result import ProfileResultfrom featuresmith.rules.base import BaseRulefrom featuresmith.core.rule_finding import RuleFindingclass ZeroVarianceRule(BaseRule): """Detect numeric columns with zero standard deviation.""" @property def id(self) -> str: return "statistical.zero_variance" @property def name(self) -> str: return "Zero Variance Columns" @property def description(self) -> str: return "Flags numeric columns with no observed variance." @property def category(self) -> str: return "statistical" @property def severity(self) -> str: return "warning" @property def enabled_by_default(self) -> bool: return True def evaluate(self, profile: ProfileResult) -> list[RuleFinding]: findings: list[RuleFinding] = [] for col_name, numeric_profile in profile.numeric_profiles.items(): if numeric_profile.std_dev == 0.0: findings.append( RuleFinding( rule_id=self.id, rule_name=self.name, category=self.category, severity=self.severity, column_name=col_name, title="Zero Variance Detected", description=f"Column '{col_name}' has standard deviation of 0.0.", evidence={"std_dev": numeric_profile.std_dev} ) ) return findingsThe Titanic dataset is bundled in the repository (no setup needed). The other datasets are generated or downloaded with the example preparation scripts in the project root:
Source: scikit-learn (load_iris)
Canonical clean benchmark dataset containing 0 missing values and clean feature ranges. Serves as a 100/100 baseline.
Source: OpenML (titanic)
Historical survival dataset containing cabin null spikes, free text columns, and age missingness. Triggers missing value and data type findings.
Source: scikit-learn (fetch_california_housing)
Continuous spatial housing metrics for regression. Triggers distribution skewness and kurtosis structural findings.
Source: Telco Churn Dataset
Telecom subscriber records containing synthetic target leakage columns. Triggers 4 intelligent leakage pattern detectors.
Source: Superstore Transactions
Transactional sales dataset used to demonstrate Dataset Diff (fs.diff) snapshot version comparisons (v1 vs v2).
Review Featuresmith's loading, profiling, review, and score benchmarks.