Featuresmith Icon
Featuresmith
  • Docs
  • SDK
  • CLI
  • Examples
  • Roadmap
v0.4.0
Documentation

Getting Started

  • Introduction
  • Installation
  • Quick Start
  • Tutorial Notebooks
  • Benchmarks
  • Development Setup
  • Contributing

Core Concepts

  • Architecture Overview
  • Dataset Layer
  • Connectors
  • Profiling Engine
  • Rule Engine
  • Dataset Review Engine
  • ML Readiness Score
  • Target Leakage Detection
  • Dataset Diff Engine
  • Target Column Concept
  • Mental Model & Workflow
  • Interpreting Findings
  • Workflow Cheat Sheet
  • Beginner Glossary

Python SDK

  • load()
  • profile()
  • analyze()
  • review()
  • diff()
  • score()
  • plan()
  • Dataset
  • Data Models
  • Profile Models
  • Rule & Finding Models
  • Review Models
  • Score Models
  • Leakage Models
  • Diff Models
  • Exceptions
  • Plugins

CLI Reference

  • analyze
  • review
  • diff
  • score
  • plan
  • Configuration

Guides

  • CI/CD Integration
  • Custom Rules
  • Writing Plugins

Resources

  • Release Notes
  • FAQ
  • Troubleshooting
DocsGuide

Getting Started

  • Introduction
  • Installation
  • Quick Start
  • Tutorial Notebooks
  • Benchmarks
  • Development Setup
  • Contributing

Core Concepts

  • Architecture Overview
  • Dataset Layer
  • Connectors
  • Profiling Engine
  • Rule Engine
  • Dataset Review Engine
  • ML Readiness Score
  • Target Leakage Detection
  • Dataset Diff Engine
  • Target Column Concept
  • Mental Model & Workflow
  • Interpreting Findings
  • Workflow Cheat Sheet
  • Beginner Glossary

Python SDK

  • load()
  • profile()
  • analyze()
  • review()
  • diff()
  • score()
  • plan()
  • Dataset
  • Data Models
  • Profile Models
  • Rule & Finding Models
  • Review Models
  • Score Models
  • Leakage Models
  • Diff Models
  • Exceptions
  • Plugins

CLI Reference

  • analyze
  • review
  • diff
  • score
  • plan
  • Configuration

Guides

  • CI/CD Integration
  • Custom Rules
  • Writing Plugins

Resources

  • Release Notes
  • FAQ
  • Troubleshooting
HomeDocsDataset Diff Engine

Core Concepts

Dataset Diff Engine

Detecting schema drift and quality regressions across dataset snapshots

In production ML environments, datasets evolve continuously as new data batches arrive daily or weekly. Upstream pipeline updates, database migrations, or third-party vendor changes can introduce silent regressions into fresh snapshots.

What Dataset Diff Compares

Featuresmith's fs.diff(old, new) engine profiles both snapshots and computes deterministic deltas across:

  • Schema Changes: Added, removed, or renamed columns, and data type changes.
  • Structure Changes: Row count deltas and column count shifts.
  • Missing Value Spikes: Per-column missingness shifts classified as new, resolved, regressed, or improved.
  • Duplicate Shifts: Changes in duplicate row counts and percentages.
  • Constant Column Changes: Newly constant or no longer constant columns.
  • Cardinality & Statistic Deltas: Shifts in unique values, mean, median, min, max, and standard deviation.
  • Leakage Status Deltas: Target leakage findings that were added, removed, escalated, or de-escalated.

Health Verdicts

  • unchanged: No material structural or quality changes between snapshots.
  • improved: Quality metrics improved (e.g. missingness decreased, leakage eliminated).
  • regressed: Quality degraded (e.g. columns dropped, missingness spiked, schema broke).

Explore

  • Quick Start
  • Python SDK
  • CLI Reference
  • Examples
Featuresmith Icon
Featuresmith

Open-source data profiling and validation for Python engineers.

Documentation

  • Introduction
  • Quick Start
  • Python SDK
  • CLI Reference

Community

  • GitHub
  • Discussions
  • Issues
  • Contributing

Project

  • Roadmap
  • Release status
  • Benchmarks
  • Changelog
  • Examples

Legal

  • Apache 2.0 License
  • Code of Conduct
  • Security

© 2026 Featuresmith Contributors. Released under the Apache 2.0 License.

Built by Aditya Gangwani in the open.