Site diffing

Site diffing is the technique of comparing two saved versions (snapshots) of a website to identify what changed between them. It's the underlying technology behind website change monitoring.

Modern site diffing typically uses content hashes (SHA-256 of the rendered text) to detect changes cheaply, then performs a more expensive textual diff only on pages where the hash changed. This lets a system check thousands of pages weekly while only doing expensive analysis on the ones that actually moved.

Naive site diffing (just comparing raw HTML) produces too many false positives because most sites have dynamic content — timestamps, randomized testimonials, A/B test variations — that changes constantly without strategic meaning. Good diffing extracts the meaningful text content first and ignores presentation noise.

The output of site diffing is a list of changes: added pages, removed pages, modified pages. The strategic value comes from layering analysis on top of the raw diff — identifying which changes matter and what they likely mean. The diff alone is data; the analysis is intelligence.