RunBSD

← Minecraft Backups

MCA Selector, ZFS, and Tarsnap: A Chunking Investigation

18 August 2026

Why this page exists

This is a side-quest that came out of the ZFS-to-Tape Backup project, not a branch of the pipeline in its own right. It started as "why is tape storage growing so fast?" and ended up as a real answer to "how do ZFS and Tarsnap each decide what counts as a changed chunk of data?" — which in turn shaped where MCA Selector's world-pruning step is allowed to run today. Worth its own page because the story doesn't fit cleanly on either the tarsnap or the tape page, and because most of it was already independently confirmed on two of those pages during earlier work — this page consolidates that into one narrative.

The trigger: ZFS incrementals nearly as big as full backups

While setting up the ZFS-to-tape pipeline, daily ZFS incrementals for the Minecraft world were coming in nearly as large as a full backup — roughly 700–800MB/day out of an ~800MB total dataset. That's not what an incremental is supposed to look like for a world that isn't being completely rewritten every night, so the natural question was: what's actually changing?

The suspect was MCA Selector, the tool that prunes rarely-visited chunks (via Minecraft's own InhabitedTime NBT field) to shrink the world before it goes into cold storage.

Finding 1 — MCA Selector rewrites the entire region file, not just the deleted chunks

Confirmed via MCA Selector's own source (GitHub issue stack traces referencing MCAFile.java, and a GitHub PR code snippet): MCA Selector prunes chunks by filtering on InhabitedTime — it only reads that value, it doesn't create or modify it. But deleting any chunk from a region file triggers a deFragment() step that repacks the remaining chunks to reclaim the freed space. The kept chunks' raw byte data is copied verbatim (dos.write(data)) — their actual NBT content, including InhabitedTime and LastUpdate, is untouched. The catch: the whole region file gets rebuilt as a new file, written to a temp file and atomically swapped in via Files.move.

Net effect: deleting a single chunk anywhere in a region file forces the entire file to be rewritten from scratch, dragging every other unrelated chunk in that file along for the ride — from any block-level backup system's point of view — even though none of those other chunks' content actually changed.

This was confirmed directly against real data, not just inferred from the source: a zfs diff between two snapshots showed literally every changed file (584 of 584) reported as a matched delete+recreate pair, not an in-place modification. That's the ZFS-incremental bloat, explained.

Finding 2 — Tarsnap doesn't have the same problem

Tarsnap uses content-defined chunking — a rolling-hash approach, confirmed via an academic paper analyzing its algorithm specifically — rather than ZFS's fixed-offset, block-level copy-on-write. Content-defined chunking finds chunk boundaries based on the data's own content rather than fixed byte offsets, so it can recognize the same byte sequence even after it's been relocated within a file — exactly what MCA Selector's deFragment() rewrite does. ZFS's fixed-offset scheme can't do this: once a file is rewritten at a new set of offsets, ZFS sees an entirely different set of blocks, dedup or not.

Confirmed empirically with tarsnap --print-stats -f <archive> against real archives, all created from MCA-Selector-pruned data — this was still true of every tarsnap archive at the time of the investigation; see "How this changed backup strategy" below for why that's no longer the case today:

How this changed backup strategy

This is the actual point of the investigation, not just a fun debugging story — though the real answer turned out to have two layers, and getting them straight matters (a mix-up here is exactly what a supervisor review caught on 18 August 2026, see the callout below).

Where things stand today

Sources