RunBSD

Minecraft Backups

2 August 2026 (updated 18 August 2026)

Overview

The Minecraft world gets backed up through a chain of unattended steps spread across a few machines. Each branch of the process now has its own page:

  1. Shockbyte Backup Download — every night at 2AM, a Mac mini logs into the Shockbyte panel (behind Cloudflare bot protection) and downloads the latest instance.zip, then runs check-version.sh — see item 2.
  2. backup.sh & DVD Archiving — on that same Mac mini, immediately after the download: check-version.sh reads the downloaded zip's version.txt and dispatches to either backup-fabric.sh (confirmed live) or backup-vanilla.sh (exists, not yet verified). The fabric branch does world-integrity checks, rsyncs the trimmed (no longer nightly-pruned) world to both gmktec and seneca, archives a dated copy to iCloud (GFS-rotated), and — on a weekly cadence — burns a chunk-pruned DVD and runs a lightweight restore verification. Also handles alerting on failure via Mail.app. gmktec forwards its own copy on to batcave over SSH (ZFS snapshots, no SMB mount from the Mac), and all SSH auth uses a dedicated automation key instead of an agent-loaded personal one.
  3. ZFS-to-Tape Backup — a daily ZFS snapshot of the world, sent incrementally to tape, on seneca (Debian Linux — migrated off gmktec/FreeBSD in early August 2026 after ongoing USB/CAM driver issues there). As of 18 August, this is two parallel branches: Fabric (confirmed live, /dev/nst0) and Vanilla (built, using a second physically separate tape drive, /dev/nst1, that's been sourced but hasn't arrived yet — not currently scheduled).
  4. Tarsnap Backup — a nightly encrypted offsite archive via tarsnap, running on gmktec. As of 18 August, dispatched from dedicated tarsnap-create-fabric.sh/ tarsnap-rotate-fabric.sh scripts rather than an inline crontab command; a Vanilla-side rotate script exists but is still an unedited copy of the pre-split original.
  5. Daily Pipeline-Health Digest — added 8 August 2026, once nightly: pulls today's log lines from all four machines above and emails one combined summary, closing the "Needed improvements" gap below once it's proven stable over a few nights. Started as a small addition tracked alongside the Mac backup script's own files; as of 9 August 2026 it's grown into its own dedicated project, same as every other branch listed here. As of 18 August 2026, renamed to daily-digest-fabric.sh and covers only the Fabric branch of the pipeline — no Vanilla-side digest coverage yet.
  6. MCA Selector, ZFS, and Tarsnap: A Chunking Investigation — added 18 August 2026, a side-quest rather than a pipeline branch: the story of tracking ZFS-incremental bloat back to MCA Selector's chunk-pruning, why Tarsnap doesn't have the same problem, and how that reasoning shaped where pruning is and isn't allowed to run across the pipeline today.

Added 18 August 2026: the Fabric/Vanilla split

Shockbyte's instance.zip can now come back as either a Fabric or a Vanilla Minecraft world backup, instead of always being the same single world. Every branch above now runs (or is being built to run) two parallel, independently-named copies of itself — separate scripts, separate ZFS datasets (java-fabric-minecraft-conundra / java-vanilla-minecraft-conundra), separate log files — dispatched from a single new check-version.sh step on the Mac that reads the downloaded zip's version.txt to decide which flavor it's looking at. As of this update, the Fabric branch is confirmed live end-to-end across all four hosts; the Vanilla branch is partially built (some scripts confirmed present, some not yet deployed or scheduled — see each branch's own page for exactly what's confirmed where).

Why split like this

Each branch now runs on different hardware and fails independently, so each gets its own troubleshooting page, dependencies, and exit codes rather than one long document covering all of it.

Needed improvements

Identified during a full-pipeline review (August 2026). Individually the branch pages are solid, but looking at the whole flow at once surfaced a few structural gaps. Update (5 August 2026): the first three are now addressed — see backup.sh & DVD Archiving for the details.

  1. Alerting on the Shockbyte/backup.sh branch. Done. backup.sh now has its own mail_alert(), driving the signed-in Mail.app account via AppleScript (no MTA available on this Mac, unlike the tape-backup host) — see "Alerting" on the backup.sh & DVD Archiving page. Fires on unzip/zip/rsync/DVD/ iCloud/forward-to-batcave.sh-trigger failures, plus advisory alerts from the integrity checks below. (There's no ssh-agent failure mode left to alert on as of 8 August — see "SSH authentication" on that page.)
  2. Integrity check before fan-out. Done — and more thoroughly than originally scoped. Three checks now run against the raw unzipped world before it fans out: a size/region-file-count trend, an rsync dry-run delete-count check (per destination host), and a level.dat Data.Time check (the most precise of the three — a documented-always-increasing NBT tick counter). See "World integrity checks" on the backup.sh & DVD Archiving page.
  3. Re-evaluate the DVD burn cadence. Done. DVD burns moved from nightly to weekly, and chunk pruning (via mcaselector) now only happens for the DVD's own clone rather than for every destination — gmktec, seneca, iCloud, and batcave all get the unpruned world now. See "DVD burn cadence & DVD-only pruning" on the backup.sh & DVD Archiving page.
  4. One daily pipeline-health digest. Confirming a clean night now means checking logs across four machines (Mac mini, gmktec, seneca, and — as of 8 August — batcave, which now runs its own snapshot/prune script and log) rather than two, since the tape-backup branch moved hosts and batcave stopped being a passive SMB target. Built 8 August 2026 — see the Daily Pipeline-Health Digest page — but not yet confirmed stable across multiple nights in production, so this item stays open until then.

Two related additions came out of the same review pass, not on the original gap list: a GFS-style rotation policy for iCloud's dated folders (batcave moved off dated-folder rotation entirely on 8 August, onto native ZFS snapshot pruning instead — see backup.sh & DVD Archiving), and a weekly lightweight restore-verification check (the "0" in 3-2-1-1-0) against the latest iCloud copy. Both are covered on the backup.sh & DVD Archiving page.