XferDB v0.5.0 — Preflight details, support bundles, and showing our work on compatibility testing
DevOpsSeptember 10, 20267 min read

XferDB v0.5.0 — Preflight details, support bundles, and showing our work on compatibility testing

XferDB v0.5.0 — Preflight details, support bundles, and showing our work on compatibility testing

I just shipped v0.5.0 of XferDB. It's a smaller release than v0.4.0 — no new adapter, no new migration path — but it's the one that makes running XferDB in anger easier: more information before you migrate, a way to package diagnostics when something goes wrong, and a couple of MySQL bugs closed out.

I also want to use this post to be more specific about what "tested" means on the README's migration-path table, because "27/27 passing" is a number that's easy to over-read.

Download XferDB v0.5.0

PlatformBinary
Linux x86_64xferdb-linux-amd64
macOS Apple Siliconxferdb-darwin-arm64
macOS Intelxferdb-darwin-amd64

If you'd rather build from source:

git clone https://github.com/bchan77/XferDB.git
cd XferDB && git checkout v0.5.0 && go build -o xferdb ./cmd/xferdb

Release notes and CHANGELOG: XferDB v0.5.0 on GitHub.

What's in v0.5.0

Database info in preflight

xferdb project preflight now shows you what you're actually connecting to before any data moves: database type and version, host, database name, table count, size, and SSL status — for both source and target, side by side. Small thing, but it's caught more than one "oh, that's pointed at the wrong host" for me already.

Support bundles

xferdb support-bundle <project>

Packages a tar.gz with everything you'd want to hand to someone debugging a failed migration: system info, project config, source/target metadata, preflight results, migration and checkpoint history, both schemas, and the MongoDB schema plan if there is one. Credentials and DSN passwords are redacted automatically before anything gets written to disk. I built this because I was tired of asking "can you paste me your config" and getting back a screenshot with a password visible in it.

Bug fixes

  • MySQL adapter no longer chokes on URL-format DSNs (mysql://user:pass@host:port/db) — it rebuilds them into the native go-sql-driver/mysql format instead of handing the URL to sql.Open() directly, which just failed.
  • MySQL → PostgreSQL migrations now map MySQL types to their Postgres equivalents (datetimetimestamp, tinyint(1)boolean, jsonjsonb, and a few others) instead of dying on pq: type "datetime" does not exist partway through a table.

Showing our work on "27/27 passing"

The CHANGELOG says the compatibility matrix is at 27/27 passing — PostgreSQL 14 through 17 in every direction, MongoDB 6.0/7.0/8.0/8.2/8.3 into PostgreSQL 14–17, and MySQL 8.4 into PostgreSQL 17. That's true, and it's now backed by an actual pipeline rather than a note I made after a few manual runs: every one of those 27 pairs runs in our internal Jenkins CI, each pair gets its own Kubernetes namespace and a fresh pair of databases, and the migration is verified value-for-value on a control table, not just by comparing row counts.

I want to be specific about what that does and doesn't prove, though, because it's easy to read "tested" as "tested on data like yours."

  • The schema is intentionally generic. Ten tables — compat_users, compat_orders, compat_products, compat_reviews, compat_inventory, compat_transactions, compat_events, compat_sessions, compat_metrics, and a 3-row compat_control table used purely to catch a migration that moves the right row count with the wrong contents. None of it comes from a real production schema — it's synthetic data built to exercise XferDB's mechanics (batching, checkpointing, type mapping, upserts), not to represent what your actual tables look like.
  • The scale is real, though. Each of the 27 cases migrates roughly 1.5 million rows across those 10 tables — the big ones (compat_events, compat_transactions) are scaled up specifically to stress batching and checkpoint recovery, not just prove a SELECT works on five rows.
  • It runs on every release, seeded fresh each time, cases run in parallel batches, and a failure in one pair doesn't hide failures in the others — the whole matrix reports together.

So "tested" here means: the migration mechanics have been exercised, value-for-value, across every one of those 27 engine/version pairs, at real scale, automatically, on every release. It does not mean XferDB has seen a schema shaped like yours. If your schema has deeply nested arrays, unusual constraint patterns, or exotic column types, that's still new territory — please file an issue if you hit something, that's exactly how the matrix grows.

The Jenkins orchestration is internal infrastructure for now, so I can't hand you a link to click through, but the matrix definition itself is what determines what "tested" means for this project, and I'd rather describe it honestly here than let the changelog line speak for itself.

Thanks

To everyone still filing issues with real xferdb migrate --history output — that's what keeps turning "it worked on my machine" into an actual regression test. If you hit something on the MySQL path now that the DSN and type-mapping fixes are in, I'd like to hear about it either way.

— Billy

XferDB v0.5.0 on GitHub

← Back to Home