The problem
The source publishes results per event, per day, per region. There is no way to ask “how has this athlete done over three seasons?”. The platform answers that by polling every event, normalising names and clubs, and indexing everything by person.
It started on SQLite, which was the right call for a first version. As the data passed a million rows and traffic grew during the competition season, the limits showed: one writer at a time, slow aggregate queries, and no clean way to run the ingest and the website on separate machines.
What we did
Planned the move as a migration with a way back. We ported the data layer to PostgreSQL behind a thin wrapper, ran both databases side by side, and compared query results page by page before switching. At cutover, all 1,258,217 rows were copied and verified by checksum. The old database stayed untouched for two weeks as a rollback path, with the steps written down.
Fixed what the cutover surfaced. Two operational bugs appeared in the first hours: a cron wrapper that broke on an unquoted value, and a backup tool one major version behind the server. Both were fixed the same morning, and the first PostgreSQL backup was restored and checked before anyone called it done.
Made it fast where users feel it. The athlete comparison page took 13 seconds because it aggregated the whole results table on every request. Scoping the query to the athlete’s own competitions brought it to 0.2 seconds with byte-identical output. The home page leaderboards were moved out of user requests into a background job, taking a cold load from about 2.7 seconds to under 0.2.
What it shows
Ingest, data modelling, database migration, performance work and operations, done by the same people. The data side knew which numbers had to match. The platform side knew how to move them safely.