The community site Lobsters completed its migration from MariaDB to SQLite over the weekend, finishing a path started in 2018 that initially aimed for PostgreSQL. The site now runs on a single VPS with four SQLite databases: primary content (3.8 GB), cache (1.1 GB), queue (218 MB), and Rack::Attack for rate limiting (555 MB). CPU and memory down, costs halved, and the site feels snappier to users.

This matters to you because it demonstrates a contrarian architectural pattern: you don’t need a traditional RDBMS to run a site serving an active community. SQLite with the right architecture handles production loads that many still associate with more complex systems. For those building with agents, it’s a concrete case of how much you can do with a single server and SQLite, as we covered on July 14 with DOOMQL.

The pattern applies: if your system has many concurrent writes to the same data, SQLite probably won’t cut it. But if you write to separate databases by function (as Lobsters does with content, cache, queue, and rate limiting), the operational simplicity pays off.

In detail

The context

Lobsters had planned its exit from MariaDB since 2018, initially toward PostgreSQL. In 2025 they instead decided to evaluate SQLite, and this weekend the migration went to production.

The architecture

The system now runs on a single VPS and uses four separate SQLite databases, each with its own responsibility:

• Primary content (3.8 GB): stories, comments, votes, users
• Cache (1.1 GB): acceleration layer
• Queue (218 MB): async jobs
• Rack::Attack (555 MB): rate limiting and abuse blocking

Separating by function avoids contention on concurrent writes, SQLite’s most known limitation. Each database can grow and vacuum independently.

The work behind it

The migration is documented in PR #1949 by Thomas Dziedzic: 30 commits, 188 files touched, +735/-593 lines. It builds on three earlier PRs (#1705, #1871, #1924) that laid the groundwork. Rails code was adapted, tests verified, and the deploy process rewritten.

What changed for users

Numbers reported by the team:

• CPU usage down
• Memory usage down
• Perceived latency better
• Operational costs halved (once the MariaDB VPS was shut down)

No formal benchmarks, but the effect is clear enough to make the migration permanent.

When SQLite isn’t enough

SQLite has a simple concurrency model: only one process can write to a database at a time. If your application has multiple processes writing to the same data simultaneously, it becomes a bottleneck. Lobsters avoids this by separating writes by database and using cache to reduce pressure on primary content.

If your load has frequent and concurrent writes to the same dataset, PostgreSQL or MariaDB remain better choices. But if you can partition by function like Lobsters, or if your writes are serializable, SQLite handles surprisingly heavy loads.

Implications for builders

This migration is a case study for those designing systems that use agents: an agent reading and writing to local SQLite can skip the complexity of a remote database, with lower latency and operational costs. If the architecture separates responsibilities (one database for sessions, one for results, one for logs), SQLite’s single-writer model isn’t a limit but a consistency guarantee.

For more on using SQLite in agentic architectures, see the multilingual static site scaffolding and the data analysis assistant, both built on SQLite.