The evolution of distributed SQLite from rqlite to managed cloud services

In September 2014, Philip O'Toole released rqlite, an open-source system that combined SQLite with the Raft consensus protocol to create a fault-tolerant clustered database. Twelve years later, O'Toole reflects on how this initial experiment sparked a broader category of distributed SQLite solutions, ranging from local-first CRDTs to venture-funded managed services.

What happened

O'Toole originally built rqlite to solve a specific infrastructure problem at jut.io: providing relational storage for a clustered system without designating a single node as the permanent data holder. By placing a Raft log in front of SQLite, he ensured that every write was replicated across a cluster, preventing data loss if a node failed. The source code was published on GitHub that same weekend, marking what O'Toole believes was the first open-source combination of Raft and SQLite. For several years, rqlite stood alone in this niche, with no other major projects attempting similar architecture.

The landscape began to shift in 2016 when Expensify released Bedrock, a Paxos-based SQLite replication system they had used internally for years. Around the same time, ActorDB appeared, offering Raft-replicated SQLite databases for individual actors in an Erlang environment. While neither gained widespread adoption outside their origin companies, they demonstrated that the concept of replicating SQLite was gaining traction. However, rqlite remained distinct as a standalone, general-purpose tool that any developer could deploy independently of a specific application framework.

By 2017, Canonical developed dqlite to provide an embedded replicated store for LXD and MicroK8s, avoiding the operational overhead of running etcd. Although O'Toole and Mark Shuttleworth discussed potential collaboration, the projects diverged in their packaging models. Dqlite became a C library embedded directly into applications, while rqlite remained a separate server process. This distinction defined their respective use cases: dqlite for database functionality inside products, and rqlite for database infrastructure beside them.

How it works

The core mechanism of rqlite relies on the Raft consensus algorithm to manage state across multiple nodes. In this CP (Consistency and Partition tolerance) system, all write operations must be directed to the leader node. The leader then appends the change to its log and replicates it to follower nodes. A write is only considered successful once a majority of the cluster acknowledges it. If a network partition isolates a minority of nodes, those nodes refuse writes to prevent data divergence, ensuring strong consistency at the cost of availability during partitions.

This design contrasts sharply with later approaches like Litestream, released by Ben Johnson in 2021. Litestream does not use consensus protocols. Instead, it monitors the SQLite Write-Ahead Log (WAL) and streams changes to object storage like S3. This provides disaster recovery for single-node deployments at minimal cost, addressing the common need for backup rather than real-time clustering. Rqlite answers the question of high availability where downtime is unacceptable, while Litestream addresses data durability for simpler architectures.

More recent innovations have explored different points in the design space. Cr-sqlite introduces Conflict-free Replicated Data Types (CRDTs) as a SQLite extension, allowing multiple writers to diverge and merge changes without a central leader. This AP (Availability and Partition tolerance) approach suits local-first applications where immediate consistency is less critical than offline capability. Meanwhile, projects like libSQL forked the SQLite engine itself to enable deeper integrations, powering services like Turso that offer edge-replicated databases as a managed product.

Key details

• Rqlite was released in September 2014, initially using the goraft library before switching to Hashicorp’s Raft implementation in early 2016.
• Canonical’s dqlite, created in 2017, is a C library embedded in applications, whereas rqlite operates as a standalone server.
• Litestream, launched in 2021, streams SQLite WAL files to S3 for disaster recovery without using consensus protocols.
• Cr-sqlite enables multi-writer support through CRDTs, catering to local-first applications that require eventual consistency.
• Turso, built on the libSQL fork, was acquired by Supabase in 2026, highlighting the commercial viability of managed distributed SQLite.
• Cloudflare’s D1 and Durable Objects represent the shift toward serverless, managed SQLite replication at the edge.

Why it matters

For software engineers and infrastructure leads, understanding the history of distributed SQLite clarifies the trade-offs between different architectural patterns. The emergence of three distinct corners in the design space—consensus-based clustering, WAL streaming, and CRDT merging—means that choosing a solution requires precise definition of requirements. Developers must decide whether they need strong consistency with high availability, cost-effective disaster recovery, or offline-first collaboration capabilities. Misapplying a CP system like rqlite to a use case that only needs backup, or vice versa, can lead to unnecessary complexity or data risk.

The trajectory from open-source tools to managed platforms also reflects a broader trend in infrastructure software. While rqlite, dqlite, and Litestream remain free and open-source, the significant venture capital and acquisitions have flowed toward services that abstract away operational complexity. Companies like Turso and Cloudflare sell replicated SQLite as a utility, allowing teams to bypass the maintenance burden of managing clusters. This shift validates the underlying technology but changes the economic model for developers who previously relied on self-hosted solutions.

What you can do

• Evaluate your consistency requirements: choose Raft-based systems like rqlite for strong consistency, or CRDTs like cr-sqlite for offline-first apps.
• Consider Litestream if your primary need is disaster recovery for a single-node SQLite database rather than real-time clustering.
• Use dqlite if you are building an application that requires an embedded, replicated database without external service dependencies.
• Explore managed services like Turso or Cloudflare D1 if you want to offload the operational overhead of managing distributed database clusters.
• Review the WAL streaming capabilities in rqlite v8.0, which were inspired by Litestream, to optimize backup and recovery processes.
• Assess whether your team has the expertise to maintain a self-hosted Raft cluster or if a managed platform offers better resource allocation.