Why SmartHoldem Replaced PostgreSQL with Sled in Our Rust Core
In the evolution of blockchain infrastructure, every millisecond and every megabyte counts.
For years, SmartHoldem relied on a proven, traditional stack: Node.js and PostgreSQL. It served us well during the Web 3.0 era. But as we transitioned to building the Web 4.0 Operating System (Netfory), we realized that a general-purpose relational database is a poor fit for the workload of a high-performance blockchain core.
To achieve true decentralization, lightning-fast sync times, and a "Zero-Infra" footprint, we completely rewrote our core in Rust (sth-core-pq) and made a bold architectural decision: we replaced PostgreSQL with Sled, an embedded, lock-free key-value store.
Here is the technical deep-dive into why this was the only logical choice.
The PostgreSQL Bottleneck in Blockchain
PostgreSQL is a phenomenal database for web applications. But a blockchain ledger is not a typical web app. It has a highly specific workload:
- Append-heavy & Immutable: Blockchains primarily write new data sequentially and rarely update old records.
- Massive State Reads: Nodes constantly read the latest state (balances, delegates, UTXOs) while simultaneously writing new blocks.
- External Dependency: Running PostgreSQL requires a separate background service, connection pooling, schema migrations, and routine maintenance (like VACUUM to prevent bloat).
In a Node.js + PostgreSQL architecture, the overhead of serializing/deserializing data across process boundaries, managing connection pools, and dealing with relational ORM abstractions creates a hard ceiling on throughput.
Why Sled is the Perfect Fit for sth-core-pq
Sled is a modern, embedded, persistent key-value database written in Rust. It was designed from the ground up for high-concurrency, low-latency workloads. Here is why it is the ideal backbone for SmartHoldem:
1. Embedded & Zero-Configuration
Sled runs in-process. There is no separate database server to install, configure, or maintain. When a user downloads the sth-core binary, the database is ready instantly. This is the essence of Zero-Infra: anyone can run a full validating node on a laptop or Raspberry Pi behind a home NAT router, without needing a degree in database administration.
2. Lock-Free Concurrency (Epoch-Based Reclamation)
Blockchain nodes must read state (e.g., serving API requests or verifying transactions) while simultaneously writing new blocks. Traditional databases use heavy mutexes or MVCC that can cause contention. Sled uses epoch-based reclamation, a lock-free concurrency model. This means readers never block writers, and writers never block readers, allowing our Rust core to process blocks in parallel (via Rayon) with near-zero overhead.
3. B-Tree Architecture for Blockchain State
At its core, a blockchain is a massive, append-only key-value store:
- block_height → block_data
- address → account_state
- app_id → SmartObject_metadata
Sled is built on a highly optimized, concurrent B-Tree. It handles sequential writes and point-lookups with exceptional efficiency, perfectly matching the access patterns of a DPoS consensus engine.
4. Radical Storage Efficiency (zstd Compression)
In the old architecture, a full node with 11.7+ million blocks and comprehensive indexes could easily swell to tens of gigabytes.
With Sled and built-in zstd compression, the entire SmartHoldem blockchain history now fits into a remarkably compact ~8 GB footprint. This dramatically lowers the barrier to entry for new node operators and reduces disk I/O bottlenecks.
The Real-World Impact
The switch to Rust + Sled is not just theoretical; the metrics speak for themselves:
Sync Time: Reduced from days to hours (or even minutes on fast hardware).
Block Application: A block of 500 transactions is now verified and applied locally in ~50 milliseconds.
Memory Footprint: Active RAM usage for a full node hovers around a minimal ~800 MB, freeing up resources for Web 4.0 P2P tasks (like Iroh QUIC routing and dVPN relaying).
Building for Web 4.0
Web 4.0 demands infrastructure that is sovereign, resilient, and accessible. Relying on heavy, centralized database services contradicts the very ethos of decentralization.
By embedding Sled directly into our Rust core, SmartHoldem has eliminated external dependencies, shattered throughput bottlenecks, and created a blockchain engine that is as lightweight as it is powerful.
We didn't just optimize our database. We optimized the future of decentralized infrastructure.
Explore the code: github.com/smartholdem/sth-core-pq
#SmartHoldem #Rust #Sled #PostgreSQL #BlockchainArchitecture #Web4 #Netfory #STH #DPoS #ZeroInfra #Decentralization #P2P #Iroh #Database #Crypto
