DISTRIBUTED DATA
Distribute reads, writes, and data with explicit architecture choices.
Halo documents separate mechanisms for read scaling, write forwarding, distributed shared storage, and sharded data placement. Each addresses a different constraint; none is presented as a universal distributed-database mode.
Four capabilities, four different jobs
| Capability | Primary role | Key boundary |
|---|---|---|
| Halo DLB | Read/write separation and read-pool expansion | An ordinary DLB read-only node rejects writes |
| TWR | Database-aware write redirection from a read-only connection | Requires an established primary/standby topology |
| HSM | Distributed shared storage managed outside the database process | Reference topology and optional RDMA require deployment validation |
| HDS | Partitioned data placement behind one logical database | The manual does not define every production concern around worker HA, resharding, or cross-shard transactions |
Halo DLB scales reads inside the database path
DLB is implemented in the database kernel rather than as a separate SQL proxy. A read/write node accepts changes, streaming replication supplies read-only replicas, and the read pool distributes query work.
DLB endpoint
The application reaches Halo through the configured database path.
Read/write node
Changes execute here and feed streaming replication.
Read-only replicas
New RO nodes can enlarge the read pool and join the documented design automatically.
dlb_fdw
Provides the node connection channel; scheduling remains in the kernel.
Ordinary DLB read-only nodes do not forward writes. TWR provides that separate behavior. Oracle-mode DLB depends on aux_oracle; the native mode does not.
TWR keeps reads local and redirects writes
TWR is embedded in the Halo kernel. It identifies operations that require a write, forwards them from an RO node to the RW node, and returns the result.
Application reaches RO
The connection object can remain the read-only endpoint.
Classify the work
Statement · transaction · database object semantics.
Execute locally
Read work stays on the RO node.
Forward to RW
The result returns through the original connection.
TWR documents statement-level and transaction-block routing and can sit behind SLB, LVS, or F5. These are integration options, not latency guarantees.
TWR and an external proxy are different architectures
| External proxy pattern | Halo TWR |
|---|---|
| Routing happens outside the database | Routing is embedded in the Halo kernel |
| Decisions use proxy-visible traffic | Halo can use statement, object, and transaction state |
| Applications or proxy rules maintain read/write separation | A read-only connection can forward detected writes |
| Proxy is a separate operating component | TWR participates in the database topology |
HSM separates database access from shared-storage management
HSM/hsmd documents a distributed storage architecture with metadata masters, data nodes, client SDKs and libraries, and etcd coordination.
Database instance
Accesses storage through the HSM client layer and SDK.
Two hsmmaster nodes
The documented 1.0.16 deployment uses two master nodes with etcd coordination.
Three data nodes
The documented deployment assigns three mounted disks to each data node, with disk-group and filesystem operations.
Rebalance and task visibility
The management client covers node lifecycle, rebalance, tasks, and detailed storage state.
For Halo 1.0.16, the manual states a deployment prerequisite of two hsmmaster nodes and three data nodes with three mounted disks per data node; optional RDMA remains deployment-specific. Confirm later-build requirements rather than inferring active-active/RAC behavior, encryption, or a storage-durability policy.
HDS places partitioned data across nodes
Applications connect through a worker that presents one logical database. Partitioned foreign tables and shard_fdw map ranges to independent Halo data nodes.
Standard JDBC
One logical database connection.
SQL and partition routing
A partitioned parent table represents the logical data set.
shard_fdw
Predicate pushdown and partition pruning reduce irrelevant remote work.
Independent Halo instances
Each node stores the mapped subset through matching base tables.
The manual demonstrates date/range placement and automatic routing. It does not document a universal worker-HA design, online resharding guarantee, or cross-shard transaction contract.
Confirm the production boundary
- Read freshness and replication lag for DLB
- Statement, transaction, and object behavior through TWR
- Load-balancer and failure-routing behavior
- HSM node count, storage protection, network path, and RDMA availability
- HDS worker availability and recovery
- Shard-key and partitioning design
- Expansion or resharding procedure
- Cross-shard transaction requirements
Do not substitute linear-scaling, millisecond-routing, zero-change, or single-point-elimination claims for an architecture review.