A swarm of AIs that grows when you add hardware.
Control plane in Rust and Tokio, data plane in Python and PyTorch, contract in Protobuf. Local models hand over what they learn without handing over the data.
The state of the repository
HECATE is not a pipeline. A pipeline is a single model running in a chain: when one node dies the service stops, when you add hardware nothing happens, and when a laptop learns something it stays on the laptop.
HECATE is a swarm. Several complete copies of the model run side by side, the topology grows while it runs, and every node can hand over what it learned as an adapter. Three pillars carry that.
Pillar A
With enough nodes the scheduler instantiates several concurrent pipelines out of replicated fragments. The redundancy map stops being a safety net and becomes the deployment map: the same layer lives on several nodes and feeds several pipelines at once.
Up to 100 nodes deploy 10 complete copies of the model: ten replicas per layer, ten times the service capacity, and the loss of any node has a replacement.
Pillar B
A new node does not restart the cluster. The scheduler runs an expansion protocol and decides whether that node becomes a replica of a fragment, extends the model with new layers, or joins as an expert bringing its own model.
The extension score follows the FSENNL frame: it combines the device profile (VRAM, RAM, NPU, latency) to decide where to expand.
Pillar C
Every node can run a local model on its own. Telomerase lets it hand a neural payload to the swarm: a LoRA adapter trained on its local data, which the swarm merges by federated averaging and returns applied.
The node keeps its adapter and stays autonomous. The swarm takes in a merged copy. The local data never leaves the node.
Two planes and one contract. The split is explicit: if something needs to know who runs what, it lives in Rust. If something needs to know how large a tensor is, it lives in Python.
| Module | Plane | Responsibility |
|---|---|---|
| Control plane · Rust + Tokio | ||
mesh-proto |
Rust | Contract and activations. Generates the stubs from the .proto and validates activations
before sending them. |
mesh-scheduler |
Rust | Assignment, capacity and growth. Node profile, plan_swarm with N pipelines,
growth_decision and apply_growth. Pure and deterministic.
|
mesh-router |
Rust | Cost and failover by replica. route_request per model, failover_in_swarm
and the degraded route. |
mesh-discovery |
Rust | Registry and mDNS discovery. Announces capabilities and telomerase, and keeps the node registry. |
mesh-worker-net |
Rust | Transport and peer pool. Message framing, connection pool and gRPC clients. Never runs a fragment. |
mesh-orchestrator |
Rust | Lifecycle and telomerase. Swarm state, telemetry, growth application and the adapter merge round. |
mesh-profiler |
Rust | Hardware profiling. Detects the hardware, benchmarks it and returns a node capability profile with a tier. Nothing is assumed: a field that cannot be measured stays at zero, marked absent, and a declared figure is marked declared. |
hecate-node |
Rust | Command line: serve, orchestrate, plan, swarm,
swarm-sim, swarm-live, fine, moe,
profile, zoo, guard, probe and
bench.
|
| Data plane · Python + PyTorch | ||
ncm_tokenizer |
Python | Residual vector quantizer: turns frames into discrete codes and rebuilds them by adding centroids. |
ncm_transformer |
Python | The model fragment in PyTorch: RMS norm, causal attention with SDPA and SwiGLU activations over a frozen substrate. |
telomerase |
Python | LoRA adapter, federated merge weighted by sample count, and the gRPC service for extract, merge and apply. |
dataplane |
Python | gRPC server of the data plane: describe, health, forward and profile. |
| Contract | ||
proto/hecate.proto |
Contract | Single source of truth. The Rust and Python stubs are generated from it, and verification fails if they drift. |
| f32 little-endian | Format | 32-bit floats in little-endian order, no header and no padding: the same bytes Rust writes and the same bytes Python writes. The adapter payload reuses that codec. |
A swarm of nodes and the pipelines that cover them. Add a node and the plan is recomputed; kill one and the route is re-spliced.
Idle: the swarm is stopped. Press “Start swarm”.
12 layers in the model chainAn illustration of the routing model, not a benchmark: latency and throughput are
measured with hecate-node bench and published in the methodology notes.
Step 1
old-tablet · 0 GB VRAM · 60 ms
The node cannot hold a new segment, but it can hold a replica. The redundancy map gains a holder for the first layer and the risk of a failed failover drops.
Step 2
beast · 20 GB VRAM · 14 TOPS
There is capacity to make the model longer. New layers are added at the end of the chain, on a frozen substrate, and the plan is validated before anything is deployed.
Step 3
seer · 16 GB VRAM · telomerase and a local model
The node brings its own model rather than more of the same one. It joins as an expert on a second chain, and the router keeps the two models apart.
The literal commands from the repository. They need Rust 1.88 or later and Python 3.10 or later, with the data plane dependencies installed.
cargo build --release -p hecate_node./target/release/hecate-node swarm --topology examples/swarm.json
./target/release/hecate-node swarm-sim --topology examples/swarm.json --kill gpu --pipelines 2
./target/release/hecate-node swarm-live --topology examples/swarm-live.json --merge
./target/release/hecate-node bench --mockThe simulation returns every step and the state of the swarm after it. The live mode discovers profiles, integrates the new node and runs one telomerase round: it collects from every node, merges and hands the result back.
python -m hecate.dataplane \
--node-id ncm-a --listen 127.0.0.1:50251 \
--layers 0-5 --model ncm-base \
--telomerase --adapter-rank 4 \
--vram-gb 6 --ram-gb 32 --npu-tops 2 --latency-ms 2 \
--local-model ncm-baseThe node announces its profile and exposes extract, merge and apply on the same port as the inference.
The repository does not use cloud CI: there is no .github/workflows/, no
CircleCI, no GitLab CI and no Travis. Quality is guaranteed on the machine of whoever writes the code,
with two entry points that live in the repository itself.
just checkLint is cargo fmt --all -- --check,
cargo clippy --workspace --all-targets -- -D warnings, ruff check,
ruff format --check and mypy. Test is cargo test --workspace and
pytest -q.
./scripts/verify.shEvery section, in order, aborting at the first failure: ruff check,
ruff format --check, mypy, pytest, the Protobuf stub parity, the
quantization parity, cargo fmt --check, cargo clippy,
cargo test, cargo build --release, the release-only bench check, the
version alignment against Cargo.toml, the scaffolding policy, llms.txt and the
documentation check. It exits 0 when everything passes and 1 when anything fails. With
SKIP_RUST=1 the Rust toolchain is left out and 15 sections remain.
What the two entry points measure today:
cargo test --workspace runs the control plane suite: unit, integration and documentation
tests.
pytest -q runs the data plane suite, the telomerase and the model zoo.
pytest --cov fails below that floor, and it moves with
COVERAGE_MIN. The measured percentage is not printed here on purpose: it
changes with every test and cannot be re-derived in the same step as the other figures,
so it would be the first number to drift.
The COVERAGE_MIN, SKIP_RUST, PYTHON and CARGO variables
adjust the behaviour. The pre-push hook runs the verification and aborts the
git push if anything fails.
In a hybrid architecture, verifying only half of it would be a mirage: the network loop, the swarm scheduler and the growth protocol are in Rust, and the tensors are in Python. Verifying only Python would leave the most critical code without a network.
The documents live under docs/ in the repository. They are not reproduced here:
they are linked. These are the ones that answer a question you are likely to have.
architecture.mdwhat lives in Rust, what lives in Python, where the
frontier is and what each layer guaranteesscm.mdwhere each crate lives, what guarantee it gives and how one is
promoted into corehosting.mdthe organization as the canonical owner, and what happened
to the old accountrouting.mdcost-aware routing: the cost model, the shortest path and
failoverswarm-routing.mdselection by model, load spread between copies and
failover by redundancyredundancy.mdcapacity score, layer assignment and head
replicationgrowth-protocol.mdthe FSENNL score and the three growth
actionsncm.mdthe model, the residual vector quantizer and the transformer
fragmentstelomerase.mdLoRA, federated merging and the gRPC service
data-architecture/model-zoo.mdadapted variants and device-aware
selection, over the catalog both planes readverify.mdlocal verification, the two entry points, the environment
variables and troubleshootingbenchmarks/methodology.mdwhat was measured, with which binary, how
many
times, and what was not measureddata-architecture/profiling.mdprofiling the nodes: measure, do not
assumedata-architecture/host-coexistence.mdcoexisting with the host: the
local model gives the node backperformance/hot-paths.mdhot paths, locks and spawn: the inventory of
what runs per requestWhat the design does not solve, without dressing it up.
Merging an adapter improves behaviour in the direction of the local data; it does not turn a small model into a large one. It takes many rounds and representative data.
Federated averaging does not guarantee that every node improves: two nodes with opposite distributions produce a midpoint that can be worse than both ends. Fixing it needs loss weighting or adapter clustering, and neither is implemented.
Every hop adds milliseconds; over WiFi the queue explodes. Ethernet is a requirement, not a recommendation.
A misreported video memory makes the scheduler place a segment that does not fit. A profile can be faked and the system does not verify it.