Documentation · Releases · Charon

Charon — Triton Panel + fleet/disk benching refinement

v.Chronos.Caduceus.0 — legacy v.H.1.19

Why “Charon”

Charon is the ferryman of the Greek underworld — the figure of orderly, gated passage. He carries each soul across the Styx one at a time, in sequence, and will not let anyone cross until the proper conditions are met. The codename is the benchmark/scoring subsystem’s permanent name (the Hipparchus → Charon rename landed under the versioning foundation), and this release captures the shape of the work: it brings order to a benching surface that had drifted into duplicated affordances. The benching panel regains its Triton Panelidentity; the whole-fleet rebench stops living in two stray windows and moves into the one tab it belongs in; disk entities are ferried across grouped by their host so they read in order; and nothing is re-benched on a host that won’t answer — the trigger is gated on genuine host liveness, the fare that must be paid before the crossing. The new Bench Progress Panel is the ledger of every crossing in flight: one row per bench, advancing phase by phase, gone the moment it lands.

Headline changes

Charon.10 — the single-source score program is complete

• Every score entity now has one canonical server-side computation. Charon.7, .8 and .9 gave the virtual prime, the real node and EarnScore each a single server-side resolver rendered verbatim everywhere. Charon.10 finishes the remaining four entities: virtual slaves, segregated chainweb children and disk entities each gain one canonical server-computed score and colour. Every score entity (real node, segregated child, virtual prime, virtual slave, disk) is now computed once and rendered identically on the Argus panel, the Triton panel, the per-server benchmark page and the /hub/nodes registry.
• The benchmark score card no longer runs its own competing colour system. The per-server ServerScoreCard previously had a second, parallel way of deciding red / orange / gold that could disagree with the rest of the hub. It now renders the server-computed colour strings verbatim for the headline, the five per-dimension tiles and the physical-data-drive banner — one colour system, no local re-derivation.
• No two code paths produce a score entity’s headline number or colour independently. Each score entity has exactly one server-side resolver; every endpoint ships its result and every UI surface renders it without recomputing — cross-surface divergence is impossible by construction.
• Charon.10a — a disk’s score propagates to its containers automatically. Re-benching a disk entity now updates the disk sub-score on every container that uses that drive — the virtual prime and the real chainweb container both live-recompute from the one canonical disk entity, with no separate container rebench. Disk-entity eligibility is one global predicate (a chainweb is pinned to the drive, or it is the host’s largest non-boot drive), so a disk’s score is the same value on every surface and the “no disk bench” notice reflects the real state.
• Charon.10c — segregated chainweb children no longer suspend themselves as duplicates of their own host. A segregated chainweb child legitimately runs on the same physical hardware as its parent host, so it shares the host’s fingerprint by design. The duplicate-host detector treated that as a conflict and permanently suspended the child from scoring — the chainweb-001 of every multi-chainweb host had been frozen out of warmup and earning entirely. The detector now recognises parent-child and sibling relationships as legitimate same-host partners and excludes them; the two stuck containers (StoaNodePrime-chainweb-001 and StoaNodeTwo-chainweb-001) are unstuck in the same patch.
• Charon.10b — the Disk Entities tab no longer lists phantom root drives. Charon.10a made the per-drive disk resolver global, which inadvertently flooded the fleet Disk Entities tab and the bulk disk rebench with one spurious /-root entry per chainweb node. The fleet enumeration now lists only the chainweb-pinned data drives; the per-node Triton Disk view and disk-score propagation stay global per-drive and are unchanged.

Charon.9 — single-source EarnScore

• One canonical EarnScore. The EarnScore math is extracted into a single dependency-free helper. Both the /hub/nodes registry and the /hub/earnings page call that one helper, so the same entity shows a byte-identical EarnScore on both pages — the two surfaces can no longer disagree. Previously each page defined “EarnScore” differently.
• EarnScore is the take-home capacity score everywhere. EarnScore is the gold-only aggregate ServerScore after the Tunnelee-fee carve-out plus any Tunneler-fee income — a red entity has EarnScore 0. The /hub/earnings page previously derived its “EarnScore” from the live accrual rate; it now shows this one capacity number, identical to /hub/nodes.
• The earnings page’s live accrual rate is a separate, clearly-labelled figure. The per-second / per-hour / per-day accrual rate, the accrual-rate medallion, and the per-node rate line are a rate — labelled as a rate, never as “EarnScore”. The red / orange / gold earning-state (Not earning / Vigilance / Stoicism) stays — that is the earning state, separate from the EarnScore number.

Charon.8 — single-source real-node display

• One canonical computation of a real node’s display. A real full-host node’s score and colour are now computed exactly once, server-side, by a single canonical function. Every surface — the /hub/nodes registry, the Argus fleet panel, and the Triton per-node panel — renders that one result verbatim. Previously each surface re-computed the score and colour on its own.
• A real container with an incomplete disk benchmark is now orange everywhere. The /hub/nodes registry used to colour a real container with a binary gold-or-red rule that could never produce orange, so a container whose data drive had an incomplete benchmark showed gold there while showing orange on the Triton and Argus panels. /hub/nodes now ships the same three-state colour (gold / orange / red) the other surfaces use, so the same container reads the same colour on every page.
• The three duplicate live-score wrappers collapse to one. A real node’s live score was recomputed by three separate copies of the same breakdown-parsing code. They are now a single shared helper, so the three surfaces can no longer drift apart.

Charon.7 — single-source virtual-prime display

• One canonical computation of a virtual prime’s display. A virtual prime’s score and colour are now computed exactly once, server-side, by a single canonical function. Every surface — the Argus fleet panel, the Triton per-node panel, and the per-server benchmark page — renders that one result verbatim. Previously each surface re-computed the score and colour on its own, so the same virtual prime could read red on one page and golden on another.
• The Argus and Triton panels can no longer disagree. Both endpoints now feed the canonical resolver the same breakdown data and the same live target version, so they compute byte-identically. Previously one graded against a fixed version and a raw breakdown while the other used the live version and a recomputed breakdown.
• The per-server card stopped recomputing. The per-server benchmark page no longer recomposes a virtual prime’s score or colour locally from the drive picker — it renders the server-computed result. Picking a different drive now saves and refetches so the server recomputes, instead of deriving a divergent local preview.

Charon.6 — per-server virtual-prime redness consistency + always-one-eligible-disk

• Per-server virtual-prime redness now matches Argus. The per-server benchmark page now colours a virtual prime’s score red consistently with the Argus Triton panel. The “cannot commit the 10 GB a chainweb container needs” signal is now carried through to the per-server card’s score-colour compute; previously the per-server card omitted it, so the same prime read golden there while Argus correctly read red.
• A server always has at least one eligible disk. Every server now resolves to at least one eligible disk — the largest available, even if it is too small for a chainweb container. On a host whose drives were never SSH-discovered (a Tunneler VPS onboarded without the chainweb install wizard), the disk is synthesized from the host probe data. A synthesized disk has no benchmark of its own, so the host still renders its virtual prime red — it has a disk, but cannot run a chainweb container until that disk is actually benched.
• The “↻ Force-fresh re-bench” button now actually forces a rebench. The force-fresh re-bench control now sends the force flag, so it re-benches regardless of gold-reuse.
• The Mount Capacity card is readable. The low-contrast text on the Mount Capacity card (mount totals, used / free, the system-partitions header) is now a readable contrast on the dark card.
• Charon.6a — a host’s largest drive is always a benchable disk entity. A host’s largest non-boot drive now always shows as an eligible disk — on the Mount Capacity card and as a benchable row in the Triton Disk Entities tab — even on a small Tunneler VPS whose only drive is below the 50 GB data-drive threshold. The drive being too small to host a chainweb container is reflected in its commitment score, not by hiding the disk.

Charon.5 — virtual-prime commitment-redness + Argus Status-icon column

• A 0-commitment virtual prime renders RED. A virtual prime that cannot commit the 10 GB minimum a chainweb container needs — no usable / eligible disk at all, or an eligible disk too full to commit ≥ 10 GB — now shows a red score. Previously the virtual-prime commitment sub-score was hard-coded golden, so a 0 commitment never turned the composite red; the “any red sub-score ⇒ red score” rule now covers the commitment axis too.
• Argus Status column is an icon. The Triton-panel Status cell now renders one glyph — a gold ✓ when the score is current, a red ✗when it is missing or not current — plus a ↻ rebench icon. The crowded status text and the separate disk force-rebench button are retired.
• Every Status-cell rebench is a forced rebench. Clicking the ↻ icon benches the entity anew regardless of gold-reuse, for all three entity kinds (disk, virtual, real). The ↻ icon is disabled (greyed) when the owning host is unreachable.
• Charon.5a — the Virtual Prime commitment mirrors the real prime. On a host that runs a real prime chainweb container, the Virtual Prime now shows the same commitment sub-score as that real prime — instead of a flat placeholder that read identically for every server. A host with no real prime keeps the forward-looking commitment projection.

Charon.4 — Argus Triton-Panel bulk-benching refinement

• Bench-all skips unreachable hosts. The bulk bench-all planner now checks each entity’s owning host against the cached SSH-reachability flag. An SSH-unreachable (red-bullet) host’s entities are reported as skipped — reason host unreachable — SSH not established— instead of being enqueued to fail. This applies to all three waves (disk, virtual, real).
• Real score instead of “missing score”. When a virtual prime’s projected score cannot be computed, the Triton panel and the fleet endpoint now fall back to the persisted last score — so the operator sees the real (red) number and knows to rebench, instead of an unhelpful “missing score”.
• No “✓ current” on a broken score. The entity Status cell is now driven by the score’s composite colour, not just the version stamp: an entity with a missing or red composite score shows ⛔ rebench (actionable), never “✓ current”.
• Score column live-updates after a bench. The Triton panel’s active-benches tracker now also follows virtual benches, so the Virtual tab’s Score column refetches automatically the moment a bench-all finishes — on every subtab.
• Bench-all uses a lightweight confirm. The Bench-all action no longer opens a password modal; the inline confirm dialog is the only confirmation step.

Charon.3 — Stoicism / earnings-page rehaul on /hub/earnings/

• Phantom-pending fix. The earnings page used to accrue a “pending stoicism” figure for servers that were earning nothing — a red or version-stale entity still ticked up a Pending balance that would never materialise. The earnings reader now applies the same red gate the scoring worker uses: a red or version-stale entity earns 0 and never shows a phantom pending balance.
• Red/orange/gold earning-state model. Every entity is classified into one of three states — red Not earning (gated out, earns nothing), orange Vigilance (eligible but still warming up / not yet fully synced), and gold Stoicism (fully eligible and earning). The colour says at a glance whether a server is actually contributing.
• EarnScore is the headline metric. The page leads with EarnScore (not ServerScore) — the figure that reflects real earnings — with a per-container breakdown beneath it so an operator can see which container earns and which does not.
• Entities reordered to Personal Server Pool order. The earnings entities now render in the same order as the /hub/nodes Personal Server Pool table, so the two pages read consistently top-to-bottom.
• De-noised rate display. The earning rate shows one /day figure by default; the /hr and /sec figures move to the hover tooltip, cutting the per-row numeric clutter.

Charon.2 — stoicism-score display rework on /hub/nodes/

• New CONTAINERSCORE column between Role and ServerScore. Every chainweb container’s individual score appears on its own line in its own composite colour — gold for fully eligible benches, orange for eligible-but-incomplete, red for stale/mismatched. Operators can finally see which container is dragging the eligible aggregate down.
• ServerScore is the sum of non-red (gold + orange) containers, shown gold. A red container is excluded from the aggregate but still listed on the CONTAINERSCORE column with its red score. Pre-Charon.2 the entire aggregate read red whenever any contributor was red, masking the eligible total.
• EarnScore = sum of GOLD-only containers (orange contributes to ServerScore but not to actual earnings), plus the tunneler-bonus on a tunneler whose own score is golden. A tunnelee’s EarnScore is slightly smaller than its ContainerScore because of the fee paid to its tunneler. Renders in yellow.
• Sync + flags gate still applies. The seven-gate Stoicism eligibility engine (benchmark stamped, ouronet set, commitment ≥ fleet-min, warmup complete, cut within tip-tolerance, peer count, not in breach) gates whether EarnScore actually accrues Stoicism. A non-synced container reads 0 EarnScore even when its container colour is gold.
• Charon.2l — clock-skew-immune Stoicism projection. Operator-observed after Charon.2k didn’t fully fix the skip-ahead. Root cause: the projection compared the CLIENT clock to the SERVER’s tick-anchor timestamp directly. If the client clock is even modestly behind the server (NTP drift, OS offset, a docker container with skewed time), the difference goes negative, the Math.max(0, ...) clamp drops it to 0, and the projection collapses to persistedonly — visible as exact 60-second chunked jumps. Charon.2l decomposes elapsed time into two pieces that never cross-compare clocks: server-side (snap.baseAt − tickAnchorAt) (both server timestamps) plus client-side (now − snapArrivalMs) (pure client delta). Refresh-safe.
• Charon.2k — the Stoicism page’s live counters no longer skip ahead at 60-second intervals. The data was correct — the worker banks pending in exact 60-second chunks and the projection formula displayed = persisted + (now - anchor) × rate is mathematically continuous across tick boundaries. The visible skips meant the projection ticker wasn’t smoothing: requestAnimationFrameis throttled or paused by modern browsers in many scenarios (background tabs, unfocused windows, low-power modes, CPU pressure, and various heuristics even in focused tabs). When rAF paused, the projection’s nowMs froze; on resume it snapped to current time and the display jumped to catch up. The ticker now uses setInterval(100ms)instead — spec-guaranteed to keep firing (throttled to 1 Hz min even in backgrounded tabs but never fully paused), with lower CPU than the prior 60 Hz rAF. Math is unchanged.
• Charon.2j — per-host SSH channel semaphore: no concurrent operation can ever overwhelm sshd MaxSessions. Charon.2i’s concurrency cap lived only in the scoring worker. Every other SSH consumer (trusted-tip poller, bench jobs, fleet rebench, force-fresh, autonomic commit, ad-hoc admin probes) was still independently scheduling channels. Any operator action that fired SSH-heavy work concurrently with the scoring tick could re-create the silent-freeze pattern. Charon.2j weaves a per-host channel semaphore into runRemoteitself — every SSH op in the hub routes through it, so this is the single authoritative chokepoint. Cap = 5 channels per host (5 channels of headroom under default sshd MaxSessions=10 for operator sessions / monitoring / external consumers). Excess callers queue FIFO; nothing fails silently because no channel is ever rejected by sshd. The 32-container-per-host cap still holds (~4 s probe phase).
• Charon.2i — chainweb-005 unfrozen: returned-error SSH probes are now logged and per-host concurrency safer. A targeted follow-up to Charon.2h after operator-instrumented probe traces showed StoaNodePrime-chainweb-005 still frozen post-deploy. The probe helper returns a status object with error set (and cutHeight: null) on SSH-channel-open failure rather than throwing — so Charon.2h’s thrown-only warning never fired and the silent-skip behaviour persisted. The worker now also logs the returned-error case for visibility. Per-host probe concurrency lowered from 6 to 4 (more headroom under MaxSessions=10) — the prod capture showed concurrent SSH consumers (trusted-tip poller, audit-pulse, autonomic-commit) eating Charon.2h’s 4-channel headroom. Concurrency 4 leaves 6 channels free and still accommodates the 32-container-per-host cap.
• Charon.2h — the scoring worker no longer exhausts SSH MaxSessions on hosts running many chainweb containers, and a 32-per-host hard cap is enforced. The hub probes every node’s chainweb cut height over SSH each ~60 seconds. The pre-Charon.2h code fired ALL probes in one parallel batch — for a host running 7+ chainweb children, all those SSH channels opened at once on the pooled connection, default sshd MaxSessions=10only left ~4 channels of headroom for concurrent users, and the daily 04:00 UTC audit archive’s extra load pushed channel demand past the cap. Surplus channels were silently rejected, the worker had no live status for the losing nodes, and those nodes stuck “Not earning” permanently because the same nodes kept losing the race every cycle. Charon.2h batches probes BY SSH target host with at most 6 concurrent per host (sshd-headroom-safe and clean up to the new 32-container-per- host cap — ~3 s probe phase); across hosts parallel as before. Worker-side probe failures now log via console.warnso the next freeze of this class is visible in the worker log instead of needing prod-DB forensics. A hard cap of 32 segregated children per host is enforced at the create-self-container API — the 33rd install attempt is refused with a clear 409 (even 32 requires a massive server: 64-core CPU, 200 GB RAM, 16 NVMes; beyond that the host’s resource pressure becomes structural).
• Charon.2g — per-hour rate inline, and the Tunneler relay row stops claiming it’s waiting to reach tip. Two small refinements. The per-row “Stoicism rate” line now shows BOTH /day and /hrinline (native browser title tooltips were flaky / often resolved to nothing, so the previous “hover for /hr” prompt wasn’t actually delivering); /secstays on the row’s title attribute for high-precision detail. StoaTunnelZero (a container-less Tunneler relay) also stops displaying “Waiting to reach tip… to start warmup” — a relay with no chainweb has nothing to reach the tip with; the row now reads “Tunneler relay — no own chainweb; waiting for Tunnelees to earn fees”.
• Charon.2f — the /hub/earnings page now matches /hub/nodes. The /hub/earnings half of the rework. The headline EarnScore for a multi-chainweb host or a Tunneler relay was inflated — the page added the host’s leftover pre-delegation score on top of its children, so a host could read 33.4 where /hub/nodes correctly read 23.8. It now mirrors /hub/nodes exactly: a host with no chainweb of its own contributes 0 to its own EarnScore. The Pending tile also stopped oscillating — nodes still waiting to reach tip (which accrue nothing) were given a phantom pending rate the scoring worker never banks; the rate is now gated on warmup having actually started. The three ledger buckets are renamed to the Stoicism vocabulary: Redeemed → Minted Stoicism, Current → Stoicism, Pending → Vigilance.
• Charon.2e — a container-less server’s ServerScore reads 0. A server that runs no chainweb container (a Tunneler relay) now shows a ServerScore of 0 — a genuine “no containers, no score” — instead of a stray hardware-only number leaking into that column.

Charon.1 — Triton Panel + fleet/disk benching refinement

• The four-tab benching panel is the Triton Panel again. The admin benching page’s header is retitled to a visible Triton Panel heading over its four tabs — Fleet Distribution, Real Container Entities, Virtual Container Entities, and Disk Entities. The panel’s identity is restored without changing what each tab does.
• The whole-fleet rebench is folded into the Fleet Distribution tab. The two standalone rebench windows that used to sit outside the tabs (a “Trigger fleet rebench” entry and a force-fresh fleet rebench wrapper) are removed; the whole-fleet rebench now lives inside the Fleet Distribution tab as its own Bench-all, exactly the way each per-kind tab carries its own. It preserves the full Disk → Virtual →  Real reconcile and its force-fresh toggle — no capability is dropped, only relocated. Exactly three sub-windows remain: Promotion, the Triton Panel, and the new Bench Progress Panel.
• Disk Entities are grouped by their host server. The Disk Entities tab sorts disks server-grouped: server groups ordered A→Z by the shared host-identity resolver’s server label, and within each group the disks alphabetically. All of one server’s disks read together instead of being interleaved across the fleet.
• Every entity row shows whether its host is reachable, and that gates rebench. A new leading bullet on every entity row across every entity tab is greenwhen the entity’s host is genuinely reachable and red when it is not. This is true host liveness (the cached reachability signal), nota benchmark-version staleness indicator: a never-benched but reachable entity is green and keeps its rebench trigger enabled — it is precisely the entity that most needs a first bench. An unreachable host’s rebench trigger is disabled, with an explanatory tooltip, until the host answers again.
• A new always-present Bench Progress Panel. Mounted directly after the Triton Panel, it mirrors the panel with three tabs — Real Container Entities Progress, Virtual Container Entities Progress, and Disk Entities Progress. Each benched entity gets one persistent row with a progress bar and a Phase N/M — step label; a queued bench shows queued. The panel handles any number of parallel benches at once, keeps a row visible while that entity benches, and drops the row the moment its bench finishes. It is empty — not hidden — when nothing is in flight.
• Triggering a rebench surfaces the entity in the panel immediately. The Status-column rebench trigger is click-to-rebench; enqueuing it makes that entity appear in the Bench Progress Panel right away and walk queued → running → phases → gone, driven by a live server-sent stream that reconnects transparently and tidies up on unmount.
• Charon.1y — the Drive Breakdown shows real free space. The virtual provisioning Drive Breakdown panel reads the drive’s measured free space from the host probe, instead of deriving it from the autonomic-pool figures — which collapsed to “Free 0 GB” for an entity with no committed allocation.
• Charon.1z — an in-flight bench survives navigation. Leaving the per-node benchmark page and returning while a bench is still running now re-shows the “Benching…” state and the live progress, instead of an idle card — the in-flight state is re-hydrated from the shared bench-progress stream on mount.
• Charon.1aa — a chainweb child that never slice-benched now always runs a real bench. A segregated chainweb container whose install-time slice bench failed was stuck permanently red and labelled “(pre-versioning)” because the version-gold skip gate kept skipping every rebench without checking that a prior score actually existed. The skip gate now applies only once a real slice benchmark has been recorded, so a never-slice-benched child always runs a genuine bench.

For operators

Everything in this release is on the admin-area benching surface. Nothing about how a benchmark is measured or how ServerScore is computed changes — the public ServerScore reference is unchanged. What changes is the admin workflow: the benching panel has its Triton Panel name back, the whole-fleet rebench is where you expect it (inside Fleet Distribution), disk entities read grouped by server, and you can see at a glance whether a host is reachable before you try to re-bench it.

Charon is a feature quickunder the benchmark/scoring subsystem codename — future benchmark-subsystem refinements are forever this codename at the next patch-number, never a new codename. The version stamp on this page is read live from @/lib/version (model-derived, never frozen into the page), so it always shows the running build’s coordinate.

Related

• §15 Benchmarking — the public reference for what a benchmark measures, the hardware classification model, and the ServerScore formula. That math is unchanged by this release; Charon refines only the admin-area benching workflow around it.
• The canonical release forest — the unified page rendering the entire Release→Era→Name→fix forest in one place, every shipped codename at its canonical coordinate with its re-draw / legacy-token provenance inline.
• Cadmus — the ChronVer versioning foundation, under which the benchmark subsystem’s codename was renamed to Charon (a rename, not a slug re-draw): from there forward this subsystem is forever Charon at the next patch-number.