Signal

XCAP's core intuition is brutal and counterintuitive to anyone coming from SaaS: in a trading-research system, the network is not a convenience, it's the largest attack surface you have. Every keyed endpoint is a credential that can leak; every write call is a trade that can be triggered by a bug. That's why XCAP collapses its entire network exposure to a single point: a read-only, keyless, GET-only ingest edge. The rest of the system is deterministic from disk. Once you grasp this, you understand why `binance_public.py` only requests klines (historical OHLCV candles) over HTTP GET, with no API key, signing nothing, never touching an account or order endpoint.

Concretely, the pattern is: an abstract source (in XCAP, an `OHLCSource` hierarchy) with two implementations. `LocalCSVSource` reads CSVs already on disk — zero network, the default case, the one ALL automated loops use. And `HttpOHLCSource` (backed by `binance_public.py`), which only activates when a caller explicitly passes `allow_network=True` or invokes the `data-fetch` command. The asymmetry is deliberate: the safe path is the default and the networked path demands an explicit option on every call. Never a global ambient flag, never an environment variable that stays switched on.

The Binance endpoint you use is the public market one: `GET https://api.binance.com/api/v3/klines?symbol=BTCUSDT&interval=1d&limit=1000`. It returns an array of arrays: `[openTime, open, high, low, close, volume, closeTime, ...]`. Notice what is NOT here: no `X-MBX-APIKEY` header, no HMAC signature query param, no signed timestamp. That endpoint physically cannot read your balance or place an order, because Binance's account API lives on a different route and requires a signature. Security isn't your promise to behave well: it's a property of the endpoint you chose.

The second pillar is normalization at the edge. Real-world data arrives dirty: timestamps in milliseconds, numbers as strings, duplicate candles, gaps. Your edge must normalize to ONE canonical format before anything downstream touches it. In XCAP, the CSVs in `data/real` are lowercase (`open,high,low,close,volume`), timestamps parsed to dates, and `LocalCSVSource` reads them the same way every time. This discipline is what makes the system reproducible: the regime classifier (SG-02) and Market Memory (SG-03) read a stable format from disk, not a volatile HTTP response that changes between runs.

Verification is the closing move. XCAP has a `check` command (run with `PYTHONPATH=src python3 -m xcap.control check`) that asserts as an invariant that the connector is keyless, read-only, cannot trade, and cannot move money. It's not a test that passes once: it's an assertion that runs on every `check`, so that if someone adds a key or a write method, the `check` turns red. That's how you turn a design intention into a property the system defends on its own. The golden rule: security that depends on you remembering something is not security; security that a red test forces you to respect is.

An important operational nuance: the XCAP dashboard binds only to `127.0.0.1`, never `0.0.0.0`. The ingest edge is the only outbound surface; the inbound one (the dashboard) doesn't listen on the network. And FileVault (AES-256) encrypts the disk at rest. The full chain — encrypted disk, keyless read-only ingest, local-only dashboard, Capital Gate closed — is what lets you keep a trading-research system on a laptop without it being a ticking bomb.

The conceptual error XCAP corrected in its June 2026 audit is subtle and worth its weight in gold: confusing operational health with price regime. `market_state.py` measures system health (do I have fresh data? are the flags right?). `regime.py` measures something entirely different: what STATE is the market itself in? They are orthogonal axes, and mixing them produces a classifier that doesn't know what it's saying. The first discipline is: a regime classifier speaks only of price, never of the state of your infrastructure.

XCAP's design classifies along four independent axes. Trend (up, down, sideways? — via the slope of moving averages or cumulative return over a window). Volatility (calm or agitated? — via the standard deviation of returns, normalized). Behavior (persistent/trending or mean-reverting? — via autocorrelation of returns). And Stress (is a drawdown developing?). Each axis is computed purely: same input, same output, no hidden state, no randomness. Then a documented priority cascade derives a single headline `Regime` from the four axes.

XCAP's sharpest insight is in how it defines Panic, and it's where almost everyone gets it wrong. The temptation is to define panic as 'high volatility' (an elevated stdev ratio). That's incorrect. A monotonic crash — price falling in an almost straight line — has LOW internal volatility, because each candle resembles the previous one. If you define panic by stdev, you miss exactly the most dangerous crash. XCAP's correct definition: panic is a large drawdown developing FAST, measured by a sharply negative recent return, not by dispersion. This is what distinguishes a classifier designed by someone who looked at real crashes from one that copied a formula out of a book.

Purity isn't an aesthetic whim, it's the foundation of trust. `regime.py` is a pure function: `classify(bars) -> Regime`. It doesn't read from disk inside itself, doesn't call the network, doesn't mutate anything global, has no random branch. This means you can run it a thousand times over the same window and get the same label, you can test it with deterministic fixtures, and you can audit exactly why it said 'Panic' on a specific date. A classifier with hidden state is a classifier you can't trust to build Market Memory on top of.

The most important rule, and the one that connects to the whole constellation: the classifier is label-only. It NEVER sizes or trades. It returns a label and nothing more. The reason is one of honesty architecture: if the same module that diagnoses the regime also decides position size, you have an incentive for the diagnosis to justify the trade you already wanted to make. Separating diagnosis from action is what makes it possible, in SG-03, to honestly measure whether the regime has predictive value — because the label was generated without knowing what you'd do with it.

To implement it well, document the priority cascade as readable code, not as a tangle of ifs. For example: if Stress triggers Panic, Panic wins over everything else; if not, and Volatility is high with a strong Trend, it's Trending-Volatile; if Behavior is mean-reverting and Trend is sideways, it's Range; and so on. Each branch must have a comment explaining the market reasoning, not just the threshold. That comment is what a reviewer — or you yourself in six months — needs in order to trust the label.

The central finding of XCAP's audit was devastating: every prior module was either stateless computation or a passive accumulator; NOTHING recorded a falsifiable prediction before the outcome and then scored its own calibration. That missing loop is exactly what makes a research system hard to fool, and the only honest source of evidence of edge. Market Memory (`market_memory.py`) is that loop. If you don't build this module, everything else is theater: pretty calculations that are never confronted with reality.

The heart is `advance_market_memory(universe, state_dir, horizon_days=5, ...)` and its sequence is sacred in this order: observe new dates → classify the regime (using the `regime.py` from SG-02) → record a LOCKED forecast (a baseline derived from the regime, with its probability and its horizon) → resolve the forecasts that have already matured against the realized forward return → score. The lock is the key word: once you record 'on date T I predict prob_up=0.62 over 5 days,' that record is immutable. You can't go back and touch it when you see the result. That immutability is what turns the ledger into evidence rather than after-the-fact rationalization.

The metrics aren't a single one; they're an honest dashboard. Hit-rate (what fraction of directional predictions were right?). Brier score (how well calibrated were the probabilities? — it penalizes both overconfidence and underconfidence). Calibration by confidence bucket (when you said 70%, were you right 70% of the time?). And the jewel: hit-rate BY REGIME. This last one is what truly informs: it tells you whether your prediction has value in a trend but is garbage in a range, or whether you only get it right in calm conditions. A global hit-rate of 55% can hide 70% in a trend and 40% in panic — and that decomposition is what tells you where, if anywhere, there's edge.

The anti-fabrication property is by construction (we go deeper in SG-04, but here it's structural): only forecasts whose horizon has already matured are resolved, with data that existed AFTER the forecast date. It's out-of-sample by design. You can never 'predict' a date whose result you already know, because the forecast was locked at T and is only resolved when T+horizon arrives with real data. The state dir is gitignored — the history isn't versioned, so there's no temptation to edit it by hand and dress up the results.

The three integrity properties XCAP enforces, and which you must replicate: it accumulates (the ledger grows between runs, never resets), it's idempotent (running the tick twice over the same dates doesn't duplicate forecasts or resolve them twice), and Gate-closed (it never trades). This is the same family of invariants as the capital invariant in SG-05: the system's state is the sum of real events, never a replay or a fresh baseline. The `market-memory-tick` CLI orchestrates this and reads `data/real`-format CSVs (lowercase) via `LocalCSVSource` — zero network, fully deterministic from disk.

A point that separates the engineer from the amateur: the baseline you record must be honest and modest. XCAP records a 'regime_baseline' — a probability derived from the regime, not an elaborate model you've already optimized by looking at the past. Why start humble? Because the baseline is your measuring stick. If a sophisticated forecaster can't beat the regime baseline out-of-sample, you don't have edge, you have overfitting. Market Memory exists to refute your ideas, not to confirm them. And a good forecast record is evidence, never proof — which is why the Capital Gate stays closed no matter how good the hit-rate looks.

The question that defines this module: how do you build a system that CAN'T lie to you about whether it has edge? XCAP's answer isn't 'be careful.' It's introducing structural adversaries — null controls — that every edge claim must compete against. The Hypothesis Lab (`hypothesis_lab.py`) has a registry of real forecasters (regime_baseline, trend_follow, momentum_follow, mean_revert) AND two deliberate null controls: `always_up` (the DRIFT null — the market rises on average, is your forecaster just capturing that?) and `coin_flip` (the CHANCE null — seeded and deterministic, do you beat a coin toss?). A forecaster that doesn't beat BOTH nulls doesn't have edge, it has an illusion.

The second pillar is significance with multiple-comparison correction. If you test twenty forecasters, one will look good by pure chance — that's the data-snooping problem. XCAP neutralizes it with a Bonferroni-corrected z threshold that SCALES with K (the number of forecasters evaluated): the more hypotheses you test, the higher the bar each one must clear. `evaluate_forecasters(universe, names, horizon_days, min_samples, alpha)` compares each real forecaster against both nulls (Δchance and Δdrift) and only grants `candidate_edge` if it beats chance AND drift AND clears the corrected threshold AND has at least `min_samples` observations. All four conditions, jointly. Removing any one opens a door to fabrication.

The detail that proves the system is honest and not rigged: `edge_demonstrated` is ALWAYS False. The lab can grant `candidate_edge` — 'this deserves more investigation' — but it never declares demonstrated edge, because no backtest, however good, proves future edge. This is a design decision that protects against your own optimism: the system literally has no code path that says 'yes, you have edge, trade.' The Capital Gate stays closed by construction, not by your discipline.

The verification that the scoreboard isn't rigged is elegant and you must replicate it: XCAP checked that on synthetic trends, `trend_follow` wins big (it detects the real signal), `mean_revert` goes significantly NEGATIVE (it detects an anti-edge — betting against the trend loses), and `coin_flip` lands at ~50% (chance isn't rewarded). If your coin_flip came out with a 60% hit-rate, your pipeline is broken or you have leakage. The nulls aren't just adversaries: they're calibration probes for your own measurement system. A coin_flip that doesn't give ~50% is a system that's lying to itself, and you must stop everything until you understand why.

The shared foundation prevents the trick of moving the goalposts: `scoring.py` is the single source of truth for the primitives (prob_up, is_hit, brier, conf_bucket, hit_rate_z). Both Market Memory (SG-03) and the Hypothesis Lab score with THE SAME code. If you had two implementations of 'hit', you could — consciously or not — use the more generous one for your favorite forecaster. A single shared primitive closes that door. This is anti-fabrication by construction: you can't pick the rule that suits you because there's only one rule.

Every evaluation is out-of-sample by the same mechanics as SG-03: each bar is classified once, the forecast is generated without seeing the future, and it's resolved against the real forward return. The `docs/HYPOTHESIS_LAB.md` doc and the 4 invariants of `check` encode these guarantees. The underlying lesson, which is the thesis of the entire Signal constellation: the binding constraint for trading real capital is DEMONSTRATED edge, and these tools measure edge honestly (null controls + significance) instead of asserting it. A system that can only emit `candidate_edge` and never `edge_demonstrated` is a system that respects what it doesn't know.

This module is born from a real and painful failure in XCAP, which is why it teaches so well. A scheduled autorun was silently RESETTING the ledger to the initial balance of 500 and replaying the same identical fixture every 5 minutes. The system showed activity, numbers changing, it looked alive — and it was completely fake. Three rules were violated at once, and they're the three that define the capital invariant. Memorize these rules because they're the core: accumulate, realized-only, and re-entry continues previous sessions.

Rule 1 — Accumulate. Capital does NOT reset between runs or sessions. If you ended yesterday's session at 547.30, today you start at 547.30. A fresh baseline at every startup isn't accounting, it's a demo in disguise. The ledger loads from disk on startup and persists on shutdown; the state lives between executions.

Rule 2 — Realized-only. The balance only moves when a position is CLOSED/sold. Unrealized P&L (mark-to-market, the floating value of an open position) never inflates the balance. The exact invariant, which you must be able to assert at any moment: `current_balance == starting_balance + realized_pnl`. An open position with +200 of floating profit does NOT change the balance; it only does so when you close it and those +200 become realized. This avoids the oldest psychological trap in trading: counting your gains before you've banked them.

Rule 3 — Re-entry continues. On restart, the capital is the accumulated result of previous sessions' trades, not a new baseline. The system must be able to shut down and start up a thousand times with the capital always being the honest sum of every real close that occurred. This is the same as Market Memory's accumulation property (SG-03): the state is the sum of real events, one direction only, no replay.

The architectural lesson is the distinction between two commands XCAP keeps deliberately separate. The scheduled autorun runs `sim-autopilot-tick`: it loads the existing ledger, rotates synthetic regimes, and ACCUMULATES on top of what came before. There's also `sim-run-agent`, which resets and replays a fixture — but that one is kept ONLY for deterministic demos, and must NEVER be what the autorun runs. The bug was precisely pointing the scheduler at the wrong command. The lesson: a command that resets and a command that accumulates must never be confusable; name them so the dangerous one shouts what it does.

How this is hardened so it never breaks again: the invariant is LOCKED by `tests/test_sim_autopilot_tick.py`. If you touch the ledger or the autorun, those tests must stay green. It's not documentation, it's an executable lock. The operational rule the user imposed emphatically: if you touch the accounting, you run that test, and if it turns red, you've broken capital integrity and you stop. This is the same philosophy as the Capital Gate and the `check` from SG-01: the invariants that truly matter are encoded as tests that refuse to let you break them. XCAP's accounting isn't trustworthy because the author is careful; it's trustworthy because a red test stops him from being careless.

An autopilot tick is code that runs without a human watching, and that's exactly why it must be the MOST disciplined code in the system. The error XCAP suffered firsthand teaches everything: a scheduled tick that every 5 minutes reset the ledger and replayed an identical fixture. Seen from outside, the system looked alive — numbers moving, constant activity. Inside it was a lie on loop. The question that defines a good tick: does this loop, running a thousand times, ACCUMULATE something true, or does it only generate the appearance of activity?

Property 1 — Deterministic from disk. The tick reads its input from disk (`data/real` CSVs via `LocalCSVSource`), not from the network. XCAP's automated loops NEVER enable the network — `allow_network` stays False, always. This is a direct continuation of SG-01: the network is explicit opt-in per human call (the `data-fetch` command), never something an autonomous loop activates on its own. A tick that touches the network unsupervised is a tick that can leak, fail non-reproducibly, or hang waiting on a socket. Deterministic from disk means the same state on disk produces the same result, bold and reproducible.

Property 2 — Idempotent. Running the tick twice over the same dates must not duplicate forecasts, re-resolve what's already resolved, or count a close twice. This matters enormously in loops because schedulers fail, retry, overlap. If your tick isn't idempotent, a retry corrupts state silently. The way to achieve it: each event (a forecast, a close) has a natural key (date+symbol) and the tick checks for existence before writing. `advance_market_memory` and `sim-autopilot-tick` are idempotent for this reason.

Property 3 — Accumulate, don't reset. The tick loads the previous state and builds ON TOP of it. `sim-autopilot-tick` loads the ledger, rotates synthetic regimes, and accumulates — exactly the capital invariant rule from SG-05 and Market Memory from SG-03. This is the same invariant appearing for the third time in the constellation, and it's no accident: the honesty of an autonomous system IS its refusal to reset. A loop that starts from zero every time learns nothing; a loop that accumulates is the only thing that can build a calibration history worth anything.

Property 4 — Gate-closed by construction. The tick NEVER trades real capital. The Capital Gate (`capital/broker/live_trading/automation` all False) is verified by `python3 -m xcap.control check`. An autopilot is precisely where a trading system kills its owner: it runs on its own, and if it had the ability to trade live, a bug at 3am empties the account. That's why in XCAP the autopilot is sim-only, no exceptions, and `check` asserts it on every run. Automation and the ability to trade live are two things that must NEVER coexist without an extremely deliberate human decision.

How all this is operated and hardened: the tick runs via the scheduler (in XCAP, a scheduled task that invokes `sim-autopilot-tick`, the command that accumulates — NEVER `sim-run-agent`, the one that resets for demos). The behavior is LOCKED by `tests/test_sim_autopilot_tick.py`: if you touch the autorun, those tests stay green or you've broken the loop's integrity. And every run ends with a `check` that reasserts the invariants (keyless, Gate-closed, accumulation). The final principle, which seals the entire constellation: an honest loop is one whose guarantees don't depend on someone watching it, because they're encoded as tests and invariants the loop itself verifies on every pass. Build loops that refuse to lie even when no one is looking — that's the only automation a research system can trust.