SharedLLM research

Community-owned distributed LLM inference + a federation of centrally-trained specialists. Why federated frontier pretraining is impossible and what to do instead.

Why we structure research as falsifiable bets with strict / lenient / catastrophic criteria, written before the experiment runs.

Run a node, train a specialist, propose a bet, or submit a hardened replacement for an existing one. The repository is open.

RFC-0006 manifest → registry entry → live model → forward pass. The end-to-end loader is the foundation everything else builds on.

Sum-of-log-probs is unsafe when specialists disagree (joint ppl 1277 vs worse single 339). Mixture (logsumexp − log N) is bounded by the best specialist by Jensen's inequality. Use mixture in production.

Wire-protocol envelope (prefix tokens + intermediate activation tensor) round-trips and lets specialist B resume from A. Architectural KV cache is separate work.

5e-5 lr, 100 steps, MPS → personal ppl 114 → 1.04 in 5.6 seconds. General ppl rises only 25%. The 'specialist per Kerala student' thesis is feasible at this scale (caveat: hardened in Bet 43+60+61).

N=10, 20 trials, worst-case 3 rounds (theoretical bound 9). The coordinator is removable.

Deferred capture (no .item() inside hook) is sub-1% overhead on MPS; naive per-layer .item() is 40%. Critical implementation rule.

Joint mixture log-prob reconciles to logsumexp(per_specialist) − log(N) within 3e-7. Per-token attribution is mathematically sound for UI.

Train only the gain parameters of every RMSNorm layer (~2300 params, 9 KB). Beats full fine-tune by 6× to 100,000× on held-out same-author text.

STRICT 15/15 PASS. Norm-only median ratio 0.0066 of full fine-tune (~150× lower ppl). Full FT overfits catastrophically (84k–400k ppl) on diverse eval texts.

STRICT 3/3 PASS. Norm-only Pareto-dominates LoRA-r4 and full fine-tune across 3 distinct users. The federation default per-user adapter format.

Single-layer last-block steering vector beats norm-only by 32% for the programmer user; all-layer steering overshoots. Sub-norm-only secondary primitive.

Mean diverges in 31 steps under 1/8 byzantine actor at scale 1000×; coordinate-wise median holds within 1.084× clean under churn + byzantine + partition combined.

Initial reading suggested K=100 outperforms K=1 by 24%. Bet 62 disambiguated this with multi-seed + early stopping; the real story is more nuanced.

External reviewer pushback: K=100 might be K=1 overfitting. Bet 62 confirmed: with early stopping on K=1, K=1 wins by 5–15% across seeds. K=100 reframed: tradeoff for bandwidth, not Pareto win. Headline retracted.

Bit-exact tokens across no-throttle, steady-throttle, ramp-throttle. Tightened framing: validates scheduler tolerates slow workers, NOT phone deployment — that requires on-device testing.

After ternary PTQ of base (10× saving), norm-only adapter still gives 50–65% ppl improvement on top. STRICT 3/3. The two cheapest compressions COMPOSE.

5/5 wins vs random pruning across diverse eval texts; 100–1700× lower ppl. Magnitude pruning is the right default for cheap-wire formats.

BORDERLINE. int8 within 1.5× raw fidelity but 2/3 compressed adapters end up worse than no adapter. Don't blind-compress.

Pairwise averaging only helps both contributors in 1/3 user pairs. Federation must route to the right user adapter; can't deduplicate via averaging.

Train norm-only on RANDOM uniform tokens (no language signal) and compare to real-text training. Random BEAT real for the programmer user. Real wins 2/3 by 1.10–1.36× margin. Honest framing required.

Train one adapter per user, evaluate every adapter on every user's holdout. Own adapter wins by 20–29% margin (programmer + novelist), 5% (scientist). Personalization is real signal. Bonus: adapters cluster by domain similarity.

Sweep Gaussian noise on hidden states. Stable to σ_rel = 1e-1 (10% relative noise) — 4 orders of magnitude beyond fp16 drift. Numerical precision is NOT the phone bottleneck.

Every nonzero α ∈ [0,1] for soup(α) = α·late + (1−α)·early hurt held-out ppl. Linear interpolation regresses toward the worse parent.

Averaging 16 experts per layer to 1 destroys quality (520× blowup, 2 unique tokens). Bet 39 showed even 200 recovery steps don't get back within 3×. MoE expert diversity is real, not nominal.

Every layer skipped produces ≥ 10× blowup. Federation can't ship N−1 layers and expect coherent output.

Logsumexp − log(N) over per-user adapters' logits doesn't help unknown users. Federation must route, can't deduplicate via averaging.

Merkle commitment over inference logs. STRICT 3/3 PASS — every truncation, substitution, and replay forgery caught (300/300). Trust layer beneath glass-box.

Conflict-detection-and-quarantine resists Sybil coalitions up to 50%. LENIENT pass; zero leaks at any Sybil level. STRICT propagation deadline relaxed to realistic 12-round envelope.

Quorum-based revocation works under 25% Byzantine + 10% per-round churn — slowly. 19 rounds to 95% propagation, zero false positives. Calibration result, not a security failure.

1.4M ops × 1000 nodes × 48 simulated hours. Falsifies the unsigned-no-ack baseline; 3.7% drift p95 from one-sided packet loss. Mandates acknowledgement-protocol receipts in production.

200k ops, 99.9% fraud rejection, but 4 swap-attacks slip through unsigned receipts when Byzantine servers are co-registered. Mandates client-signed receipts in RFC-0006.

Cognitive-estate primitive. Strict pass at 0.5+ overlap, borderline-harmful at orthogonal. Linear-average + concatenation cover the regime; UI must surface the overlap to users before accepting inheritance.

500 trainers × 50 years × 600 monthly settlements. Borderline failure at 89% correctly attributed; cause is unattended estates (21% void). Mandates beneficiary-nomination at trainer onboarding + default-pool fallback.

Strict pass. Per-community endorsement preserves all four minority communities (12/8/6/4% population) with proportional 30% query share. The routing-protocol design choice is load-bearing — global aggregation would collapse to majority-mode.

Pure-joule pricing collapses network quality from 0.85 to 0.02 at user weight ≥ 0.8. Bimodal equilibrium with sharp transition. Mandates joule-pressure cap (w ≤ 0.5) + quality-floor for routing.

Strict pass at 50% Byzantine routers. Zero leaks across 135k query-trials, even for 1.2%-of-federation jurisdictions like Kerala. Directory-as-truth + signature verification provides clean cryptographic locality.

Naive K-of-N majority debate hurts accuracy by 11.88pp vs single-routed-specialist. Confidence-weighting recovers most of the loss. The federation's alignment story is good routing, not debate.

All bets used in-process FastAPI test clients or LAN. Cross-ISP latency, jitter, and throughput remain unmeasured. The deployment story depends on this.

Norm-only adapter wins are validated at 30M params on short corpora. Whether the primitive survives at 1B+ on heterogeneous real-user data is the production-readiness experiment.

Bet 63 establishes numerical precision is sufficient. Bet 45 establishes scheduler correctness. Neither validates real phone deployment for thermal-driven model eviction, OS process kill, memory pressure, sustained-load hardware degradation.

215k laptops × ~1 GB pooled RAM each = 210 TB. The political path needs warm intros to KITE, K-DISC, IndiaAI Compute Portal. Technical foundation is now ahead of what the institutional foundation can use.