Skip to main content

Blog

Research, engineering, and insights from Kroonen AI.

Featured Run Finished 8 min read

Genesis 1B: Run 2 Extended — 20k → 40k Steps

Genesis 1B Run 2 extended to 40,000 steps (~21B tokens, above Chinchilla-optimal). Step 38,081 / 40,000 — 95.2% complete, ETA April 7.

Genesis Run 2 pretraining finishing soon

🧬 Genesis 1B

5 min read

Genesis 1B, Run 2: Deeper, Faster, Smarter

Same 1B parameters, 3x throughput. How torch.compile, real-valued RoPE, a deeper architecture (32 layers vs 20), and batch tuning tripled training speed on the same 2x RTX 4090 hardware.

Genesis Run 2 architecture torch.compile
10 min read

The Genesis Manifesto: Sovereign Intelligence for the Post-Generative Era

Data sovereignty, constitutional alignment, and the case for training language models on consumer hardware. Why the future of AI is local, private, and personality-first.

Genesis philosophy alignment

🔧 Postmortems

8 min read

The Optimizer State Bug: A Silent Failure in DCP Resume

A silent AdamW state bug during Run 1 that produced a false recovery on poisoned weights. The load path didn't crash or hang -- it just silently ruined the model.

Genesis Run 1 postmortem optimizer dcp
8 min read

Fixing FSDP Checkpoint Deadlocks on 2x RTX 4090

How we fixed FSDP checkpoint deadlocks during Run 1 on consumer GPUs without NVLink using DCP sharded checkpoints and decoupled evaluation.

Genesis Run 1 postmortem fsdp dcp

🔬 Research

5 min read

Mapping the Mind of Qwen 3.5 9B: A Sparse Autoencoder for Mechanistic Interpretability

A sparse autoencoder trained on the internal activations of Qwen 3.5 9B. Zero dead features, 16,384 interpretable dimensions, trained on a single RTX 4090.

interpretability sparse-autoencoder research