BDH just grew a little clock.

Since starting to work with the BDH architecture to build Ninereeds, I’ve been thinking about improvements. One thing that always bothered me: Vanilla BDH has no explicit mechanism for selective forgetting. Context eventually falls out of the window, and training dynamics can weaken unused pathways over time, but that is not the same as active forgetting. It’s more like neglect than forgetting. The human brain does something more active. It clears out memories during downtime. The brain seems to use sleep for memory consolidation, pruning, and reorganization: strengthening some traces, weakening others, and integrating what matters.

Sleep is something I plan to address later, as part of the cognitive OS I want to build around a mature Ninereeds. But the forgetting problem felt like something I could start working on now.

Around the same time, I looked at a paper from Sakana AI called the Continuous Thought Machine. The central idea is simple and appealing: what if neurons didn’t just fire once and vanish into the next layer? What if they had a memory of their recent activities? What if the model could spend more or less internal processing time depending on how hard a question actually is?

CTM and BDH are different architectures, and I wasn’t trying to merge them. But the overlap was obvious enough to suggest a few experiments. The question was how to run those experiments without fooling myself.

That’s where it gets tricky. Architecture experiments are easy to misread. If you change three things at once and the model improves, you don’t know what helped. Was it the forgetting? The memory? The extra processing time? Or did something change the numbers without the model actually getting smarter? So the actual work wasn’t adding features, it was adding them as carefully isolated options that can be tested one at a time.

The baseline BDH is untouched. On top of it, there are now two experimental variants. The first tests only forgetting: attention to distant tokens fades over time, so the model doesn’t drag stale context forward indefinitely. The second adds a small causal memory: each neuron can now “remember” a little of what it was doing in recent steps, rather than treating every position as if it arrived from nowhere.

Adaptive compute – the idea of letting the model think for longer on harder inputs – is now treated as a generation policy rather than a fixed part of the architecture. That’s an important distinction. If adaptive compute is bundled into the same variant, you can no longer tell whether better outputs came from a better architecture or simply from letting generation run extra passes. Separating them keeps the question clean.

The memory mechanism also became more careful. Instead of mixing activation history into everything at once, the experiment can now control exactly where that memory enters the processing pipeline. Starting narrowly and widening only when the narrow version is understood.

Measurement improved too. Training loss alone was never enough; I already had chat evaluations and a brain map tool to visualize what the model had actually learned. Now there’s also held out evaluation data for checkpoint selection, and checkpoints save much richer metadata: settings, random seeds, corpus fingerprints, loss values at both training and evaluation time, and a set of lightweight diagnostics. If a variant improves loss but changes or destroys the sparse activation patterns that make BDH what it is, the diagnostics should show that immediately.

The experiment plan is deliberately conservative. Run the unchanged baseline first. Then test forgetting alone, at a few different rates. Only after that, add the activation memory. Only after that, consider letting the model run multiple internal passes. Each step depends on the previous one making sense.

From the outside, this might not look like much. BDH is still BDH. Ninereeds is still Ninereeds. But what changed is the nature of the questions I can ask. Before, I could ask whether the training was working. Now I can also ask whether the architecture itself has room for improvement – and get an answer I can actually trust.