Learning by rebuilding
I'm an engineer in Japan trying to understand how intelligent systems learn by rebuilding them from first principles. Everything below is something I built to find out how it actually works, plus notes on what broke along the way.
Experiments
all →A real transformer with hand-written backprop, trained to copy from context. Watch the attention arc jump back to the matching token. Gradients verified against finite differences.
Train a small neural network from scratch to classify 2-D data and watch the decision boundary form. Backprop and Adam by hand; pick a dataset and tune the learning rate.
A generative adversarial network training in real time: two hand-written neural nets (backprop and Adam from scratch) fight until an amber cloud snaps onto a ring.
Scaled dot-product attention you can grab and move. Drag the query vector and watch the softmax weights and the attended output update live.
Writing
all →Two tiny networks, hand-written backprop, and a ring. What a 2-D GAN makes visible that image GANs hide, including why a falling loss is bad news.
Why I built this, and what I'll share here: machine learning, frontier LLM research, reinforcement learning, and interactive experiments.
A tiny gradient-descent toy and what it shows about learning rates.