mytorch

I was tired of treating autograd as a black box. I could read about backpropagation, but that wasn't quite the same as knowing what happened between a loss value and an updated weight.

So I wrote the pieces myself. MyTorch is a small Python library with NumPy underneath it: tensors, reverse-mode differentiation, and enough layers and training code to run a neural network. I deliberately kept the API close to PyTorch. The familiar shape let me look at what was happening underneath the calls.

I kept the scope to dense linear layers. No convolution kernels, no CUDA. Linear layers and ReLU were enough to connect the differentiation code to a model; cross-entropy gave it a loss, and I implemented SGD and Adam to update the weights.

The useful part was having the whole training loop in code I could step through line by line. The gradient calculation was no longer hidden inside a framework, and neither was the optimizer using it.

On MNIST, MyTorch reached 97.16% test accuracy. That gave me a working example of the pieces training together, rather than just separate implementations of the equations.

I keep the repository (opens in a new tab) available as a teaching reference. It isn't a package for production training. The small scope is part of what makes it useful to read.