George Hotz starts tinygrad, a framework that ports to an accelerator in about 25 ops

George Hotz pushed the first commit of tinygrad on October 18, 2020, with the message “start tinygrad”, according to the repository’s commit history. The README describes an end-to-end deep learning stack with a tensor library, an autograd, a compiler that fuses and lowers kernels, and a JIT, positioned between PyTorch and karpathy/micrograd. It also states that an accelerator port needs a total of about 25 low level ops. tiny corp, the company Hotz founded, maintains it.

The README lists backends for OpenCL, CPU, Metal, CUDA, AMD, NVIDIA, QCOM and WebGPU. The runtime documentation names the requirements for each one, Adreno 6xx series GPUs for the QCOM backend and M1 or newer Macs for Metal, with Metal 3.0 needed for bfloat support. The QCOM backend, which drives the Qualcomm GPU through the Adreno kernel driver interface instead of OpenCL, entered the repository on September 2, 2024, and the repository also carries a backend for Qualcomm’s Hexagon DSP in ops_dsp.py.

tiny corp states on its site that tinygrad is used in openpilot to run the driving model on the Snapdragon 845 GPU, where it replaces Qualcomm’s SNPE. The company says its framework is faster than SNPE, loads ONNX files, supports training and allows attention, which SNPE does not because it only allows fixed weights. The repository’s test workflow compiles an openpilot model with an image pitch alignment set to match the Adreno 630, the GPU in the Snapdragon 845.

The same workflow enforces the project’s size, failing the build when the repository goes over 26,500 lines as counted by sz.py. The repository ships a language model tool in tinygrad/llm that loads GGUF files, serves an OpenAI compatible API and benchmarks tokens per second, with a built-in model list that starts at quantised Llama 3.2 1B and Qwen3 0.6B. tinygrad is published under the MIT license, and the project recommends installing from source rather than from PyPI.