Open source
27 updates on Open source.
Sherpa runs LLaMA on an Android phone through a Flutter chat app
The Flutter app loads a ggml model file off the phone and chats with it offline, and development stopped three weeks after it started.
Georgi Gerganov publishes llama.cpp, LLM inference in plain C and C++
The dependency-free C and C++ inference engine supports 1.5-bit to 8-bit quantisation and treats Apple silicon as a first-class target.
George Hotz starts tinygrad, a framework that ports to an accelerator in about 25 ops
The tiny corp framework caps its repository at 26,500 lines, ships Metal, Adreno and WebGPU backends, and runs openpilot on a Snapdragon 845 GPU.