PyTorch
5 updates on PyTorch.
Meta's MobileLLM-Flash goes shallow and wide for 1.8 times faster prefill
Meta designed 350M, 650M and 1.4B models by measuring latency on a Galaxy S25, reversing the deep-and-thin rule of the first MobileLLM.
ExecuTorch 1.0 reaches general availability for on-device PyTorch models
The PyTorch edge runtime promotes Core ML, Qualcomm Hexagon, Arm Ethos-U, Vulkan and XNNPACK backends to production status.
Apple team finds H100 last on tokens per dollar for models up to 2B
Seven Apple authors measured tokens per dollar for LLaMA-style models from 100M to 2B and found H100s last at every size, behind cheaper A100s.
torchchat runs Llama 3 8B on a Galaxy S23 and iPhone at more than 8 tok/s
PyTorch's demonstration codebase exports Llama models to ExecuTorch and ships iOS and Android sample apps for running them on a phone.
ExecuTorch alpha runs Llama 2 7B on iPhone 15 Pro and Galaxy phones
PyTorch's edge runtime brought 4-bit Llama 2 7B to iPhone and Galaxy handsets, added early Llama 3 8B support and leaned on Apple, Arm and Qualcomm.