Meta's ParetoQ compares five bit widths and puts the accuracy cliff at 1 bit
Meta published ParetoQ on February 4, 2025, a training framework that puts 1-bit, 1.58-bit, 2-bit, 3-bit and 4-bit quantisation through the same recipe so the bit widths can be compared against each other rather than against methods built for one width. The result the authors draw from it is that 1.58-bit, 2-bit and 3-bit land close together and generally beat 4-bit on accuracy against model size, while 1-bit drops away. On LLaMA-3 8B the paper’s table gives an average of 71.2 across five commonsense tasks at 2 bits and 69.0 at 1.58 bits against 74.6 for the full-precision model, then 64.1 at 1 bit.
That ranking only holds when size is counted in bytes rather than parameters, which is how a phone’s memory budget works. The authors measure each model by its effective quantised size, every weight counted at its own bit width plus the embeddings at theirs, and compare across that axis. Their example is a 2-bit MobileLLM-1B scoring 61.0 on an eight-task average against 59.2 for a 4-bit MobileLLM-600M, 1.8 points higher at a smaller effective size, and the paper reports the same pattern across LLaMA models up to 8B.
The paper also describes where the training behaviour changes. At 3 bits and above the fine-tuned weights stay within 10 to 20 percent of the full-precision weights they started from and recover most of the accuracy after about 10B tokens, while at 2 bits and below they move by roughly 40 percent and need around 30B tokens, which the authors read as the network rebuilding its representations instead of nudging them. On that basis they argue against training ternary models from scratch, the route taken by BitNet b1.58 and Spectra, and put their ternary 600M model at 58.7 on a six-task average, against 57.5 for the ternary 3B model of the 1-bit Era and 58.2 for Spectra’s ternary 3.9B, both computed from the same table.
For speed the authors wrote a 2-bit CPU kernel and timed five MobileLLM models on an Apple M1 MacBook Pro with 32 GB of memory using 6 threads, 5 prompt tokens and 122 generated tokens, and report the 2-bit curve sitting above the 4-bit one on accuracy against decoding speed without naming a factor. The one speedup they put a number on is on an H100 NVL GPU, where their 2-bit kernel built on CUTLASS runs 4.14 times faster than FP16 and 1.24 times faster than the 4-bit Machete kernel from vLLM at a 16384 by 16384 weight shape. At 1B, 3B and 8B the authors say the in-kernel conversion overhead cancels that gain and 4-bit gives the better speed-accuracy trade-off.
Hardware is also why the paper favours 2-bit over ternary despite the two sitting together on its accuracy-size front. Storing three values needs either sparsity above 90 percent to pay for the indexing or packing that complicates the matrix multiply, and the authors note that some implementations store ternary weights as 2-bit signed integers anyway, which gives up the storage saving. They write that broad adoption will need “community-wide efforts, such as INT2 support in NVIDIA tensor cores”, and the paper itself names no weight release.
Update, May 30, 2025. Meta published the ParetoQ checkpoints on Hugging Face, five MobileLLM sizes from 125M to 1.5B, each as a BF16 model plus 1-bit, 1.58-bit, 2-bit, 3-bit and 4-bit versions, 30 repositories in all. They sit behind a gated form under Meta’s FAIR Noncommercial Research License, which permits research use only. The training code had gone up on GitHub on March 3, 2025.