Benchmarks
52 updates on Benchmarks.
Microsoft trains phi-1 to 50.6 percent on HumanEval with 1.3B parameters
Microsoft Research reports 50.6 percent on HumanEval and 55.5 percent on MBPP for a 1.3B code model trained on under 7B tokens of textbook-quality data.
LaMini-LM distils models from 61M parameters up on 2.58M instructions
MBZUAI distilled 17 instruction models from 61M to 7B parameters on 2.58M synthetic instructions, and reports the 248M model matching LLaMA-7B.
MobileBERT runs in 62 ms on a Pixel 4 with 25.3M parameters
Carnegie Mellon and Google Brain distilled BERT into a 25.3M-parameter model that answers in 62 ms on a Pixel 4 and scores 77.7 on GLUE.
TinyBERT shrinks BERT-base to 14.5M parameters and runs 9.4x faster
Huawei's distilled BERT keeps more than 96.8% of its teacher's GLUE score at 14.5M parameters, 7.5x smaller and 9.4x faster on inference.