BUPT proposes one 9.2B model in the OS that all apps call through adapters
Researchers at Beijing University of Posts and Telecommunications proposed that a phone ship one shared multimodal model instead of letting every app bundle its own, in a paper published in the ACM MobiCom 2024 proceedings on May 29, 2024. The operating system and the hardware co-manage that model like firmware, unchangeable by apps or by the OS itself, exposed to applications as a system service, and each app reaches it through a small adapter fine-tuned offline for its own task. Their prototype, called M4, holds 9.2B parameters and needs 7.5 GB of peak memory, and the authors report it reaching accuracy comparable to purpose-built models on 85% of the 50 datasets in a benchmark they assembled from 38 mobile AI tasks across five input types.
What the shared model replaces is one small model per app per task. The paper’s baselines are 50 task-specific models of 1M to 500M parameters each, one per dataset, against which M4’s adapters run from 1,000 to 10 million parameters, so each added task costs under 10 MB. Measured on an Nvidia Jetson Orin NX, 4-bit M4 needs 6.1 GB of storage to serve all 50 tasks against 15.2 GB for the 50 separate models, with the crossover at about 15 tasks, and 7.5 GB of peak memory against roughly five times that. The authors state that on a device with 12 GB of memory the 4-bit model plus all 50 adapters fits, where only 20 of the 50 task-specific models would.
The prototype is slower than the models it replaces. On the Jetson Orin NX with 16 GB, the authors measured M4 averaging 18 times the inference latency of the task-specific models across the 50 tasks and 19 times the energy, 3.6 s against 0.2 s. On a Pixel 7 Pro CPU they measured an average of 6.8 s against 0.54 s, and their per-task breakdown puts image classification at 2.10 s and question answering at 6.34 s to the first token and 0.24 s per token after it. They state that M4 cannot currently run on a stock smartphone GPU or NPU at all, because those processors lack support for the operators it uses.
The NPU numbers in the paper are a projection rather than a measurement. The authors estimate that M4 on an NPU would average 0.48 s and 1.3 J, under the 0.54 s and 2.9 J they measured for task-specific models on the Pixel 7 Pro CPU, but they derive that by applying the CPU-to-NPU ratio they observed for task-specific models, not by running M4 on an NPU. Their case for a simpler accelerator rests on a separate Pixel 7 Pro measurement, where they converted 110 downloaded models to TensorFlow Lite and only 8% ran entirely on the NPU, those gaining a median speedup above 20 times over the CPU. M4 itself uses 39 operator types against the 156 that the 50 task-specific models need between them.
The authors name their own limits. They write that the accuracy results come from an A100 and the Jetson board rather than from phones, that M4 underperforms task-specific models on some tasks including translation, and that a prototype assembled from off-the-shelf pre-trained models is “still highly inefficient in terms of accuracy and model parameter size”. Its backbone is Meta’s LLaMA-7B at 8-bit, with encoders taken from ImageBind and Whisper, and they note that adapters trained against one backbone stop working when the backbone is upgraded, so the design still needs a stable interface between the two. Code and benchmark are published at github.com/UbiquitousLearning/MobileFM, and the paper carries ACM copyright rather than an open license.