Taobao's Mnn Team Open-sources Mnn-pocket-0.3b, A 173mb On-device Model Built For Phone Apps

Alibaba's MNN team at Taobao has open-sourced MNN-Pocket-0.3B, a roughly 300-million-parameter language model designed to run inside mobile apps, IT Home reported on November 6. The model is built around the MNN inference engine and targets large, all-purpose shopping and service apps such as Taobao, where an on-device model has to share limited compute, memory and battery with everything else on the phone.

The team set four design goals for what it calls an app-native large model. Decoding should exceed 200 tokens per second on specified devices. Peak memory should stay under 300MB, leaving room inside a model budget of about 500MB for context growth and integration overhead. A single typical inference should use no more than one ten-thousandth of a reference phone's battery, so 100 runs cost under 1%. And on specified business tasks, the model should match the results of an 8B-class model.

According to the team's tests, MNN-Pocket-0.3B runs fully offline on an iPhone 18 Pro through the MNN engine at an average decode speed of about 350 tokens per second. The INT4 deployment package, which includes weights, tokenizer, compute graph and inference configuration, is 173MB. On an iPad Pro with the M5 chip using the Metal backend, the team measured 466 tokens per second for decoding and 8,721 tokens per second for prefill, with peak memory rising 228MB over baseline. It reported 251 tokens per second on the CPU backend of a Snapdragon 8 Gen 5 phone, and an incremental energy cost of about 0.3% of battery for 100 runs on a OnePlus 13.

The model was trained from scratch with Chinese and English as its focus languages. After testing dense configurations from about 50M to 1.3B parameters, the team found around 300M to be the largest size that met both the memory and speed targets. It settled on 28 layers with grouped-query attention, noting that going to 36 layers did not raise average scores but slowed decoding. The vocabulary was trimmed from about 151,000 tokens in the Qwen3 tokenizer to 64,000, cutting embedding and output-layer costs.

The team reports an IFEval score of 73.20, a Chinese IFEval score of 74.59 and an average of 50.00 across 11 Chinese and English benchmarks, ahead of the same-size and larger models it compared, including Qwen3-0.6B. The model card notes that, like other sub-billion-parameter models, it is not recommended for advanced math or long-horizon code generation.

The variants published on Hugging Face cover a base model for continued pretraining, a supervised fine-tuning checkpoint, the instruction-tuned model, a GGUF build, the INT4 MNN build and a decision-head model for choosing among candidates. The release also includes a DFlash2 speculative decoding draft model, which the team says can double decoding throughput on certain tasks.

SEE ALSO: ModelBest Open-Sources MiniCPM-V 4.7, Its First MoE Multimodal Model, Running in 3GB on a Phone