Microsoft Research published results for Phi-4 Mini, a 3.8-billion parameter language model that achieves state-of-the-art performance on the MMLU and GSM8K benchmarks among models under 5B parameters — and in several categories outscores models nearly twice its size. The model's architecture incorporates dense attention sharing, aggressive weight tying, and a novel tokenizer optimized for instruction-following tasks, reducing inference memory requirements to under 2GB for full-precision deployment. On grade-school math reasoning (GSM8K), Phi-4 Mini scores 87.2% — three points above Mistral 7B and within two points of Llama 3.3's 8B variant.
The research team attributes the efficiency gains to training data quality rather than architecture alone. Phi-4 Mini was trained on a curated mixture of textbook-quality synthetic data, academic papers, and high-signal web content, filtering out roughly 85% of the raw corpus used in Phi-3's training pipeline. This "textbook is all you need" philosophy, first articulated by Microsoft researchers in 2023, continues to yield returns even as competing labs pursue scale. The team notes that per-token reasoning accuracy improved most on multi-step problems requiring working memory — suggesting the data quality gains compound on tasks that demand chained inference.
Practical deployment implications are substantial. A 3.8B model running on consumer hardware or mobile NPUs makes real-time on-device inference viable for a class of applications previously requiring cloud roundtrips. Microsoft is integrating Phi-4 Mini into Windows Copilot+ features and Surface devices with Qualcomm's Snapdragon X Elite chips, targeting latency-sensitive tasks like real-time document summarization, offline transcription, and privacy-preserving code completion. The model weights and training code are released under MIT license on HuggingFace, with quantized variants available for Ollama and llama.cpp deployment from launch day.