Get Ready for AI Like Never Before!

In a groundbreaking move, Arm is bringing its Scalable Matrix Extension 2 (SME2) to Android devices! Integrated into the Armv9-A architecture, this cutting-edge technology is set to supercharge on-device AI, allowing mobile developers to run sophisticated AI models directly on CPUs with lightning-fast efficiency—no app modifications necessary!

Unleashing the Power of SME2!

Building on the foundation of the original SME extension, SME2 introduces an array of complex CPU instructions specifically tailored for matrix-heavy computations. With enhancements like multi-vector data-processing instructions and advanced load/store capabilities, the performance leap is astonishing. While iOS users have already experienced these benefits with the latest Apple M4-series chips, Android users can soon join in on the action, according to Arm's VP of AI and Developer Platforms, Alex Spinelli.

Why It Matters for Mobile Inference!

Matrix workflows are crucial for real-time tasks such as image recognition, language processing, and voice synthesis. According to Arm, the advantages of SME2 are clear: on SME2-powered devices, Google's Gemma 3 model can deliver an astounding 6x faster chat response times and summarize up to 800 words in less than a second using just one CPU core!

Boosting Speed on Leading Smartphones!

The speed enhancements don't stop there—flagship smartphones like the vivo X200 Pro have shown a remarkable 2.6x boost in prompt processing when using a sophisticated 3.8B parameter Phi-3 Mini model, showcasing the transformative impact of SME2.

Empowering Developers with KleidiAI!

To fully leverage SME2, Arm has introduced KleidiAI, a powerful library seamlessly integrated into Google's XNNPACK framework. This toolkit powers various machine learning and AI platforms including Alibaba's MNN, Microsoft's ONNX Runtime, and Google’s LiteRT, ensuring maximum accessibility for developers.

Innovative Micro-Kernel Architecture!

What sets KleidiAI apart is its micro-kernel architecture, designed for easy integration into C and C++ codebases. Unlike standard functions, each micro-kernel tackles a portion of the output tensor, allowing for efficient multi-threaded operations. Plus, developers will appreciate that KleidiAI requires no external dependencies, avoids dynamic memory allocation, and boasts a modular design, with each micro-kernel easily contained in .c and .h files.

The Future of Mobile AI is Here!

With Arm's Scalable Matrix Extension 2 paving the way for faster, more efficient AI applications on Android, the future of mobile technology looks brighter than ever. Get ready to experience features you never thought possible on your smartphone!