A New Era of On-Device AI Performance

Get ready, Android users! The highly anticipated Arm Scalable Matrix Extension 2 (SME2) is about to make waves in the world of mobile AI. Built on the robust Armv9-A architecture, this groundbreaking technology is designed specifically to turbocharge matrix-heavy computations right on your device, enhancing AI model performance without requiring any tweaks to existing apps.

What's New with SME2?

SME2 is taking the reins from its predecessor, the Arm Scalable Matrix Extension, and is packed with advanced capabilities. This new extension not only introduces multi-vector processing instructions but also refines the way data is loaded and stored, all while implementing an innovative multi-vector predication mechanism.

Faster, Smarter, Better!

While Apple devices have already experienced the performance boost that SME2 delivers, Android enthusiasts will soon enjoy similar enhancements. According to Alex Spinelli, Arm’s VP of AI and Developer Platforms and Services, the transition is underway.

SME2 is a game-changer for real-time mobile AI applications like image processing, language translation, and voice synthesis. Arm highlights that the advantages of SME2 are staggering—Google's Gemma 3 model on SME2-enabled devices responds to queries an impressive 6 times faster, capable of summarizing a staggering 800 words in less than a second on just one CPU core.

Unleashing the Power of KleidiAI

To help developers harness the full potential of SME2, Arm is rolling out a power-packed library called KleidiAI. This tool is seamlessly integrated into Google’s XNNPACK, which already supports a range of popular machine learning frameworks like Alibaba’s MNN and Microsoft’s ONNX Runtime.

With SME2 activated, XNNPACK smartly directs matrix-intensive tasks to KleidiAI without requiring any alterations to the app's existing structure. Developers will appreciate how easily they can incorporate this library into their C and C++ projects, thanks to its micro-kernel architecture.

What is a Micro-Kernel?

In the world of Arm, a micro-kernel is the essential software needed to accelerate machine learning operations with stellar performance. Each micro-kernel focuses on processing specific parts of output tensors, allowing complex operations to be executed across multiple threads for enhanced efficiency.

Seamless Integration, No Headaches!

KleidiAI is designed with developer convenience in mind. It eliminates the need for external dependencies, negates dynamic memory use, and simplifies memory management. Each micro-kernel operates independently, packaged neatly into manageable .c and .h files.

As SME2 prepares to launch on Android, it promises to redefine the boundaries of mobile computing, making advanced AI more accessible than ever!