At this year's WWDC25, Apple dropped a bombshell — announcing revolutionary upgrades to its on-device and cloud-based AI models. Now, they've released a groundbreaking tech report that spills the secrets on how these advanced models were trained, optimized, and evaluated, packed with fascinating behind-the-scenes details.

Deep Dive into Apple's Tech Report

The tech report, titled "Apple Intelligence Foundation Language Models – Tech Report 2025," explores various aspects of their latest models: from architecture and data sources to tool use and performance benchmarks. If you're a tech enthusiast, this is a treasure trove of information.

Smart Design: The Split Local Model

Previously, we learned that Apple's on-device model boasts around 3 billion parameters. But here's the kicker: it's divided into two distinct blocks! Block 1 encompasses 62.5% of the total transformer layers, while Block 2 has 37.5% but cleverly omits key and value projections. This ingenious split reduces memory requirements by 37.5% and accelerates the time taken to output the first token.

Though Apple considered other approaches years ago, this method illustrates their ongoing innovation in delivering stellar local performance even for devices with limited memory.

Revolutionary Cloud-Based AI Architecture

On the server side, Apple has crafted a custom architecture known as Parallel-Track Mixture-of-Experts (PT-MoE). Instead of relying on one gargantuan AI model, this approach activates only relevant subnetworks or 'experts' based on the task at hand. For example, if you're asking about cooking, only the culinary experts spring to life.

This modular setup enhances speed and precision, allowing the model to provide lightning-fast responses while utilizing the expertise of each specialized subnetwork.

A Leap in Multilingual Capabilities

One of the big complaints about Apple's original AI rollout was its limited language support. Not anymore! This time around, the company has expanded its multilingual data dramatically from 8% to 30%. The tokenizer capacity has also surged by 50%, from 100,000 to a whopping 150,000 tokens.

These enhancements have resulted in significant performance improvements, especially in non-English benchmarks, giving features like Writing Tools a major upgrade.

Sourcing Data: Where Does It Come From?

As before, a large portion of training data comes from web scraping via Applebot, which respects site exclusions. Apple utilized a layered filtering process to eliminate any low-quality or irrelevant content.

They also incorporated licensed data from prominent publishers, likely including big names like Condé Nast and NBC News. Furthermore, synthetic data generated through smaller models played a pivotal role in fine-tuning and bolstering multilingual support.

The Road Ahead for Apple AI

Despite facing competition and discussions around its AI momentum, this tech report highlights Apple's strides towards innovation and privacy in AI. As they evolve their capabilities, it’s clear that Apple is determined to reshape the AI landscape with cutting-edge technology. Stay tuned for more exciting developments!