Meta now runs AI on Google chips

700 tokens per second. That is the speed a new setup achieved for a single user on a specific AI model. It marks a significant performance benchmark in the race to process large language models efficiently. The figure comes from a tuned configuration of the vLLM engine running on Google’s seventh-generation Ironwood TPU. This speed outpaces a comparable setup on Nvidia hardware, which reached 452 tokens per second in the same test. The result highlights how specialized chips can outperform general-purpose graphics cards in certain inference tasks.

The shift involves a new version of the SGLang inference engine. It is now PyTorch-native. Previously, running these engines on Google’s tensor processing units required using Google’s own JAX framework. The new versions remove that requirement. Developers can now use PyTorch directly on the hardware. This change means less rewriting is needed for existing code. A matching version of vLLM was also released as open source. Both engines are widely used in the industry for serving AI models.

Meta, the company behind the PyTorch framework, is the first to run this new stack in production. Zongwei Zhou, the PyTorch-TPU lead at Meta, confirmed the deployment. He noted that the work was a three-way collaboration between Meta, Google, and the SGLang team. The team works at RadixArk, a startup founded by the creators of SGLang. Zhou described the engine as a remarkably mature serving stack. It gives Meta a native option for its workflows. This marks a significant change in how the company manages its AI infrastructure.

Jana van Greunen, director of PyTorch engineering at Meta, provided further context. She stated that Meta cannot be constrained by a single hardware architecture. The new TorchTPU tool enables the company to deploy its PyTorch workloads to Google TPUs. Google co-developed this tool with Meta’s PyTorch team. This partnership aims to reduce dependency on any one vendor. It allows Meta to choose the best hardware for specific tasks. The software bridge makes this choice easier to implement.

This is not the first time Meta has looked at Google’s chips. Google calls the older PyTorch/XLA bridge a predecessor to the new system. Reports last year indicated that Meta research teams used TPUs through that older bridge from 2020 to 2023. However, Meta canceled those contracts. The current push by Google was aimed mainly at Meta. The company did not want to migrate to the JAX framework. The new native support addresses that preference directly. It offers a smoother transition for existing PyTorch users.

The location of these production TPUs is not publicly confirmed. Talks last year reportedly included renting Google Cloud TPUs in 2026. Purchases were discussed for 2027. Meta later reportedly signed a multi-year rental agreement. The exact details of the hardware placement remain private. What is clear is that the infrastructure is now live. The production status was announced on Google’s developer blog. This confirms the move from testing to real-world use.

The announcement came one day after a similar move by Amazon. The company said it was open-sourcing TorchNeuron. This is a native PyTorch backend for its Trainium2 and Trainium3 chips. Both moves appear aimed at loosening Nvidia’s software lock-in. The industry is seeing a trend toward more flexible software stacks. This allows companies to run AI workloads on diverse hardware. It reduces the friction of switching between different chip architectures.

Performance details extend beyond the 700 tokens per second figure. The test used 16 Ironwood chips with speculative decoding. This method speeds up inference using a small draft model. It was compared against vLLM’s published setup on 16 Nvidia GB200s. The new kernels for Kimi K3 and Qwen 3.8 27B deliver higher throughput. At small batch sizes, the new setup is roughly 1.4 to 2 times faster than the GB200 baseline. This advantage holds even without speculative decoding in some scenarios.

The team plans to add more models to the SGLang-TorchTPU stack. The first additions will be DeepSeek V4.1 Flash and Reflection Beam. RadixArk is working toward Day 0 support for new open models. This means support will be available on release day. The goal is to achieve this by the first quarter of 2027. Google, Meta, and RadixArk will share more details at the SGLang Summit. The event is scheduled for November 12 and 13 in San Francisco. The industry will watch closely to see how this partnership evolves.