Google's TPU Hardware Finds a New Home at Meta
The shift allows Meta to run large language models on Google silicon without rewriting its core PyTorch codebase.
By Marco Kapoor ·
For the first time, Meta’s artificial intelligence infrastructure now runs production workloads on Google’s custom silicon. This change means the social media giant is no longer reliant on a single chip architecture to process its massive data loads. The new setup integrates Google’s tensor processing units directly with the PyTorch framework, a move that simplifies the technical stack for one of the world’s largest tech companies.
The integration was made possible by a new version of the SGLang inference engine that is native to PyTorch. Previously, running these models on Google hardware required the use of a different framework known as JAX. The new engine eliminates that dependency, allowing developers to keep their existing code intact. This reduces the friction for engineers who prefer the PyTorch ecosystem and want to deploy models on alternative hardware.
Zongwei Zhou, who leads the PyTorch team at Meta, confirmed that the system is already live. He noted that the collaboration involved a close partnership between Meta, Google, and the SGLang team. The team behind the engine is RadixArk, a startup founded by the creators of the SGLang tool. Zhou described the result as a remarkably mature serving stack that fits naturally within Meta’s current operations.
Jana van Greunen, a director of engineering at Meta, emphasized the strategic value of this hardware diversity. She stated that the company cannot be constrained by a single hardware architecture. The new TorchTPU tool enables Meta to deploy its specific workloads to Google’s chips. This represents a significant step forward from older bridging tools that Meta had used in previous years before canceling some contracts.
The timing of this announcement is notable because it followed a similar move by Amazon. Just one day earlier, Amazon revealed it was open-sourcing its own backend for its Trainium chips. Both companies are working to loosen the grip that Nvidia has had on the AI software market. By making their specific hardware compatible with standard tools like PyTorch, they aim to create a more competitive environment for AI developers.
Performance tests indicate that the new setup is highly efficient. A startup called Inferact reported that a tuned version of another inference engine called vLLM achieved speeds over 700 tokens per second on Google’s seventh-generation chips. This test used a large open-weight model and compared the results against Nvidia’s GB200 hardware. The Google setup outperformed the Nvidia baseline in several scenarios, particularly when using specific decoding techniques to speed up processing.
The development teams plan to continue expanding the list of supported models. They intend to add support for new releases as they become available, aiming for immediate compatibility on release day by early 2027. Google, Meta, and RadixArk are scheduled to share more details at a summit in San Francisco next month. This ongoing collaboration signals a long-term commitment to making Google’s chips a viable option for major AI players.
The move also hints at broader hardware agreements between the two companies. While the exact location of the production hardware is not public, reports suggest that Meta has signed multi-year rental agreements for these chips. The inclusion of these units in Meta’s data centers marks a significant shift in the industry, as Google sells its custom processors to external customers in a major production capacity for the first time.
