Arm touts growing ecosystem of sustainable AI datacenter silicon

Arm said that a year from its introduction, the Arm Total Design ecosystem has doubled in size, driving
global silicon innovation for sustinability.

Datacenters are constantly challenged to balance power demands with the growth of AI workloads, the increasing cost and complexity of developing chips, and the need for sustainability, Arm said.

Eddie Ramirez, vice president of go-to-market, infrastructure line of business at Arm, said in a blog post the company introduced Arm Total Design a year ago to address these challenges by creating an ecosystem of partners to accelerate the development of custom silicon, bringing together key
industry players to build solutions for the datacenters of the future with Arm Compute Subsystems (CSS).

It has quickly grown into a multivendor, Arm-based chiplet and SoC ecosystem, bringing together capabilities from design through to foundry manufacturing and doubling to more than 30 participating companies, with Alcor Micro, Egis, PUF Security and SemiFive as the latest companies to join the ecosystem.

New Arm-based solutions to power the AI datacenter sustainably

Arm Total Design has sparked global collaboration, leading to real-world CSS-powered solutions for GenAI computing. Arm said an example is the news today that Arm, Samsung Foundry, ADTechnology and Rebellions are partnering to bring to market an AI CPU chiplet platform.

This platform, targeting cloud, HPC, and AI/ML training and inference workloads combines Rebellions’ Rebel AI accelerator built with a Neoverse CSS V3-powered compute chiplet from ADTechnology to be implemented with Samsung Foundry 2nm Gate-All-Around (GAA) advanced process technology.

The platform promises to deliver performance and optimal power efficiency, with an estimated two times to three times efficiency advantage for GenAI workloads (Llama3.1 405B parameter LLMs).

“AI and HPC designs require technology solutions that deliver maximum performance, high transistor density, and energy efficiency,” said Taejoong Song, head of foundry business development at Samsung Electronics, in a statement. “Samsung Foundry’s 2nm GAA process is designed precisely to satisfy the most stringent HPC and AI design requirements, and we’re excited to leverage the flexibility of Arm CSS and the power of the Arm Total Design ecosystem to deliver an AI CPU chiplet platform, which will further accelerate adoption of our leading-edge technology and design solutions for hyperscalers and cloud service providers.”

This exemplifies the unique value of Arm Total Design and standards-based compute subsystems in accelerating AI silicon development by integrating Arm-optimized EDA tools, global design expertise and foundry partnerships to facilitate easy integration by AI accelerator designers, Arm said.

With the rapid evolution of AI workloads, tightly coupled CPU compute is essential for supporting the complete AI stack. Data pre-processing, orchestration, database augmentation techniques, such as Retrieval-augmented Generation (RAG), and more all benefit from performance-efficiency of Arm Neoverse CPUs. We’ve baked support for these requirements into our CSS and through Arm Total Design, the ecosystem is already benefitting from these innovations.

A new standard for purpose-built AI infrastructure

CSS and Arm Total Design are helping to create the hardware foundation for a sustainable AI datacenter. Arm Total Design is already accelerating the development of Arm-based test chips and chiplet products powered by Neoverse N-series or V-series CSS. The diversity of chiplet solutions spanning cloud to edge and the pace at which they are being developed is a direct result of reducing barriers to entry by enabling broad, preferential access to the latest CSS.

Just today Alcor Micro announced they are building a chiplet powered by CSS and targeting AI/ML training and inference use-cases. And recently Alphawave announced its own advanced compute chiplet built on CSS for AI/ML, HPC, datacenter and 5G/6G applications. These Arm-based chiplets exemplify the diversity, flexibility and global supply chain that only the Arm partnership can deliver.

Additionally, Arm Total Design partners including Alphawave, Cadence, and proteanTecs, are validating their third-party IP products with CSS on advanced nodes to ensure compliance with Arm specifications and standards. This means partners can build CSS-based custom silicon on leading edge nodes and have a seamless out-of-the-box software experience.

This type of software readiness remains a critical gateway to the potential of AI and for more than 30 years Arm has invested in ensuring software on Arm “just works.”

Today all major frameworks and OSes run on Arm. For the Arm Total Design ecosystem, this means the diverse set of silicon solutions that partners are bringing to market can leverage an equally vibrant and cohesive software ecosystem.

One of the latest examples of this ongoing investment is the introduction of Arm Kleidi technology which optimizes CPU-based inference on Arm to open source projects like PyTorch and Llama.cpp. This is especially important for the Arm Total Design partners who are building CSS-based chiplets for edge AI computing without the need for an accelerator.