Azure Maia 100 and Cobalt 100. What these chips mean for the AI industry
At the beginning of the year, information appeared that Microsoft had almost completed the development of its own chip designed to work in the AI industry. The project is called Athena, and its main purpose is to enable chatbots with AI.
The development itselfthe company led since 2019, and secretly (well, almost, because sometimes there were almost random leaks of information about the project). About what happened in the end, what the company’s plans for the chips are, plus what all this will mean for the AI industry, is under the cut.
They exist!
The fact that the company decided to develop its own chips is not particularly surprising. After all, Microsoft, in partnership with other companies, created chips for Xbox and Surface, as described by Rani Borkar, head of hardware systems and Azure infrastructure at Microsoft.
Yes, AI processors from Microsoft are quite real, moreover, they work. Moreover, as far as one can judge, exactly as intended. The corporation has developed not one processor, but two: one for training large language models so as not to depend on chip suppliers, the second for data centers that support the company’s cloud services. Both chips are intended to satisfy both the company’s own needs, Azure data centers, and for provision to corporate clients.
The chips are called Azure Maia and Azure Cobalt. Both are based on ARM architecture and will become more or less widely used in the company’s data centers. As far as we can understand, the company plans to make money by supplying chips to corporate clients. There’s probably good money to be made, judging by the success of Nvidia, which also supplies high-performance chips for AI services. Graphics chips from Nvidia quickly became a scarce commodity; now retail prices for them reach $60,000 US and even higher.
It is not yet clear how much the chips from the Redmond corporation will cost, but it is clear that it is a lot. As for Azure Maia 100, this development is designed to optimize work with large language models, which are the basis for a wide variety of AI applications. The chip is manufactured using a 5nm process technology and contains 105 billion transistors, which is about 30% less than the AMD MI300X AI chip. It supports the company’s first implementation of eight-bit data types (MX data types) for hardware and software co-design.
Maia is Microsoft’s first liquid-cooled server processor. It will allow for higher server density. According to Microsoft, this is not just an optimization, the company is redesigning the concept of its entire IT infrastructure.
The corporation is now testing this processor both with the Bing search engine and its AI elements, and with Office AI products. In addition, the same chip is also being tested by the Open AI company, about which a lot has been written lately. And not least in connection with the interaction between Open AI and Microsoft.
The Cobalt 100 processor, as mentioned above, is designed to optimize the operation of cloud services on the Azure platform. It is a 128-core chip built on Arm’s Neoverse CSS design and customized by Microsoft. The corporation plans to massively use this ARM chip in its data centers, which will increase overall performance and reduce power consumption.
“We thought a lot about not only how to achieve high performance, but also about energy management,” explains Borkar. “We made several thoughtful design decisions that added the ability to control performance and power consumption per core on each individual virtual machine.”
The chip is also being tested primarily in Microsoft Teams and Microsoft SQL Server. In 2024, the corporation plans to begin selling access to virtual machines on Cobalt to corporate clients. According to company representatives, systems based on the new processor are 40% more productive in some tasks than the systems currently used in Microsoft data centers.
In addition to working on the chips themselves, the company is deeply optimizing its cloud server stack to increase performance and power while reducing the cost of operating such systems. The company’s developers said that they had thought about the ability to control performance and power consumption for each of the cores and each individual virtual machine.
Well, processors and processors, what is it?
The company operates in the AI industry not herself, it is part of a group along with such giants as AMD, Arm, Intel, Meta, Nvidia and Qualcomm. Together, the participants are trying to standardize the next generation of data formats for AI models. As before, Microsoft uses the Open Compute Project (OCP) in its work.
Thanks to new chips and systems based on them, the company plans to quickly deploy servers that provide AI services without the need to allocate a separate space in data centers around the world. A special rack has been developed specifically for this equipment, which contains an individual coolant cooler.
Maia-based hardware is already being tested against GPT 3.5 Turbo (the same model as ChatGPT), Bing AI and GitHub Copilot workloads. All developments obtained by the company during testing will be provided to partners along with data on server equipment, operating parameters, etc.
Unfortunately, there is no testing data in the public domain yet; only the information provided by Microsoft is available there. But it is already known that right now work is underway on the second generation of chips.
If the project with the new IT infrastructure proceeds as planned, then all this could have a strong and lasting impact on the future of artificial intelligence technologies. This is reinforced by the fact that the corporation is very closely associated with OpenAI, which is now one of the leaders in AI solutions.
