The most powerful Supermicro server in Moscow based on AMD Epyc

For many years in the server processor and chipset market, Intel ruled unconditionally with its xeons, but in 2017 AMD finally competed with them with the release of the first line of their new EPYC server processors. Since then, their position has only strengthened, and the lineup has been replenished with a new version of “epics”. AMD supplies its modern processors to many well-known companies, including Dropbox, Hetzner, Linode, CERN, OVHcloud, Scaleway, several universities and smaller organizations, including data centers. Everyone praises the platforms at EPYC and recommends them for organizing and scaling a cluster of high-performance servers.

When the news began to arrive that AMD began to make processors that bypass the
performance and price / quality ratio of Intel processors, we began to think about a separate service on this platform. The choice fell on Supermicro A + Server 2124BT-HNTR… The server consists of 4 nodes, each node has the following specification:

  • Processors: 2x AMD EPYC 7742 64-Core Processor 2.25GHz;
  • Memory: 16x64GB RAM 3200 MT / s;
  • NVMe disks for the cluster: 4x INTEL SSDPE2KE076T8;
  • System disk: 1x INTEL SSDSC2KB240G8;
  • Network controller: AOC-MH25G-m2S2T.

All characteristics

There was no such server in Moscow and Russia, we had to order it directly from
manufacturer. After waiting, which was complemented by some logistical problems associated with COVID-19, the server ended up with us in early June.

Assembly

At first glance, it seemed to us that there was nothing particularly epic about the server, since we have experience working with platforms of 12 nodes for a service Hi-CPU servers… The same platform represents only four independent nodes, but we did not yet understand how much power they give in aggregate.

Tests

After assembly, of course, we wanted to load the server, after all, we are dealing with a monster, in
the amount, which on 4 nodes: 1024 cores, 4TB RAM:


Clickable picture

The performance test results impressed us. The build quality from Supermicro engineers is also pleasing.

Mounting

Rack mounted and started assembling the cluster:

The cluster is assembled on these 4 nodes. In the future, the cluster can be scaled on a similar
equipment or similar from other vendors. The public network of the cluster is located on
interfaces 25Gbps (SFP28), private network also 25Gbps (SFP28).

Software revision

The fundamentals and principles laid down in the customer service management software made it possible to refine the code for the new architecture in a short time, in mid-June the service Epic servers was launched.

Benefits for us and customers

AMD EPYC allows you to create tariffs up to EPIC 128 x 3.4 GHz CPU / 512 GB RAM / 4000 GB NVMe, which no one can offer on a platform with processors from Intel. In addition, we use Only NVMe NAS with triple data replication.

For 2 months of working with this platform, we felt all its charms – you can provide customers with the same resources as in the case of Intel, but for less money. One node with 2 CPUs from AMD replaces 10 nodes with 2 CPUs Intel Scalable Silver.

A spoon of tar

Such a server is very expensive – the cost is about $ 150,000 and, of course, it was bought on full prepayment, in fact, we were buying a pig in a poke since we had not previously worked with AMD servers.

The second point, QEMU-KVM does not allow migrating virtual machines from AMD to Intel and vice versa on the fly, but using our functionality backups you can switch from one type of server to another without data loss at any time.

The cluster showed us previously unknown performance indicators, which is natural
began to affect the temperature of processors. Even with a 30% processor load on the node, the temperature was approaching 95 degrees. We have optimized the cooling somewhat in
rack, but it did not give any significant result. Then, in the form of a test, they removed the cover on the server case:

The downward temperature jump was more than 10 degrees and does not exceed 80 degrees, even with
load more than 70%, which more than satisfies us. There are also plans to put additional cooling in the rack.

Outcome

As a result, the server more than met our expectations, the processor AMD EPYC 7742 it seems
us unkillable in the load. The scheme based on similar solutions will be further
scale, we plan to launch this service in the Netherlands over time. We enjoyed working with EPYC and hope you will enjoy it too!


Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *