Microsoft launches Azure ND H200 v5 series cloud-based AI supercomputing clusters

As AI continues to advance, the demand for scalable, high-performance infrastructure is on the rise. Microsoft is meeting this need by introducing cloud-based AI supercomputing clusters powered by Azure ND H2100 v5 series virtual machines (VMs). Now generally available, these clusters are designed to support the increasing complexity of advanced AI workloads, from foundational model training to generative inferencing. The capabilities of the ND H2100 v5 VMs are already gaining traction, driving adoption among customers and Microsoft AI services like Azure Machine Learning and Azure OpenAI Service.

Benefits of using Azure ND H200 v5 series cloud-based AI supercomputing clusters:

• Boost efficiency with Microsoft's systems approach, featuring eight NVIDIA H200 Tensor Core GPUs.

• Access model parameters faster with increased High Bandwidth Memory (HBM), reducing overall latency.

• Seamlessly handle complex Large Language Models (LLMs) in a single VM, eliminating the need for distributed processing.

• Optimize GPU memory for improved throughput, latency, and cost-efficiency in LLM-based AI workloads.

• Achieve higher batch sizes for better GPU utilization and enhanced performance for both SLMs and LLMs.