NL 110FM-OS: 200GbE LAN Module to Accelerate Data Throughput

The Trend

Data drives the modern always-connected society. From a simple search for your favorite recipe on your mobile phone or connecting with friends through social media to powerful business intelligence tools and interconnected web apps, almost every facet of our digital lives hinges on fast and efficient data transfer. But now, the data centers that drive those applications are straining to keep up with the data load.

The typical corporate data center, usually designed to serve a single business or corporation's needs, is often just a part of an organization’s other functions and not designed with the flexibility to scale up and meet increasing bandwidth demands. As they’ve grown in size and complexity, they’ve created a massive overhead for businesses that now find themselves burdened by resources needed to maintain and upgrade those data centers, diverting critical attention away from their core business expertise into IT. Many enterprises are moving to a cloud or hybrid cloud solution to alleviate this problem and reduce the burden on their corporate data centers.

Cloud services offered by big names in the field provide dedicated data centers that allow organizations to take advantage of the scale of the cloud service’s infrastructure and in essence, outsource the majority of the IT infrastructure workload to the provider. The cloud provider handles the upgrades and maintenance, while the corporation pays an ongoing fee depending on how many resources they use. These cloud services are amongst the offerings of the world’s biggest data-producing companies like Amazon, Google, Facebook, IBM, and Microsoft, an offshoot of technologies those companies developed for their own needs and now offer out as a service. To ensure that their infrastructure can flexibly grow, those world-leading companies use a new style of data center called a hyperscale data center.

Hyperscale data centers are bigger than enterprise data centers and house thousands of servers in thousands of square meters of facilities dedicated to those servers. Hyperscale data centers benefit from economies of scale and custom engineering to vastly outperform the typical enterprise data center. Most hyperscale data centers are Class 4 data centers, meaning they are fault-tolerant and have 2(N+1) redundancy. They mostly use fiber-optic cables for all connections, apart from perhaps a few copper connections for local networks or control centers. And hyperscale data centers are on the rise.

Hyperscale data centers doubled from 2016 to 2021^[1]. In 2022, Amazon Web Services (AWS), Microsoft Azure, Alibaba Cloud, and Google Cloud Platform (GCP) are forecast to generate 167.3 billion US dollars^[2], with AWS as the largest IaaS and PaaS hyperscale vendor projected to command a 49 percent market share. As these hyperscalers grow and expand, and the data volumes increase, they are looking for faster networking to increase efficiencies and maximize throughput per node by getting more bandwidth over fewer data lanes.

The Challenge

With hyperscale data centers on the rise, there are clear benefits for efficiency and economies of scale. And as unrelenting growth pushes new competitors into the market, it is becoming imperative that providers continue to push forward to adopt new technologies that give them an edge. To gain and maintain that edge, they must keep pace with the general increase in data production and demands for faster speeds. In simple terms, how do you squeeze more bandwidth into the same space?

To keep pace, hyperscale data centers must handle the explosive growth of data needed for cloud services, including PaaS, SaaS, and IaaS, successive generations of Ethernet technologies have sought to increase the data throughput both through the speed per network port and the overall speed of a node on the network. Speed increases are achieved by increasing the number of network ports, the number of lanes per port, or the speed per lane.

Port density: This means more network ports per network module. Adding more ports is a straightforward way to add more bandwidth, but with the limitation on how many ports can physically fit in the space, and the complexity of adding lots of cables.

Data lanes: SFP+ supported a single data lane and QSFP+ quadrupled the data lanes for four times the throughput. QSFP28 and QSFP56 retained the four data lanes and increased the bandwidth per lane. Finally, there is also QSFP56-DD, which increases the lane count to eight.

Modulation technology: New modulation technologies have opened the door to reliable double-bit data transmission. QSFP+ and QSFP28 both use NRZ modulation, which, very roughly, allows single bit transmission so that a 1 or 0 is transmitted and received. QSFP56 utilizes a new modulation technology called PAM4 and allows for double-bit data transmission, so instead of a single-bit at a time, a 00, 01, 10, or 11 double-bit can be sent and received. QSFP-DD is backward compatible with both.

To overcome the physical limitations and increase the overall bandwidth, hyperscalers can look to 200GbE, which combines those technologies to provide the advantage they need to keep ahead of the pack.

NEXCOM Solution

NEXCOM developed the NL 110FM-OS 200GbE LAN module as part of its latest network appliance portfolio. This new 200GbE module meets new market challenges and utilizes the latest network technologies to support the requirements of today’s most advanced networks, delivering more data throughput at higher speeds over fewer data lanes. As an early stage leading-edge technology, this 200GbE module fits perfectly into NEXCOM’s branded range of network appliances where we can roll out new technologies quickly, even before they see widespread adoption in COTS servers.

The NL 110FM-OS features a PCIe Gen4 x16 connection that delivers 16 GT/s data transfer, double the speed of PCIe Gen3. The network connection is enabled through a QSFP56 port that provides 200 Gb/s (4x50 Gb/s) data speeds. And at the heart of the LAN module is the NVIDIA^® ConnectX^®-6 Dx SmartNIC, a world-class, award-winning network adapter that provides best-in-class hardware capabilities that accelerate and secure cloud and data center workloads.

The performance of all NEXCOM’s LAN modules over 10GbE are optimized with the Data Plane Development Kit (DPDK), providing an Environment Abstraction Layer (EAL) that delivers higher computing efficiency and higher packet throughput. The DPDK data plane libraries and network interface controller polling-mode drivers offload TCP packet processing from the kernel to user space, allowing developers to implement powerful functionality into their applications quicker than before.

The NVIDIA^® ConnectX^®-6 Dx Ethernet Smart Network Interface Controller (SmartNIC) is designed to support advanced networking and security for the most demanding cloud and data center workloads. The ConnectX-6 delivers 200 Gb/s throughput with 200 million messages per second at under 600 nanoseconds of latency end-to-end.

The ConnectX^®-6 Dx built into the NL 110FM-OS 200GbE powers a single port of 200 Gb/s connectivity through 50Gb/s (PAM4) SerDes technology and a QSFP56 connector. The ConnectX^®-6 Dx also includes some other useful features for virtualization, privacy, and memory performance.

Virtualization: Supports single-root I/O virtualization (SR-IOV) and VirtIO in hardware to address different application needs. This technology offloads cloud networking workloads and frees up CPU resources for other applications.

Privacy: With data security as a primary concern, Inline encryption and decryption, stateful packet filtering, and other capabilities bring advanced security down to every node with unprecedented performance and scalability.

RDMA: NVIDIA’s ConnectX^® line brings best-in-class RDMA over Converged Ethernet (RoCE), enabling scalable, easy-to-deploy and resilient RoCE solutions. And for data storage the ConnectX^®-6 Dx optimizes a suite of storage accelerators.

The ConnectX^®-6 Dx SmartNIC provides a secure and advanced cloud network interface to accelerate mission-critical data-center applications, such as security, virtualization, SDN, NFV, big data, machine learning, and storage.

Conclusion

For hyperscalers that want to maintain their edge as first-movers it is paramount that they keep up with the latest technology trends that improve performance through lower latency and higher transmission speeds over fewer data lanes. The NEXCOM NL 110FM-OS allows hyperscale data centers to tap into 200 Gb/s network speeds to simplify their network infrastructure and maximize the interface speed per network node and stay ahead of their competition. At the same time, they can leverage the power of the DPDK to implement new applications with better responsiveness and scalability.

The NEXCOM NL 110FM-OS is also an ideal choice for network performance testing. With the latest technologies and high speeds, it is a prime candidate to perform benchmark tests, identify bottlenecks, and increase the efficiency of all networking subsystems.

References
[1] Number of hyperscale data centers worldwide from 2015 to 2021. https://www.statista.com/statistics/633826/worldwide-hyperscale-data-center-numbers/
[2] Worldwide infrastructure as a service (IaaS) and platform as a service (PaaS) hyperscaler market share from 2020 to 2022, by vendor. Hyperscaler cloud market market share 2022 | Statista https://www.statista.com/statistics/1202770/hyperscaler-iaas-paas-market-share/