IDC

February 2022

Taking Hyperscale Networking to the Enterprise Datacenter

Brad Casemore

Brad Casemore

Research Vice President, Research Vice President, Datacenter and Multicloud Networks

Product Type:
IDC: White Paper
Sponsored by: Dell Technologies
Binary data on monitor screen panels in data center

IDC Opinion

As enterprises continue to pursue strategic initiatives associated with digital transformation, they increasingly find that modernization of datacenter infrastructure is essential to the realization of their goals. Nowhere in the datacenter is that more necessary than in relation to the network, where traditional vertically integrated switches — with their closed architectures, lack of open APIs for automation and programmability, and dependence on manual configuration and management — often impede enterprise efforts to make infrastructure as agile and flexible as that of the public cloud.

The rise of network disaggregation, including Linux as a network operating system (NOS), offers encouragement to enterprises seeking a more modern and automated approach to datacenter networking. Many hyperscalers have adopted this approach, including Microsoft, which deploys Software for Open Networking in the Cloud (SONiC), an open source Debian-based Linux NOS, on the switches in its datacenters to support the scale, agility, flexibility, programmability, and performance that its applications and services require.

SONiC has considerable potential beyond the realm of hyperscale environments. For it to be able to succeed in the wider world of enterprise datacenters, SONiC had to be adapted for the needs of enterprise customers. With its Enterprise SONiC Distribution by Dell Technologies, Dell has sought to adapt and enhance SONiC for enterprise customers and their environments.

Situation Overview

As the lifeblood of digital business, applications are responsible for generating revenue, creating competitive advantage, and providing customers and employees with engaging digital experiences. It is no wonder that applications are increasingly defining and driving decisions relating to datacenter infrastructure. To an unprecedented degree, enterprise IT buyers seek truly digital datacenter infrastructure that aligns with and supports applications through automation, programmability, and greater flexibility.

For many organizations, the datacenter network is often the long pole in the tent, the element of datacenter infrastructure that inhibits the agility and flexibility of business-critical applications. Digitizing organizations want their datacenter network to be more like their datacenter servers, which increasingly run a Linux stack and are extensively automated, resulting in streamlined operations, consolidated tooling, and adherence to DevOps principles.

The network can and must be agile and flexible, but the traditional network operating system — a closed, monolithic code base, vertically integrated with switch hardware and interfaces — does not accommodate application-centric datacenter automation or modern cloud-native applications (built with containers and microservices). As a result, the network has been unable to benefit from continuous innovation and the community support of a broader open source ecosystem.

Although networking was slow to embrace Linux and other open source software, developments in the past few years have resulted in significant new opportunities for enterprises to benefit significantly from both. In the realm of datacenter network operating systems, SONiC has the potential to bring hyperscale efficiencies to enterprise environments.

Brief Overview of SONiC

SONiC is an open source Linux network operating system, first released by Microsoft in 2016 and contributed to the Open Compute Project (OCP) that same year. It is deployed widely in hyperscale datacenters at Microsoft and increasingly at other hyperscale and cloud providers worldwide. In fact, SONiC has been adopted by tier 2 cloud operators, webtechs, and digital natives, as well as communications service providers, cable multiple system operators (MSOs), and a growing number of large enterprises. Consequently, IDC forecasts that SONiC-based Ethernet datacenter switches will account for worldwide revenue of approximately $2.5 billion in 2025.

Initially deployed on top-of-rack Ethernet switches in cloud-scale datacenters, SONiC has extended its reach to leaf-spine datacenter networks, converged networks, telco clouds, WANs, various edge environments, EVPN multihoming, and other routing use cases such as datacenter interconnect (DCI). SONiC is built atop the Switch Abstraction Interface (SAI), which allows SONiC to run across hardware platforms powered by a wide range of network silicon, giving customers flexibility and choice. SONiC supports streaming network telemetry and works with configuration automation tools such as Ansible, Puppet, and Chef.

Although SONiC originated in the cloud-scale datacenter network of a hyperscaler, its evolution has dovetailed with market dynamics to make it increasingly applicable to enterprise customers. Enterprises aren’t hyperscalers, but digital transformation compels datacenter network modernization that is driven by needs similar to those that first emerged at hyperscalers.

Benefits of an Enterprise SONiC Distribution

An enterprise SONiC distribution can deliver a wide range of business and operational benefits: 

  • Reduced cost of datacenter networks:
    • Capex and opex IT cost reductions are possible with SONiC, which provides choice and flexibility in the selection and procurement of datacenter network infrastructure and allows for a greater degree of programmability and API management than traditional network operating systems.
  • Accelerated network provisioning, allowing self-service for known repetitive processes associated with developer and DevOps pipelines
  • Faster ROI on networking investments, resulting from cost savings and operational efficiencies (as noted previously)
  • Continuous innovations and contributions from the open source SONiC community that improve operational efficiency, performance, and productivity of the software and datacenter networking
  • Long-term investment protection, resulting from the inherent flexibility of SONiC and the aforementioned continuous innovations and contributions of the SONiC community
  • Greater operational efficiencies, deriving from feature standardization and consistency across multivendor network environments
  • A modern microservices-based Linux NOS that works well with popular Linux tooling and processes
  • Cloud-native applications, built with containers and leveraging microservices, allowing for continuous integration and continuous development (CI/CD)
  • Support for pipelines and agile workflow-based processes, allowing the network to move in lockstep with developers at the speed required by digital business:
    • By leveraging a modern Linux network operating system, the network and its processes can better align with the needs of developers. As such, the network becomes an enabler of cloud-native agility and flexibility rather than a siloed inhibitor to faster application development and delivery.
  • Delivers network support for modern applications and their requirements, including cloud-native Kubernetes environments
  • Compatible with popular Linux automation tools, including Ansible, allowing tool consolidation across datacenter infrastructure:
    • IT generalists with Linux backgrounds are allowed to function in NetOps roles.
  • Facilitates network programmability and day 2 troubleshooting and optimizations, including rapid change management
  • Provides compatibility with open source logging and telemetry tools

An Enterprise SONiC: Enterprise SONiC Distribution by Dell Technologies

Dell has developed its own enterprise-oriented SONiC distribution designed and developed to address enterprise use cases and markets beyond hyperscale environments. Enterprise SONiC Distribution by Dell Technologies is based on open source SONiC, providing all the benefits associated with the innovations in the SONiC community, with enhancements and extended for enterprises and their datacenter environments. Dell’s objective is to provide an enterprise SONiC experience from procurement to deployment to support, software development, testing, and solution validation.

Dell has a long history and familiarity with SONiC. In 2015, Dell and Microsoft jointly submitted the SAI to the Open Compute Project. Since then, Dell has further adapted and enhanced SONiC for the enterprise market, resulting in Enterprise SONiC Distribution by Dell Technologies. Dell retains its status as a leading SONiC contributor — having provided more than 1 million lines of feature code and more than 5,000 bug fixes — and is dedicated to contributing future SONiC enhancements to the community.

Enterprise SONiC Distribution by Dell Technologies is distinguished by the following capabilities: 

  • Support for heterogeneous environments. Enterprise SONiC can operate across network hardware and work with a range of higher-layer automation and orchestration tools like Ansible. Dell SONiC’s container-based microservices architecture and Switch Abstraction Interface also provide for multivendor openness in network functions, network silicon, and switch environments.
  • Agility. Feature velocity, including faster upgrades with no downtime, enabled by capabilities such as nondisruptive warm reboot and user container management, allows new features and functionality to be added through modular container upgrades, thus removing the need for a complete system-level software upgrade.
  • Flexibility and extensibility. Dell SONiC can provide the programmable flexibility to evolve in support of new features and capabilities.
  • Reliability. Robustness and integrity are provided partly through eliminating the “code bloat” associated with traditional monolithic NOSs. In addition, software testing, solution validation, and validations of solutions from ecosystem partners further contribute to reliability.
  • Standardization. Leveraging industry-standard network protocols and standards, such as Free Range Routing (FRR) and OpenConfig, provides added flexibility and choice in network functions and tools.
  • Enterprise-grade service and support. Dell offers 24 x 7 support for the entire Dell SONiC solution stack, inclusive of hardware, software, and validated technologies from ecosystem partners.

Designed for Enterprise Needs

The Dell SONiC distribution includes regularly optimized features that support Layer 2 and Layer 3 topologies, container networking (with CNI plug-in support for Kubernetes), and usability features such as MF-CLI and GUI, providing suitability for a range of enterprise IT teams.

Dell’s SONiC is currently positioned to address three key use cases in datacenter networking: 

  • A flat, highly scalable, leaf-spine Clos datacenter fabric (underlay) using Layer 3 Border Gateway Protocol (BGP) is designed to support modern cloud-native applications, such as Kubernetes container environments, similar to hyperscale datacenter networks but adapted for enterprise use.
  • A highly scalable, leaf-spine Clos datacenter fabric enables Virtual Extensible LAN (VXLAN) tunneling of Layer 2 traffic over a Layer 3 Border Gateway Protocol Ethernet VPN (BGP-EVPN) network for enterprise datacenters that must support both traditional and modern applications.
  • Bare metal automation provides automated provisioning and closed-loop network automation for bare metal environments.

Forthcoming support also will be offered for use cases such as datacenter interconnect and edge environments, both of which benefit from Dell SONiC’s automation, standardization, and API-based extensibility. In addition, SONiC works well as a switch OS in datacenter networks where SDN overlays such as VMware’s NSX are deployed.

In addition to the use cases and capabilities discussed previously, Dell provides packaging and services designed to enhance SONiC’s appeal and applicability to enterprise needs: 

  • Enterprise customers can choose one of the two bundles (Cloud or Enterprise), depending on which is best suited to their architectural needs and environment.
  • Professional services and support for migration and deployment are available.
  • Validated integrations are supported, along with cabling and optics.
  • Ecosystem partner offerings are available, including support for popular open source projects such as OpenStack, Prometheus, and Telegraf.
  • Enterprise SONiC Distribution by Dell Technologies also comes with customer use case validations, training, hands-on labs, and enterprise documentation.
  • Functionality and support for network life-cycle management from day 0 through day 2/day n, from design and deploy through to troubleshooting and remediation, change management, and network optimizations, are included.

Dell also has established several ecosystem partnerships to Enterprise SONiC Distribution by Dell Technologies. Those partnerships extend across a range of open source and third-party tools and platforms, including validated integrations, providing enterprise-grade tools that leverage SONiC’s inherent automation and telemetry capabilities. These tools include predictive analytics using AI/ML, which enable network operators to adopt predictive and proactive monitoring rather than reactive approaches. Other tools facilitate declarative intent-based configuration management, while some allow customers to manage multivendor network environments, including heterogeneous hardware and software, through a common controller or orchestrator.

Challenges

Although Linux is long established as an operating system for servers in the datacenter, datacenter networking has traditionally featured vertically integrated switches — often including a proprietary network ASIC, a proprietary network operating system, and proprietary interfaces. Even today, enterprises tend to procure these traditionally architected switches, although a considerable proportion are now powered by merchant Ethernet silicon rather than proprietary ASICs.

For enterprise IT buyers and network operators, cultural inertia, including fear of and resistance to change, can be a major factor governing selection and procurement of datacenter networking technologies. This reticence to change, and the reluctance to adopt new technologies, represents a challenge that organizations must overcome if they are to embrace network disaggregation and open source, Linux-based network operating systems such as Enterprise SONiC Distribution by Dell Technologies.

Still, even as many enterprises tend to resist change, others are seeking to emulate the datacenter networking best practices and technology choices of hyperscalers and other cloud giants, most of which have adopted Linux-based network operating systems, including SONiC, in unprecedentedly demanding and scalable environments. The challenge here is for vendors such as Dell to ensure that SONiC is adapted for enterprise deployment, requirements, and ongoing management and support.

Conclusion

Networking need not be a drag on the agility and speed of enterprise datacenter infrastructure. A modern approach to datacenter networking, which puts long overdue emphasis on agility and flexibility, must include a modern network operating system. Linux, which has already proven its mettle on datacenter servers, is well placed to extend its mandate to datacenter switching. In the form of SONiC, Linux has already achieved that goal in hyperscale datacenters, the most scalable and demanding networking environments in the world.

SONiC was born in some of the world’s largest datacenters, supporting networking at unprecedented levels of reliability and scale. It was also built with automation as a core capability, which allowed it to deliver operational agility and flexibility, along with simplified network management, including streaming telemetry for enhanced fabric visibility and actionable network insights.

Enterprise SONiC Distribution by Dell Technologies has been designed to build on SONiC’s already considerable strengths by extending its feature set and enhancing its usability for enterprise-centric requirements and use cases. If Dell can overcome the challenges discussed in this paper, the company is well positioned to bring SONiC to new markets and to help enterprises turn their datacenter networks into integral elements of next-generation digital infrastructure.

Taking Hyperscale Networking to the Enterprise Datacenter