Cisco Catalyst 9000 Switches

A new era of networking

2nd edition
Preface 7
Authors 8
Acknowledgements 10
Organization of this book 11
Intended audience 12
Book writing methodology 13
What is new in this edition of the book? 14

Introduction 15
Executive summary 16
Industry trends 19
Business use cases 21

Catalyst 9000 family of switches 25

Overview 26
Catalyst 9200 Series switches 29
Catalyst 9300 Series switches 33
Catalyst 9400 Series switches 38
Catalyst 9500 Series switches 47
Catalyst 9600 Series switches 54
Packaging, licensing and support 60

ASICs - the power of programmable silicon 65

What is an ASIC? 66
Why programmable ASICs? 69
Cisco UADP - programmable ASIC silicon 73
The Cisco UADP family 79
Cisco IOS XE 85
Cisco IOS evolution 86
Cisco IOS XE architecture 89
Cisco IOS XE benefits 92

High availability 95
Overview 96
Catalyst 9200 Series high availability 98
Catalyst 9300 Series high availability 102
Catalyst 9400 Series high availability 108
Catalyst 9500 Series high availability 115
Catalyst 9600 Series high availability 116
StackWise Virtual 120
Graceful insertion and removal 126
Patching Cisco IOS XE 129

Security and identity 131

Overview 132
Encrypted traffic analytics 133
Trustworthy solutions 136
MACsec encryption 139

QoS and queuing 145

Overview 146
Buffers and queues 148
QoS and queuing in the Cisco UADP ASIC 153
Hierarchical QoS 159
QoS for StackWise Virtual 164
QoS for overlay technologies 165

Application visibility and control 169

Overview 170
Application recognition 171
Application control 174

IoT 175
Overview 176
Power over ethernet innovations 177
Audio video bridging - AVB 180
Cisco DNA Service for Bonjour 182

User-centric platform design 185

Overview 186
RFID 187
Blue beacon 188
Bluetooth console 189
WebUI 190
Flexible templates 191

Programmability and automation 193

Overview 194
Device provisioning 196
Open programmable device APIs 198
Model-driven telemetry 208
Scripting 211
Configuration management tools 213
Cisco DevNet 215

Application hosting 217

Application hosting operation 218
Hardware resources 220

Campus network design 221

Overview 222
Physical infrastructure 225
Multi-layer campus 230
Collapsed core 233
Routed access 235
Campus MPLS 237
Campus wireless 239
Software-Defined Access 243

Appendix 247
References 248
Acronyms 250
Preface
8 Preface

Authors

This book represents a collaborative effort between Technical Marketing, Product

Management, Engineering, and Sales teams during a week-long intensive session at
Cisco Headquarters in San Jose, CA.

• Bob Sayle - Sales

• Dave Zacks - Technical Marketing

• Dimitar Hristov - Technical Marketing

• Fabrizio Maccioni - Technical Marketing

• Ivor Diedricks - Product Management

• Jay Yoo - Engineering

• Kenny Lei - Technical Marketing

• Mahesh Nagireddy - Technical Marketing

• Minhaj Uddin - Technical Marketing

• Muhammad Imam - Technical Marketing

• Sai Zeya - Technical Marketing

• Shawn Wargo - Technical Marketing

Another group of authors contributed to the most recent update to this book, which
was completed in April 2019.

• Gary M Davis - Customer Experience

• Ginger Liu - Product Management

• Jay Sharma - Technical Marketing

 Preface 9

• Jeffrey Meek - Marketing

• Kenny Lei - Technical Marketing

• Minhaj Uddin - Technical Marketing

• Sai Zeya - Technical Marketing

• Shawn Wargo - Technical Marketing

10 Preface

Acknowledgements

A special thanks to Cisco’s Enterprise Networking Business Product Management,

Engineering, and Sales teams who supported the realization of this book. Thanks to Carl
Solder and Muninder Sambi for supporting this effort. We would also like to thank
Cynthia Resendez for her exceptional resource organization and support throughout
our journey, and Sehjung Hah for his help with planning and logistics.

We are also genuinely appreciative to our Book Sprints (www.booksprints.net) team:

• Adam Hyde (Founder)

• Barbara Rühling (CEO and Facilitator)

• Faith Bosworth (Facilitator)

• Henrik van Leeuwen and Lennart Wolfert (Illustrators)

• Juan Carlos Gutiérrez Barquero (Technical Support)

• Agathe Baëz (Book Producer)

• Raewyn Whyte and Susan Tearne (Proofreaders)

Barbara and the team created an enabling environment that allowed us to exercise our
collaborative and technical skills to produce this technical publication to meet a
growing demand.

Cover photo: Satpal Ranu

 Preface 11

Organization of this book

This book is best read in the order presented. However, based on the roles of the reader
and their interests, some chapters can be reviewed out of sequence. The book is
organized into sections, with each section having multiple chapters.

First we introduce the Cisco® Catalyst® 9000 switching family, review the business
drivers for enterprises, and illustrate how Catalyst 9000 switches address the
challenges faced by enterprise IT. Next we review the architectural foundations of the
Catalyst 9000 switching platform, both from a hardware perspective with the
innovative Cisco Unified Access Data Plane (UADP) ASIC, as well as the cutting-edge
capabilities provided by Cisco IOS® XE software. These foundational elements enable
the Catalyst 9000 switching family to address the many demands placed on enterprise
networks today.

How Catalyst 9000 switches meet these demands is outlined in the next sections
covering high availability, security, quality of service, application visibility and control,
IoT and user-centric platform design. Cisco IOS XE software brings an open, standard
and model-based approach to network management interfaces. These capabilities are
reviewed in the chapter Programmability
Programmability and automation Then an examination of
and automation
automation.
application hosting on the Catalyst 9000 switching family is provided. Finally, the book
examines the present state and future evolution of network design, and how Catalyst
9000 switches lead the way towards the ongoing transformation of enterprise network
architectures.
12 Preface

Intended audience

Network administrators, engineers, and architects are always under pressure to meet
the business needs of their organizations. This book focuses on the innovative Cisco
Catalyst 9000 family of switches, and how they help to solve the many challenges that
networking professionals face today.

Catalyst 9000 switches provide state-of-the-art technologies driven by open, flexible,

and powerful hardware and software. Networking professionals will be able to utilize
this book to understand the Catalyst 9000 switching family, delve deep into its
architecture, and understand how it provides a strong foundation for next-generation
networks.

This book assists network professionals, IT managers, executives, and anyone with an
interest in the latest and greatest networking technologies to understand and embrace
the new era of networking that the Catalyst 9000 switching family enables.
 Preface 13

Book writing methodology

Fix your eyes on perfection and you make almost everything

speed towards it – W.E. Channing

Simplicity, consistency and performance have been overriding themes in designing the
Catalyst 9000 switching family. The idea of this book is to present readers with the
current challenges in enterprise networking and explore how the Catalyst 9000
switching platform solves these challenges. Catalyst 9000 switches provide cutting-
edge hardware and software capabilities, easily adapting to future protocols and
network architectures without losing sight of simplicity and security. This book
explores this powerful next generation networking platform - the basis for the new era
of intent-based networking.

A group of Cisco Engineers from diverse backgrounds accepted the challenge of writing
a book about a platform that changes the paradigm of enterprise networking. At the end
of day one, the task seemed even more daunting, given the breadth of capabilities that
Catalyst 9000 switches bring to networks. However the team persisted, and after
hundreds of hours of diligent penmanship, this book was born! The Book Sprints (www.
www.
booksprints.net methodology captured each of our unique strengths, fostered a team-
booksprints.net
booksprints.net)
oriented environment, and accelerated the overall time to completion.

#Cat9K
#CiscoDNA
#NewEraOfNetworking
14 Preface

What is new in this edition of the book?

This book has been updated to reflect several of the new and improved features that are
available with Cisco Catalyst 9000 switches.

Here are the highlights of this revision:

Catalyst 9200 Series switches - the latest addition to the Catalyst 9000 fixed
enterprise access-layer switching portfolio. The Catalyst 9200 Series offers full PoE+,
power and fan redundancy, stacking bandwidth support up to 160 Gbps, fixed or
modular uplinks, and Layer 2 and Layer 3 features. This includes updates to multiple
chapters, including UADP 2.0 mini, Cisco IOS XE Lite, high availability
(StackWise®-160/80), user design, security, QoS, etc.

Catalyst 9600 Series switches - the next generation purpose-built 40G/100G modular
enterprise core/distribution switching platform. The Catalyst 9600 Series provides
security, resiliency and performance at scale with a comprehensive set of industry
leading, Layer 2 and Layer 3 features. This includes updates to multiple chapters,
including UADP 3.0, high availability (SSO and StackWise® Virtual), user design,
security, and QoS.

XFSU - Extended FSU builds on the standalone Catalyst 9300 Series FSU capability and
provides a mechanism to upgrade and downgrade the software image by segregating
the control plane and data plane update, to allow less than 30 seconds of traffic impact
during the upgrade.

gRPC - gRPC remote procedure call is an open source remote procedure call (RPC)
system initially developed by Google.

The book's revised edition of the Catalyst 9000 switching family addresses all of the
above areas and capabilities.

Continue reading to know more!

Introduction
16 Introduction

Executive summary

The world is rapidly changing. The demands of ubiquitous mobility, evolving IoT, cloud
adoption and rapidly advancing security threats are making IT managers rethink how
their networks are designed and implemented. Enterprises of all sizes around the world
are replacing their legacy systems with new and evolving digital technologies to create
a competitive advantage, enable higher productivity and lower operating costs. As more
businesses embrace this change, networks have to adapt. Businesses cannot build
networks the same way they have for the past 30 years. Organizations need to create
flexible networks that can constantly learn, adapt, protect and evolve.

A new era of intent-based networking

Cisco Digital Network Architecture (Cisco DNAtm) and Cisco Software-Defined Access
(SD-Access) help organizations unlock opportunities, enhance security, be more agile,
and operate more efficiently. Designed to be intuitive, the network recognizes intent,
mitigates threats through segmentation and encryption, learns and adapts over time.
The Cisco Catalyst 9000 family of switches are the next generation in the legendary
Catalyst portfolio of enterprise LAN access, distribution and core switches. These are
the first purpose-built platforms designed to take advantage of Cisco DNA and SD-
Access.

Cisco Catalyst 9000 switches have been designed as the foundation for an entirely new
era of intent-based networking. This book explores the Catalyst 9000 family of switches
and examines how these platforms meet the ever-changing needs of the enterprise
network, today and well into the future.

For the first time in the industry, a single family of fixed and modular LAN switches can
run a single software code base with a common ASIC across every platform in campus
and branch networks. Design considerations can now be focused entirely on the scale
requirements for different places in the network. This provides significant reduction in
total cost of ownership (TCO) for enterprise networks.
 Introduction 17

Catalyst 9000 switches are based on two foundational aspects:

• Common hardware - built with a flexible, programmable ASIC and CPU

architecture.

• Common software - built with an open, modular operating system, with simple
feature licenses.

DIAGRAM Foundational attributes of Catalyst 9000 switches

Catalyst 9000 switches are built on a common ASIC architecture powered by the Cisco
Unified Access Data Plane (UADP) ASIC. This serves as an innovative, programmable and
flexible silicon foundation for the platform. The Cisco UADP ASIC enables network
infrastructures to adapt to new technologies, trends and business needs over time.
Catalyst 9000 switches are also built on a standard multi-core 64-bit x86 CPU
architecture. Note that Catalyst 9200 Series switches use an ARM CPU integrated into
the UADP, for greater cost efficiency and lower power consumption. A common CPU
architecture provides predictable software processing and control-plane management,
providing the horsepower to tackle next-generation network architectures and
providing a platform for application hosting.

Every Catalyst 9000 switch runs on the open and modular Cisco Internet Operating
System - Cisco IOS XE. This improves portability across Cisco enterprise platforms
(including Catalyst switches, ISR/ASR routers, access points and wireless LAN
controllers). It increases feature development velocity, improves high availability, and
18 Introduction

makes it easier to consistently deploy features across the campus network. Cisco IOS
XE provides a well-defined set of APIs, improving management and simplifying
automation and programmability.

ᄂthe bottom line

Catalyst 9000 switches are built on common hardware and software, and the
foundation for the new era of intent-based networking.
 Introduction 19

Industry trends

The common trends seen in the industry today fall into four main categories - mobility,
IoT, cloud and security.

Mobility trends and considerations

The need for untethered, uninterrupted access enabled by new wireless and mobility
technologies is driving the enterprise network infrastructure market. BYOD and mobile
applications make it possible for workers to access corporate data from nearly
anywhere from nearly any device. This creates significant challenges for back-end IT
infrastructures and those who manage the network. Mobility is now not just a cost of
doing business, but a strategic business asset, making it an integral part of the future
enterprise network.

IoT trends and considerations

The digital transformation of business processes and operations includes connecting
new devices, sensors, and machines in an effort to improve productivity, reduce risk,
and increase security. Billions of machine-to-machine connections will emerge over the
next several years that require machine learning intelligence based on analytics and
business policy. Enterprise campus networks will be required to support this influx of
machine connectivity.

The Cisco “Internet of Things: Workloads and Key Projects 2017” survey predicts
organizations will undertake IoT data aggregation, filtering, and analysis at the network
edge. The primary drivers for processing IoT data at the network edge are to improve
security and speed up data analysis. The network needs to evolve to support the
current and future demands of IoT.
20 Introduction

Cloud trends and considerations

Enterprises are augmenting internal IT with cloud services, be it on-premises,
collocated private cloud, or public cloud services. Cloud infrastructures mix virtual and
physical elements, with workloads moving between on-premises and off-premises
resources. Campus networks have to not only interface with off-premises and public
clouds but ensure the same application performance, security, and policy adherence for
those workloads as if they were still on-premises.

Security trends and considerations

All of these new connections open up profound security implications. Each new
connection is a potential attack vector. Attacks are becoming more and more
sophisticated, and worse, they are often obscured via encryption. Campus networks
must be able to secure these new connections by detecting anomalies and recognizing
potentially malicious behaviors and patterns in real-time at scale.
 Introduction 21

Business use cases

Catalyst 9000 switches extend Cisco networking leadership with breakthrough

innovations in mobility, IoT, cloud and security.

Enabling the mobility use case

Wired and wireless networks have historically been built and operated by different
teams. Catalyst 9000 switches with SD-Access deliver central orchestration and the
assurance of a single, integrated wired and wireless network. This allows the network to
scale seamlessly without bottlenecks as more wireless clients are added to the network.

The SD-Access architecture delivers simplified and seamless roaming for devices across
the network. Catalyst 9000 switches with SD-Access support the embedded wireless
LAN controller capability for small, branch deployment. With SD-Access, access points
connect directly to Catalyst 9000 switches for data plane forwarding directly in
hardware.

Furthermore, SD-Access policies for wired and wireless are the same in this network
architecture. Network segmentation and group-based policies are consistent between
wired and wireless traffic, making operations simple.

The Catalyst 9000 family of switches deliver industry leading multigigabit (mGig) and
PoE capacity allowing customers to build the densest wireless environments, leveraging
Wi-Fi 6 (802.11ax) and 802.11ac Wave 2 and future wireless innovations. For more
information, refer to https://cs.co/wirelessbook
https://cs.co/wirelessbook

ᄂthe bottom line

Catalyst 9000 switches o er the optimal foundation for converging wired and
wireless access.
22 Introduction

Enabling the IoT use case

There are many new devices being connected to the network such as sensors, alarm
systems, HVAC systems, badge readers, etc. which have not traditionally been
connected or have been using proprietary protocols. Catalyst 9000 platforms, together
with Cisco Identity Services Engine (ISE), are able to automatically profile these devices,
provide security and segmentation and apply policies to them.

For example, many devices are starting to advertise their services using the Bonjour
(mDNS) protocol. Cisco DNA Service for Bonjour delivers visibility to these services
across locations and segments of the network, assigns policy based on these services,
and orchestrates all of this from a centralized point with Cisco DNA Centertm.

Some IoT devices, such as LED lighting, require 'always-on' power over ethernet (PoE).
Catalyst 9000 switches support perpetual PoE and fast PoE to keep the lights on, even
while the switch reloads.

For professional media and audio applications, Catalyst 9000 access switches support
audio video bridging (AVB) and IEEE 1588 timing.

ᄂthe bottom line

Catalyst 9000 switches are the ideal platform for connecting the Internet of
Things.

Enabling the cloud use case

In order to make deployment and operation of the network more agile, Cisco has added
a programmatic framework and tools to drive use of automation through NETCONF,
RESTCONF, and gNMI APIs with YANG models.

Streaming telemetry facilitates near-real-time visibility to operational data.

Catalyst 9000 switches support application hosting with local storage enabling fog
computing and network function virtualization. This supports distributed intelligent
 Introduction 23

agents embedded into the network for analytics, assurance, security, and cloud-
connected applications. Customers are able to host third-party applications on Catalyst
9000 switches, making this the most flexible platform in the industry. For more
information, refer to https://cs.co/programmabilitybook
https://cs.co/programmabilitybook

ᄂthe bottom line

Catalyst 9000 switches are an open and fully programmable platform for enabling
the move to cloud.

Enabling the security use case

There are a diverse and growing set of devices connecting to enterprise networks.
Network segmentation may be used to constrain devices and users so that
communication is only possible once allowed. Catalyst 9000 switches support
numerous segmentation capabilities at a macro (network segment) and micro (user or
device group) level with support in hardware for SD-Access, MPLS, VRF-Lite, and
TrustSec.

As more network traffic is becoming encrypted, it is critical that these threats are
detected and mitigated at the point where it connects to the network. Catalyst 9000
switches can detect and mitigate malware hiding in encrypted traffic using encrypted
traffic analytics (ETA). Even better, ETA detects anomalies in encrypted traffic without
decrypting it.

Catalyst 9000 switches collect metadata in hardware about all flows traversing the
network, using full Flexible NetFlow. Combining this with Cisco security solutions, such
as Cisco Stealthwatch®, provides detection of denial-of-service attacks and other
malicious activity.

With the Catalyst 9000 family of switches, the links between switches can be encrypted
using up to 256-bit AES MACsec, operating at line rate. This encryption can also be
used for connections between the switch and endpoints.
24 Introduction

Finally, Cisco trustworthy solutions security technology protects the network switches
themselves. A holistic approach provides comprehensive verification of hardware and
software integrity by securing the device, network communications, and hosted
applications.

ᄂthe bottom line

Catalyst 9000 switches provide the most secure switching environment in the
networking industry.
Catalyst 9000 family of
switches
26 Catalyst 9000 family of switches

Overview

The Cisco Catalyst 9000 switching family is the next generation in the legendary Cisco
Catalyst portfolio of enterprise LAN access, distribution, and core switches. Catalyst
9000 switches extend Cisco networking leadership with breakthrough innovations in
mobility, security, IoT, and cloud.

Designed from the ground up for higher performance, greater flexibility, security and
resiliency, Catalyst 9000 switches start with feature parity from day one, while adding
many new features and functionality. Leveraging the programmability of the Cisco
UADP ASIC and Cisco IOS XE as well as the policy-based automation and assurance of
Cisco Software-Defined Access, the Catalyst 9000 switching family addresses the
challenges of today's always-on network, to help you focus on the needs of the
business, not on the network - now and in the future.

DIAGRAM Catalyst 9000 family of switches

Catalyst 9000 switches are built on a common architecture with a strong hardware and
software foundation. This commonality and consistency brings simplicity and ease of
 Catalyst 9000 family of switches 27

operations for network architects and administrators, reducing total operational cost
and creating a better user experience.

Common hardware
The Catalyst 9000 switching hardware uses a common design, both internally and
externally.

Internally the hardware uses a common ASIC, the Cisco Unified Access Data Plane
(UADP) ASIC, providing flexibility for packet handling. The hardware also uses a
common control plane CPU. For the first time in the history of Catalyst switches, Cisco
has also introduced an onboard x86-based CPU to allow the switch to host additional
applications (beyond those normally possible on a network switch).

Externally the hardware is designed by one of the best industrial design firms in the
world - Pininfarina, designer of the famous Ferrari. This level of design focus brings an
enhanced user experience for the Catalyst 9000 switching family. It provides
ergonomic design and common attributes that simplify device operations. More details
are provided in the chapter User-centric
User-centric platform
platform design
design.
design

Common software
The Catalyst 9000 family of switches run a common operating system, the Cisco
Internet Operating System IOS XE. Cisco IOS XE is an enhanced, open and
programmable OS. With a 30 year history and thousands of features, Cisco IOS XE is
arguably the most feature-rich OS in the networking industry. Having a common code
base shared across Catalyst 9000 switching platforms enables end-to-end feature
support and feature parity throughout the network.

The strong hardware and software foundation of the Catalyst 9000 family of switches
enables it to face the challenges of the modern enterprise network, while bringing
consistency and simplicity for customers.

The Catalyst 9000 switching family has five members, broadly segregated into two
types of network design models:
28 Catalyst 9000 family of switches

• Simple branch deployment

• Business-critical deployment

More details are provided in the chapter Campus

Campus network
network design
design.
design

• Catalyst 9200 Series - fixed, stackable access (simple branch)

• Catalyst 9300 Series - fixed, stackable access (business-critical)

• Catalyst 9400 Series - modular access and distribution (business-critical)

• Catalyst 9500 Series - fixed core and distribution (business-critical)

• Catalyst 9600 Series - modular core and distribution (business-critical)

These platforms are discussed in further detail in the following chapters.

 Catalyst 9000 family of switches 29

Catalyst 9200 Series switches

Catalyst 9200 Series switches focus on offering right-sized switching for simple branch
deployments, extending the power of intent-based networking and Catalyst 9000
hardware and software innovations to a broader set of deployments. With its family
pedigree, Catalyst 9200 Series switches offer simplicity without compromise – it is
secure, always on and IT simplified. Catalyst 9200 Series switches offer full PoE+
capability, power and fan redundancy, stacking bandwidth up to 160 Gbps, modular
uplinks, Layer 3 feature support and cold patching. Catalyst 9200 Series switches are
purpose-built for cost-effective branch-office access.

DIAGRAM Catalyst 9200 Series switches

Platform overview
All Cisco Catalyst 9200 Series switches are fixed configuration 1RU switches with dual
power supplies and redundant fans. The Catalyst 9200 Series offers two model options:
Catalyst 9200 switches with fixed uplinks and fans (C9200L SKUs) and Catalyst 9200
switches with modular uplinks and fans (C9200 SKUs), offering various levels of scale
and redundancy. Both models have 24 and 48 port copper options with three
configurations:

• Data-only 1G models - optimized for devices that primarily require data

connectivity such as printers and desktop computing. All ports in these models
support 10/100/1000Mbps.
30 Catalyst 9000 family of switches

• PoE+ 1G models - provide all capabilities of the data-only models with added
support for PoE (15.4W) and PoE+ (30W) power to devices such as access points,
IP phones capable of consuming power over ethernet. All ports can provide PoE+
power simultaneously with dual power supplies.

• Multigigabit models - provide multiple speeds up to 10Gbps on mGig ports,

supporting Wi-Fi 6 and 802.11ac Wave 2 access points and desktops that demand
higher speeds above the 1Gbps. All ports (mGig and 1G) on these models are
PoE+ capable. Full PoE+ is supported with dual power supplies.

Architecture
Catalyst 9200 Series switches have been optimized for a simple branch deployment
with an architecture that is simple yet powerful. Catalyst 9200 Series switches also
have an embedded 4-core ARM CPU on the Cisco UADP 2.0 mini ASIC which is utilized
to run the operating system. The Cisco IOS XE operating system has been optimized for
the Catalyst 9200 Series as Cisco IOS XE Lite, providing a smaller image size and faster
boot time accommodating the optimized hardware without compromising on the
benefits of Cisco IOS XE.

Non-mGig models are powered by a single UADP 2.0 mini ASIC, whereas the mGig
models have two UADP 2.0 mini ASICs. All ports on Catalyst 9200 Series switches are
line rate for all packet sizes.
 Catalyst 9000 family of switches 31

DIAGRAM Series switch architecture

StackWise-160/80
Catalyst 9200 Series switches provide the ability to stack up to eight switches,
providing a centralized control plane while allowing distribution of the data plane. This
allows network engineers to manage, configure and troubleshoot a stack of switches as
one logical unit. The modular uplink C9200 models support a stacking bandwidth of 160
Gbps, whereas fixed uplink C9200L models have a stacking bandwidth of 80Gbps.

Additional details on benefits and operations of StackWise-160/80 can be found in the

chapter High
High availability
availability.
availability

Network modules
Catalyst 9200 Series modular uplink C9200 models have an optional slot for uplink
network modules. The fixed uplink C9200L has fixed uplink configuration for each
model. There are three variants of uplink modules which can be used not only to
provide connectivity to an uplink switch but can also be used to connect hosts.

• 4x 1G SFP ports

• 4x 10G SFP ports

32 Catalyst 9000 family of switches

• 2x 25G SFP ports

• 2x 40G QSFP ports

Uplink modules are field replaceable units (FRU) which enable a swap of network
modules without interrupting switch operations, thereby providing investment
protection without compromising on availability.

Power supply and fan

All Catalyst 9200 Series switch models support dual redundant power supplies. The
data-only model uses 125W AC power supply; 24 port PoE+ and mGig models use 600W
AC and 48 port PoE+; and mGig models use 1000W AC supply. All three power supplies
are highly efficient, rated by 80Plus as Silver (125W) and Platinum (600W, 1000W)
efficiency. Such high efficiency along with innovations such as EEE leads to significantly
lower cost of ownership.

Catalyst 9200 Series switch models are equipped with dual variable-speed fans. All
models are capable of cooling the switch with a single fan in case of one fan failure. On
modular uplink C9200 models, the fans can be replaced on an operational switch. On
fixed uplink C9200L models, the fans are fixed.
 Catalyst 9000 family of switches 33

Catalyst 9300 Series switches

Cisco Catalyst 9300 Series switches are the leading business-critical stackable
enterprise fixed access switching platform. At 480 Gbps of stacking bandwidth and up
to eight devices in a stack, it is the industry’s highest-density stacking bandwidth
solution.

DIAGRAM Catalyst 9300 Series switches

Platform overview
All models of Catalyst 9300 Series are 1RU high with dual power supplies and redundant
fans. Different models offer a variety of connectivity and scale. These models can be
organized into four configurations, each with 24 port and 48 port copper options:
34 Catalyst 9000 family of switches

• Data-only models - Optimized for devices such as desktops and printers that
just need data connectivity from 10 Mbps to 1 Gbps

• PoE/PoE+ models - Provide the same capability as the data models plus added
support for 30W of power over Ethernet (PoE+). All the ports support PoE /
PoE+ and all ports can be active simultaneously with PoE+

• Universal PoE (UPOE) models - Provide the same capability as the PoE+ models
with the added support of 60W of PoE. Any of the ports can be configured with
UPOE, but the maximum available total PoE power per switch is 1800W

• Multigigabit (mGig) models - Provide connectivity at multiple speeds up to 10

Gbps on mGig ports. Wireless access points supporting Wi-Fi 6 (802.11ax) and
802.11ac Wave 2 are the most common devices requiring mGig connectivity, but
wired connections to desktops can also benefit. All ports on these models
support UPOE, but the total available PoE per switch is 1800W. There are three
mGig variants:

- 24 port mGig - All 24 ports support 100 Mbps, 1 Gbps, 2.5 Gbps, 5 Gbps
and 10 Gbps

- 48 port mixed mGig - The first 36 ports support 100 Mbps, 1 Gbps and
2.5 Gbps. The last 12 ports support full mGig speeds

- 48 port 5G mGig - All 48 ports support 100 Mbps, 1 Gbps, 2.5 Gbps and
5 Gbps

Architecture
Catalyst 9300 Series switches operate at line rate and offer configurable system
resources to optimize support for specific features. The switch architecture consists of
three main components:
 Catalyst 9000 family of switches 35

• UADP ASIC

• x86 CPU complex

• ASIC interconnect / Stack interface

UADP ASIC
The Catalyst 9300 family of switches are built with UADP 2.0 ASIC. The non-mGig
models of the Catalyst 9300 Series are powered by a single Cisco UADP 2.0 ASIC. The
mGig models are equipped with two UADP 2.0 ASICs. All ports on Catalyst 9300 Series
switches operate at line rate for all packet sizes.

UADP 2.0 ASIC is built using 28 nanometer (nm) technology with two cores, with each
core capable of supporting up to 80 Gbps of bandwidth for a total of 160 Gbps. Switches
equipped with the UADP 2.0 ASIC support a total of up to 32K IPv4 / 16K IPv6 hardware
tables, up to 18K of security ACL TCAM, and 2 x 8MB of buffer.

X86 CPU complex

Catalyst 9300 Series switches are all equipped with the same CPU, system memory and
flash storage. Catalyst 9300 Series switches come with a 1.8 Ghz x86 quad core CPU,
8GB DDR4 RAM, and 16 GB of internal flash storage. For application hosting or general
purpose storage, these switches support USB 3.0 SSD storage

ASIC interconnect
Catalyst 9300 Series switches consist of an internal stack interface (240 Gbps) acting as
ASIC interconnect on the switches with dual ASICs. This interface is internally used for
transporting traffic between front-panel ports of two ASICs on the same switch.
36 Catalyst 9000 family of switches

DIAGRAM Catalyst 9300 Series switch architecture

StackWise-480
Catalyst 9300 Series switches provide the ability to stack up to eight switches using
special cables on the back, combining them together to operate as a single, logical
switch. This allows network engineers to manage, configure and troubleshoot the stack
of switches as one. the chapter High availability provides additional details on the
High availability
operation of StackWise-480.

Network modules
All Catalyst 9300 Series switches have an optional slot for uplink network modules.
There are four variants of uplink modules. The ports on these modules can be used for
both uplink and downlink connectivity.

• 4x 1G RJ45 ports (10/100/1000Mbps)

• 4x mGig RJ45 ports (no PoE)

• 8x 10G SFP ports

 Catalyst 9000 family of switches 37

• 2x 25G SFP ports

• 2x 40G QSFP ports

Catalyst 9300 Series switches are compatible with Catalyst 3850 Series uplink modules.
However, Catalyst 9300 Series switch uplink modules are not compatible with Catalyst
3850 Series switches.

Power supply and fan

Catalyst 9300 Series switches support dual redundant power supplies. These power
supplies are available in 350W AC, 715W AC, 1100W AC, and 715W DC options. The power
supplies can be mixed in any combination, for example, AC and DC.

Catalyst 9300 switches are equipped with three field-replaceable fans. These fans are
operated in an N+1 redundant mode.

StackPower®
Catalyst 9300 Series switches provide the ability to create a shared pool of power using
dedicated stack power cables. In the event of power supply failure or more PoE power
draw, the switch can utilize the power from the shared pool to support the extra load.
StackPower can be deployed in two modes: power-sharing and redundant mode.
Additional details are provided in the chapter High
High availability
availability.
availability
38 Catalyst 9000 family of switches

Catalyst 9400 Series switches

Cisco Catalyst 9400 Series switches are the leading business-critical modular
enterprise switching access platform. Catalyst 9400 Series switches provide
unparalleled investment protection with a flexible chassis architecture capable of
supporting up to 9 Tbps of system bandwidth. They also offer unmatched power
delivery for high-density Power over Ethernet deployments, delivering 60W UPOE to
endpoints. Catalyst 9400 Series switches deliver state-of-the-art high availability with
capabilities such as dual supervisors and N+1/N+N power supply redundancy. The
platform is campus-optimized with an innovative dual-serviceable fan tray design and
side-to-side airflow and is closet-friendly with a ~16-inch depth. A single system can
scale up to 384 access ports.

DIAGRAM Catalyst 9400 Series switches

 Catalyst 9000 family of switches 39

Platform overview
Catalyst 9400 Series switches provide up to 480G per slot bandwidth. There are three
models that offer different densities to fit different size requirements: 4 slot, 7 slot, and
10 slot chassis. All three chassis options provide dual supervisor slots for maximum
availability. Each chassis is designed to support up to 720G of bandwidth between the
two supervisor slots, allowing future supervisors to support multiple 100G ports. With
the growing need for increased PoE, the chassis has the capability of providing more
than 4,800W of PoE power per slot.

Architecture
Catalyst 9400 Series switches are based on a centralized architecture, which means all
forwarding, services and queuing are done on the supervisor while the line cards are
considered transparent, containing only stub ASICs and PHYs. The simplicity of this
centralized design allows easy upgrade of features by just upgrading the supervisor
while keeping the existing line cards. This provides significant investment protection.

DIAGRAM Catalyst 9400 Series switch architecture

40 Catalyst 9000 family of switches

Supervisors
There are currently three versions of supervisor available for Catalyst 9400 Series
switches:

• C9400-SUP-1

• C9400-SUP-1XL

• C9400-SUP-1XL-Y

All Catalyst 9400 supervisors are powered by 3x UADP 2.0 XL ASICs and x86 CPU
Processor. The three ASICs are interconnected through a 720G ASIC interconnect for
packets passing between the ASICs.

The Catalyst 9400 Series SUP-1 provides 80 Gbps of bandwidth per slot for all chassis
models and is optimized for access deployments.

The Catalyst 9400 Series SUP-1XL/1XL-Y provide 80 Gbps of bandwidth per slot in the
10 slot chassis, 120 Gbps of bandwidth per slot for the 7-slot chassis and 240 Gbps per
slot for the 4-slot chassis. SUP-1XL/1XL-Y also adds support for different flexible
templates to accommodate various deployment models such as access, distribution,
core, SD-Access or NAT.

Cisco UADP use switch link interfaces (SLIs) to connect line card stub devices through
the backplane. Each SLI (running at 10G rate with SUP-1/1XL/1XL-Y) aggregates a
group of front panel ports, known as an SLI port group. Future supervisors can run the
SLIs at a higher speed and provide more bandwidth for the existing line cards. This
provides additional investment protection for the existing line cards.
 Catalyst 9000 family of switches 41

DIAGRAM Catalyst 9400 Series Supervisor-1XL architecture

Catalyst 9400 Series switches are all equipped with the same CPU, system memory, and
flash storage. Catalyst 9400 Series switches come with a 2.4 GHz x86 quad-core CPU,
16GB DDR4 RAM and 16 GB of internal flash storage. For application hosting or general-
purpose storage, these switches support USB 3.0 SSD storage and additionally support
up to 960GB M2 SATA SSD storage options.

Supervisor uplinks
All supervisors have 8x SFP / SFP+ ports and 2x QSPF+ ports on the front. The
architecture of SUP-1 and SUP-1XL provides 80G total uplink bandwidth shared
between 1G / 10G / 40G interfaces.

On SUP-1XL-Y, port 1 and 5 of the SFP/SFP+ ports are also capable of 25G (SFP28). On
SUP-1XL-Y, the uplink total bandwidth is also 80G shared between 1G / 10G / 25G /
40G interfaces.

Line cards
Catalyst 9400 Series switches offer mGig, Cisco UPOE, data and 10G line cards for
different connectivity requirements.
42 Catalyst 9000 family of switches

• Copper RJ45 modules:

- 48-port Data-only line card - All 48 ports on this module support 10

Mbps, 100 Mbps, and 1 Gbps.

- 48-port PoE+/PoE line card - All features on the data-only line card
with added support for PoE+ (30W) and PoE (15.4W).

- 48-port UPOE line card - All features on PoE+/PoE line card with added
support of UPOE (60W).

- All 48 ports within the slot can provide UPOE simultaneously.

- 48-port mGig line card - The first 24 ports are the traditional
10/100/1000 Mbps ports and the last 24 ports are mGig ports.

- All 48 ports on this module support UPOE (60W), PoE+ (30W)

and PoE (15W).

- All 48 ports within the slot can provide UPOE simultaneously.

• Fiber SFP/SFP+ modules:

- 24-port SFP line card - Supports 100 Mbps and 1 Gbps speeds.

- 48-port SFP line card - All features on 24-port SFP line card with
double the density.

- 24-port SFP+ line card - Supports 100 Mbps, 1 Gbps and 10 Gbps.

Line card slot bandwidth

The three diagrams below illustrate how the SUP-1XL bandwidth is being utilized for
line card support in the 4 slot, 7 slot, and 10 slot chassis and also shows the distribution
of the number of SLIs to each line card.
 Catalyst 9000 family of switches 43

DIAGRAM Catalyst 9400 Series 4 slot chassis

Catalyst 9400 Series 4 slot chassis: 24 SLIs are active for each line card slot. Each UADP
services 1 line card.

DIAGRAM Catalyst 9400 Series 7 slot chassis

44 Catalyst 9000 family of switches

Catalyst 9400 Series 7 slot chassis: 12 SLIs are active for each line card slot. Each UADP
services 2 line cards.

DIAGRAM Catalyst 9400 Series 10 slot chassis

Catalyst 9400 Series 10 slot chassis: 8 SLIs are active for each line card slot. Each UADP
services 3 line cards.

Line card oversubscription

All variants of 1G line cards operate at line rate for all packet sizes.

The 10G fiber line card and mGig line cards are oversubscribed with Catalyst 9400
Series Supervisor-1, 1XL and 1XL-Y. Also important to note is that line cards are
designed to take advantage of higher per-slot bandwidth with future supervisors by
running a larger number of SLIs.

Line card performance mode

With the oversubscribed 10G modules, if 10G line rate performance is needed, both the
24-port SFP / SFP+ and mGig line card can be enabled for maximum performance.
 Catalyst 9000 family of switches 45

When performance mode is enabled, the system uses only one 10G port in each SLI port
group.

The following diagram shows the SLI port group mappings for the 10G fiber line card
with the 7 slot and 10 slot chassis.

DIAGRAM Catalyst 9400 Series 7 slot and 10 slot SLI port groups

Stateful switchover (SSO)

Catalyst 9400 Series switches support redundant supervisors with SSO. In SSO mode,
the redundant supervisor engine is fully synchronized with configuration on the active
supervisor. It subsequently maintains states for different protocols and minimizes the
time the switch is unavailable during a supervisor failure or switchover. Additional
details are provided in the chapter High
High availability
availability.
availability

In-service software upgrade (ISSU)

ISSU is an upgrade process available on Catalyst 9400 Series switches allow upgrading
supervisor software while traffic forwarding continues. This process is built on top of
stateful switchover. ISSU increases network availability and reduces downtime caused
by planned software upgrades. Additional details are provided in the chapter High
High availa
availa
bility
bility.
bility
46 Catalyst 9000 family of switches

Power supply
The power supplies for Catalyst 9400 Series switches come in small form factor while
providing high capacity and efficient output. The 7 slot and 10 slot chassis provide eight
power supply bays while the 4-slot chassis provides four power supply bays. The
Catalyst 9400 Series combines N+1, and N+N redundant options for power supplies.
Additional details are provided in the chapter High
High availability
availability.
availability

Fan tray
The fan tray of Catalyst 9400 Series switches contain multiple individual fans operating
in an N+1 redundant mode. Fans operate at variable speeds based on the system
temperature and altitude. This makes efficient use of the power and provides lower
noise levels. The fan tray on Catalyst 9400 Series switches can be replaced from the
front or the rear of the chassis. This is a tremendous help with operations and reduces
downtime since the cable management for wiring in a typical wiring closet could make
it unwieldy to remove the cables from the front of the chassis to service the fan tray.

DIAGRAM Catalyst 9400 Series fan tray

 Catalyst 9000 family of switches 47

Catalyst 9500 Series switches

Cisco Catalyst 9500 Series switches are the industry’s first purpose-built business-
critical fixed 100G/40G core and distribution layer switches. These switches deliver
exceptional table scales and buffering capabilities. Catalyst 9500 Series switches deliver
up to 3.2 Tbps of switching capacity and up to 2 billion packets per second of
forwarding performance. The platform offers non-blocking 100 Gigabit Ethernet
(QSFP28), 40 Gigabit Ethernet (QSFP+), 25 Gigabit Ethernet (SFP28) and 10 Gigabit
Ethernet (SFP+) switches with granular port densities.

DIAGRAM Catalyst 9500 Series switches

Platform overview
Catalyst 9500 Series switching platform consists of 1RU fixed configuration switches
based on the Cisco Unified Access Data Plane (UADP) ASIC architecture. The platform
runs on the Cisco IOS XE operating system. The platform also supports all the
foundational high-availability capabilities including dual redundant power supplies and
variable-speed highly efficient redundant fans.

Catalyst 9500 Series switches are built on two variants of UADP ASIC: UADP 2.0 XL and
UADP 3.0. The architecture of both ASICs are similar, but they differ in switching
capacity, port density, port speeds, buffering capability and forwarding scalability.
48 Catalyst 9000 family of switches

100 Gigabit Ethernet switches:

• C9500-32C - Catalyst 9500 Series high-performance switch with 32x 100GE

ports (2x UADP 3.0)

• C9500-32QC - Catalyst 9500 Series high-performance switch with 32x 40GE or

16x 100GE ports (1x UADP 3.0)

40 Gigabit Ethernet switches:

• C9500-24Q - Catalyst 9500 Series switch with 24x 40GE ports (4x UADP 2.0 XL)

• C9500-12Q - Catalyst 9500 Series switch with 12x 40GE ports (2x UADP 2.0 XL)

25 Gigabit Ethernet switches:

• C9500-48Y4C - Catalyst 9500 Series high-performance switch with 48x 25GE +

4x100/40 GE ports (1x UADP 3.0)

• C9500-24Y4C - Catalyst 9500 Series high-performance switch with 24x 25GE +

4x100/40 GE ports (1x UADP 3.0)

10 Gigabit Ethernet switches:

• C9500-40X - Catalyst 9500 Series switch with 40x 1/10GE ports (2x UADP 2.0
XL)

• C9500-16X - Catalyst 9500 Series switch with 16x 1/10GE ports (1x UADP 2.0 XL)

Architecture
Catalyst 9500 Series switches operate at line rate and offer configurable system
resources to optimize support for specific features.
 Catalyst 9000 family of switches 49

The switch architecture consists of three main components:

• UADP ASIC

• x86 CPU complex

• ASIC interconnect

UADP ASIC
Cisco UADP 2.0 XL ASIC is built using 28-nanometer technology with two cores, with
each core capable of supporting up to 120 Gbps of bandwidth for a total of 240 Gbps
supporting a maximum forwarding capacity of 375M packets per second. Switches
equipped with the UADP 2.0 XL ASIC support a total of up to 224K IPv4 / 112K IPv6
hardware tables, up to 54K of security ACL TCAM, and 2 x 16MB of shared buffer.

Cisco UADP 3.0 ASIC is built on 16-nanometer technology using two cores, with each
core capable of supporting up 800 Gbps of bandwidth for a total of 1.6 Tbps supporting
a maximum forwarding capacity of 1B packets per second. Switches equipped with the
UADP 3.0 ASIC support a total of up to 416K for IPv4 / IPv6 hardware tables entries, up
to 54K of security ACL TCAM, and 36MB of unified buffer.

X86 CPU complex

Catalyst 9500 Series switches are all equipped with the same onboard CPU, system
memory, and flash storage. Catalyst 9500 Series switches come with a 2.4 Ghz x86 quad
core CPU, 16GB DDR4 RAM, and 16 GB of internal flash storage. For application hosting
or general purpose storage, these switches support USB 3.0 SSD storage, and models
equipped with UADP 3.0 support up to 960 GB M2 SATA SSD storage options.

ASIC interconnect
Catalyst 9500 Series switches use high-speed ASIC interconnect links for inter-ASIC
communication. UADP 2.0 XL has up to 720 Gbps (360 Gbps full duplex) of interconnect
bandwidth and UADP 3.0 has up to 1.6 Tbps (800 Gbps full duplex) of interconnect
bandwidth between two ASICs. Packets destined to local ports within the ASIC do not
use ASIC interconnect links.
50 Catalyst 9000 family of switches

DIAGRAM Cisco Catalyst 9500 Series switch block diagram - Cisco UADP 2.0 XL
 Catalyst 9000 family of switches 51

DIAGRAM Cisco Catalyst 9500 Series high-performance switch block diagram -

Cisco UADP 3.0

Network modules
Cisco Catalyst 9500 Series switches support optional network modules for uplink ports
on the C9500-40X and C9500-16X switch models. The default switch configuration
does not include the network modules. There are two network module options
available: 8x 1/10G and 2x40G.

All ports on the network module are line rate and all software features supported on
switch downlink ports are also supported on network module ports. The network
modules support online insertion and removal (OIR).

Power supply
Catalyst 9500 Series switches support up to two AC or DC small form factor platinum
rated power supply units for a total system capacity up to 650W, 950W & 1600W. Power
52 Catalyst 9000 family of switches

supplies can be installed in the following combinations: two AC, two DC or a mix of AC
and DC power supplies. The power supplies work together in redundant load-sharing
mode, in which each power supply operates at approximately 50 percent of its capacity.
If one power supply fails, the other power supply can provide power for the entire
system. These switches support OIR for power supplies.

Fan and fan tray

Cisco Catalyst 9500 switches have a total of five independent fans or dual fan trays
depending on the SKU. Each individual fan operates at variable speeds. The hardware is
capable of accommodating a failure of up to one individual fan or fan tray. The
remaining fans will automatically increase their RPM to compensate and maintain
sufficient cooling. These switches support OIR of the fans or fan trays for up to 120
seconds.

25G and 100G - enabling higher speeds in enterprise

Cisco has been pioneering several initiatives to bring new Ethernet technologies to
market. These include 100GBASE quad small form factor pluggable QSFP28 and
25GBASE small form factor pluggable SFP28. The 25GBASE options include dual-rate
optics (25G and 10G) supporting enterprise campus distances to facilitate next-
generation network speed and architecture transformations. These innovations enable
flexible options and backwards compatibility to drive network speeds beyond the
current 10G and 40G capabilities while minimizing cost and real estate changes.

Cisco Catalyst 9500 Series 25G & 100G switches

Catalyst 9500 Series switches are the foundation for the next-generation 25G and
100G-enabled high-speed enterprise campus network. To support these newer speeds
in core and distribution, Cisco offers a full suite of switch options with industry-leading
port density and flexibility.

Cisco 25G optics portfolio

The Cisco 25GBASE SFP28 portfolio offers customers a wide variety of high-density and
low-power 25 Gigabit Ethernet connectivity options for the evolving campus network.
 Catalyst 9000 family of switches 53

Features and benefits of Cisco 25G optics:

• Support for dual rate optics in enterprise networking to provide unsurpassed

investment protection

• Interoperable with other IEEE-compliant 25G interfaces where applicable

• Certified and tested for superior performance, quality, and reliability

• High-speed electrical interface compliant to IEEE 802.3by

For more information, refer to Cisco 25GBASE SFP28 optics and copper modules
https://cs.co/SFP28
https://cs.co/SFP28

Cisco 100G optics portfolio

The Cisco 100GBASE QSFP28 portfolio offers customers a wide variety of high-density
and low-power 100 Gigabit Ethernet connectivity options for enterprise core and
distribution layers. In a 3-tier campus network, as the distribution layer moves to 25G
or 40G, it is desirable to have a 100G-enabled core. It is important to note that form
factors of 40G and 100G optics are compatible.

Features and benefits of Cisco QSFP28 optics:

• Hot-swappable - input/output device that plugs into a 100G Gigabit Ethernet

Cisco QSFP port

• Interoperable - with other IEEE-compliant 100GBASE interfaces where

applicable

• Certified and tested - for superior performance, quality, and reliability

• High-speed electrical interface - compliant to IEEE 802.3bm

For more information, refer to Cisco 100GBASE QSFP optics and copper modules
https://cs.co/QSFP
https://cs.co/QSFP
54 Catalyst 9000 family of switches

Catalyst 9600 Series switches

Cisco Catalyst 9600 Series switches are the next generation of the industry leading
business-critical modular enterprise campus core and distribution platform. The
Catalyst 9606R chassis is hardware-ready to support a switching capacity of up to 25.6
Tbps. Catalyst 9600 Series switches support granular port densities that fit diverse
campus needs, include non-blocking 40 and 100 GE (QSFP28) and 1, 10, and 25GE
(SFP28). The platform delivers high availability with field replaceable dual supervisors,
redundant power supplies and fans. The platform is campus-optimized with an
innovative dual-serviceable fan tray design and side-to-side airflow and is closet-
friendly with a ~16-inch depth.

DIAGRAM Cisco Catalyst 9606R chassis

 Catalyst 9000 family of switches 55

Platform overview
Catalyst 9606R is a 6 slot chassis, with two middle slots dedicated for the supervisors
and four slots dedicated for the line cards. Each line card slot has a dedicated total
bandwidth of up to 6.4 Tbps. The Catalyst 9606R chassis can provide a maximum of 128
x 40G/100G (QSFP) or 192 x 1G/10G/25G (SFP) ports.

Architecture
Catalyst 9600 Series switches are based on a centralized architecture. All fowarding,
security, and queueing are done on the supervisor while the line cards are considered
transparent, containing only PHYs and control logics. The simplicity of this centralized
design allows easy upgrade of features as well as additional bandwidth by just
upgrading the supervisor while keeping the existing line cards. The combination of the
centralized architecture and transparent line card also provide uninterrupted
supervisor switchover which is the foundation for in-service software upgarde (ISSU).

DIAGRAM Catalyst 9600 Series switching architecture

56 Catalyst 9000 family of switches

Supervisors
The Catalyst 9600 Series Supervisor-1 is powered by 3x UADP 3.0 ASICs and x86 CPU
Processor. The three ASICs are interconnected with 3.2 Tbps ASIC interconnect on
each ASIC. The ASIC interconnect uses a broadcast network to ensure traffic reaches all
other ASICs. Due to high performance line card requirements, the Supervisor module
does not have dedicated uplink ports (any of the line cards ports can be used for
uplinks).

The Catalyst 9600 Series Supervisor-1 provides 2.4 Tbps (1.2 Tbps full-duplex) of
bandwidth per slot for all the line card slots.

DIAGRAM Catalyst 9600 Series Supervisor-1 architecture

The Catalyst 9600 Series switches are all equipped with the same CPU, system memory,
and flash storage. Catalyst 9600 Series switches come with a 2.0 GHz x86 8-Core CPU,
16GB DDR4 RAM, and 16 GB of internal flash storage. For application hosting or general
purpose storage, these switches support USB 3.0 SSD storage, and models equipped
with UADP 3.0 support up to 960GB M2 SATA SSD storage options.

Note Due to the bandwidth requirements of the high-density C9600 line cards,
the C9600-SUP-1 does not support uplink ports.
 Catalyst 9000 family of switches 57

Line cards
Cisco Catalyst 9600 Series switches offer two types of line cards for different
connectivity requirements.

• Fiber QSFP module:

- 24-port QSFP+/QSFP28 line card

- up to 24 ports of 40GE nonblocking

- up to 12 ports of 100GE nonblocking (upper ports)

• Fiber SFP module:

- 48-port SFP/SFP+/SFP28 line card

- up to 48 ports 1/10/25GE nonblocking

The diagram below illustrates how the front panel ports are mapped to the 3x UADP 3.0
ASICs.
58 Catalyst 9000 family of switches

DIAGRAM Catalyst 9600 Series port mapping

Stateful switchover (SSO)

Catalyst 9600 Series switches support a redundant supervisor with SSO. In SSO mode,
the redundant supervisor engine is fully synchronized with configuration on the active
supervisor. It subsequently maintains states for different protocols and minimizes the
time the switch is unavailable during a supervisor failure or switchover. Additional
details are provided in the chapter High
High availability
availability.
availability

In-service software upgrades (ISSU)

ISSU is an upgrade process available on Catalyst 9600 Series switches that allows
upgrading supervisor software while traffic forwarding continues. This process is built
on top of the stateful switchover. ISSU increases network availablity and reduces
downtime caused by planned software upgrades. Additional details are provided in the
chapter High
High availability
availability.
availability

Power supply
The power supplies for Catalyst 9600 Series switches come in small form factor while
providing high capacity and efficient output. The Catalyst 9606R chassis provides four
 Catalyst 9000 family of switches 59

power bays. Both AC and DC power supplies are available for Catalyst 9600 Series
switches. The platform supports both combined and N+1 redundant mode. Additional
details are provided in the chapter High
High availability
availability.
availability

Fans and fan tray

The fan tray for Catalyst 9600 Series switches contains multiple individual fans
operating in an N+1 redundant mode. The fans operate at variable speeds based on the
system temperature and altitude. This makes efficient use of the power and provides
lower noise level. The fan tray on Catalyst 9600 Series switches can be replaced from
the front or the rear of the chassis. This is a tremendous help with operations and
reduces downtime since the cable management for wiring in a typical wiring closet
could make it unwieldy to remove the cables from the front of the chassis to service the
fan tray.

DIAGRAM Catalyst 9600 Series switch fan tray

60 Catalyst 9000 family of switches

Packaging, licensing and support

The Catalyst 9000 switching family uses a simplified licensing model. The previous
generation of switches had different license types (LAN Base, IP Base, IP Services, and
Enterprise Services) with the added complexity of differences across the multiple
Catalyst switch families. The Catalyst 9000 switching family uses the same software
packaging and licensing model across all platforms.

The new model provides the following benefits:

• simplifies the packaging of features

• delivers a more cost-effective solution for consuming features

• lowers up-front costs by adding more features and support

Software packages and licenses

The Catalyst 9000 switching family offers two mandatory term-based software
licensing options:

• Cisco DNA Essentials - provides the baseline network functionality used to

operate a network and includes the perpetual Network Essentials functionality

• Cisco DNA Advantage - includes all the functionality in the Cisco DNA
Essentials package (including Network Essentials) plus advanced capabilities
such as advanced security, availability, automation, and assurance and the
perpetual Network Advantage functionality

Each software feature license option is offered with 3, 5, or 7-year term and includes
solution support, the Cisco Enhanced Limited Lifetime hardware Warranty (E-LLW) and
Smart Net Total Care™ support for the life of the term. While renewal of the term
 Catalyst 9000 family of switches 61

license is not mandatory, if the term license is allowed to expire, switch functionality
will revert to the perpetual base network functionality and solutions support will end.

Smart Accounts are also mandatory for purchase. If a customer does not have a Smart
Account set up prior to the purchase, a new Smart Account must be created at the time
of purchase.

Note Cisco DNA Essential license option is not available on the Catalyst 9600
Series

Additional information:
For Smart Account overviews and training sessions,
visit http://cs.co/operationsexchange
http://cs.co/operationsexchange

To learn more about end-to-end Smart Account and Smart License management,
visit http://cs.co/smartmanager
http://cs.co/smartmanager
62 Catalyst 9000 family of switches

A high-level grouping of functionality in each package is shown in the following

diagram:

DIAGRAM Cisco DNA Advantage and Cisco DNA Essentials packages

Note Not all functionality is available across all platforms.

For a complete list of the features of each package, use the Cisco Feature Navigator
https://cisco.com/go/cfn
https://cisco.com/go/cfn

Technical support
Technical support for the Catalyst 9000 switching family covers both the hardware and
the feature packaging just discussed. For technical assistance troubleshooting hardware
problems and providing replacement components or chassis, Cisco provides the
following general options:
 Catalyst 9000 family of switches 63

• An Enhanced Limited-Lifetime hardware warranty covering:

- 90 days of technical support (beginning on the date of initial purchase)

from Cisco's Technical Assistance Center (TAC).

- Hardware troubleshooting and replacement honored for the life of the

switch

For terms and conditions, please refer to Enhanced Limited-Lifetime Warranty on

www.cisco.com. https://cs.co/ELLW
https://cs.co/ELLW

• Premium services available through Cisco or a Cisco partner include both

technical support and hardware replacement.

- Consult your Cisco or Partner sales teams for available offers.

Software support comes in the following forms:

• The switch base functionality enabled by its network license is valid for the
device's lifetime. Software updates for network licensed components are
perpetual.

• 90 days of technical support for the switch base functionality (beginning on the
date of initial purchase) from Cisco's Technical Assistance Center (TAC).

For terms and conditions, please refer to Enhanced Limited-Lifetime hardware

Warranty on www.cisco.com. https://cs.co/ELLW
https://cs.co/ELLW

• Software support for those features enabled through a Cisco DNA Essentials or
Cisco DNA Advantage license is included while the subscription is valid. Again,
support for these features ends once the subscription expires.
ASICs - the power of
programmable silicon
66 ASICs - the power of programmable silicon

What is an ASIC?

An application specific integrated circuit (ASIC) is a silicon microchip designed for a

specific task (e.g. bridging or routing packets), rather than being used for general-
purpose processing such as a CPU. ASICs are fundamental to how an Ethernet switch
works.

ASICs are custom-designed for the products they are part of and the solutions they
support. In a network switch, an ASIC handles packet recognition, manipulation and L2
/ L3 forwarding at extremely high speeds (tens or hundreds of gigabits per second,
trending towards terabits per second), while also allowing a rich set of services for the
traffic, such as prioritization (e.g. QoS), accounting (e.g. NetFlow), segmentation (e.g.
VRFs and SGTs), traffic filtering and enforcement (e.g. ACLs), path selection (e.g. PBR),
and much more.

ASIC microchips are measured in nanometers (billionths of a meter). This is the size of
the various components, such as transistors, that the ASIC is built from. The three main
advantages of smaller ASICs are:

• increased speed (electrons have shorter distances to travel)

• lower power consumption and less energy wasted as heat

• lower cost (improved chip yield by decreasing the chance of hitting a silicon
defect)

Modern ASICs are generally manufactured at sizes ranging from 45 to 28 nanometers

with some newer models as small as 16 nanometers.
 ASICs - the power of programmable silicon 67

DIAGRAM Microchip packaging

Why do we need ASICs?

A generic CPU is too slow for forwarding traffic in a switch. While a general-purpose
CPU may be fast at running random access applications on a laptop or server,
manipulating and forwarding network traffic is a different matter. Traffic handling
requires constant lookups against large memory tables (e.g. L2 tables for MAC
addresses, L3 tables for IP routes, L4 ACLs for Security and QoS, etc)

In a generic CPU, all of these tables are held in off-chip memories (not located on the
CPU itself) and incur significant performance penalties for frequent memory access.
There are also limited data paths and buffers to handle incoming packets (remember,
this is millions or even billions of packets per second). Once packets have been received
and queued, the CPU must perform the actual processing functions, finding destination
lookups, rewriting packet formats, etc. For these reasons, a CPU is not well-suited for
this purpose.
68 ASICs - the power of programmable silicon

DIAGRAM Traditional CPU - more exibility, less performance, cost neutral

ᄂthe bottom line

CPUs are exible but slow. ASICs are necessary to meet the requirements of
enterprise networks.

The following chapters examine both traditional types of network ASICs and the latest
state-of-the-art programmable ASICs. Administrators will discover not only why ASICs
are central to how a switch operates but also how modern ASICs form the foundation of
the enterprise network, now and in the future.
 ASICs - the power of programmable silicon 69

Why programmable ASICs?

Traditional ASICs
Many ASICs have been used in Cisco switches and routers over the years. Each of these
ASICs were designed and developed for the specific features and scale needed for
different roles in the campus network. Each also has different capabilities, speeds and
scaling properties suitable for their roles in the network.

However, this class of networking ASICs are known as fixed ASICs. All aspects of these
ASICs (behavior, speed, scale, etc.) are hard-wired (fixed) into them as part of the
manufacturing process and cannot be changed without creating a whole new version of
the ASIC.

Another reason they are called fixed ASICs is their processing behavior. As the name
suggests, all incoming packets are subject to a fixed series of steps known as a
processing pipeline. The typical fixed ASIC processing pipeline stages are similar to the
following:

1 Parse incoming packets (examine headers)

2 Layer 2 processing (e.g. MAC lookup)

3 Layer 3 processing (e.g. IP lookup)

4 Policy processing (e.g. ACL lookup)

5 Packet rewrite and traffic counters

6 Queue scheduling and transmission

70 ASICs - the power of programmable silicon

DIAGRAM Traditional ASIC - processing pipeline

It is also worth noting that due to the way fixed ASICs are designed and manufactured,
along with the time needed to integrate the ASIC into a network switch, it can often
require many years before delivering the final product. Fixed ASICs are very cost-
effective and efficient but are not flexible nor adaptable. They are only able to handle
the types of packets that the chip is hard-wired to process.

DIAGRAM Traditional ASIC - more performance, less exibility, cost neutral

Network and protocol evolution

Why do ASICs need to change? To provide an example, the ASIC in Catalyst 3750 can
only forward IPv4 and IPv6 packets in hardware. It was designed before VXLAN was
developed, so it cannot handle VXLAN in hardware. Since it is a fixed ASIC, it is not
possible to change the processing pipeline to handle new protocols such as VXLAN. An
entirely new ASIC would need to be created.
 ASICs - the power of programmable silicon 71

This lack of flexibility may have been acceptable when networks, and the related
protocols, did not change much. In the new era of networking, everything is "software-
defined", with ever-evolving protocols and scale requirements. This requires ASICs to
support new packet formats and encapsulations such as VXLAN-GPO, GPE and NSH.

ᄂthe bottom line

Traditional xed ASIC architecture, and the network itself, requires hardware
replacement to adopt innovative capabilities that the new software-de ned world
demands.

Programmable ASICs
How to get the best of both worlds? How to get the speed we need for multi-gigabit or
multi-terabit network devices and also the flexibility to keep pace with new network
innovations? These questions led to the concept of programmable ASICs - flexible
network microchips designed to adapt to new capabilities as the need emerges, yet still
offer the performance networks demand.

Early attempts led to the development of the field programmable gate array (FPGA).
These are essentially simplified ASICs, with reprogrammable logic gates, that can
change the original behavior after manufacturing. Although FPGAs do provide a level of
flexibility, they are actually very expensive to develop and support. They are not built
for any particular task and have little or no onboard memory requiring other chips to
provide memory access.

These limitations typically relegate FPGAs to a special-purpose role in most network

devices. An FPGA may be used to augment the packet forwarding capabilities of a fixed
ASIC for that "one special feature" the fixed chip does not have. For example, the
Catalyst 4500 Series Supervisor-8 uses an FPGA to provide VXLAN encapsulation,
which the switch ASIC does not support. It is usually too expensive to use FPGAs as the
primary forwarding engine for a switch (design and manufacturing costs, board space,
heat, and power). Ultimately, this raises the total cost of a switch using combined FPGA
and ASIC designs to achieve flexibility.
72 ASICs - the power of programmable silicon

DIAGRAM FPGA - more cost, moderate exibility, moderate performance

In summary, CPUs are flexible but do not scale for high-speed forwarding; fixed ASICs
are fast and scalable but inflexible, and FPGAs are flexible and scalable but very
expensive. What is the answer?

Cisco saw this need coming several years ago, and as a result of that foresight, designed
and developed the flexible, programmable Cisco Unified Access Data Plane (UADP)
ASIC.

The Cisco UADP ASIC combines the flexibility needed to address new and emerging
networking protocols and encapsulations, with the speed of a fixed ASIC, and the
appropriate cost and scalability to address multiple different areas of the campus
network: core, distribution, and access. With UADP, Cisco has truly begun an entirely
new era of networking.

The following chapters explore the Cisco Unified Access Data Plane ASIC, which is at
the heart of the Catalyst 9000 family of switches.
 ASICs - the power of programmable silicon 73

Cisco UADP - programmable ASIC silicon

UADP - Cisco's flexible, programmable switching ASIC

The Cisco Unified Access Data Plane (UADP) ASIC forms the heart of the Catalyst 9000
family of switches. Catalyst 9000 switches are based on the latest generation of UADP
ASIC which are evolved from earlier versions used in the Catalyst 3850/3650 Series
switches.

Flexibility is the key attribute that makes UADP the ideal foundation for the world's
most advanced switches. This enables the Catalyst 9000 family of switches to:

• handle new frame encapsulations, allowing new features and protocols.

• reprogram their memory tables, allowing switches to adapt to changing needs.

• support multiple interface types and chassis configurations, to address evolving

network designs.

• maintain consistent high performance, to address a growing diversity of

applications.

• provide a rich, integrated set of flexible traffic handling and accounting services.

Flexibility at every stage

Flexibility has been designed into every aspect of the Cisco UADP from the beginning.
In UADP, almost every processing stage that would be present in a fixed-configuration
chip has been replaced with a flexible counterpart.

The job of the parser stage is to recognize packet types and headers and analyze them
for further processing in the ASIC pipeline. In traditional ASICs, the parser stage is
fixed, making it impossible to upgrade the fixed ASIC to recognize or process new
74 ASICs - the power of programmable silicon

packet types and headers in hardware. The Cisco UADP ASIC contains a
reprogrammable FlexParser that can parse a packet for different types of headers.

Unlike the traditional fixed-processing pipeline, the Cisco UADP multi-stage flexible
pipeline (L2 / L3 forwarding, policy, rewrite, queuing, etc) is also completely
reprogrammable (via firmware microcode). There is an ingress pipeline and an egress
pipeline, which is not available in most fixed ASICs.

DIAGRAM Programmable Cisco UADP - processing pipeline

Packet recirculation
Traffic tunneling is a common design in modern networks. GRE, MPLS and VXLAN are
considered tunnels because they add an additional header to the outer portion of a
packet when sending (known as encapsulation), and remove the header when the
packet is received (known as decapsulation). Any time packets need to be tunneled in an
ASIC, the original packet needs to be processed more than once (known as
recirculation) to add or remove the additional header(s).

A quick review of what happens during tunneling reveals why. When a packet arrives
and the ASIC decides (based on the configuration) that the packet needs to be sent
through a tunnel (e.g. VXLAN), a new tunnel header needs to be added in front of the
original packet. This new header will use the source IP of the local side of the tunnel
and the destination IP of the remote side of the tunnel. Since the destination IP address
has now changed to that of the remote side of the tunnel, the packet needs to be
recirculated through the processing pipeline again to forward to this new destination,
along with any services or policies that may apply to the tunnel.
 ASICs - the power of programmable silicon 75

DIAGRAM Programmable Cisco UADP - packet recirculation

In the Cisco UADP, a packet can be recirculated in approximately 500 nanoseconds (half
a microsecond). The bandwidth available for recirculation is flexible, meaning packet
recirculation can also use the spare bandwidth not currently being used by the front-
panel ports. In the event that tunneling is required, the impact to forwarding
performance is minimal. A packet can be recirculated up to 16 times, while only 2 or 3
passes are normally required.

This ability to recirculate packets many times, if necessary, enables even complex use-
cases to be accommodated via the UADP flexible pipeline architecture. Now that traffic
tunneling is common, it is clear that UADP was purpose-built and optimized for
tunneling.

Integrated micro-engines
Certain advanced functions executed by UADP may be very processing-intensive.
Several tasks, such as fragmentation and encryption, are based on well-known fixed
algorithms, and it does not make sense to waste cycles within the UADP pipeline. In
such cases, an on-chip micro-engine is available that can process these well-known
functions, in parallel, saving the valuable UADP pipeline performance for other
functions.
76 ASICs - the power of programmable silicon

Some examples of micro-engine functions built into the Cisco UADP include:

Encryption and decryption - Security and data confidentiality are paramount in

modern networks. Every UADP comes with two levels of hardware encryption built in.
The first, operating at the port level at line rate on all UADP ports, is MACsec (802.1AE).
MACsec provides link-level encryption to guarantee data confidentiality, meaning
packets are encrypted during transmission to, and decrypted when received from, a
MACsec-enabled link.

The Cisco UADP also provides a datagram transport layer security (DTLS) micro-engine
which can encrypt traffic based on packet formats such as CAPWAP and VXLAN. This
can serve as the basis for encryption of tunnel overlay traffic. The UADP provides
hardware MACsec encryption using the AES alorithm with up to 256-bit keys and Galois
Counter Mode (AES-256-GCM).

Fragmentation - Any time the maximum transmission unit (MTU) size of a link is
exceeded in the network, the original packet may need to be fragmented and then
reassembled at the other side, for example, when traffic is tunneled and the output
interface MTU is too small to accommodate the tunnel header plus the original packet.
UADP can handle fragmentation actions in hardware, unlike many other ASICs.

Integrated Flexible NetFlow - Accounting for all traffic flowing through the network is
important for multiple use cases. For example, network baselining and capacity
planning, or basic application visibility. Using Flexible NetFlow, the entire state of the
end-to-end session (TCP or UDP) is tracked by the switch, allowing important
information about the entire packet flow to be extracted and analyzed. UADP
implements full Flexible NetFlow (FNF) collection capability in hardware. Catalyst 9000
switches are capable of collecting NetFlow statistics for every packet transiting the
switch, as an inherent part of overall packet handling.

Cisco encrypted traffic analytics (ETA) utilizes the Flexible NetFlow capability and
inspects encrypted transactions, without decryption, to extract vital information about
them such as the initial data packet (IDP) exchange, as well as information about the
sequence of packet lengths and times (SPLT). By integrating this with Cisco
Stealthwatch and cloud-based machine learning capabilities using cognitive threat
analytics, a high-accuracy transaction 'fingerprint' analysis can be performed to
 ASICs - the power of programmable silicon 77

determine if the encrypted flow represents 'normal' network traffic, or whether it may
represent a threat posed by encrypted malware.

Policy and ACL - Using integrated ternary content addressable memory (TCAM) blocks
located on-chip for maximum performance, the Cisco UADP ASIC provides multiple
options for traffic classification and policy enforcement. TCAM matching provides the
ability to match traffic flows using IPv4 or IPv6 addresses, special tags such as virtual
network (VN) ID and scalable group tag (SGT), QoS, CoS or DSCP values, or other
packet markings. The UADP flexible pipeline can reference up to two packet matches,
for multiple parallel actions, without degrading performance. UADP can then apply the
appropriate policies configured by the network administrator. Examples include
permit/deny, QoS remarking, path selection, packet copy, and other actions.

Packet replication - Certain application traffic types may require packet replication
(creation of multiple copies). For example, an ingress multicast stream may require
replication to multiple receivers on the same switch. The UADP architecture is
optimized for replication, because each packet is held in a central buffer memory
during processing, and then a single or multiple copies can be transmitted to all
receivers.

Integrated stacking capability

The Cisco UADP is a very powerful and flexible ASIC. In many cases (based on port
types and densities), an entire switch may be built around a single UADP ASIC.

However, it may be necessary to connect multiple UADP ASICs together into a larger,
integrated system. UADP was designed with a dedicated high-speed ASIC Interconnect
interface, in addition to the front-panel switch ports, to provide these flexible design
options.

UADP supports the following inter-ASIC connection options:

• stacking multiple switches together via external cables, to build an integrated

stack.

• linking multiple ASICs together on the baseboard, within a fixed switch.

78 ASICs - the power of programmable silicon

• linking multiple ASICs together on a Supervisor module, within a modular

switch.

Programming UADP with microcode

Cisco IOS XE uses multiple layers of software. Some software layers are more closely
associated with the hardware than others. For example, features that a user interacts
with (e.g. the CLI) are typically at a higher layer and less dependent on the hardware.
Meanwhile, hardware drivers and other infrastructure pieces of the software (known as
microcode) directly interact with the hardware. This microcode layer of the software
actually programs the ASIC. Refer to the chapter Cisco
Cisco IOS XE for more details on the
IOS XE
software architecture.

The microcode for programming the Cisco UADP ASIC is included in the Cisco IOS XE
image. Any changes in the microcode come with the image that runs on Catalyst 9000
switches. From a user perspective, the microcode upgrades are seamless and do not
require any additional tasks to enable.
 ASICs - the power of programmable silicon 79

The Cisco UADP family

The history of Cisco UADP ASIC began in 2013 when Cisco introduced Catalyst 3850
Series switches. As discussed, the ASIC design and manufacturing process is very
complex and can take several years for any individual component or product. Several
years of innovative work went into developing UADP ASIC.

Cisco UADP 1.0 took longer to design than most other fixed ASICs at the time, as many
components were entirely new and designed to be flexible. UADP 1.0 was built on a 65
nanometer (nm) process, while the latest UADP 3.0 was built on 16 nm. Cisco UADP has
progressed significantly in terms of ASIC technology and has incorporated more
transistors with each generation. Each additional transistor means additional
performance, scalability, features and functionalities can be built into the ASIC.

Cisco UADP 1.0/1.1

Cisco UADP 1.0 is a single-core 65nm ASIC with 1.3 billion transistors, capable of
56Gbps of aggregate bandwidth. It was the first to deliver many of the programmable
features discussed in previous sections. Due to its flexible nature, UADP 1.0 was one of
the first ASICs to enable support for different, flexible types of packet encapsulations.
The first generation of Catalyst 3850 and 3650 Series switches use UADP 1.0.

By 2015, a newer 36nm version of the same ASIC design (version 1.1) was introduced.
The main elements and architecture of the ASIC remained essentially unchanged, but
with several important improvements were introduced. The key difference between
Cisco UADP 1.1 and 1.0 is the use of a dual-core architecture inside the ASIC.

Unlike UADP 1.0, the UADP 1.1 has two ASIC cores and 3 billion transistors. The result is
similar to using two UADP 1.0 chips in a single ASIC package. UADP 1.1 also provides
higher aggregate bandwidth and performance of up to 160Gbps (80Gbps per core), as
well as some new and updated micro-engines. Some of the new features that UADP 1.1
supports include IEEE 1588 timestamps and MACsec 256-bit encryption (AES-256-
GCM). The second generation of Catalyst 3850 and 3650 MultiGigabit and SFP+
switches use UADP 1.1.
80 ASICs - the power of programmable silicon

Cisco UADP 2.0

Catalyst 9000 switches are built on the next generations of UADP - the UADP 2.0 and
3.0 ASICs.

The Cisco UADP 2.0 is a dual-core 28nm ASIC with 7.46 billion transistors to provide
even higher aggregate bandwidth up to 240Gbps. UADP 2.0 also has larger, more
flexible memory tables that can be reprogrammed (using purpose-built SDM templates),
giving the option to deploy the same device in multiple network areas, as discussed in
the chapter Campus
Campus network
network design
design.
design

DIAGRAM Comparison of Cisco UADP 1.0, 1.1, 2.0 and 3.0

Cisco UADP 2.0 ASICs have three variants: UADP 2.0, 2.0 XL and 2.0 mini.

Both 2.0 and 2.0 XL have the same architecture, but the UADP 2.0 bandwidth, table
scale and overall performance has been optimized for business-critical access layer
switches. The Catalyst 9300 Series switches use UADP 2.0.

Cisco UADP 2.0 XL has been optimized for modular access and / or distribution layer
switches. It has larger memory table sizes (hence the XL designation) with greater
aggregate bandwidth and overall performance to support the port speeds and density
of these roles. UADP 2.0 XL also has dual data paths of 720Gbps inter-ASIC connectivity,
 ASICs - the power of programmable silicon 81

making it more suitable for platforms where multiple ASICs may be required. The first-
generation Catalyst 9500 Series switches and the Catalyst 9400 Series Supervisor-1 and
Supervisor-1XL use UADP 2.0 XL.

Cisco UADP 2.0 mini has a modified single-core architecture with an integrated quad
core ARM CPU, and the bandwidth, table scale, overall performance and power
consumption has been optimized for simple access layer switches. UADP 2.0 mini
supports up to 80Gbps of inter-ASIC connectivity, for platforms with lower bandwidth
requirements. Catalyst 9200 Series switches use UADP 2.0 mini.

TABLE Cisco UADP 2.0, 2.0 XL and 2.0 mini comparison

Cisco UADP 2.0 mini Cisco UADP 2.0 Cisco UADP 2.0 XL

MAC entries 16K-32K 32K 64K

Host routes 8K-10K 24K 48K

LPM routes 3K-4K 8K 64K

Multicast routes 1K 8K 16K

QoS entries 1K 5K 18K

ACL entries 1K 5K 18K

NetFlow entries (per ASIC) 16K 64K 128K

SGT entries 2K 8K 16K

Bu ers 6MB 16MB 32MB

Inter-ASIC bandwidth 80G 240G 720G

Cisco UADP 3.0

The need for network speed is never-ending, driven by new wireless speeds, the
increasing number of attached devices (IoT) and high-definition video. New
82 ASICs - the power of programmable silicon

technologies are appearing every day and driving new requirements for network
performance and scale.

The Cisco UADP 3.0 is a dual-core 16nm ASIC with 19.2 billion transistors, to provide a
significant increase of aggregate bandwidth up to 1.6Tbps. Cisco UADP 3.0 is the most
recent version of UADP, designed to address the challenges brought on by new
interface speeds (e.g. 25G and 100G) and new network designs and solutions. In
addition to increased bandwidth and performance, UADP 3.0 also incorporates several
new improvements that make it the ideal ASIC for campus core and distribution layer
switches.

Cisco UADP 3.0 has larger memory tables and greater reprogramming flexibility, with
larger shared packet buffers (36MB) to support the interface speed increases. It also has
double-wide memory table sizes to store both IPv4 (32bit) and IPv6 (128bit) addresses in
a single entry. Many other ASICs and previous generations of UADP only support
single-width tables, requiring an additional lookup cycle to support IPv6. The second
generation, high-performance, Catalyst 9500 Series switches and the Catalyst 9600
Series Supervisor-1 use UADP 3.0.
 ASICs - the power of programmable silicon 83

TABLE Cisco UADP 2.0 XL and 3.0 comparison

Cisco UADP 2.0 XL Cisco UADP 3.0

(distribution SDM template) (core SDM template)

MAC entries 64K 32K

Host routes (IPv4/v6) 48K/24K

212k/212K
LPM routes (IPv4/v6) 64K/32K

Multicast routes (IPv4/v6) 16K/8K 32K/32K

QoS entries 18K 16K

ACL entries 18K 27K

NetFlow entries (per ASIC) 128K 64K

SGT entries 16K 32K

Bu ers 32MB 36MB

Inter-ASIC bandwidth 720G 1600G

Cisco IOS XE
86 Cisco IOS XE

Cisco IOS evolution

The history of the Cisco Internetwork Operating System (Cisco IOS) goes back to
Cisco’s first product, the AGS multi-protocol routers launched in 1986. At the time
Cisco IOS was a pretty rudimentary, monolithic operating system (OS). It was one of the
very first network operating systems in the industry. Thousands of features have since
been added to Cisco IOS and over the last 30 years and as the industry has gone
through different transitions, Cisco IOS has evolved into a more feature rich OS.

Over time, Cisco IOS software has branched out into many different versions for
different products. Meanwhile, the Cisco product portfolio has also expanded into
various kinds of switches and routers. Purpose-built network areas have evolved, such
as data centers and service providers and new operating systems were introduced for
these areas, such as Cisco NX OS and Cisco IOS XR.

A few years ago, Cisco introduced Cisco IOS XE, designed to restructure the monolithic
code infrastructure of Cisco IOS into a more modular and modern software
architecture. With Cisco IOS XE, the OS is subdivided into multiple components to
achieve modularity and portability of the features. A low-level Linux kernel was
introduced to provide CPU load-balancing, memory management, and enhanced
hardware resource management. Cisco IOS now runs as a modular process on top of
the Linux kernel (known as Cisco IOSd). This approach allows other modular functions
to be introduced, such as Wireshark and a wireless LAN controller (WLC).

More applications will be embedded on Cisco IOS XE in the future, following a similar
approach.

Cisco IOS XE is continually evolving. With new applications appearing, the established
models for configuration and monitoring, such as CLI and SNMP, are beginning to be
replaced by standardized APIs for configuration and monitoring data models.
 Cisco IOS XE 87

DIAGRAM Classic Cisco IOS to Cisco IOS XE comparison

The latest Cisco IOS XE software addresses several key customer needs:

• A common OS for enterprise networks

• Rapid introduction of new features and technologies

• A secure, trustworthy OS to protect the network

• Modularity and high availability

• Programmability and automation

Considering these customer needs, Cisco IOS XE 16.1 was first introduced on Catalyst
3850 / 3650 Series switches, which use the programmable Cisco UADP 1.0 ASIC. Since
then, several other enterprise network platforms, including Catalyst 9000 switches,
have also adopted Cisco IOS XE for its software flexibility and scalability.

This unified OS software release brings multiple advantages:

• Consistent experience across platforms

• Fewer software images to manage

88 Cisco IOS XE

• Faster certification of software features

• Ability to run any feature anywhere

In addition, if there is a need to bring a feature from a core layer platform to an access
layer platform, this is much easier due to the use of a unified code release. In most
cases, importing the feature from one platform to another only requires platform
dependent code changes.

Catalyst 9000 family of switches have taken this one step further. The entire Catalyst
9000 switching family runs on the same code base and same image release. This
provides for a faster delivery of innovation, along with consistent feature behavior,
bringing added value by simplification of software image selection, deployment, and
use.

Note Catalyst 9200 Series has a different binary based on optimised version of
Cisco IOS XE called Cisco IOS XE Lite.
 Cisco IOS XE 89

Cisco IOS XE architecture

Cisco IOS XE is built on top of Linux OS. Various components of Cisco IOS XE run as
individual sub-processes and share a common information database that stores the
operational state of all the features in a consistent format. This modular OS
architecture not only provides key features such as process restartability and patching
but also enables the use of containers or virtual machines (VMs) for hosting Cisco and
third-party applications.

The Cisco IOS XE architecture has three significant enhancements:

• Cisco IOS modularity

• Cisco IOS XE database

• VMs and containers

Modular OS
With Cisco IOS XE, the classic Cisco IOS code is divided into multiple modules. The
majority of the base Cisco IOS code is hosted as a daemon (Cisco IOSd) which is
comprised of traditional Cisco IOS features and components such as switching and
routing protocols.

Cisco IOSd is further divided into multiple Cisco IOS subsystems, providing the
capability to service one of the sub-systems without affecting the remaining Cisco IOSd
code. Cisco IOSd also provides resiliency in case of individual subsystem failure as it is
completely segmented from the remaining Cisco IOS code.

This particular OS modularization helps with updating Cisco IOS by applying software
patches (known as software maintenance upgrades, or SMUs), without affecting the
running system.
90 Cisco IOS XE

DIAGRAM Modular Cisco IOS XE

Cisco IOS XE Database

Previously, Cisco IOS stored all switching and routing protocol state and related
forwarding information in a distributed manner. The process state information was
stored in many different parts of memory, in different formats, making it non-optimal
nor consumable outside the switch.

The Cisco IOS XE architecture decouples the data from the code. A new feature in the
OS is the Cisco IOS XE database that stores the configuration and operational state of
the system. The stored data is in a standardized format. Major benefits of storing the
state information in a centralized database include being able to share information
easily between different components of Cisco IOS XE.

This standard Cisco IOS XE database makes system data easier to express as data
models. Cisco IOS XE has an interface to convert the database to common data models
such as YANG, and provides efficient export using model-driven telemetry (MDT). MDT
is explained in greater detail in the chapter Programmability
Programmability and
and automation
automation.
automation

Containers and VMs

Cisco IOS XE supports containers and VMs hosting capability on Catalyst 9000
switches. Please refer to the chapter Application
Application hosting
hosting.
hosting
 Cisco IOS XE 91

Cisco IOS XE Lite

With Cisco IOS XE being modular, feature sets not applicable on a simple branch (such
as MPLS, BGP and ETA) could be removed from operating system libraries. Cisco IOSd
and bash were optimized to further reduce the footprint in Cisco IOS XE Lite. Even with
a 50% decrease in image size, core benefits of Cisco IOS XE are maintained and all
modern features such as streaming telemetry, failure isolation and status recovery are
available.
92 Cisco IOS XE

Cisco IOS XE bene ts

With the modern ever-changing software-defined environment, it is imperative that

the OS software foundation of Catalyst 9000 family of switches be open, easy to use,
flexible, and secure. Cisco IOS XE is an open and modular operating system common
across multiple enterprise network products and brings a number of benefits to
customers. The modularity, standard database, object-based models, and containers of
Cisco IOS XE provide key capabilities that help network administrators and engineers
with operational tasks and reduce operational costs.

Benefits include a single software image across Catalyst 9000 switches, simplifying
network administration and improving software lifecycle management. This provides a
consistent format and experience, with consistent provisioning interfaces across all
devices. A "run any feature anywhere" approach means that features can be ported very
quickly to other platforms. Recent examples of software imported to Catalyst platforms
in a short amount of time are MPLS, NAT, and NBAR2.

Some additional key benefits include Cisco IOS XE install mode, a new WebUI, and
Cisco trustworthy solutions.

Cisco IOS XE install mode consumes less memory because the packages are already
extracted from the .bin file. With install mode, Catalyst 9000 switches boot Cisco IOS
faster compared to bundle mode. Install mode is the recommended mode, and
advanced high-availability features such as ISSU, patching, and xFSU are only supported
with install mode.

Cisco IOS XE WebUI was introduced to help customers navigate the device through a
standard Web browser. Users can perform simple configurations, troubleshooting, and
monitoring high levels of CPU and memory utilization. Users can also configure
advanced feature such as AVC to monitor the various applications.

Cisco built the Catalyst 9000 family of switches to be trustworthy to help prevent
attacks against a network. As a Trustworthy solution, Catalyst 9000 switches verify the
 Cisco IOS XE 93

authenticity of the platform, prevent malicious code execution, establish run-time

defenses, and secures communication. For more information, please refer to the
chapter Trustworthy
Trustworthy solutions
solutions.
solutions
High availability
96 High availability

Overview

Building networks and network equipment with high availability (HA) is essential to
ensuring business continuity. The Catalyst 9000 family of switches offer several
traditional techniques for achieving HA and even introduces some new ones. This
section explores these techniques:

• High availability on the Catalyst 9200 Series - stacking, power

• High availability on the Catalyst 9300 Series - stacking, power, fast software
upgrade, extended fast software upgrade

• High availability on the Catalyst 9400 Series - chassis, supervisors, power,

ISSU

• High availability on the Catalyst 9500 Series - power, StackWise Virtual, ISSU

• High availability on the Catalyst 9600 Series - chassis, supervisors, power,

StackWise Virtual, ISSU

• Graceful insertion and removal

• Cisco IOS XE patching

The Catalyst 9000 family of switches utilizes two HA techniques:

Stateful switchover
Stateful switchover (SSO) offers minimal disruption to Layer 2 sessions for redundant
device configuration. SSO replicates forwarding tables and both the running and start-
up configuration between an active and a standby component. In the event that the
active device fails, the system immediately switches control over to the standby device.

Non-stop forwarding
Usually, when a networking device restarts, all routing peers of that device detect that
it went down and then came back up. This transition results in what is called a routing
 High availability 97

flap, which could spread across multiple routing domains. Routing flaps caused by
restarts create routing instabilities, which are detrimental to the overall network
performance. Non-stop forwarding (NSF) helps to suppress routing flaps in SSO-
enabled devices. NSF allows for the forwarding of data packets to continue along known
routes while the routing protocol information is being restored following a switchover.
With NSF, peer networking devices do not experience routing flaps.

The sub-sections in this chapter explain how each Catalyst 9000 switch series utilizes
SSO and NSF in slightly different ways for high availability.
98 High availability

Catalyst 9200 Series high availability

Catalyst 9200 Series switches deliver access layer redundancy with features such as
StackWise-160/80 and power supply redundancy.

Catalyst 9200 Series switch stacks

Fixed configuration Cisco Catalyst 9200 Series switches include stacking to expand
port density, switching capacity, and enable redundancy in wiring closets. Moreover,
stacking delivers operational simplicity by combining multiple switches together to
form a single logical switch.

Cisco IOS XE Lite on Catalyst 9200 Series switches support stacking individually among
C9200 and C9200L models for up to eight members but mixed stacking between them
is not supported. They are physically connected in a ring with special stacking cables
connected to the back of each switch using their stacking ports.

Catalyst 9200 Series switch stacks deliver deterministic and non-blocking switching
performance for up to 416 ports. The switching performance delivers hardware-
accelerated, integrated network services such as:

• PoE up to 15.4W

• PoE+ up to 30W

• Quality of service (QoS)

• Access control lists

• Flexible NetFlow

Note All switches in a stack must run the same version of Cisco IOS XE and
licensing.
 High availability 99

StackWise-160/80 architecture
Catalyst 9200 Series switches enable stacking using a stack-ring fabric known as either
StackWise-160 or StackWise-80. StackWise-160 is supported on C9200 switch models
with the support of 160 Gbps stack bandwidth. StackWise-80 is supported on C9200L
switch models with the support of 80 Gbps stack bandwidth. The fabric consists of two
counter-rotating rings (40/20 Gbps/ring), and the system's throughput is a function of
the aggregated throughput of these rings (80/40 Gbps). Throughput doubles by
employing spatial reuse on the stack's rings. Spatial reuse is enabled by destination
packet-stripping. Normally, within ring architectures, packet stripping from the ring
happens on the source switch where the packet originated and when ring members are
processing a packet, no other data may be passed into the ring. Spatial reuse, however,
allows multiple flows to co-exist. Spatial reuse frees available bandwidth on the ring as
the destination switch strips the packet destined to itself allowing insertion of
additional packets onto the ring by other stack members.

DIAGRAM StackWise-160/80 architecture

StackWise-160/80 creates a unified control and management plane by electing one

switch in the stack as an active switch and another switch as a hot-standby. Remaining
switches become stack members. The active switch is responsible for all Layer 2 and
Layer 3 network control processing and for synchronizing all state information with the
hot-standby. The active switch unifies management for the entire stack; administrators
perform all configuration and monitoring via the active switch.
100 High availability

The forwarding architecture is designed to provide distributed switching across all

member switches in the stack. Each switch in the stack optimizes data plane
performance by utilizing its local hardware resources. This includes forwarding tasks
and network services such as QoS and ACLs. Distributing stack processing delivers
wire-speed performance, increases overall system resource capacity, prevents
overloading of the active switch processor and optimizes stack-ring bandwidth
capacity.

StackWise-160/80 SSO support

Catalyst 9200 Series switches support a wide range of Layer 2 and Layer 3 stateful
capabilities to provide non-stop network communication. The active switch in a stack
synchronizes the protocol state machines, software forwarding tables, and system
configuration to the stack's standby switch. Supported protocols are listed in the table
below:

TABLE StackWise-160/80 stateful protocol support

Layer HA-aware protocols

STP, VLAN, VTP, DTP, CDP, UDLD, SPAN and RSPAN, 802.1x, PAgP and LACP, IGMP
Layer 2
snooping

Services QoS, ACL, PBR, NetFlow, port security

Power redundancy
Cisco Catalyst 9200 Series switches have two power supply bays with the support of
hot-swappable power supplies. The power supplies operate in two modes based on the
PoE and non-PoE models. PoE models support 1+1 combined mode, where system and
PoE power is shared by both power supplies. If one of the power supply fails, then the
remaining available power from the budget is utilized and there is no impact on either
the system components or the PoE Devices. If there is not enough power in the budget