Cisco 8711-32FH-M Architecture White Paper
Introduction
The Cisco 8000 Series combines the revolutionary Cisco Silicon One™, IOS XR® software, and a set of clean sheet chassis to deliver a breakthrough in high-performance routers. High-performance networking systems have historically been divided into routing or switching classes, with distinct hardware and software. But, as the necessity for reliable networks proliferates, it is imperative that traditional networks shift to a new architecture to address exponential bandwidth needs, ubiquitous connectivity, security, better quality of service, low latency coupled with high reliability, availability, and serviceability of the infrastructure. The Cisco® 8000 Series portfolio completes this journey with the Cisco 8700 products. The Cisco 8700 series is an addition to the Cisco 8000 Series Routers powered by Cisco Silicon One™ ASICs. 8700 products embody this cutting-edge innovation of the 8000 portfolio, offering advanced features engineered for seamless integration and scalability. Whether enhancing existing infrastructure or enabling new capabilities, the 8700 series products empower organizations to achieve their goals with unmatched efficiency and effectiveness.
Cisco 8711-32FH-M Overview
Cisco 8711-32FH-M is a 1RU 12.8T fixed system based on a single P100 ASIC. With 16 ports of QSFP-DD800 and 16 ports of QSFP56-DD on the faceplate, Cisco 8711-32FH-M uses only 4 of the 6 slices of the P100, optimizing for higher scale and a broader L2 feature set than is currently capable on Q200 based fixed systems.
Video
Cisco 8711-32FH-M addresses scaled requirements for spine/leaf and collapsed core, Aggregation, Peering, DC Core, 3rd party colocation, and RON use cases.
Figure 1. Front view of the Cisco 8711-32FH-M
Figure 2. Rear view of the Cisco 8711-32FH-M
In the below table high level Cisco 8711-32FH-M overview is listed.
| Items | Descriptions |
|---|---|
| Data Plane | Single 12.8T Cisco Silicon One P100 (7nm) 72 MB fully shared packet buffer & 8 GB HBM packet buffer |
| Control Plane | Intel 6 Core x86 CPU 64 GB DRAM / 480 GB SSD |
| Port Configuration | 16x QSFP-DD800 + 16x QSFP56-DD |
| Management Ports | RS-232 console, 10 GbE Control Plane expansion, 10/100/1000Mbps Management, 1x USB 3.0, GPS (ToD, 10MHz, 1PPS), GNSS |
| Product Capabilities | 800G (QDD-2X400G-FR4, QDD-8X100G-FR) Line-rate MACsec(XR support)/IPsec(HW ready) encryption at all ports Class C timing 400G RON ready (Supports DCO optics) |
| Fans and PSU | 6x Fan Trays 5+1 Redundancy 2x PSUs 1+1 Redundancy Port side intake airflow, Port side Exhaust airflow 2kW AC & 2kW DC PSUs PSU and Fan Trays Field Replaceable Unit |
| Software | Supports Cisco IOS XR Software Release 24.3.1 and later Enhanced Automation for onboarding, service provisioning and monitoring Programmable infrastructure with segment routing (SR, SRv6) and Multi-homing EVPN |
| Typical Power | 740W w/o Optics |
| Dimension | 1RU, (H) 1.73 x (W) 17.3 x (D) 23.6 in. (4.40 x 43.9 x 60 cm) 35.4 lbs (16.07 kg) |
| Use Case | Core, Aggregation, Peering, DC Core |
Table 1. Cisco 8711-32FH-M key components
Cisco 8711-32FH-M Use Cases
Aggregation, Core, Peering and DC Core with RON(Routed Optical Networking)
As the user applications are becoming more dynamic, requiring differentiated user experiences network resources needs to be optimized in terms of power and real estate. Cisco 8711-32FH-M has been designed to cater the market requirement for total system bandwidth, port diversity, and MACsec encryption. The requirements far exceeded what could be accomplished with a single P100 NPU. Cisco 8711-32FH-M supports 12.8 Tbps of forwarding capacity. Supported various configurations of “400G only”, “800G Only” or “800G plus 400G combo” in 1RU height and 600mm depth form factor. It provides 32 ports of high speed 400G or 128 ports of 100G or 128 ports of low speed 10G/25G in the system.
Cisco’s Converged SDN Transport Solution, is an architecture that delivers efficient network utilization, Simplified single layer, reduced network complexity, faster time to market, automation empowerment, and differentiated service offering. The solution works by merging IP and Optical onto a single layer where all the switching is done at Layer 3. Routers are connected with standardized 400G ZR/ZR+/Bright(0dBm) ZR+ coherent pluggable optics. With a single service layer based upon IP, flexible management tools can leverage telemetry and model-driven programmability to streamline lifecycle operations.
Secure High-speed Data Center/Cloud interconnect and large Regional Hub/Branch router
The requirements for securing the connection of multiple data center and cloud links typically target a smaller amount of links, while requiring the highest level of bandwidth requirements. Sometimes called data center interconnect (DCI), WAN MACsec is an ideal encryption solution for interconnecting data center or multilocation cloud environments.
The backhauling of remote branch sites for government, enterprise, or commercial organizations is critical to any business. In the consumer space, this can be important for remote stores and point-of-sale kiosks, and in the enterprise and government space, it is crucial for remote agencies and offices.
Understanding the Cisco 8711-32FH-M Naming Logic
Let us have a quick refresher of the Cisco 8711-32FH-M’s PIDs and description.
| PIDs | Product Description |
|---|---|
| 8711-32FH-M | Cisco 8711 1RU 12.8T P100 System |
| 8000-QSFP-DCAP | QSFP Dust CAP |
| 8700-1RU-4P-KIT | Rack Mount Kit for Cisco 8700 1RU Chassis 19” and 23”, 4 POST |
| 8700-1RU-2P-KIT | Rack Mount Kit for Cisco 8700 1RU Chassis 19” and 23”, 2 POST |
| FAN-1RU-PI-V2 | 1RU Fan with Port-side Air Intake |
| FAN-1RU-PE-V2 | 1RU Fan with Port-side Air Exhaust |
| PSU2KW-ACPI | 2000W AC Power Module with Port-side Air Intake |
| PSU2KW-ACPE | 2000W AC Power Module with Port-side Air Exhaust |
| PSU2KW-DCPI | 2000W DC Power Module with Port-side Air Intake |
| PSU2KW-DCPE | 2000W DC Power Module with Port-side Air Exhaust |
| PWR-2KW-DC-CBL | 2000W DC Power Cable |
| ESS-8KE-100G-RTU | Essential Right-to-Use 100G for Cisco 8000E Series |
| ADN-8KE-100G-RTU | Advantage Right-to-Use 100G for Cisco 8000E Series |
| PRM-8KE-100G-RTU | Premium Right-to-Use 100G for Cisco 8000E Series |
| 8KSW-B-SIA-3 | 8000 Type B Device SIA for 3-year term FCM 2.0 |
| 8KSW-B-SIA-5 | 8000 Type B Device SIA for 5-year term FCM 2.0 |
Table 2. Cisco 8711-32FH-M PIDs
The RTUs (Right-to-Use licenses) provide customers with the ability to access and utilize specific perpetual software functionalities without the requirement to purchase the complete software package.
The SIAs (Software Innovation Access licenses) are term-based agreements that provide customers with access to specific software benefits. The SIAs enable customers to optimize their software usage, easily manage licenses across their network infrastructure, and ensure seamless upgrades to the latest versions of IOS XR software.
Platform Description
Cisco 8711-32FH-M is a 1RU fixed router belonging to Cisco 8700 family of products, a variant of the Cisco 8000 business, intended to be the primary routing solution for next generation Core, Aggregation, Peering, and DC Core networks.
Figure 3. Front view of Cisco 8711-32FH-M Router
As shown in Figure 3, The front of the chassis has 16 QSFP-DD800 + 16 QSFD56-DD ports along with the IO ports used for management, console, USB, and timing synchronization. Each QSFPDD port has its own bi-color LED that can be used to display the link status indication.
Notes : QSFP-DD800 is 800G capable version of the QSFP56-DD. It is backward compatible to legacy QSFP56- DD and all QSFP modules.
Figure 4. Mgmt/Timing Interface details
Front panel ports include management and Timing interfaces:
- Management interfaces
- 1x 10/100/1000 Mbps Management Ethernet port
- 1x USB 3.0 (Type A)
- RJ45 console port directly to CPU
- 1x 10 GbE Control plane Expansion Ethernet port (Not used)
- Timing interfaces
8711-32FH-M is capable of frequency, time, and phase synchronization. These can be sourced from dedicated interfaces GPS port or from timestamped 1588 packets received on a normal data port in the system. The time, phase and frequency are distributed through the system with a frequency and PPS signal to devices near the physical port where timestamps are applied on ingress and egress packets.
In 8711-32FH-M, network synchronization clock can be sourced from any of the QSFPDD ports and from GPS (ToD, 10MHz, 1PPS ports). - 1x GPS interface
- ToD (Time of Day) with RJ45 port
- 1 PPS coax port
- 10 MHz coax port
- 1x GNSS (Global Navigation Satellite System) receiver ( PRTC(Primary Reference Time Clock) Class B Accuracy)
Notes: No BITS interface support on 8711-32FH-M
As shown in Figure 5, There are also 4 LEDs on the front-panel that can be used to display the system status. The front panel faceplate is filled with perforations that provide airflow for cooling.
Figure 5. System status LEDs and descriptions details
As shown in Figure 6, on the rear side of the chassis, there’re the followings:
- 2 Power Supply Units (PSUs) with support of (1 + 1) redundancy
- Pull-out label: Serial number and PSU/Fan Tray location
- 6 Fan Tray (FT) with support of 1 Fan Failure condition
- PSUs and FTs are hot-swappable
Figure 6. Back view of Cisco 8711-32FH-M Router with 6 FTs and 2 PSUs
This router supports both AC and DC Power Supply Unit providing the system power. The Fan Trays provide system cooling via dual airflow direction mechanism – front to back(PSI) and back to front(PSE).
Fan Tray
Cisco 8711-32FH-M supports six Fan trays for system cooling. It supports N+1 redundancy. All the Fan Trays are hot swappable and get inserted from the rear side of the router. There are two types of fans available.
- Port Side Intake or Front to back Airflow (Burgundy color)
- Port Side Exhaust or back to front Airflow (Royal Blue)
All Fan Trays must be the same color (direction) to work correctly.
Figure 7. Back view of 6x FAN-1RU-PI-V2 + 2x PSU2KW-ACPI (Burgundy Latches)
Figure 8. PSI (Port-side Intake) Airflow of Cisco 8711-32FH-M Router
Figure 9. Back view of 6x FAN-1RU-PE-V2 + 2x PSU2KW-ACPE (Blue Latches)
Figure 10. PSE (Port-side Exhaust) Airflow of Cisco 8711-32FH-M Router
Fan Tray has tricolor LED (one for each Fan Tray) of color Red, Yellow, and Blue
Figure 11. Fan Tray LEDs of Cisco 8711-32FH-M Router
Power Supply Unit (PSU)
871-32FH-M has two PSUs supporting 1+1 redundancy. Redundancy will be supported for High Line Voltage mode(200Vac to 240Vac) of AC supply. When AC supply is used in low line volage(100Vac to 127Vac), redundancy is not supported.
Note: The Cisco 8711-32FH-M router doesn’t support a mix of AC and DC Power Supply Units (PSUs)
AC Power Supply Unit (PSU2KW-ACPI, PSU2K-ACPE)
- The AC power modules are single feed with 2000W capacity at 220V. They also support operation at 110V with half the capacity
- AC power modules are rated at 2000W maximum at 220V AC high line input, and 1000W maximum at 110V AC low line input
- AC nominal range: 100 to 127V AC and 200 to 240V AC
- AC full range: 90 to 140V AC and 180 to 264V AC; with extended range to 300V AC
DC Power Supply Unit (PSU2KW-DCPI, PSU2K-DCPE)
- The DC power modules are single feed with 2000W capacity at any specified input voltage
- DC nominal range: -40 to -72V DC
| PSU PID | Input Voltage | Input Current(Max) | Output Power | Output |
|---|---|---|---|---|
| PSU2KW-ACPI | 100Vac to 127Vac 200Vac to 240Vac | 12A 12A | 1,000W 2,000W | 12V/84A 12V/167A |
| PSU2KW-ACPE | 100Vac to 127Vac 200Vac to 240Vac | 12A 12A | 1,000W 2,000W | 12V/84A 12V/167A |
| PSU2KW-DCPI | -40Vd to -72Vdc | 60A | 2,000W | 12V/167A |
| PSU2KW-DCPE | -40Vd to -72Vdc | 60A | 2,000W | 12V/167A |
Table 3. PSUs types of Cisco 8711-32FH-M
RP/0/RP0/CPU0:8711-32FH-M#show platform
Node Type State Config state
--------------------------------------------------------------------------------
0/RP0/CPU0 8711-32FH-M(Active) IOS XR RUN NSHUT
0/FB0 8711-32FH-M[FB] OPERATIONAL NSHUT
0/FT0 FAN-1RU-PI-V2 OPERATIONAL NSHUT
0/FT1 FAN-1RU-PI-V2 OPERATIONAL NSHUT
0/FT2 FAN-1RU-PI-V2 OPERATIONAL NSHUT
0/FT3 FAN-1RU-PI-V2 OPERATIONAL NSHUT
0/FT4 FAN-1RU-PI-V2 OPERATIONAL NSHUT
0/FT5 FAN-1RU-PI-V2 OPERATIONAL NSHUT
0/PM0 PSU2KW-ACPI OPERATIONAL NSHUT
0/PM1 PSU2KW-ACPI OPERATIONAL NSHUT
On this router, there are two Power Supply Units in Cisco 8711-32FH-M
RP/0/RP0/CPU0:8711-32FH-M#show environment power
================================================================================
CHASSIS LEVEL POWER INFO: 0
================================================================================
Total output power capacity (Group 0 + Group 1) : 2000W + 2000W
Total output power required : 1187W
Total power input : 913W
Total power output : 789W
Power Group 0:
================================================================================
Power Supply ------Input---- ------Output--- Status
Module Type Volts Amps Volts Amps
================================================================================
0/PM0 PSU2KW-ACPI 207.5 2.3 12.1 34.6 OK
Total of Group 0: 477W/2.3A 418W/34.6A
Power Group 1:
================================================================================
Power Supply ------Input---- ------Output--- Status
Module Type Volts Amps Volts Amps
================================================================================
0/PM1 PSU2KW-ACPI 207.5 2.1 12.1 30.7 OK
Total of Group 1: 435W/2.1A 371W/30.7A
================================================================================
Location Card Type Power Power Status
Allocated Used
Watts Watts
================================================================================
0/RP0/CPU0 8711-32FH-M 917 - ON
0/FT0 FAN-1RU-PI-V2 45 9 ON
0/FT1 FAN-1RU-PI-V2 45 9 ON
0/FT2 FAN-1RU-PI-V2 45 10 ON
0/FT3 FAN-1RU-PI-V2 45 9 ON
0/FT4 FAN-1RU-PI-V2 45 9 ON
0/FT5 FAN-1RU-PI-V2 45 9 ON
RP/0/RP0/CPU0:8711-32FH-M#show environment fan
=============================================================
Fan speed (rpm)
Location FRU Type FAN_0 FAN_1
-------------------------------------------------------------
0/FT0 FAN-1RU-PI-V2 19080 16650
0/FT1 FAN-1RU-PI-V2 19020 16770
0/FT2 FAN-1RU-PI-V2 19230 16860
0/FT3 FAN-1RU-PI-V2 19110 16740
0/FT4 FAN-1RU-PI-V2 19050 16680
0/FT5 FAN-1RU-PI-V2 19080 16650
0/PM0 PSU2KW-ACPI 14656 13312
0/PM1 PSU2KW-ACPI 14560 13408
8711-32FH-M CPU card
8711-32FH-M uses common CPU card designed for Cisco 8000 fixed router.
CPU complex card is designed with Intel 6 core processor. This processor running IOS XR operating system manages the system by setting up the control plane, NPU, and communicates with various subsystems.
| Device | Configuration |
|---|---|
| CPU | Intel 6 Core @ 2.5GHz |
| Memory | 64 GB DDR4 SODIMM |
| M.2 SATA Drive | 480 GB SATA SSD |
| USB | 1 x USB3.0 |
| MGMT | MGMT copper |
| Console | |
| Aikido FPGA | Aikido Secure Boot |
IOS XR Software
The Cisco 8711-32FH-M is launched with IOS XR 24.3.1
In the software download center, the search engine will not help with 8711 keywords:
But you can pick any 8000 (8200, 8600, 8800) image to install your Cisco 8711-32FH-M routers:
RP/0/RP0/CPU0:8711-32FH-M#show version
Cisco IOS XR Software, Version 24.3.1 LNT
Copyright (c) 2013-2024 by Cisco Systems, Inc.
Build Information:
Built By : sajshah
Built On : Wed Sep 04 18:28:32 UTC 2024
Build Host : iox-ucs-002
Workspace : /auto/srcarchive11/prod/24.3.1/8000/ws
Version : 24.3.1
Label : 24.3.1-iso
cisco 8000 (Intel(R) Xeon(R) CPU D-1633N @ 2.50GHz)
cisco 8711-32FH-M (Intel(R) Xeon(R) CPU D-1633N @ 2.50GHz) processor with 64GB of memory
8711-32FH-M uptime is 25 minutes
Cisco 8711 1RU 12.8T P100 System
From a security / integrity perspective, the Cisco 8711-32FH-M implements the latest secureboot features with TAm (Trust Anchor module) FPGA.
For a complete list of supported features, refer to the Cisco Feature Navigator.
Cisco Software Download page to download the Cisco IOS XR software images.
Detailed information on IOS XR can be found in the IOS XR data sheet.
You can refer to Upgrade Guide on CCO as well.
Cisco 8711-32FH-M Port Identification
Port Numbering
Cisco 8711-32FH-M port numbering follows that of physical interfaces as {Type}{R/S/I/P} or {Type}{R/S/I/P/B}
Native Interface Use-case with 100 GbE at 400 G port
RP/0/RP0/CPU0:8711-32FH-M#show ip interface brief
Interface IP-Address Status Protocol Vrf-Name
HundredGigE0/0/0/0 40.0.0.1 Up Up default
HundredGigE0/0/0/0.1 11.0.1.1 Up Up default
HundredGigE0/0/0/0.2 11.0.2.1 Up Up default
HundredGigE0/0/0/0.3 11.0.3.1 Up Up default
HundredGigE0/0/0/0.4 11.0.4.1 Up Up default
HundredGigE0/0/0/0.5 11.0.5.1 Up Up default
HundredGigE0/0/0/0.6 11.0.6.1 Up Up default
HundredGigE0/0/0/0.7 11.0.7.1 Up Up default
HundredGigE0/0/0/0.8 11.0.8.1 Up Up default
HundredGigE0/0/0/0.9 11.0.9.1 Up Up default
HundredGigE0/0/0/0.10 11.0.10.1 Up Up default
Breakout Interface Use-case with 2x400 GbE at 800G port ( Even port, Top row ports)
hw-module port-range 0 1 instance 0 location 0/RP0/CPU0 mode 2x400
RP/0/RP0/CPU0:8711-32FH-M#show ip interface brief
Interface IP-Address Status Protocol Vrf-Name
FourHundredGigE0/0/0/0/0 unassigned Up Up default
FourHundredGigE0/0/0/0/1 unassigned Up Up default
Breakout Interface Use-case with 8x100 GbE at 800G port ( Even port, Top row ports)
hw-module port-range 2 3 instance 0 location 0/RP0/CPU0 mode 8x100
RP/0/RP0/CPU0:8711-32FH-M#show ip interface brief
Interface IP-Address Status Protocol Vrf-Name
HundredGigE0/0/0/2/0 unassigned Down Down default
HundredGigE0/0/0/2/1 unassigned Down Down default
HundredGigE0/0/0/2/2 unassigned Down Down default
HundredGigE0/0/0/2/3 unassigned Down Down default
HundredGigE0/0/0/2/4 unassigned Down Down default
HundredGigE0/0/0/2/5 unassigned Down Down default
HundredGigE0/0/0/2/6 unassigned Down Down default
HundredGigE0/0/0/2/7 unassigned Down Down default
System Details
System Block Diagram
The Cisco 8711-32FH-M has one common CPU complex board, P100 NPU, 2 Power Supply Units (PSU) with support of (1 + 1) redundancy and 6 Fan Tray (FT) with support of 1 Fan Failure condition. – These PSUs and FTs are field replaceable units (FRUs).
Figure 12. Cisco 8711-32FH-M Block Diagram
P100 is a family of high-capacity packet switching devices. P100 consists of 6 slices. Each slice has a Receive and a Transmit part (Rx slice and Tx slice). Each slice supports two groups of interfaces (i.e., a pair of IFGs). Performance wise, each slice handles up to one packet per clock and has a data bandwidth of up to 3.2 Tbps. The total capacity of a P100 device is determined by the number of slices. With four slices used in Cisco 8711-32FH-M and up to 5.047 Bpps forwarding performance per system.
Cisco Silicon One P100 features:
- 19.2 Tbps full-duplex and 8.1 Bpps Forwarding Capacity NPU
- 192x 100G SerDes with each capable of operating at 10G/25G/50G/100G using NRZ or PAM4 modulation
- Flexible port configuration supporting 10/25/40/50/100/200/400/800 Gbps
- 72 MB fully shared on-die packet buffer
- Expandable in-package packet buffer using 8 GB HBM
- On-chip, high-performance, P4-programmable host NPU for high-bandwidth offline packet processing
- Multiple embedded processors for CPU offloading
- 64K VoQ (Virtual Output Queue) for Network slice and 16K VoQ for Fabric slice
- Dedicated 132 Counters banks (1M counters) and built-in counters in HCAM(Hash-based TCAM)
- 6M IPv4 or 3M IPv6 FIB qualified scale without compression (Hardware capable much more)
Figure 13. Cisco Silicon One P100
Figure 14. Cisco Silicon One P100 Details
Cisco 8711-32FH-M uses only 4 of the 6 slices of the P100, optimizing for higher scale and a broader L2 feature set than is currently capable on Q200 based fixed systems.
Figure 15. Cisco Silicon One P100 in 8711-32FH-M
For more understanding on Cisco Silicon One P100 refer to Cisco Silicon One P100-Info.
Port/IFG/Slice assignment to P100 based Cisco 8711-32FH-M Router
There is the output of Port-IFG(Interface Group)-Slice mapping information.
Figure 16. Port-IFG-Slice mapping of Cisco 8711-32FH-M
RP/0/RP0/CPU0:8711-32FH-M#show controllers npu voq-usage interface all instance all loc 0/rp0/cpu0
----------------------------------------------------------------------------------------
Node ID: 0/RP0/CPU0
Intf Intf NPU Slice IFG Sys VOQ Flow VOQ Port Global
name handle # # # Port base base port speed IFG #
(hex) type
----------------------------------------------------------------------------------------
Hu0/0/0/0 78000130 0 3 1 4 576 0 local 100G 7
Hu0/0/0/1 78000158 0 3 1 9 584 0 local 100G 7
FH0/0/0/2 78000228 0 3 1 35 824 0 local 400G 7
FH0/0/0/3 78000220 0 3 1 34 816 0 local 400G 7
FH0/0/0/4 78000218 0 3 0 33 808 0 local 400G 6
FH0/0/0/5 78000210 0 3 0 32 800 0 local 400G 6
FH0/0/0/6 78000160 0 3 0 10 640 0 local 400G 6
FH0/0/0/7 78000208 0 3 0 31 648 0 local 400G 6
FH0/0/0/8 78000168 0 2 1 11 656 0 local 400G 5
FH0/0/0/9 78000170 0 2 1 12 664 0 local 400G 5
FH0/0/0/10 78000178 0 2 1 13 672 0 local 400G 5
FH0/0/0/11 78000180 0 2 1 14 680 0 local 400G 5
Hu0/0/0/12/0 78000360 0 2 0 15 696 0 local 100G 4
Hu0/0/0/12/1 78000368 0 2 0 16 832 0 local 100G 4
Hu0/0/0/12/2 78000370 0 2 0 39 840 0 local 100G 4
Hu0/0/0/12/3 78000378 0 2 0 40 848 0 local 100G 4
Hu0/0/0/12/4 78000380 0 2 0 41 856 0 local 100G 4
Hu0/0/0/12/5 78000388 0 2 0 42 864 0 local 100G 4
Hu0/0/0/12/6 78000390 0 2 0 43 872 0 local 100G 4
Hu0/0/0/12/7 78000358 0 2 0 38 688 0 local 100G 4
FH0/0/0/14 78000198 0 2 0 17 704 0 local 400G 4
FH0/0/0/15 780001a0 0 2 0 18 712 0 local 400G 4
Hu0/0/0/16 78000150 0 1 1 8 632 0 local 100G 3
FH0/0/0/17 780001a8 0 1 1 19 720 0 local 400G 3
FH0/0/0/18 780001b0 0 1 1 20 728 0 local 400G 3
Hu0/0/0/19 78000148 0 1 1 7 624 0 local 100G 3
FH0/0/0/20/0 780001b8 0 1 0 21 616 0 local 400G 2
FH0/0/0/20/1 780001c0 0 1 0 22 736 0 local 400G 2
FH0/0/0/22 780001c8 0 1 0 23 744 0 local 400G 2
Hu0/0/0/23 78000140 0 1 0 6 608 0 local 100G 2
Hu0/0/0/24 78000138 0 0 1 5 592 0 local 100G 1
FH0/0/0/25 780001d0 0 0 1 24 600 0 local 400G 1
FH0/0/0/26 780001d8 0 0 1 25 752 0 local 400G 1
FH0/0/0/27 780001e0 0 0 1 26 760 0 local 400G 1
FH0/0/0/28 780001e8 0 0 0 27 768 0 local 400G 0
FH0/0/0/29 780001f0 0 0 0 28 776 0 local 400G 0
FH0/0/0/30 780001f8 0 0 0 29 784 0 local 400G 0
FH0/0/0/31 78000200 0 0 0 30 792 0 local 400G 0
From the above output, we can see Port 0 to port 7 is connected to slice 3 and Port 8 to port 15 is connected to slice 2.
PHY
IFG of P100 connects to the PHYs through 128 SerDes each operating at 100G data rate The PHY can supports the followings:
- Up to full 1.6 Tbps capacity per PHY
- 16 SerDes in host (P100 NPU) side each SerDes operating at 100G
- 16 SerDes in the line (Front Panel Ports) side supporting at 100G (Retimer/800G mode)
- 32 SerDes in the line (Front Panel Ports) side support at 50G (RGB mode)
- PHY controlled from FPGA
- SynE clock recovery and PTP (Class C timing) supported by PHY
- Line-rate MACsec (XR support) or IPsec (HW ready)
Figure 17. NPU to PHY to QSFPDD Serdes Connectivity
Front Panel Port combo configurations
Cisco 8711-32FH-M can support several interface combinations.
The first of the configurations is 32x Grey 400 GbE QSFP56-DD, in this configuration the QSFPDD ports are populated with 400G Grey optics. This is the default mode(without manual configuration). There is no restriction.
The second of the configurations is 32x Grey 100 GbE QSFP, in this configuration the QSFPDD ports are populated with 100G optics not 400G, the QSFP28 ports are populated with 100G optics without manual configuration. There is no restriction.
The third of the configurations is 400G DCO (Digital Coherent Optic) optics, in this configuration 400G DCO ports are populated with 400G ZR/ZR+/Bright ZR+ into even ports (0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30 : Top).
The fourth of the configurations is 400G DCO (Digital Coherent Optic) and Grey 400G optics combinations, in this configuration 400G DCO ports are populated with 400G ZR/ZR+/Bright ZR+ into even ports (0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30 : Top) and 400G Grey optic into Odd ports (1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31: Bottom).
The 5th of the configurations is 16x Grey 800G Grey Optics (QDD 2X400G FR4 or QDD 8X100G FR), in this configuration 800G ports are populated with 800G optics into even ports (0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30 : Top) and all other Odd ports(Bottom) are shutdown automatically.
The 6th of the configuration is Grey 800G Optics and 400G DCO (ZR/ZR+/Bright ZR+) Optics, in this configuration the number of 800G Grey and 400G DCO (ZR, ZR+, Bright ZR+) must sum to 16 on Even ports (0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30: Top).
The 7th of the configuration is Grey 800G Optics, 400G DCO Optics (ZR, ZR+, Bright ZR+), and Grey 400G Optics combination, in this configuration the number of 800G Grey and 400G DCO must sum to 16 on Even ports (0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30: Top). For every even port with 400G DCO (ZR, ZR+, Bright ZR+), the corresponding odd port can also have Grey 400G optics.
To verify optics supported, please use the Cisco Optics-to-Device Compatibility Matrix tool: the TMG matrix.
It contains details on the connector types, the reach, the minimum release required, etc.
Maximum port scale and support on Cisco 8711-32FH-M
The following table represents the maximum number of ports the Cisco 8711-32FH-M router can support.
| 10 GbE | 25 GbE | 40 GbE | 100 GbE | 400 GbE | 800G* | |
|---|---|---|---|---|---|---|
| Native | Roadmap | Roadmap | 32 | 32 | 32 | N/A |
| Breakout | 128 | 128 | N/A | 128** | 32** | 16 Even ports |
* 2x400G & 8x100G only
** 2x400G only
*** 4x100G, 8x100G
- 10 GbE
- As Native 10G Optic with CVR-QSFP-SFP10G (QSA), fully populated into all 32 Ports in roadmap
- As Breakout, fully populated into all 32 Ports. 32 ports x (4x10 GbE) = 128 10 GbE total
- 25 GbE
- As Native 25G Optic , fully populated into all 32 Ports in roadmap
- As Breakout, fully populated into all 32 Ports. 32 ports x (4x25 GbE) = 128 25 GbE total
- 40 GbE
- As Native 40G Optic , fully populated into all 32 Ports
- 100 GbE
- As Native 100G Optic, fully populated into all 32 Ports
- As Breakout (Use case-2: 2x100 GbE), fully populated into all 32 Ports. 32 ports x (2x100 GbE) = 64 100 GbE total or
- As Breakout (Use case-2: 4x100 GbE), fully populated into all 32 Ports. 32 ports x (4x100 GbE) = 128 100 GbE total or
- As Breakout (Use case-3: 8x100 GbE), populated into 16 even ports(Top) x (8x100 GbE) = 128 100 GbE total
- 400 GbE
- As Native 400G Optic, fully populated into all 32 Ports
- As Breakout with 2x 400 GbE, Populated into even 16 Ports (Top row). 16 Even ports (Top) x (2x400 GbE) = 32 400 GbE total
- 800 G
- As Native 800GbE Optic, Not supported
- As Breakout, Used with 2x400 GbE or 8x100 GbE optics at Even ports (Top row)
Other things to know?
- We will support 10 GbE optics (SFP-10G-LR-S, SFP-10G-SR-S, SFP-10G-ER-S) via QSFP to SFP Adaptor (QSA: CVR-QSFP-SFP10G). Not supported in XR 24.4.1, in the roadmap
- 1G will not be supported
800G Transceiver
2x400G (QDD-2X400G-FR4) Optical Transceiver
Figure 18. QDD-2X400G-FR4 Optical connector
- Fiber: SMF
- Connector: Dual Duplex LC
- 4 Wave (nm): 1270,1290,1310,1330
- Use-Case: 2 ports 400 GbE
8x100G (QDD-8X100G-FR) Optical Transceiver
Figure 19. QDD-8X100G-FR Optical connector
- Fiber: SMF
- Connector: Dual MPO-12 (APC)
- Wave (nm): 1310 - 1 wavelength per fiber
- Use-Case: 8 ports 100 GbE with 2x MPO-4xLC SMF breakout cable or 2 ports 400 GbE with MPO-MPO SMF
QSFP-DD800 Optics Use Cases
Sample 800G Breakout config on Cisco 8711-32FH-M
Use-Case 1. QDD-2X400G-FR4 at Port 0 and QDD-8X100G-FR at Port 2
hw-module port-range 0 1 instance 0 location 0/RP0/CPU0 mode 2x400
hw-module port-range 2 3 instance 0 location 0/RP0/CPU0 mode 8x100
RP/0/RP0/CPU0:8711-32FH-M#show interfaces brief | inc FH0/0/0/0
Intf Intf LineP Encap MTU BW
Name State State Type (byte) (Kbps)
--------------------------------------------------------------------------------
FH0/0/0/0/0 down down ARPA 1514 400000000
FH0/0/0/0/1 down down ARPA 1514 400000000
RP/0/RP0/CPU0:8711-32FH-M#show interfaces brief | inc Hu0/0/0/2
Intf Intf LineP Encap MTU BW
Name State State Type (byte) (Kbps)
--------------------------------------------------------------------------------
Hu0/0/0/2/0 down down ARPA 1514 100000000
Hu0/0/0/2/1 down down ARPA 1514 100000000
Hu0/0/0/2/2 down down ARPA 1514 100000000
Hu0/0/0/2/3 down down ARPA 1514 100000000
Hu0/0/0/2/4 down down ARPA 1514 100000000
Hu0/0/0/2/5 down down ARPA 1514 100000000
Hu0/0/0/2/6 down down ARPA 1514 100000000
Hu0/0/0/2/7 down down ARPA 1514 100000000
Use-Case 2. QDD-2X400G-FR4 at Port 0 and Grey 400G optics at Port 2 and Port 3
hw-module port-range 0 1 instance 0 location 0/RP0/CPU0 mode 2x400
RP/0/RP0/CPU0:8711-32FH-M#sh interfaces brief | inc FH0/0/0/0
Intf Intf LineP Encap MTU BW
Name State State Type (byte) (Kbps)
--------------------------------------------------------------------------------
FH0/0/0/0/0 down down ARPA 1514 400000000
FH0/0/0/0/1 down down ARPA 1514 400000000
RP/0/RP0/CPU0:8711-32FH-M#sh interfaces brief
Intf Intf LineP Encap MTU BW
Name State State Type (byte) (Kbps)
--------------------------------------------------------------------------------
FH0/0/0/2 down down ARPA 1514 400000000
FH0/0/0/3 down down ARPA 1514 400000000
Use-Case 3. QDD-8X100G-FR at Port 2 and Grey 400G Optics at Port 0 and Port 1
hw-module port-range 2 3 instance 0 location 0/RP0/CPU0 mode 8x100
RP/0/RP0/CPU0:8711-32FH-M#show interfaces brief
Intf Intf LineP Encap MTU BW
Name State State Type (byte) (Kbps)
--------------------------------------------------------------------------------
FH0/0/0/0 down down ARPA 1514 400000000
FH0/0/0/1 down down ARPA 1514 400000000
RP/0/RP0/CPU0:8711-32FH-M#show interfaces brief | inc Hu0/0/0/2
Intf Intf LineP Encap MTU BW
Name State State Type (byte) (Kbps)
--------------------------------------------------------------------------------
Hu0/0/0/2/0 down down ARPA 1514 100000000
Hu0/0/0/2/1 down down ARPA 1514 100000000
Hu0/0/0/2/2 down down ARPA 1514 100000000
Hu0/0/0/2/3 down down ARPA 1514 100000000
Hu0/0/0/2/4 down down ARPA 1514 100000000
Hu0/0/0/2/5 down down ARPA 1514 100000000
Hu0/0/0/2/6 down down ARPA 1514 100000000
Hu0/0/0/2/7 down down ARPA 1514 100000000
For more understanding on Cisco QSFP-DD800 Transceiver Modules, refer to Cisco QSFP-DD800 Transceiver Modules Data Sheet.
MACsec Support
Starting IOS-XR 24.4.1, MACsec is supported on all ports of Cisco 8711-32FH-M. All ports are connected to PHY supporting the feature. It can be supported on 10G, 25G, 40G, 100G, 400G ports and also any breakout ports.
- Full line-rate MACsec encryption engines to enable secure and low latency transport
- Supports MACsec GCM-AES-128/256 and GCM-AES-XPN-128/256
- Supports LAN MACsec and WAN MACsec
- Supports “VLAN tag in the clear” in the roadmap
Notes: k9 package is required for crypto data plane
Cisco 8711-32FH-M can support the following MACsec Use Cases
Notes: Use Case #3b and #4 in the roadmap
| Use Cases | Use Case Descriptions | Cisco 8711-32FH-M |
|---|---|---|
| Use Case #1 | Link MACsec in MPLS/IP Topology | Supported |
| Use Case #2 | Link MACsec over LAG members | Supported |
| Use Case #3a | CE Port Mode MACsec over P2P L2VPN | Supported |
| Use Case #3b | CE Port Mode MACsec over MP L2VPN | Roadmap |
| Use Case #4 | VLAN Clear Tags MACsec over L2VPN | Roadmap |
Table 4. Cisco 8711-32FH-M MACsec Use Cases supported
Cisco 8711-32FH-M can support the following features to interoperate over the service provider.
- Ability to change MKA EAPoL Destination Address type
- Ability to change MKA Ether-type value
- Ability to configure Anti-replay window sizes
MACsec configurations on Cisco 8711-32FH-M
Fan Tray Redundancy
Cisco 8711-32FH-M provides 5+1 Fan Tray Redundancy. The system can operate in following conditions:
- 6 Fan Trays is Ideal case
- 5+1 Fan Tray, followed by major alarm. Speed up the all system Fans to 100%
Packet flow in Cisco 8711-32FH-M
Figure 20. Data path flow in Cisco 8711-32FH-M
- Packets arrive on the input slice of the receive device
- The NPU performs the packet lookup, feature (example: ingress ACL) checks, and identifies the destination for the packet
- The ingress NPU enqueues the packet on the corresponding VoQ for the {destination, traffic class}
- When credits are available for the destination VoQ, the packet is switched across the SMS to the destination (egress) slice
- On the egress slice, the transmit direction lookups and feature (example: Egress ACL) checks are performed
- The final encapsulations are added to the packet and it’s transmitted from the TX NPU out the physical interface
- All packets are switched through the same SMS regardless of whether they are bound for the same NPU slice or a different NPU slice
Conclusion
This document discussed the Cisco 8711-32FH-M architecture.
The Cisco 8711-32FH-M is the P100 based fixed router. It offers a very high 10G, 25G, 100G, and 400G port density including MACsec and RON in a very small size (1RU).
Cisco 8711-32FH-M is intended to be the primary routing solution for next generation Core, Aggregation, Peering, and DC Core networks.
References
Cisco Silicon One™ (P100)
Cisco 8711-32FH-M Datasheet
Cisco 8711-32FH-M Hardware Installation Guide
Cisco 8000 Configuration guide
Cisco Optics-to-Device Compatibility Matrix tool
Cisco 8000 Power Calculator
802.1AE MACsec White Paper
Modification History
| Version | Data | Author(s) | Comments |
|---|---|---|---|
| 1 | January 8, 2025 | Chang Soo Lee | Initial Publication |
Leave a Comment