BFD Architecture on NCS55xx and NCS5xx

Introduction

After successfully introducing the LPTS for control plane protection and the ACL’s for data-plane protection on the NCS55xx and NCS5xx portfolio, we will start this new series on faster convergence. All of us are already aware of the concept Bidirectional Forwarding Detection - BFD. BFD is fairly old technology and is widely deployed across the industry for faster network convergence. In this tech-note, we will discuss the implementation of BFD w.r.t NCS55xx and NCS5xx platforms.

Quick Refresh

Convergence of business-critical applications is a very important for any network infrastructure, whether it is service provider or any enterprise. Though today’s network have various levels of redundancy, but the convergence is dependant upon the ability of individual network devices to quickly detect failures and reroute traffic to an alternate path. Bidirectional forwarding detection (BFD) provides low-overhead, short-duration detection of failures in the path between adjacent forwarding engines. BFD allows a single mechanism to be used for failure detection over any media and at any protocol layer, with a wide range of detection times and overhead. The fast detection of failures provides immediate reaction to failure in the event of a failed link or neighbor.

Another benefit of BFD, is that it provides network administrators with a consistent method of detecting failures. Thus, one availability methodology could be used, irrespective of the Interior Gateway Protocol (IGP) or the topology of the target network. This eases network profiling and planning, because re-convergence time should be consistent and predictable. BFD function is defined in RFC 5880. BFD is essentially a Control plane protocol designed to detect the forwarding path failures. BFD adjacencies do not go down on Control-Plane restarts (e.g. RSP failover) since the goal of BFD is to detect only the forwarding plane failures. [Reference]

For understanding the BFD in details please visit the excellent articles. [Reference1 Reference2]. In this document, we will try to focus on BFD architecture and its hardware implementation on NCS55xx and NCS5xx product families.

BFD Modes Supported on NCS55xx and NCS5xx

For further details and working on the modes of operation, please Visit

BFD Single-Path and Multi-Path Sessions

BFD IPv4/IPv6 sessions over physical interfaces and sub-interfaces or bundle are referred to as Single-Path sessions. BFD sessions between virtual interfaces (BVI, BE, GRE tunnel) of directly connected peers is referred to as Multi-Path session because of possible asymmetrical routing. Both BFD Single-Path and Multipath sessions are supported on NCS55xx and NCS5xx.

BFD Single-Path Support

BFD Multi-Path Support

Note1: BFD Multi-Path (v4/v6) over BVI and BGP Multihop is supported on systems based on J2 from IOS-XR 7.4.1 in native and compatible mode.

Note2: BFD Multi-Path (v6) over BVI is not supported on NCS560. It will be supported in future releases.

BFD Hardware Implementation

BFD on the NCS55xx and NCS5xx is hardware offloaded. The hardware offload feature enables the offload of a BFD session to the network processing unit - NPU of the line cards on modular chassis or a standalone fixed chassis, in an IPv4 or IPv6 network. BFD hardware offload improves scale and reduces the overall network convergence time by sending rapid failure detection packets to the routing protocols for recalculating the routing table. NCS55xx and NCS5xx uses Pipeline architecture for packet processing. For details on the Pipeline Architecture please visit this excellent article. For further details on BFD hardware offload please visit. Let us understand the packet processing in the pipeline architecture w.r.t BFD.

RX Path

Ingress PP Pipeline

The ingress packet processing pipeline provides two main functions:

• BFD identification: Whether it is a BFD packet (after identifying it as for-us packet).
• BFD classification: Whether single path or a multi path BFD packet.

2-Pass Processing

All the BFD packets are processed in 2 cycles. In first cycle, packet is recycled on a well known port (internal) for core0 and core1 before BFD packet processing starts.

1st cycle:

• The first pass is based on L3 Lookup.
• BFD is treated as an IP packet and check will be made whether it is a ‘for-us’ packet.
• If the packet is identified as ‘for-us’ packet, it is recycled for second pass.
• The packet is then sent to the PMF stage.
• If the packet is not a for-us packet then based on the L3 lookup appropriate action is taken on the packet.

2nd cycle:

• After recycling the packet, parser qualifies the packet as BFD with parser code.
• Parser is the block which extracts ethertype, MAC addresses and determines offset (where network header starts) for next stages in the pipeline
• There is a specific parser code for BFD packet which is internal to the hardware.
• BFD FLP is hit based on the parser context and trap code is generated accordingly.
• In PMF stage, BFD packet is processed and sets the required traps & destination accordingly.
• The packets are then sent to OAMP Engine.
• When OAMP Engine receives the BFD packet, it has 2 options:
  • OAMP Engine consumes the BFD packet
  • OAMP Engine punts the packet to CPU.

OAMP Engine punts the BFD packet to CPU

• When a BFD packet is forwarded to the OAMP engine, it must receive some information on how to process the packet.
• Various blocks like FLP and PMF update this information in the pipeline and pass it on to the OAM engine.
• Upon the reception of the BFD packet, the following checks are done.
  • Verify that the packet and the traps are correct or else generate the appropriate error code.
  • Verify that the type taken from the trap is the correct.
  • There are various other checks which are internally done in the pipeline.
  • When any of the required checks fails, the corresponding failure bit is set and destination to the CPU is picked.
  • The packet is then punted to the CPU.

OAMP Engine consumes the BFD packet

• If all the checks pass without any failure bit set, the packet is consumed by the OAMP engine and processed. It is not punted to CPU.

Highlevel definition of the blocks used in the above discussion

Note: There are multiple blocks in the IRPP and ERPP in the pipeline for packet processing. Parser, FLP, PMF etc are one of those. This is specific to ASIC used, hence not exposing the internal details.

TX Path

• OAMP Engine will generate the BFD packet with complete BFD payload, L3 partial header and UDP header
• This packet is then injected into the IRPP for further processing.
• IRPP and ITPP will send the packet to the ETPP.
• In ETPP and ERPP the complete L3 header is populated and L2 header is also added to the packet
• Packet is then sent out accordingly from the network interface.

Note: OAMP Engine has various blocks which are internal and cannot be published. We have used a single block for simplication

BFD Feature Support

• BFD is supported only in Asynchronous mode. Demand and Echo mode are not supported.
• Static, OSPF, BGP and IS-IS applications are supported in IPv4 BFD.
• Only static routes are supported in IPv6 BFD.
• BFD over VRF is supported.
• BFD over BVI is supported only on fixed NCS5500 platforms.
• BFD support over VRRP interface is supported from IOS-XR 7.2.1
• BFD dampening for IPv4 is supported.
• BFD multihop is supported over an IP and non-IP core.
• BoB and BLB coexistence is not supported at the moment. It will be supported in the future.
• BFD with IPv4 supports upto 6 unique timers. With IPv6 we do not have limit of unique timers.

Note: We will dedicate separate technotes on some of the features described above for details.

BFD Scale

Scale on NCS55xx and NCS5xx is always a subject of discussion because of the hardware resources available on the chipsets. The hardware resources are limited and has to be utilised properly to achieve the best results. Let us discuss in more details on how BFD scale is determined for these platforms and how well we have managed to address the available resources.

Scale is decided on multiple things. First is obviously the hardware resources. From BFD feature perspective, the OAMP Engine resources play a critical role. The resources are equally divided between CFM and BFD feature needs. Now if we double-click on BFD, then within that resources are again divided for BFD Single-Path and BFD Multi-Path. Again one more consideration, we need to divide the resources amongst IPv4 and IPv6. For IPv4 we only need to worry from the OAMP resources. But for IPv6 we also need to take into account other internal resources which limits the scale (internal to ASIC). IPv6 needs almost three times more resources than IPv4 w.r.t OAMP engine. Again when the packets are recycled, we need to take into consideration the queue and its bandwidth shared with other features and number of recycles. All these factors goes into scale considerations. Each Asic type i.e. Q-MX/J/J+/J2 has different processing, resources and bandwidth capacity. So the scale will vary across chipsets.

Considering all the above criterias, we have succeeded in carving and the resources optimally , to provide right amount of BFD sessions (v4/v6) which suits every use cases.

Note: If you are looking for details on the scale per product please contact us or your cisco representative.

BFD Timers

NCS55xx and NCS5xx BFD Implementation supports timers which can be used for faster detection of failures. These are configurable values and users have the flexibility of configuring different timers and multiplier values. BFD v4 sessions do not have any limitations in scale w.r.t minimum timer values. But the BFD v6 scale limit does depend on the configured minimum timer. Below is the list of timers and multiplier values supported.

Support for IPv4 Timers and Multipliers

Note: Only 6 unique timers are supported with BFDv4.
Scale numbers for v4 will not be affected by the timer values

Support for IPv6 Timers and Multipliers

Note: No restrictions on number of unique timers support with BFDv6.
Scale numbers for v6 will be affected by the timer values. The above values are for J/J+/J2 compatible mode. For J2 native mode this numbers will be a bit higher. For other chipsets the numbers will vary. For detailed scale numbers please get in touch with us or your cisco representative.

Reference

• ASR9k BFD Implementation
• BFD NCS5500
• CCO Config Guide
• NCS5500 Deepdive Cisco Live

Thanks

Special thanks to Arun Vadamalai for his valueable feedback during the documentation.

Summary

This was an introductory article on BFD to get familiarise with feature with pipeline architecture. In the next article, we will focus more on the technical aspects of how to read the BFD parameters in the outputs and also see some basic debugging. We will touch upon the OAMP engine resource usage. We will discuss the concepts of BLB and BoB and look at its difference and use cases. Stay tuned !!!!