NCS5500 URPF: Configuration and Impact on Scale

12 minutes read

You can find more content related to NCS5500 including routing memory management, VRF, ACLs, Netflow following this link.

Change Log

DateChange
August 2018note added on the lack of support of S-RTBH, and to fix an error pointed by Muffadal Presswala (thanks :) related to the behavior with eTCAM systems
November 2020S-RTBH support has been added in IOSXR 7.2.1 but only for line cards and platforms powered by Jericho2 and eTCAM (NC57-18DD-SE for example)
November 2020Change of configuration in 7.x
December 2020decommission the “internet-optimized” mode, please check this article
June 2024Added section about S-RTBH support

S01E07 NCS5500 URPF Configuration and Impact on Scale

Previously on “Understanding NCS5500 Resources”

In previous posts, we presented:

In this new episode, we will cover the impact of activating URPF on the NCS5500 routers.

Definition

URPF stands for Unicast Reverse Path Forwarding.

Definition from the CCO website:

This security feature works by enabling a router to verify the reachability of the source address in packets being forwarded. This capability can limit the appearance of spoofed addresses on a network. If the source IP address is not valid, the packet is discarded.

Unicast RPF in strict mode, the packet must be received on the interface that the router would use to forward the return packet.

Unicast RPF in loose mode, the source address must appear in the routing table.

It’s a feature configured at the interface level.

URPF relevancy

Regardless of the Forwarding ASIC (Qumran-MX, Jericho or Jericho+), the NCS5500 only supports URPF in loose mode today.

Configuring URPF comes at a cost in term of scale on some of the NCS5500 family members. It will be detailed extensively in this article. That’s why it’s important to understand what are the benefits of enabling this feature.

As explained in the definition section above, the loose mode simply verify that source addresses of the packets received are in of the routable space. To bypass this “protection”, it’s fairly easy for an attacker to pick source addresses inside existing routes when forging the packet instead of totally random addresses.

We invite the operators to check how much traffic is currently dropped by the URPF loose mode if they have it enabled on production routers.

Example: to check this on an ASR9000:


RP/0/RP0/CPU0:Router#show cef drops | i RPF drops

  RPF drops            packets :               0
  RPF drops            packets :               0
  RPF drops            packets :               0
  RPF drops            packets :               0
  RPF drops            packets :               0
  RPF drops            packets :               0
  RPF drops            packets :               0
  RPF drops            packets :               0
  RPF drops            packets :           50065
  RPF drops            packets :               0
  RPF drops            packets :            1262
  RPF drops            packets :         3627918
  RPF drops            packets :            1262
  
-- SNIP --

And compare these figures to the packet count per interface to understand how much traffic it represents. The impact it could have on route scale and the protection efficiency it offers need to be put in perspective before deciding if it is worth enabling URPF.

Now said, some other very good reasons to enable URPF loose mode exist. For example, it’s a mandatory brick of a Source-based Remotely Triggered Black Hole architecture (S-RTBH).

But… S-RTBH is not supported currently on most of the NCS5500 platforms (even if the URPF loose-mode is supported, it can not be used for this particular use-case). Only exception: the NC57-18DD-SE line cards and all future platforms based on Jericho2 (with eTCAM).

S-RTBH support on NCS5500

the table below clarifies the support of S-RTBH in different deployment scenario while using NCS5500

ReleasePlatformDetails
7.2.1J2 based platform (with eTCAM)Support for S-RTBH have been introduced and enabled by default (can’t be disabled)
7.5.2J2 based platform (with eTCAM)hw-module fib urpf disable command have been introduced to provide the possibility of globally disable URPF and S-RTBH (require router reload)
24.4.1J/J+ based platformSupport for S-RTBH will be introduced for J/J+ platforms and enabled based on the ingress interface configuration command ipv[46] verify unicast source reachable-via any
TBDJ2 based platform (with eTCAM)Interface command ipv[46] verify unicast source reachable-via any requirement is enforced for J2 platform as well to achieve consistency

NCS5500 Implementation

We don’t support URPF strict mode today. URPF loose mode is available on NCS5500 since IOS XR 6.2.2 for IPv4 and IPv6. The feature is supported on Jericho and Jericho+ systems, with or without eTCAM.

The configuration implies the deactivation of some profiles, different on “base” and “scale” systems. After this preliminary operation, the configuration is applied at the interface level.

Deactivating URPF on an interface implies to do it for both IPv4 and IPv6.

Allow-self-ping is the default mode and allow-default is not supported.

Configuration and impact on base systems (no eTCAM)

On “base” systems (without external TCAM):

Since URPF requires two accesses to the LEM (lookup for source address then for destination address in the packet header), we have to disable the optimizations present by default or after a configuration:


hw-module fib ipv4 scale host-optimized-disable
hw-module fib ipv6 scale internet-optimized-disable

Note: depending on the IOS XR version, the options could be different and actually could be the opposite of “disable”, be attentive at what is availabe in the CLI.

Note: in IOS XR 7.3.1, we will decommission the “internet-optimized” mode, please check this article: https://xrdocs.io/ncs5500/tutorials/decommissioning-internet-optimized-mode/

With the optimization disabled and after the line cards / system reload, we have now:

disable-optimizations.jpg

Important: with such mode, it will no longer be possible to handle a full internet view (v4+v6 or v4-only).

The configuration can now be applied on the interfaces:


hw-module fib ipv4 scale host-optimized-disable
hw-module fib ipv6 scale internet-optimized-disable
!
interface HundredGigE0/7/0/0
 ipv4 address 192.168.1.1 255.255.255.252
 ipv4 verify unicast source reachable-via any
 ipv6 verify unicast source reachable-via any
 ipv6 address 2001:10:1::1/64

Note: Starting from 6.7.x and 7.x.y, it’s mandatory to enable both IPv4 and IPv6 URPF configuration at the same time. The configuration for just one address-family will be rejected by the commit.

Configuration and impact on scale Jericho systems (with eTCAM)

Now, let’s consider the scale systems and line cards with Jericho ASICs:

The eTCAM is a 80bit memory and in normal condition we use it in two blocks of 40 bits to double the capacity. The first access being performed on the first half and the second access in the pipeline being done on the second half.

double-cap.jpg

With URPF, we need these two accesses to check source and destination, it’s no longer possible to use the double capacity mode: it needs to be disabled.

double-cap-disabled.jpg


RP/0/RP0/CPU0:NCS5508-632(config)#hw-module tcam fib ipv4 scaledisable
RP/0/RP0/CPU0:NCS5508-632(config)#hw-module fib ipv4 scale host-optimized-disable
RP/0/RP0/CPU0:NCS5508-632(config)#hw-module fib ipv6 scale internet-optimized-disable
RP/0/RP0/CPU0:NCS5508-632(config)#commit

The impact on scale is significative since we lost 1M out of the 2M of the eTCAM capacity.

URPF-impact2.png

Let’s check with a large routing table (internet v4 + internet v6 + 435k host routes) what is the impact:


RP/0/RP0/CPU0:5508-6.3.2#sh bgp sum

BGP router identifier 1.1.1.1, local AS number 100
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0xe0000000   RD version: 1720749
BGP main routing table version 1720749
BGP NSR Initial initsync version 634456 (Reached)
BGP NSR/ISSU Sync-Group versions 1720749/0
BGP scan interval 60 secs
 
BGP is operating in STANDALONE mode.
 
 
Process       RcvTblVer   bRIB/RIB   LabelVer  ImportVer  SendTblVer  StandbyVer
Speaker         1720749    1720749    1720749    1720749     1720749     1720749
 
Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd
192.168.100.151   0  1000  706753  307270  1720749    0    0     5w0d     655487
192.168.100.152   0 45896  707144  307270  1720749    0    0     5w0d     656126
192.168.100.153   0  7018  705342  307270  1720749    0    0     5w0d     654330
192.168.100.154   0  1836  709963  307270  1720749    0    0     5w0d     658948
192.168.100.155   0 50300  687217  307270  1720749    0    0     5w0d     636208
192.168.100.156   0 50304  708316  307270  1720749    0    0     5w0d     657301
192.168.100.157   0 57381  708322  307270  1720749    0    0     5w0d     657307
192.168.100.158   0  4608  728503  812358  1720749    0    0     5w0d     677487
192.168.100.159   0  4777  717228  307270  1720749    0    0     5w0d     666213
192.168.100.160   0 37989  339686  307270  1720749    0    0     5w0d     288706
192.168.100.161   0  3549  705390  307270  1720749    0    0     5w0d     654376
192.168.100.163   0  8757  683499  307270  1720749    0    0     5w0d     632483
192.168.100.164   0  3257  705671  307270  1720749    0    0     5w0d     654661
192.168.100.166   0 10051 1186443  217145  1720749    0    0 00:28:05    1186410
 
RP/0/RP0/CPU0:5508-6.3.2#sh dpa resource iproute loc 0/7/CPU0
 
"iproute" DPA Table (Id: 24, Scope: Global)
--------------------------------------------------
IPv4 Prefix len distribution
Prefix   Actual            Capacity    Prefix   Actual            Capacity
/0       3                 20           /1       0                 20
/2       0                 20           /3       0                 20
/4       3                 20           /5       0                 20
/6       0                 20           /7       0                 20
/8       16                20           /9       14                20
/10      37                204          /11      107               409
/12      288               818          /13      557               1636
/14      1071              3275         /15      1909              5732
/16      13572             42381        /17      8005              25387
/18      14055             42585        /19      25974             86603
/20      40443             127348       /21      45082             141679
/22      83722             231968       /23      71750             207173
/24      395142            1105590      /25      2085              4299
/26      3362              4504         /27      5736              3275
/28      15909             2866         /29      17377             6961
/30      42508             2866         /31      112               204
/32      435868            20
                          NPU ID: NPU-0           NPU-1           NPU-2           NPU-3
                          In Use: 1224707         1224707         1224707         1224707
                 Create Requests
                           Total: 1224713         1224713         1224713         1224713
                         Success: 1224713         1224713         1224713         1224713
                 Delete Requests
                           Total: 6               6               6               6
                         Success: 6               6               6               6
                 Update Requests
                           Total: 341539          341539          341539          341539
                         Success: 341538          341538          341538          341538
                    EOD Requests
                           Total: 0               0               0               0
                         Success: 0               0               0               0
                          Errors
                     HW Failures: 0               0               0               0
                Resolve Failures: 0               0               0               0
                 No memory in DB: 0               0               0               0
                 Not found in DB: 0               0               0               0
                    Exists in DB: 0               0               0               0
 
RP/0/RP0/CPU0:5508-6.3.2#sh contr npu resources lem loc 0/7/CPU0

HW Resource Information
    Name                            : lem
 
OOR Information
    NPU-0
        Estimated Max Entries       : 786432
        Red Threshold               : 95
        Yellow Threshold            : 80
        OOR State                   : Green

-- SNIP --
 
Current Usage
    NPU-0
        Total In-Use                : 467163   (59 %)
        iproute                     : 435868   (55 %)
        ip6route                    : 31304    (4 %)
        mplslabel                   : 0        (0 %)

-- SNIP --
 
RP/0/RP0/CPU0:5508-6.3.2#sh contr npu resources lpm loc 0/7/CPU0

HW Resource Information
    Name                            : lpm
 
OOR Information
    NPU-0
        Estimated Max Entries       : 530552
        Red Threshold               : 95
        Yellow Threshold            : 80
        OOR State                   : Green

-- SNIP --

Current Usage
    NPU-0
        Total In-Use                : 5235     (1 %)
        iproute                     : 0        (0 %)
        ip6route                    : 5219     (1 %)
        ipmcroute                   : 0        (0 %)

-- SNIP --
 
RP/0/RP0/CPU0:5508-6.3.2#sh contr npu resources exttcamipv4 loc 0/7/CPU0

HW Resource Information
    Name                            : ext_tcam_ipv4
 
OOR Information
    NPU-0
        Estimated Max Entries       : 2048000
        Red Threshold               : 95
        Yellow Threshold            : 80
        OOR State                   : Green

-- SNIP --
 
Current Usage
    NPU-0
        Total In-Use                : 788839   (39 %)
        iproute                     : 788839   (39 %)
        ipmcroute                   : 0        (0 %)

-- SNIP --
 
RP/0/RP0/CPU0:5508-6.3.2#

So we have 13 times Internet (coming from actual internet full views provided by different customers) and a lot of host routes (435k). In a Jericho + eTCAM card, before enabling URPF it occupies:

  • LEM: 59%
  • LPM: 1%
  • eTCAM: 39%

Let’s now remove the double capacity mode and configure the URPF on interfaces


RP/0/RP0/CPU0:5508-6.3.2#sh run | i hw-m

Building configuration...
RP/0/RP0/CPU0:5508-6.3.2#sh run int hu 0/7/0/0

interface HundredGigE0/7/0/0
cdp
ipv4 address 192.168.1.1 255.255.255.252
ipv6 address 2001:10:1::1/64
load-interval 30
flow ipv4 monitor fmm sampler fsm1 ingress
!

RP/0/RP0/CPU0:5508-6.3.2#conf

RP/0/RP0/CPU0:5508-6.3.2(config)#hw-module tcam fib ipv4 scaledisable

In order to activate this new scale, you must manually reload the chassis/all line cards
RP/0/RP0/CPU0:5508-6.3.2(config)#commit

RP/0/RP0/CPU0:5508-6.3.2(config)#end
RP/0/RP0/CPU0:5508-6.3.2#admin

root connected from 127.0.0.1 using console on 5500-6.3.2
sysadmin-vm:0_RP0# hw-module location 0/7 reload

Reload hardware module ? [no,yes] yes
result Card graceful reload request on 0/7 succeeded.
sysadmin-vm:0_RP0# exit

RP/0/RP0/CPU0:5508-6.3.2#conf
RP/0/RP0/CPU0:5508-6.3.2(config)#int hu 0/7/0/0
RP/0/RP0/CPU0:5508-6.3.2(config-if)# ipv4 verify unicast source reachable-via any
RP/0/RP0/CPU0:5508-6.3.2(config-if)# ipv6 verify unicast source reachable-via any
RP/0/RP0/CPU0:5508-6.3.2(config-if)#commit
RP/0/RP0/CPU0:5508-6.3.2(config-if)#end
RP/0/RP0/CPU0:5508-6.3.2#
RP/0/RP0/CPU0:5508-6.3.2#sh run | i hw-m

Building configuration...
hw-module tcam fib ipv4 scaledisable
RP/0/RP0/CPU0:5508-6.3.2#sh run int hu 0/7/0/0

interface HundredGigE0/7/0/0
cdp
ipv4 address 192.168.1.1 255.255.255.252
ipv4 verify unicast source reachable-via any
ipv6 verify unicast source reachable-via any
ipv6 address 2001:10:1::1/64
load-interval 30
flow ipv4 monitor fmm sampler fsm1 ingress
!
 
RP/0/RP0/CPU0:5508-6.3.2#sh contr npu resources lem loc 0/7/CPU0

HW Resource Information
    Name                            : lem
 
OOR Information
    NPU-0
        Estimated Max Entries       : 786432
        Red Threshold               : 95
        Yellow Threshold            : 80
        OOR State                   : Green

-- SNIP --
 
Current Usage
    NPU-0
        Total In-Use                : 467173   (59 %)
        iproute                     : 435868   (55 %)
        ip6route                    : 31304    (4 %)
        mplslabel                   : 0        (0 %)

-- SNIP --

RP/0/RP0/CPU0:5508-6.3.2#sh contr npu resources lpm loc 0/7/CPU0

HW Resource Information
    Name                            : lpm
 
OOR Information
    NPU-0
        Estimated Max Entries       : 171722
        Red Threshold               : 95
        Yellow Threshold            : 80
        OOR State                   : Green

-- SNIP --

Current Usage
    NPU-0
        Total In-Use                : 5234     (3 %)
        iproute                     : 0        (0 %)
        ip6route                    : 5218     (3 %)
        ipmcroute                   : 0        (0 %)

-- SNIP --

RP/0/RP0/CPU0:5508-6.3.2#sh contr npu resources exttcamipv4 loc 0/7/CPU0

HW Resource Information
    Name                            : ext_tcam_ipv4
 
OOR Information
    NPU-0
        Estimated Max Entries       : 1024000
        Red Threshold               : 95
        Yellow Threshold            : 80
        OOR State                   : Green

-- SNIP --

Current Usage
    NPU-0
        Total In-Use                : 788839   (77 %)
        iproute                     : 788839   (77 %)
        ipmcroute                   : 0        (0 %)

-- SNIP --

RP/0/RP0/CPU0:5508-6.3.2#

With URPF configured (and dual capacity mode disabled) and the same very large table we have:

  • LEM: 59%
  • LPM: 3%
  • eTCAM: 77%

In conclusion of this demo, enabling URPF implies the deactivation of the dual capacity mode and reduces by half the eTCAM memory. Nevertheless, routes are also stored in LEM and LPM. A very large internet table can still fit in the system, even if the room for growth is reduced.

Configuration and impact on scale Jericho+ systems (with NG eTCAM)

The Jericho+ w/ eTCAM systems don’t need to disable the dual capacity mode to enable URPF.

The same configuration than above can be re-used (except the hw-module commands).

The impact on scale is not null but is signficantly less than it was on the Jericho-based systems. Since the J+/eTCAM systems are qualified for 4M entries which is much less than its actual capacity, the 25% impact doesn’t change the officially supported numbers: with URPF enabled we still support 4M routes in eTCAM.

Verification

Packets dropped by URPF can be counted at the NPU level with:


RP/0/RP0/CPU0:NCS5508-6.3.2#show contr npu stats traps-all instance 0 location 0/7/CPU0 | inc Rpf

RxTrapUcLooseRpfFail                          0    84   0x54        32035   0         0        
RxTrapUcStrictRpfFail                         0    137  0x89        32035   0         0   

Conclusion

URPF loose mode can be configured on all NCS5500 systems. On Jericho w/ eTCAM, the impact is significative but we demonstrated we still support a very large public table and a lot of host routes. On Jericho+ w/ eTCAM, URPF doesn’t affect the supported scale of 4M entries.

Leave a Comment