Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
cancel

Service Providers Blogs

146 Views
0 Comments

The following article will show you available packet matching criteria that can be configured on an NCS5500 platform as part of BGP flowspec feature.
In this regards, NCS5500 will act as a BGP flowspec server and advertise the criteria to its BGP neighbors.

General discussion about BGP flowspec feature on NCS5500 is outside of the scope of this article but is discussed on following page:

https://supportforums.cisco.com/t5/service-providers-blogs/bgp-flowspec-implementation-on-ncs5500-platforms/ba-p/3387443

Same configuration should be applicable as well on ASR9K and CRS.



config example for Type 1 (Destination address) criteria

... match specific host

class-map type traffic match-all ipv4_type1_dip
 match destination-address ipv4 200.255.4.2/32
 end-class-map
!

class-map type traffic match-all ipv6_type1_dip
 match destination-address ipv6 200:242:250::1/128
 end-class-map
!

... match all hosts in range

class-map type traffic match-all ipv4_type1_dip
 match destination-address ipv4 200.255.4.0/24
 end-class-map
!

 

Note:

When we're defining host range instead of specific host in flowspec Type 1 advertisement, we need to make sure that there is an existing BGP route for that destination address on client router.
This pre-existing BGP route can be an exact match as the flowspec route (200.255.4.0/24 in this example) or it can be a less specific match (e.g. 200.255.4.0/23, 200.255.4.0/22, etc).
If on client side we only have more specific match (e.g. 200.255.4.0/25, 200.255.4.0/26, etc), the client router will still accept the flowspec but it will report the flowspec-path to be invalid and will NOT program it to LC.

 


config example for Type 2 (Source address) criteria

... match specific host

class-map type traffic match-all ipv4_type2_sip
 match source-address ipv4 200.255.3.2/32
 end-class-map
!

class-map type traffic match-all ipv6_type2_sip
 match source-address ipv6 200:255:3::12/128
 end-class-map
!

... match all hosts in range

class-map type traffic match-all ipv4_type2_sip
 match source-address ipv4 200.255.3.0/24
 end-class-map
!

 


config example for Type 3 (IP protocol) criteria

https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml

... match specific protocol

class-map type traffic match-all type3_prot
 match protocol tcp
 end-class-map
!

... X or Y

class-map type traffic match-all type3_prot
 match protocol tcp udp
 end-class-map
!

 

 

config example for Type 4 (IP protocol) criteria

class-map type traffic match-any type4_prot
 match source-port 80
 match destination-port 80
 end-class-map
!

the above will fail to commit and it's a known limitation for IOS XR based routers.

commit error message:

!!% Policy manager does not support this feature: Match all is the only mode supported for match type "source-port" in class-map type "traffic"

 

 

 

config example for Type 5 (destination port) criteria

... specific port

class-map type traffic match-all type6_dp
 match destination-port 80
 end-class-map
!

... port x or port y

class-map type traffic match-all type6_dp
 match destination-port 1026 1025
 end-class-map
!

... port range

class-map type traffic match-all type6_dp
 match destination-port 80-1026
 end-class-map
!

 

 

 config example for Type 6 (Source port) criteria

... specific port

class-map type traffic match-all ipv4_type6_sp
 match source-port 1026
 end-class-map
!

... port x or port y

class-map type traffic match-all ipv4_type6_sp
 match source-port 80 1026
 end-class-map
!

... port range

class-map type traffic match-all ipv4_type6_sp
 match source-port 80-1026
 end-class-map
!

 

 

 config example for Type 7 (ICMP type) criteria

https://en.wikipedia.org/wiki/Internet_Control_Message_Protocol
https://en.wikipedia.org/wiki/Internet_Control_Message_Protocol_for_IPv6

... specific ICMP type

class-map type traffic match-all icmp_type_3
 match ipv4 icmp-type 3
 end-class-map
!

... ICMP type range is NOT supported

class-map type traffic match-all ipv6_icmp_type_and_code
 match ipv6 icmp-type 4-255
 end-class-map
!
error message will be printed when trying to commit the config:

!!% 'FlowSpec' detected the 'warning' condition 'FS MGR': Operation not supported

... multiple ICMP type can be configured and advertised, but client that runs IOS-XR will not apply the rule

class-map type traffic match-all ipv6_icmp_type
 match ipv6 icmp-type 1 4
 end-class-map
!

RP/0/RP0/CPU0:fretta-50#sh flowspec ipv6 inter   

AFI: IPv6
  Flow           :ICMPType:=1|=4
    Actions      :Traffic-rate: 0 bps  (bgp.1)
      Client Version: 0
      Local:          FALSE
      Unsupported:    FALSE
      RT:
        VRF Name Cfg:   0x00
        RT Cfg:         0x00
        RT Registered:  0x00
        RT Resolved:    0x00
    Class handles:
    Class Handle Version:     0
    Sequence:                 1024
    Match Unsupported:        ICMP type count exceeded

 

 

 

 config example for Type 8 (ICMP code) criteria

https://en.wikipedia.org/wiki/Internet_Control_Message_Protocol
https://en.wikipedia.org/wiki/Internet_Control_Message_Protocol_for_IPv6

... ICMP code must be configured along with ICMP type

class-map type traffic match-all icmp_code_0
 match ipv4 icmp-code 0 <----     Match Unsupported:        ICMP code without ICMP type
 end-class-map
!

class-map type traffic match-all icmp_type_3
 match ipv4 icmp-type 3
 match ipv4 icmp-code 0
 end-class-map
!

class-map type traffic match-all ipv6_icmp_type_and_code
 match ipv6 icmp-type 4
 match ipv6 icmp-code 1
 end-class-map
!

... multiple ICMP code can be configured and advertised, but client that runs IOS-XR will not apply the rule

class-map type traffic match-all ipv4_type8_icmp_code
 match ipv4 icmp-type 3
 match ipv4 icmp-code 0 4
 end-class-map
!

RP/0/RP0/CPU0:flowspec_client#sh flowspec afi-all internal    

AFI: IPv4
  Flow           :ICMPType:=3,ICMPCode:=0|=4
    Actions      :Traffic-rate: 500000000 bps DSCP: af33 Nexthop: 200.255.9.2  (bgp.1)
      Client Version: 0
      Local:          FALSE
      Unsupported:    FALSE
      RT:
        VRF Name Cfg:   0x00
        RT Cfg:         0x00
        RT Registered:  0x00
        RT Resolved:    0x00
    Class handles:
    Class Handle Version:     0
    Sequence:                 512
    Match Unsupported:        ICMP code count exceeded

... ICMP code range is NOT supported

class-map type traffic match-all ipv6_icmp_type_and_code
 match ipv6 icmp-code 1-2
 end-class-map
!

error message will be printed when trying to commit the config:

!!% 'FlowSpec' detected the 'warning' condition 'FS MGR': Operation not supported

 

 

 

 config example for Type 9 (TCP flags) criteria

http://rapid.web.unc.edu/resources/tcp-flag-key/

class-map type traffic match-all tcp_flag_syn
 match tcp-flag 2
 end-class-map
!

 

 

 config example for Type 10 : total IP packet length (excluding Layer 2 but including IP header) criteria


note:
If you're dealing with ingress vlan traffic, here's what your L2 header would look like:

  1. eth header: 18B
  2. dot1q header: 4B
    total L2 header = 22B

... specific value

class-map type traffic match-all ipv4_type10_total_pkt_len
 match packet length 1000
 end-class-map
!

... X or Y
    
class-map type traffic match-all ipv4_type10_total_pkt_len
 match packet length 1100 1000
 end-class-map
!

... range X to Y

class-map type traffic match-all ipv4_type10_total_pkt_len
 match packet length 1001-1100
 end-class-map
!

 

 

 config example for Type 11 (DSCP) criteria

https://www.tucny.com/Home/dscp-tos

... specific DSCP

class-map type traffic match-all type11_dscp
 match dscp 22
 end-class-map
!

... range X to Y

class-map type traffic match-all type11_dscp
 match dscp 10-22
 end-class-map
!

 

 

 config example for Type 12 (IPv4 Fragmentation bits) criteria

https://en.wikipedia.org/wiki/IPv4#Fragmentation_and_reassembly

... DF packets

class-map type traffic match-all type12_frag_bits
 match fragment-type dont-fragment
 end-class-map
!

... Last fragment packets

class-map type traffic match-all type12_frag_bits
 match fragment-type  last-fragment
 end-class-map
!

... Fragmented packets

class-map type traffic match-all type12_frag_bits
 match fragment-type  is-fragment
 end-class-map
!

488 Views
0 Comments

Feature Description

 

 


BGP flowspec in a nutshell is a feature that will allow you to receive IPv4/IPv6 traffic flow specification (source X, destination Y, protocol UDP, source port A .. etc) and actions that need to be taken on that traffic (drop, or police .. or redirect etc) via BGP update.
Inside the BGP update, the flowspec matching criteria is represented by BGP NLRI and the actions are represented by BGP extended communities.


This feature is based on RFC 5575 and can be used to mitigate against DDoS attack. When a certain host inside of a network is being attacked, we can send a flowspec update to edge routers so that attack traffic can be policed or dropped, or even redirected elsewhere, maybe to an appliance that can clean the traffic (filter out the bad traffic and forward only the good traffic toward the affected host).

Once flowspecs have been received by a router and programmed in applicable line cards, any active L3 ports on those line cards will start processing ingress traffic according to flowspec rules.
If needed, we can disable flowspec processing on specific ports of the LC via CLI config (discussed later).
Note also that flowspec will only affect ingress traffic, it won't interfere with egress direction.

Flowspec can be programmed on different kind of interfaces:

  1. regular interface, e.g. TenGigE0/0/0/0
  2. vlan interface, e.g. TenGigE0/0/0/0.1 , Bundle-Ether3.2
  3. bundle interface, e.g. Bundle-Ether3

Ingress traffic can be matched by many criteria as defined in the RFC.
We can define only one, some, or all criteria to match traffic.
Once a set of criteria in a flowspec is defined, then all criteria must match the packet for actions to take place.

      
We call the router that receives the BGP update as the "client", and the router/appliance that advertises the update as the "server" or "controller".
A router than runs IOS XR can function as either client or server, and even as both server/client at the same time. For instance, when the router also needs to take actions on the attack traffic in addition to propagating the flowspec rules to clients.

BGP flowspec feature has been supported on ASR9K and CRS for a while, and NCS5500 supports the feature starting with 6.5.1 release.
Disclaimer:
6.5.1 may not be a GA release and we’ll update as we know more.

The following is comparison of flowspec support between existing XR routers like ASR9K and NCS5500:

fs_flowspec_type.png


 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

fs_flowspec_action.png

 

This article will not discuss further details about working of BGP flowspec, because we already have great documentations that does that already:

1. basic description of BGP flowspec on ASR9K:
https://supportforums.cisco.com/t5/service-providers-documents/asr9000-xr-understanding-bgp-flowspec-bgp-fs/ta-p/3139916

2. detailed description of BGP flowspec on ASR9K, complete with config examples.

Please refer to :

Slides in PDF: https://buff.ly/2GE67rn

MP4 recording of the session: https://buff.ly/2J3gpzp



Rather, we will focus more on specific information pertaining to NCS5500 platform.

This is also a live document, we will update if new info becomes available.




Supported Hardware

    
If the router just needs to act as server only (no packet processing is required, perhaps the router is not in the attack path):
Any NCS5500 will do.

If the router needs to act as client (packets processing is required), we have two options:

  • NCS5500 modular platform
    Line card that RECEIVES traffic must be of "scale-enhanced" type and equipped with later ASIC.
    At the time of writing, only one LC satisfies above requirement: "NC55-36X100G-A-SE".
    Line card that TRANSMITS traffic can be of any flavor.
  • NCS5500 non modular, only "NCS-55A1-36H-SE-S" platform is supported.

 

Also note that when the router acts as client, it doesn't matter either on which LC that you receive BGP update on, the LC that receives BGP update from BGP peer can be of any flavor.



Supported Scale

       
3,000 flowspecs (a "flowspec" consists of packet-matching criteria and desired actions)
Where a single flowspec can't take more than 1,024 TCAM entries on the LC.

A single flowspec can take more TCAM entries than others, for instance when we define some kind of IP address range instead of a host address as matching criteria.

 



Config example
"attack traffic ingressing on interface in Global Routing Table (GRT)"


Use case:

Attack IPv4 and IPv6 traffic received on GRT interface.
Redirect attack IPv4 traffic to VRF "honeypot" for scrubbing.
Redirect attack IPv6 traffic to different NH IPv6 address in GRT for scrubbing.

Note:
If we're redirecting traffic to different VRF (VRF "honeypot" in this use case), then that different VRF must have a route toward the traffic destination. Otherwise traffic will be redirected to that VRF internally but our router won't know where to forward it to.
We can add this route via static route under VRF "honeypot".
See following config example.

Topology:

attack traffic
|
|
|
|
|
\/
GRT interface
NCS5500 ----------> redirect attack traffic to VRF or different NH IP address in GRT
|   |
|   |
|   +------ iBGP ----- flowspec server
|
|
\/
attacked host


Client config

... mandatory if we want to have flowspec IPv6 support (need LC reload after commit)
... this will configure support on all supported line cards on the chassis

hw-module profile flowspec v6-enable

... or this, if only want to configure support on specific LC

hw-module profile flowspec v6-enable location <>

 

... activate flowspec programming in the LC

flowspec
 local-install interface-all
!

... or if you want to be totally explicit:
    
flowspec
 address-family ipv4
  local-install interface-all
  !
  address-family ipv6
   local-install interface-all
   !
 !
        
... optional, disable flowspec processing on specific ingress interfaces

interface Bundle-Ether3.1
 ipv4 flowspec disable
 ipv6 flowspec disable
!
        
... basic PASS-ALL and DROP-ALL BGP policy

route-policy PASS-ALL
  pass
end-policy
!

route-policy DROP-ALL
  drop
end-policy
!

... configure BGP toward flowspec server
    
router bgp <>
 nsr
 bgp router-id <>
 address-family ipv4 flowspec
 !
 address-family ipv6 flowspec
 !
 neighbor <>
  remote-as <>
  address-family ipv4 flowspec
   route-policy PASS-ALL in
   route-policy DROP-ALL out
  !
  address-family ipv6 flowspec
   route-policy PASS-ALL in
   route-policy DROP-ALL out
  !
  update-source <>
 !
!

... define VRF to redirect the attack traffic to

vrf honeypot
 address-family ipv4 unicast
  import route-target
   4787:13
  !
  export route-target
   4787:13
  !
 !
 address-family ipv6 unicast
  import route-target
   4787:13
  !
  export route-target
   4787:13
  !
 !
!

... Define static route to forward the redirected traffic under VRF.
... Here we assume the traffic destination is any hosts under 70.0.0.0/8

router static
 vrf honeypot
  address-family ipv4 unicast
   70.0.0.0/8 200.255.55.2
  !
 !
!




Server Config

... basic PASS-ALL and DROP-ALL BGP policy

route-policy PASS-ALL
  pass
end-policy
!

route-policy DROP-ALL
  drop
end-policy
!

... configure BGP toward flowspec client

router bgp <>
 address-family ipv4 flowspec
 !
 address-family ipv6 flowspec
 !
 neighbor <>
  remote-as <>
  address-family ipv4 flowspec
   route-policy DROP-ALL in
   route-policy PASS-ALL out
  !
  address-family ipv6 flowspec
   route-policy DROP-ALL in
   route-policy PASS-ALL out
  !
 !
!

... let's configure flowspecs to be advertised to client

...... address-family ipv4 flowspec

    class-map type traffic match-all ipv4_fragment
     match destination-address ipv4 70.2.1.1 255.255.255.255
     match source-address ipv4 80.2.1.1 255.255.255.255
     match packet length 700
     match dscp af21
     match fragment-type  is-fragment
     end-class-map
    !
    
    class-map type traffic match-all ipv4_icmp
     match destination-address ipv4 70.2.1.1 255.255.255.255
     match source-address ipv4 80.2.1.1 255.255.255.255
     match packet length 700
     match dscp af21
     match fragment-type  is-fragment
     match ipv4 icmp-type 3
     match ipv4 icmp-code 2
     end-class-map
    !

    policy-map type pbr scale_ipv4
     class type traffic ipv4_fragment
      drop
     !
     class type traffic ipv4_icmp
      police rate 1 mbps
      !
      set dscp cs2
      redirect nexthop route-target 4787:13
     !
     class type traffic class-default
     !
     end-policy-map
    !

    flowspec
     address-family ipv4
      service-policy type pbr scale_ipv4
     !
    !

...... address-family ipv6 flowspec

    class-map type traffic match-all ipv6_tcp
     match destination-address ipv6 70:1:1::5a/128
     match source-address ipv6 80:1:1::5a/128
     match protocol tcp
     match destination-port 22
     match source-port 4000
     match tcp-flag 0x10
     match packet length 300
     match dscp af12
     end-class-map
    !

    class-map type traffic match-all ipv6_icmp
     match destination-address ipv6 70:2:1::1/128
     match source-address ipv6 80:2:1::1/128
     match packet length 800
     match dscp af22
     match ipv6 icmp-type 4
     match ipv6 icmp-code 1
     end-class-map
    !

    policy-map type pbr scale_ipv6
     class type traffic ipv6_tcp
      police rate 1 mbps
      !
      set dscp cs1
      redirect ipv6 nexthop 202:158:2::1
     !
     class type traffic ipv6_icmp
      police rate 1 mbps
      !
      set dscp cs3
      redirect nexthop route-target 4787:13
     !
     class type traffic class-default
     !
    !

    flowspec
     address-family ipv6
      service-policy type pbr scale_ipv6
     !
    !

 



Config example
"attack traffic ingressing on VRF interface"


Use case:


Attack traffic received on VRF "customer_1" interface.
Redirect this traffic to VRF "dirty-dancing" for scrubbing.

Note:

If we're redirecting traffic to different VRF (VRF "dirty-dancing" in this use case), then that different VRF must have a route toward the traffic destination. Otherwise traffic will be redirected to that VRF internally but our router won't know where to forward it to.
In this usecase, we will forward the traffic under VRF "dirty-dancing" using L3VPN route.
Our router gets this route from another L3VPN PE that is connected to redirect destination, so we won't need to define static routing under VRF like previous use case.



Topology:

attack traffic
|
|
|
|
|
\/
VRF "customer_1" interface
NCS5500 ----------> redirect attack traffic to different VRF "dirty-dancing"
|   |
|   |
|   +------ iBGP ----- flowspec server
|
|
\/
attacked host (VRF "customer_1")


Client Config

... mandatory if we want to have flowspec IPv6 support (need LC reload after commit)
... this will configure support on all supported line cards on the chassis

hw-module profile flowspec v6-enable

... or this, if only want to configure support on specific LC

hw-module profile flowspec v6-enable location <>


... activate flowspec programming in the LC

flowspec
 local-install interface-all
!
    
... or if you want to be totally explicit:
    
flowspec
 vrf customer_1
  address-family ipv4
   local-install interface-all
  !
  address-family ipv6
   local-install interface-all
  !
 !
!
        
... optional, disable flowspec processing on specific ingress interfaces

interface Bundle-Ether3.1
 ipv4 flowspec disable
 ipv6 flowspec disable
!
        
... basic PASS-ALL and DROP-ALL BGP policy

route-policy PASS-ALL
  pass
end-policy
!

route-policy DROP-ALL
  drop
end-policy
!

... configure BGP toward flowspec server
    
router bgp <>
 nsr
 bgp router-id <>
 address-family vpnv4 unicast
 !
 address-family vpnv6 unicast
 !
 address-family vpnv4 flowspec
 !
 address-family vpnv6 flowspec
 !
 neighbor <>
  remote-as <>
  address-family vpnv4 unicast
   route-policy PASS-ALL in
   route-policy PASS-ALL out
  !
  address-family vpnv6 unicast
   route-policy PASS-ALL in
   route-policy PASS-ALL out
  !
  address-family vpnv4 flowspec
   route-policy PASS-ALL in
   route-policy DROP-ALL out
  !
  address-family vpnv6 flowspec
   route-policy PASS-ALL in
   route-policy DROP-ALL out
  !
  update-source <>
 !
 vrf customer_1
  rd auto
  address-family ipv4 unicast
   label mode per-vrf
   redistribute connected
  !
  address-family ipv6 unicast
   label mode per-vrf
   redistribute connected
  !
  address-family ipv4 flowspec
  !
  address-family ipv6 flowspec
  !
 !
 vrf dirty_dancing
  rd auto
  address-family ipv4 unicast
   label mode per-vrf
  !
  address-family ipv6 unicast
   label mode per-vrf
  !
 !
!

... define VRF "customer_1" and "dirty_dancing"

vrf customer_1
 address-family ipv4 unicast
  import route-target
   4787:1313
  !
  export route-target
   4787:1313
  !
 !
 address-family ipv4 flowspec
  import route-target
   4787:1313
  !
  export route-target
   4787:1313
  !
 !
 address-family ipv6 unicast
  import route-target
   4787:1313
  !
  export route-target
   4787:1313
  !
 !
 address-family ipv6 flowspec
  import route-target
   4787:1313
  !       
  export route-target
   4787:1313
  !
 !
!
vrf dirty_dancing
 address-family ipv4 unicast
  import route-target
   4787:666
  !
  export route-target
   4787:666
  !
 !
 address-family ipv6 unicast
  import route-target
   4787:666
  !
  export route-target
   4787:666
  !
 !
!


Server Config

... basic PASS-ALL and DROP-ALL BGP policy

route-policy PASS-ALL
  pass
end-policy
!
route-policy DROP-ALL
  drop
end-policy
!

... configure BGP toward flowspec client

router bgp <>
 nsr
 bgp router-id <>
 address-family vpnv4 unicast
 !
 address-family vpnv6 unicast
 !
 address-family vpnv4 flowspec
 !
 address-family vpnv6 flowspec
 !
 neighbor <>
  remote-as <>
  address-family vpnv4 unicast
   route-policy PASS-ALL in
   route-policy PASS-ALL out
  !
  address-family vpnv6 unicast
   route-policy PASS-ALL in
   route-policy PASS-ALL out
  !
  address-family vpnv4 flowspec
   route-policy DROP-ALL in
   route-policy PASS-ALL out
  !
  address-family vpnv6 flowspec
   route-policy DROP-ALL in
   route-policy PASS-ALL out
  !
  update-source <>
 !
 vrf customer_1
  rd auto
  address-family ipv4 unicast
   label mode per-vrf
   redistribute connected
  !
  address-family ipv6 unicast
   label mode per-vrf
   redistribute connected
  !
  address-family ipv4 flowspec
  !
  address-family ipv6 flowspec
  !
 !
!

... let's configure flowspecs to be advertised to client

 

Note:

For a flowspec that is configured under VRF like the following, we can only use "redirect nexthop route-target <>" on the controller side to redirect this traffic to other VRF.
"redirect ipv4|ipv6 nexthop <>" is not supported in a VRF context.



...... address-family vpnv4 flowspec

    class-map type traffic match-all l3vpn_ipv4_attack
     match destination-address ipv4 202.158.3.2 255.255.255.255
     match source-address ipv4 202.158.1.2 255.255.255.255
     end-class-map
    !

    policy-map type pbr pm_cust_VRF_to_diff_VRF
     class type traffic l3vpn_ipv4_attack
      redirect nexthop route-target 4787:666
      set dscp cs6
      police rate 250 mbps
     !
     class type traffic class-default
     !
     end-policy-map
    !

    flowspec
     vrf customer_1
      address-family ipv4
       service-policy type pbr pm_cust_VRF_to_diff_VRF
     !
    !
    
...... address-family vpnv6 flowspec

    class-map type traffic match-all l3vpn_ipv6_attack
     match destination-address ipv6 200:158:3::2/128
     match source-address ipv6 200:158:1::2/128
     match protocol tcp
     match destination-port 22
     match source-port 4000
     match packet length 300
     match dscp af12
     end-class-map
    !

    policy-map type pbr pm_cust_VRF_to_diff_VRF_ipv6
     class type traffic l3vpn_ipv6_attack
     no drop
      redirect nexthop route-target 4787:666
      set dscp cs6
      police rate 250 mbps
     !
     class type traffic class-default
     !
     end-policy-map
    !

    flowspec
     vrf customer_1
      address-family ipv6
       service-policy type pbr pm_cust_VRF_to_diff_VRF_ipv6
     !
    !


more detailed config examples to define packet matching criteria can be found here:

https://supportforums.cisco.com/t5/service-providers-blogs/ncs5500-bgp-flowspec-packet-matching-criteria/ba-p/3387457

 

 


BGP Flowspec and IPv6 BGP neighbors

 

The use-cases and config examples provided previously are when we have BGP neighbors that runs on IPv4.
What about if we have IPv6 BGP neighbors instead of IPv4?
We can still advertise flowspec to that neighbor, but only for address-family "ipv6 flowspec".

The following flowspec address-families are not supported:

  • ipv4 flowspec
  • vpnv4 flowspec
  • vpnv6 flowspec

This is by design and same limitation applies to other platforms that run IOS-XR like ASR9000.

 



BGP Flowspec and bundle interface

 

As mentioned before, BGP flowspec is also supported on bundle interface.

Note the following bundle behavior when it comes to flowspec traffic policing, however.

Say, we have interface Bundle-Ether3 with member links Hu0/4/0/0 and Hu0/4/0/9.
The member links are served by different NPU on LC 0/4/CPU0.

Hu0/4/0/0 will be served by NPU 0.
Hu0/4/0/9 will be served by NPU 1.

RP/0/RP0/CPU0:fretta-50#sh controllers npu voq-usage interface Hu0/4/0/0  instance all location 0/4/cpu0

-------------------------------------------------------------------
Node ID: 0/4/CPU0
Intf         Intf     NPU NPU  PP   Sys   VOQ   Flow   VOQ    Port
name         handle    #  core Port Port  base  base   port   speed
             (hex)                                     type        
----------------------------------------------------------------------
Hu0/4/0/0    2000130   0   1   21  2421   1024   4312 local   100G
RP/0/RP0/CPU0:fretta-50#

RP/0/RP0/CPU0:fretta-50#sh controllers npu voq-usage interface Hu0/4/0/9  instance all location 0/4/cpu0

-------------------------------------------------------------------
Node ID: 0/4/CPU0
Intf         Intf     NPU NPU  PP   Sys   VOQ   Flow   VOQ    Port
name         handle    #  core Port Port  base  base   port   speed
             (hex)                                     type        
----------------------------------------------------------------------
Hu0/4/0/9    2000178   1   1   21  2521   1096   4312 local   100G

And let's say there is currently a flowspec being advertised to our router:

policy-map type pbr scale_ipv4
 class type traffic ipv4_icmp
  police rate 1 mbps
  !
 !
 class type traffic class-default
 !
 end-policy-map
!

Now what's going to happen is that the police action will be programmed to both NPU 0 and NPU 1.
Hu0/4/0/0 on NPU 0 will police at 1Mbps.
Hu0/4/0/9 on NPU 1 will police at 1Mbps.

Assuming we receive many ICMP streams such that the streams will be load-balanced pretty evenly on both member links, this will give you 2Mbps police rate total instead of 1Mbps.

 



BGP Flowspec and BGP route-policy

We can configure BGP route-policy to filter flowspec updates.
One use case is maybe to limit flowspec updates with only specific subnet masks.

For instance:

router bgp <>
!
address-family ipv4 flowspec
!
address-family ipv6 flowspec
!
neighbor <>
  remote-as <>
  update-source <>
  address-family ipv4 flowspec
   route-policy routes_flowspec in
   route-policy drop-all out
  !
  ...

route-policy routes_flowspec
  if destination in ge_17 then
    pass
  endif
end-policy

prefix-set ge_17
  0.0.0.0/0 ge 17
end-set
!

Just be careful when doing route-policy.
If you commit the above config, and the coming flowspec updates don't have Type 1 (Destination address) criteria, then no updates would match the policy and it would be implicit deny for all flowspec updates.

 

 

BGP Flowspec and ACL

When flowspec is implemented on an interface that is also having ingress ACL, ACL will come before flowspec processing.

 

  • When the ACL is permitting the traffic:
    Flowspec will take whatever passed by ACL and run flowspec processing.
  • When the ACL is denying the traffic:
    Flowspec will not process any traffic since ACL has discarded them.

 

 

 BGP Flowspec and local QoS configuration

When flowspec is implemented on an interface that is also having local QoS configuration, local config will come before flowspec processing.
Local config will police and dscp-mark the packets and pass them to flowspec.
Flowspec will then do its processing (police, redirect) except dscp marking.

Flowspec will retain dscp marking as dictated by local qos config.

Say, we have the following:

inbound qos config : police 100Mbps, mark dscp af11
 
=============================================================
 
ipv4 access-list acl_ipv4_qos_stream
6 permit ipv4 any host 200.255.5.2
!
!
class-map match-any cm_ipv4_qos_stream
match access-group ipv4 acl_ipv4_qos_stream
 end-class-map
!
 
policy-map pm_ipv4_qos_stream
class cm_ipv4_qos_stream
  police rate 100 mbps
  !
  set dscp af11
!
 class class-default
!
 end-policy-map
!
 
interface hundredGigE 0/4/0/35
service-policy input pm_ipv4_qos_stream
 
=============================================================

Then we receive the following in flowspec advertisement.
flowspec config : police 50Mbps, mark dscp af43, redir vrf.
 
=============================================================
 
RP/0/RP0/CPU0:fretta-50#sh flowspec ipv4 detail | b 200.255.5.2
  Flow           :Dest:200.255.5.2/32
    Actions      :Traffic-rate: 50000000 bps DSCP: af43 Redirect: VRF honeypot Route-target: ASN2-4787:13  (bgp.1)
    Statistics                        (packets/bytes)
      Matched             :           116570713/12822778430        
      Transmitted         :            57360817/6309689870         
      Dropped             :            59209896/6513088560   
 
=============================================================
 
Then the outcome will be:

 

  1. traffic will be policed by flowspec at 50Mbps.
  2. traffic will be redirected by flowspec to VRF honeypot.
  3. flowspec will not overwrite dscp marking, traffic will be forwarded using dscp af11 instead of af43.

 

 


BGP Flowspec and NSR


NSR RP switchover is hitless for flowspec provided all underlay protocol (BGP, ISIS, OSPF, etc) has been configured for NSR.

 


Caveats

"hw-module profile flowspec v6-enable" config will cause IPv6 linerate degradation from 835Mpps to ~700Mpps.

Flowspec processing on an ingress packet only takes place when the router does L3 lookup.
This means that flowspec won't process transit MPLS packets since it will be just a label swap instead of L3 lookup.

Put it other way, only the following ingress traffic can be processed:

  1. Plain L3 IPv4/IPv6 packets.
  2. MPLS packet with explicit-null / implicit-null label.


BGP flowspec will NOT process packets when it's received on GRE tunnel.

BGP flowspec is NOT supported on BVI interface.

BGP flowspec is NOT supported with multicast traffic.

BGP flowspec polices traffic at L2, not L1.
What this means is that the policing will not take into account L1 headers.
So, say we police at 500Mbps, then what being transmitted on the wire would be more than 500Mbps since the traffic will be transmitted with L1 headers on top of it.

Once received via BGP update, flowspec will take longer time to program to the LC if we compare with regular IPv4/IPv6 route updates.
Enhancement is being planned to make the programming faster, but it will only be provided after 6.5.1.

 

 


Related Show Commands

 

The following commands are captured from client side.

=============================================================


RP/0/RP0/CPU0:fretta-50#sh bgp ipv4 flowspec
BGP router identifier 202.158.0.1, local AS number 4787
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0   RD version: 7506
BGP main routing table version 7506
BGP NSR Initial initsync version 130 (Reached)
BGP NSR/ISSU Sync-Group versions 7506/0
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
*>iDest:70.1.1.1/32,Proto:=6,DPort:=80,SPort:=3000,Length:=200,DSCP:=10/176
                      0.0.0.0                        10      0 ?
*>iDest:70.1.1.2/32,Proto:=6,DPort:=80,SPort:=3000,Length:=200,DSCP:=10/176
                      0.0.0.0                        10      0 ?
*>iDest:70.1.1.3/32,Proto:=6,DPort:=80,SPort:=3000,Length:=200,DSCP:=10/176
                      0.0.0.0                        10      0 ?
*>iDest:70.1.1.4/32,Proto:=6,DPort:=80,SPort:=3000,Length:=200,DSCP:=10/176
                      0.0.0.0                        10      0 ?
*>iDest:70.1.1.5/32,Proto:=6,DPort:=80,SPort:=3000,Length:=200,DSCP:=10/176
                      0.0.0.0                        10      0 ?
          
=============================================================
          
RP/0/RP0/CPU0:fretta-50#sh bgp ipv6 flowspec
BGP router identifier 202.158.0.1, local AS number 4787
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0   RD version: 1503
BGP main routing table version 1504
BGP NSR Initial initsync version 2 (Reached)
BGP NSR/ISSU Sync-Group versions 1504/0
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
*>iDest:70:1:1::1/0-128,Source:80:1:1::1/0-128,NH:=6,DPort:=22,SPort:=4000,TCPFlags:=0x10,Length:=300,DSCP:=12/464
                      202:158:2::1                  100      0 i
*>iDest:70:1:1::2/0-128,Source:80:1:1::2/0-128,NH:=6,DPort:=22,SPort:=4000,TCPFlags:=0x10,Length:=300,DSCP:=12/464
                      202:158:2::1                  100      0 i
*>iDest:70:1:1::3/0-128,Source:80:1:1::3/0-128,NH:=6,DPort:=22,SPort:=4000,TCPFlags:=0x10,Length:=300,DSCP:=12/464
                      202:158:2::1                  100      0 i
*>iDest:70:1:1::4/0-128,Source:80:1:1::4/0-128,NH:=6,DPort:=22,SPort:=4000,TCPFlags:=0x10,Length:=300,DSCP:=12/464
                      202:158:2::1                  100      0 i
*>iDest:70:1:1::5/0-128,Source:80:1:1::5/0-128,NH:=6,DPort:=22,SPort:=4000,TCPFlags:=0x10,Length:=300,DSCP:=12/464
                      202:158:2::1                  100      0 i
          
=============================================================
          
RP/0/RP0/CPU0:fretta-50#sh bgp vpnv4 flowspec
BGP router identifier 202.158.0.1, local AS number 4787
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0   RD version: 0
BGP main routing table version 5
BGP NSR Initial initsync version 3 (Reached)
BGP NSR/ISSU Sync-Group versions 5/0
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 202.158.0.1:0 (default for vrf customer_1)
*>iDest:202.158.3.2/32,Source:202.158.1.2/32/96
                      0.0.0.0                       100      0 i
Route Distinguisher: 202.158.0.2:1
*>iDest:202.158.3.2/32,Source:202.158.1.2/32/96
                      0.0.0.0                       100      0 i

Processed 2 prefixes, 2 paths
          
=============================================================
          
RP/0/RP0/CPU0:fretta-50#sh bgp vpnv6 flowspec
BGP router identifier 202.158.0.1, local AS number 4787
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0   RD version: 0
BGP main routing table version 5
BGP NSR Initial initsync version 4 (Reached)
BGP NSR/ISSU Sync-Group versions 5/0
BGP scan interval 60 secs

Status codes: s suppressed, d damped, h history, * valid, > best
              i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network            Next Hop            Metric LocPrf Weight Path
Route Distinguisher: 202.158.0.1:0 (default for vrf customer_1)
*>iDest:200:158:3::2/0-128,Source:200:158:1::2/0-128,NH:=6,DPort:=22,SPort:=4000,Length:=300,DSCP:=12/440
                      0.0.0.0                       100      0 i
Route Distinguisher: 202.158.0.2:1
*>iDest:200:158:3::2/0-128,Source:200:158:1::2/0-128,NH:=6,DPort:=22,SPort:=4000,Length:=300,DSCP:=12/440
                      0.0.0.0                       100      0 i

Processed 2 prefixes, 2 paths
RP/0/RP0/CPU0:fretta-50#
          
=============================================================

RP/0/RP0/CPU0:fretta-50#sh bgp ipv4 flowspec summary
BGP router identifier 202.158.0.1, local AS number 4787
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0   RD version: 7506
BGP main routing table version 7506
BGP NSR Initial initsync version 130 (Reached)
BGP NSR/ISSU Sync-Group versions 7506/0
BGP scan interval 60 secs

BGP is operating in STANDALONE mode.


Process       RcvTblVer   bRIB/RIB   LabelVer  ImportVer  SendTblVer  StandbyVer
Speaker            7506       7506       7506       7506        7506        7506

Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd
200.255.1.5       0  4787    6956    2957     7506    0    0 04:47:52       1000 <-- this many flowspecs were received from server
200.255.1.6       0 50011    3015    3010        0    0    0 05:27:41  (NoNeg)
202.158.2.1       0  4787    1548    1648     7506    0    0    1d01h        250
202.158.3.1       0  4787    1683    1644     7506    0    0    1d01h        250
202.158.4.1       0  4787    1543    1649     7506    0    0    1d01h          0
          
=============================================================

RP/0/RP0/CPU0:fretta-50#sh bgp ipv6 flowspec summary
BGP router identifier 202.158.0.1, local AS number 4787
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0   RD version: 1503
BGP main routing table version 1504
BGP NSR Initial initsync version 2 (Reached)
BGP NSR/ISSU Sync-Group versions 1504/0
BGP scan interval 60 secs

BGP is operating in STANDALONE mode.


Process       RcvTblVer   bRIB/RIB   LabelVer  ImportVer  SendTblVer  StandbyVer
Speaker            1504       1504       1504       1504        1504        1504

Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd
200.255.1.5       0  4787    6957    2957     1504    0    0 04:48:02          0
200.255.1.6       0 50011    3015    3010        0    0    0 05:27:50  (NoNeg)
202.158.2.1       0  4787    1548    1648     1504    0    0    1d01h        750 <-- this many flowspecs were received from server
202.158.3.1       0  4787    1683    1644     1504    0    0    1d01h        751
202.158.4.1       0  4787    1543    1649     1504    0    0    1d01h          0
          
=============================================================

RP/0/RP0/CPU0:fretta-50#sh bgp vpnv4 flowspec summary
BGP router identifier 202.158.0.1, local AS number 4787
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0   RD version: 0
BGP main routing table version 5
BGP NSR Initial initsync version 3 (Reached)
BGP NSR/ISSU Sync-Group versions 5/0
BGP scan interval 60 secs

BGP is operating in STANDALONE mode.


Process       RcvTblVer   bRIB/RIB   LabelVer  ImportVer  SendTblVer  StandbyVer
Speaker               5          5          5          5           5           5

Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd
202.158.2.1       0  4787    1549    1648        5    0    0    1d01h          1 <-- this many flowspecs were received from server
202.158.3.1       0  4787    1684    1644        5    0    0    1d01h          0
202.158.4.1       0  4787    1543    1649        5    0    0    1d01h          0
          
=============================================================

RP/0/RP0/CPU0:fretta-50#sh bgp vpnv6 flowspec summary
BGP router identifier 202.158.0.1, local AS number 4787
BGP generic scan interval 60 secs
Non-stop routing is enabled
BGP table state: Active
Table ID: 0x0   RD version: 0
BGP main routing table version 5
BGP NSR Initial initsync version 4 (Reached)
BGP NSR/ISSU Sync-Group versions 5/0
BGP scan interval 60 secs

BGP is operating in STANDALONE mode.


Process       RcvTblVer   bRIB/RIB   LabelVer  ImportVer  SendTblVer  StandbyVer
Speaker               5          5          5          5           5           5

Neighbor        Spk    AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down  St/PfxRcd
202.158.2.1       0  4787    1549    1649        5    0    0    1d01h          1 <-- this many flowspecs were received from server
202.158.3.1       0  4787    1684    1645        5    0    0    1d01h          0
202.158.4.1       0  4787    1543    1650        5    0    0    1d01h          0
          
=============================================================

RP/0/RP0/CPU0:fretta-50#sh flowspec ipv4 detail

AFI: IPv4
  Flow           :Dest:70.1.1.1/32,Proto:=6,DPort:=80,SPort:=3000,Length:=200,DSCP:=10
    Actions      :Traffic-rate: 0 bps  (bgp.1)
    Statistics                        (packets/bytes)
      Matched             :            18174999/3707699796         
      Transmitted         :                   0/0                  
      Dropped             :            18174999/3707699796    
          
=============================================================

RP/0/RP0/CPU0:fretta-50#sh flowspec ipv4 internal

AFI: IPv4
  Flow           :Dest:70.1.1.1/32,Proto:=6,DPort:=80,SPort:=3000,Length:=200,DSCP:=10
    Actions      :Traffic-rate: 0 bps  (bgp.1)
      Client Version: 0
      Local:          FALSE <--- this is flowspec advertised from server
      Unsupported:    FALSE <--- flowspec data is supported
      RT:
        VRF Name Cfg:   0x00
        RT Cfg:         0x00
        RT Registered:  0x00
        RT Resolved:    0x00
    Class handles:
      Handle [0]:        30000000760013eb
    Class Handle Version:     1
    Sequence:                 1024
    Match Unsupported:        None <--- flowspec data is supported
    Synced:                   TRUE <--- flowspec is successfully synced to standby RSP (if any)
    Ref Count:                1
    Last Error:               0:Success <--- no error is seen
    Last Batch:               218
    Time Init:                May 22 11:30:13
    Time iClass Update:       May 22 11:30:13
    Statistics                        (packets/bytes)
      Matched             :            18184140/3709564560         
      Transmitted         :                   0/0                  
      Dropped             :            18184140/3709564560         
          
=============================================================

RP/0/RP0/CPU0:fretta-50#sh flowspec ipv6 detail  

AFI: IPv6
  Flow           :Dest:70:1:1::1/0-128,Source:80:1:1::1/0-128,NH:=6,DPort:=22,SPort:=4000,TCPFlags:=0x10,Length:=300,DSCP:=12
    Actions      :Traffic-rate: 1000000 bps DSCP: cs1 Nexthop: 202:158:2::1  (bgp.1)
    Statistics                        (packets/bytes)
      Matched             :            64091597/19483845488        
      Transmitted         :            33973978/10328089312        
      Dropped             :            30117619/9155756176         
          
=============================================================

RP/0/RP0/CPU0:fretta-50#sh flowspec ipv6 internal

AFI: IPv6
  Flow           :Dest:70:1:1::1/0-128,Source:80:1:1::1/0-128,NH:=6,DPort:=22,SPort:=4000,TCPFlags:=0x10,Length:=300,DSCP:=12
    Actions      :Traffic-rate: 1000000 bps DSCP: cs1 Nexthop: 202:158:2::1  (bgp.1)
      Client Version: 0
      Local:          FALSE <--- this is flowspec advertised from server
      Unsupported:    FALSE <--- flowspec data is supported
      RT:
        VRF Name Cfg:   0x00
        RT Cfg:         0x00
        RT Registered:  0x00
        RT Resolved:    0x00
    Class handles:
      Handle [0]:        30000000760005e4
    Class Handle Version:     1
    Sequence:                 1024
    Match Unsupported:        None <--- flowspec data is supported
    Synced:                   TRUE <--- flowspec is successfully synced to standby RSP (if any)
    Ref Count:                1
    Last Error:               0:Success <--- no error is seen
    Last Batch:               31
    Time Init:                May 21 15:31:39
    Time iClass Update:       May 21 15:31:39
    Statistics                        (packets/bytes)
      Matched             :            64099357/19486204528        
      Transmitted         :            33978090/10329339360        
      Dropped             :            30121267/9156865168             
          
=============================================================

RP/0/RP0/CPU0:fretta-50#sh flowspec vrf customer_1 ipv4 detail

VRF: customer_1     AFI: IPv4
  Flow           :Dest:202.158.3.2/32,Source:202.158.1.2/32
    Actions      :Traffic-rate: 250000000 bps DSCP: cs6 Redirect: VRF dirty_dancing Route-target: ASN2-4787:666  (bgp.1)
    Statistics                        (packets/bytes)
      Matched             :         37260786850/4098686553500      
      Transmitted         :         21304093027/2343450232970      
      Dropped             :         15956693823/1755236320530          
          
=============================================================

RP/0/RP0/CPU0:fretta-50#sh flowspec vrf customer_1 ipv4 internal

VRF: customer_1     AFI: IPv4
  Flow           :Dest:202.158.3.2/32,Source:202.158.1.2/32
    Actions      :Traffic-rate: 250000000 bps DSCP: cs6 Redirect: VRF dirty_dancing Route-target: ASN2-4787:666  (bgp.1)
      Client Version: 0
      Local:          FALSE <--- this is flowspec advertised from server
      Unsupported:    FALSE <--- flowspec data is supported
      RT:
        VRF Name Cfg:   0x00
        RT Cfg:         0x01
        RT Registered:  0x01
        RT Resolved:    0x01
    Class handles:
      Handle [0]:        30000000760003ba
    Class Handle Version:     1
    Sequence:                 1024
    Match Unsupported:        None <--- flowspec data is supported
    Synced:                   TRUE <--- flowspec is successfully synced to standby RSP (if any)
    Ref Count:                1
    Last Error:               0:Success <--- no error is seen
    Last Batch:               19
    Time Init:                May 21 15:31:08
    Time iClass Update:       May 21 15:31:27
    Statistics                        (packets/bytes)
      Matched             :         37263070189/4098937720790      
      Transmitted         :         21305398659/2343593852490      
      Dropped             :         15957671530/1755343868300      
          
=============================================================

RP/0/RP0/CPU0:fretta-50#sh flowspec vrf customer_1 ipv6 detail  

VRF: customer_1     AFI: IPv6
  Flow           :Dest:200:158:3::2/0-128,Source:200:158:1::2/0-128,NH:=6,DPort:=22,SPort:=4000,Length:=300,DSCP:=12
    Actions      :Traffic-rate: 250000000 bps DSCP: cs6 Redirect: VRF dirty_dancing Route-target: ASN2-4787:666  (bgp.1)
    Statistics                        (packets/bytes)
      Matched             :         16130480136/4903665961344      
      Transmitted         :          8490755776/2581189755904      
      Dropped             :          7639724360/2322476205440  
          
=============================================================
    
RP/0/RP0/CPU0:fretta-50#sh flowspec vrf customer_1 ipv6 internal

VRF: customer_1     AFI: IPv6
  Flow           :Dest:200:158:3::2/0-128,Source:200:158:1::2/0-128,NH:=6,DPort:=22,SPort:=4000,Length:=300,DSCP:=12
    Actions      :Traffic-rate: 250000000 bps DSCP: cs6 Redirect: VRF dirty_dancing Route-target: ASN2-4787:666  (bgp.1)
      Client Version: 0
      Local:          FALSE <--- this is flowspec advertised from server
      Unsupported:    FALSE <--- flowspec data is supported
      RT:
        VRF Name Cfg:   0x00
        RT Cfg:         0x01
        RT Registered:  0x01
        RT Resolved:    0x01
    Class handles:
      Handle [0]:        30000000760003bb
    Class Handle Version:     1
    Sequence:                 1024
    Match Unsupported:        None <--- flowspec data is supported
    Synced:                   TRUE <--- flowspec is successfully synced to standby RSP (if any)
    Ref Count:                1
    Last Error:               0:Success <--- no error is seen
    Last Batch:               19
    Time Init:                May 21 15:31:08
    Time iClass Update:       May 21 15:31:27
    Statistics                        (packets/bytes)
      Matched             :         16131555837/4903992974448      
      Transmitted         :          8491321864/2581361846656      
      Dropped             :          7640233973/2322631127792
          
=============================================================

RP/0/RP0/CPU0:fretta-50#sh flowspec ipv4 nlri

AFI: IPv4
  NLRI (hex)     :0x01204601010103810605815006910bb80a81c80b810a
    Actions      :Traffic-rate: 0 bps  (bgp.1)
          
=============================================================

RP/0/RP0/CPU0:fretta-50#sh flowspec ipv6 nlri

AFI: IPv6
  NLRI (hex)     :0x018000007000010001000000000000000000010280000080000100010000000000000000000103810605811606910fa00981100a91012c0b810c
    Actions      :Traffic-rate: 1000000 bps DSCP: cs1 Nexthop: 202:158:2::1  (bgp.1)
          
=============================================================

RP/0/RP0/CPU0:fretta-50#sh flowspec vrf customer_1 ipv4 nlri

VRF: customer_1     AFI: IPv4
  NLRI (hex)     :0x0120ca9e03020220ca9e0102
    Actions      :Traffic-rate: 250000000 bps DSCP: cs6 Redirect: VRF dirty_dancing Route-target: ASN2-4787:666  (bgp.1)
          
=============================================================

RP/0/RP0/CPU0:fretta-50#sh flowspec vrf customer_1 ipv6 nlri

VRF: customer_1     AFI: IPv6
  NLRI (hex)     :0x018000020001580003000000000000000000020280000200015800010000000000000000000203810605811606910fa00a91012c0b810c
    Actions      :Traffic-rate: 250000000 bps DSCP: cs6 Redirect: VRF dirty_dancing Route-target: ASN2-4787:666  (bgp.1)
          
=============================================================

RP/0/RP0/CPU0:fretta-50#sh policy-map transient type pbr                             
policy-map type pbr __bgpfs_default_IPv4
 handle:0x36000004
 table description: L3 IPv4 and IPv6
 class handle:0x760013eb  sequence 1024
   match destination-address ipv4 70.1.1.1 255.255.255.255
   match protocol tcp
   match destination-port 80
   match source-port 3000
   match packet length 200
   match dscp 10
  drop
 !
          
=============================================================

 

 

 


Logs to provide to Cisco TAC for BGP flowspec related issues on NCS5500 platform


Gather the following set of logs from client router.

Replace "NAME_OF_ROUTER"with the name of your router.

 

  1. Timestamp when the problem occurs (e.g. 16:25:15.095 GMT-7 Fri Dec 15 2017), the more exact, the better.
    It's best if the timestamp can be copied from a specific line of "show log" output.
  2. show tech flowspec file harddisk:/NAME_OF_ROUTER_sh_tech_flowspec
  3. show log | file harddisk:/NAME_OF_ROUTER_show_log.txt
    (showing the events when the problem occurs)




56 Views
0 Comments

I am sure you are all familiar with the difference between P and PX images that are seen on CCO for the ASR9000.

If not, read this https://supportforums.cisco.com/t5/service-providers-documents/asr9000-xr-what-is-the-difference-between-the-p-and-px-files/ta-p/3329745

 

With XR 4.3.0 we are making some changes to the file packaging and from that release on, we only provide

the PX version which is useable on both the RSP2/ASR9001 and RSP440.

This will simplify the SMU delivery as well as the complexity of managing multiple images.

 

Note that in order to run XR 4.3.0 combo image, you may need to upgrade the rommon of your 9022 and RSP440

to a minimum of rommon 0.61 if you do a turboboot.

If you don't do turboboot (that is running release X and upgrading to 4.3.0 with the mini-pie the rommon update is not a requirement

for boot. As always your FPD's need to be up to date after the upgrade, please

1182 Views
1 Comment

Before going further ...

This document describes "BFD over Logical Bundle" (BLB) implementation on NCS5500 platforms.
BLB is supported on these platforms since IOS XR 6.3.2.
For older platforms (like ASR9K or CRS) running IOS XR, BLB has been supported since 4.3.0.
Unless noted, details in this document are applicable across all platforms running IOS XR.

There are many documentations already about XR implementation of BLB or BFD in general, this document took some info from them while also adding new data about specific NCS5500 implementation.

 

This is also a living document, the document will be updated if we have new information.

A particularly good general reference is as follows:
https://supportforums.cisco.com/t5/service-providers-documents/bfd-support-on-cisco-asr9000/ta-p/3153191

 

 

 

A very high level overview of BLB

In the context of routing, purpose of BFD is to detect communication failure between two routers faster than what is supported by routing protocols detection timers.
BFD detects the failure by monitoring incoming BFD control packets from neighbor router.
If a number of packets are lost in transmission for whatever reason and thus not received by the monitoring router, the monitoring router will bring down routing session to the neighbor router.

Picture1.pngBFD detects failure by monitoring incoming BFD control packets from neighbor router.






Keep in mind that BFD will bring down the routing session, but it will be up to the routing protocols to bring up the routing session again (i.e. BFD is only responsible to bring down, not to bring up routing sessions).
So it is possible that the following scenario might happen:

  • BFD misses BFD control packets from a specific neighbor.
  • BFD session goes down, down BFD session brings down routing session (say, OSPF) to that neighbor.
  • After a while, OSPF for some reason manages to recover on its own (maybe the problem only affects BFD packets and not OSPF packets).
  • OSPF session to the neighbor will recover.
  • But BFD session will still be down since it still experiences missing BFD packets.
    So now we have OSPF up and BFD down.
    This outcome is counter-intuitive but expected.

BLB is a BFD implementation on bundle interface.
Since bundle interface has multiple member links, we need a somewhat more complex implementation of BFD when it runs on bundle interface, as compared with when it runs on regular physical interface.



BFD implementation on bundle interface:
BVLAN VS BOB VS BLB


There are three different BFD over bundle interface implementations on IOS XR platform:

 

  1. BVLAN ("BFD over VLAN over bundle")

    Note:

    Supported since IOS-XR 3.3, withdrawn and then supported again on 3.8.2.
    Now has reached end of development and thus not advised to be implemented anymore.
    BFD session can run only on VLAN subinterfaces (e.g. be1.1), not on main bundle interface (e.g. be1).

  2. BoB ("BFD over Bundle")

    Also known as:


    MicroBFD

    Note:

    Supported since IOS-XR 4.0.1 for ASR9K.

    Operation:

    Each bundle member link runs its own BFD session.

    142130-bob.pngIn the above figure, we will have 4 total BFD sessions running (1 session per member link).

                





    BFD Client:


    bundlemgr, i.e. down BFD session on a specific member link can potentially bring down the whole bundle interface (say when down member link would make number of available links to fall below required minimum).
    Down bundle interface will in turn bring down routing session.

  3. BLB ("BFD over Logical Bundle")

    Also known as:

    Logical bundle
        
    Note:

    Supported since IOS-XR 4.3.0 for ASR9K, since 6.3.2 for NCS5500.
    Replaces BVLAN implementation.
    Does not support echo mode (since BLB relies on bfd multipath implementation).
    BFD session can run on main bundle interface (e.g. be1) as well as on VLAN subinterface (e.g. be1.1).
    For ASR9K, BoB and BLB coexistence can be configured via "bfd bundle coexistence bob-blb <>" config, but this is not supported yet for NCS5500.
                
    Operation:

    On NCS5500 platforms, BFD is hardware offloaded, meaning processing of the BFD packets will mostly be done in LC NPU.
    LC CPU will process BFD packets only during BFD initialization process.
    No BFD packets are ever sent to RP.
    This is different than default operation of ASR9K platform in which RP and LC will work together to support BFD sessions.
                
    User will configure a specific LC to host BFD sessions, which doesn't need to be the same LC on which bundle member links reside.
    So for example, it's possible that bundle member links are on LC slot 2 and slot 3, while the BFD sessions are actually hosted by LC on slot 5.

    This is different with the way BFD on non-bundle interface works.
    For non-bundle BFD, the BFD sessions will always be hosted on the LC where the port resides.
    So for example, say we configure OSPF BFD over Hu0/6/0/32.147 interface, then this BFD session will always be hosted by LC on slot 6.
                
    In case of BLB, the host LC will:
    - Send BFD packets into bundle by querying FIB for the list of next hops for the BFD session destination address.
    This is done according to load-balance algorithm, thus different sessions may use different member links of the bundle.
    - Receive BFD packets from bundle via internal path.

    At any time, Tx packets for a particular BFD session will be on one single bundle member link.
    Rx packets, on the other hand, can be received on any bundle member link.

    When LC NPU doesn't receive BFD packets in timely manner, it will generate protection packet towards LC CPU.
    LC CPU will then bring BFD sessions down and notify routing protocol clients.
                
    Same as other XR platforms, BFD packets will use UDP with destination port of 3784. Source port might differ though.

    Here's an example of BLB operation:

    blb_1.pngblb_2.png
    In the figure above, be1 has 2 member links: link1 on LC1 and link2 on LC2.
    LC1 is configured as BFD host LC.
    We configure 4 VLAN subinterfaces: be1.1, be1.2, be1.3, and be1.4.

    BFD session X is running for be1.1 (say, to serve OSPF on be1.1).
    BFD session Y is running for be1.2 (say, to serve ISIS on be1.2).

    Based on load-balance algorithm for the next hops of the BFD session destination address:
    - BFD packets for BFD session X will be transmitted on link1
    - BFD packets for BFD session Y will be transmitted on link2

    Incoming BFD packets for any session can be received on any link (link1 or link2).

    BFD Client:
            
    Routing protocols (single-hop BGP, ISIS, OSPF, static as of IOS-XR 6.3.2), i.e. down BFD brings down routing session.
    Detection of physical bundle member link failure is done via ifmgr and/or LACP informing bundlemgr, which doesn't have anything to do with BFD.
                
    In case of member link failure (that happens to host a specific BLB session), bundlemgr will update the load-balance tables and transmit the BFD packets using different member link, which means a failure of member link will NOT bring down the BLB session.
                
    When it comes to BLB implementation, the only time bundle will bring down BLB sessions is when the whole bundle goes down.
    This is because BLB then won't be able to transmit BFD packets on any bundle member link.

 

Sample Configurations

BVLAN

configure BFD under each desired protocol.

 

-----------------------------------------------------------------------------

interface Bundle-Ether2.1
 ipv4 address 200.1.1.1 255.255.255.0
 encapsulation dot1q 1
!

router static
 address-family ipv4 unicast
  202.158.3.13/32 200.1.1.2 bfd fast-detect minimum-interval 300 multiplier 3
 !
!
router isis cybi
 interface Bundle-Ether2.1
   bfd minimum-interval 300
   bfd multiplier 3
   bfd fast-detect ipv4
   address-family ipv4 unicast
   !
 !
!
router ospf cybi
 area 0
  interface Bundle-Ether2.1
    bfd fast-detect
    bfd minimum-interval 300
    bfd multiplier 3
  !
 !
!
router bgp 4787
 neighbor 202.158.1.1
  remote-as 4787
  address-family ipv4 unicast
   route-policy PASS-ALL in
   route-policy PASS-ALL out
  !
  bfd fast-detect
  bfd multiplier 3
  bfd minimum-interval 300
 !
!

 

-----------------------------------------------------------------------------



BoB

No need for BFD config under protocol since routing protocol is NOT client to BoB.
BoB is configured on main bundle interface (i.e. be1), NOT on VLAN subinterface (i.e. be1.1).
Only ietf mode is supported for NCS5500 platform.

 

-----------------------------------------------------------------------------


interface be1
 bfd mode ietf
 bfd address-family ipv4 destination 202.158.2.1
 bfd address-family ipv4 fast-detect

 

-----------------------------------------------------------------------------



BLB


Configure BFD under each desired protocol (exactly the same with BVLAN).
Configure multipath capability under BFD since BFD needs to use multiple paths (i.e. bundle member links) to reach BFD neighbor.

assuming we pick LC on slot 6 to host BFD:

 

-----------------------------------------------------------------------------


bfd
 multipath include location 0/6/CPU0

 

-----------------------------------------------------------------------------


 Multiple LCs can be configured for load sharing / redundancy purpose:

 

-----------------------------------------------------------------------------


bfd
 multipath include location 0/6/CPU0
 multipath include location 0/2/CPU0

...

-----------------------------------------------------------------------------


There is no specific algorithm to pick the host LC for particular BLB session.
At any point in time, any configured LC that have sufficient resource (in terms of PPS, etc) can host any new BLB session.
Whenever a host LC can no longer support these session type and PPS etc, any new BLB session will be created in next LC in list.
Whenever a host LC restarts, its hosted BLB sessions will be brought down and recreated in next host LC in list.

 


Supported Scale and Timers

How many BFD sessions can run on one time depends on multiple factors as follow:

  1. How many clients served by the BFD sessions.
    The less BFD clients to support, the more BFD sessions can be run.
    For example:
    We can run more BFD sessions if they serve only OSPF clients, instead of the whole OSPF, ISIS, BGP, and static route as clients.

  2. How aggressive the BFD timers are.
    The less aggressive the timers are, the more BFD sessions can be run.
    For example:
    We can run more BFD sessions if they're configured with 300ms*3 timers, instead of 150ms*3.

On NCS5500 platforms, supported scale is as follows:

 

Quick note about LC:

Modular NCS-5500 platforms like NCS-5508 support multiple LC, while "pizza boxes" platforms like NCS-5501 is considered a single LC as a whole (LC 0/0/CPU0).

 

Per LC:

500 mixed BFD sessions (multihop, singlehop on regular interface, BLB, BoB).

 

( BLB + BFD multihop ) per LC:

Modular platforms (NCS-5508, NCS-5516 etc) : 250 sessions

"Pizza boxes" platforms (NCS-5501 etc) : 125 sessions

 

Per chassis:

Modular platforms (NCS-5508, NCS-5516 etc) : 4000 mixed BFD sessions

"Pizza boxes" platforms (NCS-5501 etc) : 500 mixed BFD sessions

 

During test, each of BLB session has OSPF, ISIS, and BGP as client.


On NCS5500 platforms, recommended timer is as follows:

Minimum of 300ms with multiplier 3.
Value more aggressive (i.e. less) than 300ms can be configured but not advised.

Multiplier less than 3 is also not supported..

"show bfd summary" command will give you the info about supported BFD PPS and sessions per chassis.

 

 


BLB and NSR

RP switch over will not tear down existing BLB sessions when NSR is configured under each desired routing protocols.

 


Caveats

Same with ASR9K, only BFD async mode is supported for BLB, echo mode is not supported.
In fact, echo mode is not supported at all with NCS5500 as of 6.3.2 release.

Specific to NCS5500, the following clients are not supported.

  1. IPv6 (e.g. OSPFv3).
  2. MPLS protocols like RSVP-TE and LDP.

Since BFD processing is hardware offloaded, BFD packet counters will not increment when we issue certain show bfd commands like "show bfd session detail".
This is expected since regular BFD CLI command will derive the counter from LC CPU, not from LC NPU.



Sample Scenario

Topology

 

NCS-5508 "potat"
|
|
|Bundle-Ether2.1 : 1 BLB session with OSPF and static route as client
|Bundle-Ether2.2 : 1 BLB session with ISIS and BGP as client
|
|
NCS-5501 "birin"



Router configurations
(only relevant config is shown)

NCS-5508 "potat"

bfd
 multipath include location 0/6/CPU0
!

interface Bundle-Ether2.1
 ipv4 address 202.158.1.1 255.255.255.0
 encapsulation dot1q 1
!

interface Bundle-Ether2.2
 ipv4 address 202.158.2.1 255.255.255.0
 encapsulation dot1q 2
!

router static
 address-family ipv4 unicast
  202.158.3.13/32 202.158.1.2 bfd fast-detect minimum-interval 300 multiplier 3
 !
!
router isis cybi
 nsr
 interface Bundle-Ether2.2
   bfd minimum-interval 300
   bfd multiplier 3
   bfd fast-detect ipv4
   address-family ipv4 unicast
   !
 !
!
router ospf cybi
 nsr
 area 0
  interface Bundle-Ether2.1
    bfd fast-detect
    bfd minimum-interval 300
    bfd multiplier 3
  !
 !
!
router bgp 4787
 nsr
 neighbor 202.158.2.2
  remote-as 4787
  address-family ipv4 unicast
   route-policy PASS-ALL in
   route-policy PASS-ALL out
  !
  update-source Bundle-Ether2.2
  bfd fast-detect
  bfd multiplier 3
  bfd minimum-interval 300
 !
!

NCS-5501 "birin"

bfd
 multipath include location 0/0/CPU0
!

interface Bundle-Ether2.1
 ipv4 address 202.158.1.2 255.255.255.0
 encapsulation dot1q 1
!

interface Bundle-Ether2.2
 ipv4 address 202.158.2.2 255.255.255.0
 encapsulation dot1q 2
!

router static
 address-family ipv4 unicast
  202.158.3.14/32 202.158.1.1 bfd fast-detect minimum-interval 300 multiplier 3
 !
!
router isis cybi
 interface Bundle-Ether2.2
   bfd minimum-interval 300
   bfd multiplier 3
   bfd fast-detect ipv4
   address-family ipv4 unicast
   !
 !
!
router ospf cybi
 area 0
  interface Bundle-Ether2.1
    bfd fast-detect
    bfd minimum-interval 300
    bfd multiplier 3
  !
 !
!
router bgp 4787
 neighbor 202.158.2.1
  remote-as 4787
  address-family ipv4 unicast
   route-policy PASS-ALL in
   route-policy PASS-ALL out
  !
  update-source Bundle-Ether2.2
  bfd fast-detect
  bfd multiplier 3
  bfd minimum-interval 300
 !
!


Show commands

(from NCS-5508 "potat")

RP/0/RP0/CPU0:potat#sh bfd session
Interface           Dest Addr           Local det time(int*mult)      State     
                                    Echo             Async   H/W   NPU     
------------------- --------------- ---------------- ---------------- ----------
BE2.1               202.158.1.2       0s(0s*0)         900ms(300ms*3)   UP        
                                                             Yes   0/6/CPU0       
BE2.2               202.158.2.2       0s(0s*0)         900ms(300ms*3)   UP        
                                                             Yes   0/6/CPU0    



RP/0/RP0/CPU0:potat#sh bfd session detail interface be2.1
I/f: Bundle-Ether2.1, Location: 0/6/CPU0
Dest: 202.158.1.2
Src: 202.158.1.1
 State: UP for 0d:21h:35m:54s, number of times UP: 1
 Session type: SW/V4/SH/BL
Received parameters:
 Version: 1, desired tx interval: 300 ms, required rx interval: 300 ms
 Required echo rx interval: 0 ms, multiplier: 3, diag: None
 My discr: 12584150, your discr: 845, state UP, D/F/P/C/A: 0/0/0/1/0
Transmitted parameters:
 Version: 1, desired tx interval: 300 ms, required rx interval: 300 ms
 Required echo rx interval: 0 ms, multiplier: 3, diag: None
 My discr: 845, your discr: 12584150, state UP, D/F/P/C/A: 0/1/0/1/0
Timer Values:
 Local negotiated async tx interval: 300 ms
 Remote negotiated async tx interval: 300 ms
 Desired echo tx interval: 0 s, local negotiated echo tx interval: 0 ms
 Echo detection time: 0 ms(0 ms*3), async detection time: 900 ms(300 ms*3)
Label:
 Internal label: 64119/0xfa77
Local Stats:
 Intervals between async packets:
   Tx: Number of intervals=3, min=160 ms, max=726 ms, avg=385 ms
       Last packet transmitted 77754 s ago
   Rx: Number of intervals=4, min=100 ms, max=270 ms, avg=183 ms
       Last packet received 77753 s ago
 Intervals between echo packets:
   Tx: Number of intervals=0, min=0 s, max=0 s, avg=0 s
       Last packet transmitted 0 s ago
   Rx: Number of intervals=0, min=0 s, max=0 s, avg=0 s
       Last packet received 0 s ago
 Latency of echo packets (time between tx and rx):
   Number of packets: 0, min=0 ms, max=0 ms, avg=0 ms
MP download state: BFD_MP_DOWNLOAD_ACK
State change time: Dec 14 18:38:06.721
Session owner information:
                            Desired               Adjusted
  Client               Interval   Multiplier Interval   Multiplier
  -------------------- --------------------- ---------------------
  ospf-cybi            300 ms     3          300 ms     3         
  ipv4_static          300 ms     3          300 ms     3         

H/W Offload Info:
 H/W Offload capability : Y, Hosted NPU     : 0/6/CPU0
 Async Offloaded        : Y, Echo Offloaded : N
 Async rx/tx            : 5/4

Platform Info:
NPU ID: 0
Async RTC ID        : 1          Echo RTC ID        : 0
Async Feature Mask  : 0x0        Echo Feature Mask  : 0x0
Async Session ID    : 0x34d      Echo Session ID    : 0x0
Async Tx Key        : 0x34d  Echo Tx Key        : 0x0
Async Tx Stats addr : 0x0   Echo Tx Stats addr : 0x0
Async Rx Stats addr : 0x0   Echo Rx Stats addr : 0x0



RP/0/RP0/CPU0:potat#sh bfd session detail destination 202.158.2.2
I/f: Bundle-Ether2.2, Location: 0/6/CPU0
Dest: 202.158.1.2
Src: 202.158.1.1
 State: UP for 0d:21h:39m:36s, number of times UP: 1
 Session type: SW/V4/SH/BL
Received parameters:
 Version: 1, desired tx interval: 300 ms, required rx interval: 300 ms
 Required echo rx interval: 0 ms, multiplier: 3, diag: None
 My discr: 12584129, your discr: 824, state UP, D/F/P/C/A: 0/0/0/1/0
Transmitted parameters:
 Version: 1, desired tx interval: 300 ms, required rx interval: 300 ms
 Required echo rx interval: 0 ms, multiplier: 3, diag: None
 My discr: 824, your discr: 12584129, state UP, D/F/P/C/A: 0/1/0/1/0
Timer Values:
 Local negotiated async tx interval: 300 ms
 Remote negotiated async tx interval: 300 ms
 Desired echo tx interval: 0 s, local negotiated echo tx interval: 0 ms
 Echo detection time: 0 ms(0 ms*3), async detection time: 900 ms(300 ms*3)
Label:
 Internal label: 64098/0xfa62
Local Stats:
 Intervals between async packets:
   Tx: Number of intervals=3, min=160 ms, max=616 ms, avg=383 ms
       Last packet transmitted 77975 s ago
   Rx: Number of intervals=4, min=100 ms, max=374 ms, avg=209 ms
       Last packet received 77975 s ago
 Intervals between echo packets:
   Tx: Number of intervals=0, min=0 s, max=0 s, avg=0 s
       Last packet transmitted 0 s ago
   Rx: Number of intervals=0, min=0 s, max=0 s, avg=0 s
       Last packet received 0 s ago
 Latency of echo packets (time between tx and rx):
   Number of packets: 0, min=0 ms, max=0 ms, avg=0 ms
MP download state: BFD_MP_DOWNLOAD_ACK
State change time: Dec 14 18:38:06.721
Session owner information:
                            Desired               Adjusted
  Client               Interval   Multiplier Interval   Multiplier
  -------------------- --------------------- ---------------------
  isis-cybi            300 ms     3          300 ms     3         
  bgp-default          300 ms     3          300 ms     3         

H/W Offload Info:
 H/W Offload capability : Y, Hosted NPU     : 0/6/CPU0
 Async Offloaded        : Y, Echo Offloaded : N
 Async rx/tx            : 5/4

Platform Info:
NPU ID: 0
Async RTC ID        : 1          Echo RTC ID        : 0
Async Feature Mask  : 0x0        Echo Feature Mask  : 0x0
Async Session ID    : 0x338      Echo Session ID    : 0x0
Async Tx Key        : 0x338  Echo Tx Key        : 0x0
Async Tx Stats addr : 0x0   Echo Tx Stats addr : 0x0
Async Rx Stats addr : 0x0   Echo Rx Stats addr : 0x0



Logs to provide to Cisco TAC for BLB related issues on NCS5500 platform

 

As BLB runs between 2 routers, we need to provide logs from both routers.

Gather the following set of logs from each router:

  • Replace "NAME_OF_ROUTER"with the name of your router.
  • Replace "HOSTLC" with the location of the LC, e.g: 00cpu0 for 0/0/CPU0.

 

  1. Timestamp when the problem occurs (e.g. 16:25:15.095 GMT-7 Fri Dec 15 2017), the more exact, the better.
    It's best if the timestamp can be copied from a specific line of "show log" output.

  2. show tech routing bfd file harddisk:/NAME_OF_ROUTER_sh_tech_routing_bfd

  3. show tech bfdhwoff location <host LC> file harddisk:/NAME_OF_ROUTER_sh_tech_bfdhwoff_loca_HOSTLC
    (if you have multiple host LCs, grab show tech from all of them)

  4. show dpa trace location <host LC>  | file harddisk:/NAME_OF_ROUTER_show_dpa_trace_location_HOSTLC.txt
    (if you have multiple host LCs, grab output from all of them)

  5. show controller fia trace all location <host LC> | file harddisk:/NAME_OF_ROUTER_show_controller_fia_trace_all_location_HOSTLC.txt
    (if you have multiple host LCs, grab output from all of them)

  6. show bfd session detail | file harddisk:/NAME_OF_ROUTER_show_bfd_sess_det.txt

  7. show bfd summary | file harddisk:/NAME_OF_ROUTER_show_bfd_sum.txt

  8. show log | file harddisk:/NAME_OF_ROUTER_show_log.txt
    (showing the events when the problem occurs)



134 Views
0 Comments

Quick note on the issues with support forums we are all experiencing

Read more...

317 Views
0 Comments

Read about the new features and fixes for CSM 3.5!!

 

Read more...

1994 Views
2 Comments

Cisco Software Manager (CSM) v2.0 is not able to retrieve platforms and releases information from Cisco.com.

Problem: Cisco.com no longer accepts HTTP protocol when querying platforms and releases information.  All queries must use the secure HTTPS protocol.

Symptom: When CSM v2.0 is open, it displays the following error message 

‘Unable to retrieve platforms and releases information from Cisco.com.  Close this Tab, check the Internet connection, and re-open the Tab.’

Workaround:  Use a text editor to include this entry in csm.ini (Windows) or .csmrc (UNIX or MAC) file

smu.meta_file_urls=https://www.cisco.com/web/Cisco_IOS_XR_Software/SMUMetaFile

 

On Windows: The csm.ini file resides in the c:\Users\<username> or c:\Documents and Settings\<username> directory

On UNIX/MAC: The .csmrc file resides in the user home directory (i.e. cd ~)

 

NOTE: CSM v2.0 (Java client application) is an End-of-Life product.  There is no planned development.  Customers are advised to migrate to CSM Server v3.4. CSM Server is a web application with centralized user authentication and database which provides end-to-end software management for IOS-XR devices.  CSM v3.4 can be downloaded at

https://software.cisco.com/download/release.html?mdfid=284012813&softwareid=284777134&release=3.4&relind=AVAILABLE&rellifecycle=&reltype=latest

Once download, unzip the file and follow the install_guide.pdf under the csm directory for installation instructions

or send us a note and we can provide a pre-installed OVA.

CSM Server Youtube Videos: 

1)       Overview

https://youtu.be/isxN08x-mr4

 

2)       Getting Started

https://www.youtube.com/watch?v=omdpr3uP_b4

Please send question to cs-software-manager@cisco.com regarding CSM Server.

xander & CSM team!

1623 Views
5 Comments

Up until 6.1.2, IOS-XR sshv2 supports only CBC ciphers (aes128-cbc,aes192-cbc,aes256-cbc,3des-cbcaes128-cbc,aes192-cbc,aes256-cbc,3des-cbc). That is, if a client were to request a CTR cipher (for e.g.: ssh -c aes128-ctr -l dpullat 1.1.1.2), IOS-XR will close the connection with:

RP/0/RSP0/CPU0:Feb 21 14:37:24.551 : SSHD_[65823]: %SECURITY-SSHD-6-INFO_GENERAL : Enc name is NULL: client aes128-ctr server aes128-cbc,aes192-cbc,aes256-cbc,3des-cbc

CBC ciphers have been well known for their security vulnerability:

SSH CBC vulnerability

As part of this effort to disable CBC ciphers and enable only CTR ciphers for SSHv2 on IOS-XR, from release 6.1.2 onwards, all CBC ciphers are disabled or not supported on IOS-XR. Only CTR ciphers are supported from 6.1.2 and up. This change was brought in by CSCvb53125.

Next, IOS-XR will have the capability to configure a specific CTR cipher to use, for customers who wish to strictly enforce a particular one. This is targeted for an upcoming release.

580 Views
0 Comments

With release 6.1.3, IOS-XR 64-bit (eXR) brings in the feature of Flexible Packaging (aka Golden ISO or GISO).

What is it:

Cisco releases IOS-XR 64-bit software shipped with a mini ISO which contains mandatory IOS-XR packages for a given platform, set of optional packages as RPMs and software patches, SMUs.

In response to Customer demands on more flexible ways to manage software on the router, Golden ISO was developed as a customised ISO which customers and field teams can build offline out of the mini ISO by using Cisco Released Golden ISO build script (written in python). The IOS-XR flexible packaging install infrastructure facilitates this feature from release 6.1.3 onwards on its 64-bit versions.

When the system is booted up with Golden ISO, additional SMUs/optional packages present in Golden ISO will be auto-installed. IOS-XR configuration, if present in Golden ISO, will be auto-applied. The router, once booted with a GISO, is ready for running traffic.

Golden ISO is not an image released by Cisco. The customer can create their own Golden ISO using Cisco released Golden ISO creation tool, Cisco released mini ISO. One can also move from one GISO to another within the same release. And, we are integrating GISO with CSM.

Where to find more info:

Cisco ASR 9000 Series Aggregation Services Router Flexible Packaging Configuration Guide

618 Views
0 Comments

Introduction

  • Virtual Machines (VMs) are running on all RP’s and LC’s
  • 1x SysAdmin VM and 1x XR VM (Default-SDR) per node
  • From the Active XR VM, connect to the SysAdmin VM by typing ‘admin’
  • For troubleshooting purposes it is possible to ‘attach’ to remote VM’s
  • From SysAdmin VM: attach location x/y
  • From XR VM: attach location x/y/cpu0

 

How does it look on a node

  • Sysadmin (also known as Calvados) offers admin plane separation from XR for better fault isolation
  • Linux Kernel for scale and performance
  • Better manageability – CLI/XML/SNMP/Netconf

Show platform in XR

RP/0/RP0/CPU0:eXR-LER1#show platform

Node              Type                       State             Config state

--------------------------------------------------------------------------------

0/RP0/CPU0        A99-RP2-SE(Active)         IOS XR RUN        NSHUT

0/RP1/CPU0        A99-RP2-SE(Standby)        IOS XR RUN        NSHUT

0/FT0             ASR-9922-FAN-V2            OPERATIONAL       NSHUT

0/FT1             ASR-9922-FAN-V2            OPERATIONAL       NSHUT

0/FT2             ASR-9922-FAN-V2            OPERATIONAL       NSHUT

0/FT3             ASR-9922-FAN-V2            OPERATIONAL       NSHUT

0/0/CPU0          A9K-8X100GE-SE             IOS XR RUN        NSHUT

0/11/CPU0         A99-12x100GE               IOS XR RUN        NSHUT

0/FC0             A99-SFC2                   OPERATIONAL       NSHUT

0/FC1             A99-SFC2                   OPERATIONAL       NSHUT

0/FC2             A99-SFC2                   OPERATIONAL       NSHUT

0/FC3             A99-SFC2                   OPERATIONAL       NSHUT

0/FC4             A99-SFC2                   OPERATIONAL       NSHUT

0/PT0             A9K-AC-PEM-V2              OPERATIONAL       NSHUT

0/PT1             A9K-AC-PEM-V2              OPERATIONAL       NSHUT

 

State, IOS XR RUN above for RP/LC indicates XR VM is up and IOS XR is running. For FC/FT/PT, the XR VM state is indicated as OPERATIONAL for it to be functional.

Show platform in Sysadmin

sysadmin-vm:0_RP0# show platform

Wed Nov  30 18:14:27.693 UTC

Location  Card Type               HW State      SW State      Config State 

----------------------------------------------------------------------------

0/0       A9K-8X100GE-SE          OPERATIONAL   OPERATIONAL   NSHUT        

0/11      A99-12X100GE            OPERATIONAL   OPERATIONAL   NSHUT        

0/RP0     A99-RP2-SE              OPERATIONAL   OPERATIONAL   NSHUT        

0/RP1     A99-RP2-SE              OPERATIONAL   OPERATIONAL   NSHUT        

0/FC0     A99-SFC2                OPERATIONAL   N/A           NSHUT        

0/FC1     A99-SFC2                OPERATIONAL   N/A           NSHUT        

0/FC2     A99-SFC2                OPERATIONAL   N/A           NSHUT        

0/FC3     A99-SFC2                OPERATIONAL   N/A           NSHUT        

0/FC4     A99-SFC2                OPERATIONAL   N/A           NSHUT        

0/FT0     ASR-9922-FAN-V2         OPERATIONAL   N/A           NSHUT        

0/FT1     ASR-9922-FAN-V2         OPERATIONAL   N/A           NSHUT        

0/FT2     ASR-9922-FAN-V2         OPERATIONAL   N/A           NSHUT        

0/FT3     ASR-9922-FAN-V2         OPERATIONAL   N/A           NSHUT        

0/PT0     A9K-AC-PEM-V2           OPERATIONAL   N/A           NSHUT        

0/PT1     A9K-AC-PEM-V2           OPERATIONAL   N/A           NSHUT        

 

HW state (OPERATIONAL) above indicates powered up state and SW state (OPERATIONAL) shows Sysadmin VM to be up and running.  

 

Managing VM reload and shutdown across nodes

Reload Type

VM Admin/XR

Test case

CLI

HW Reload

admin

Chassis reload

sysadmin-vm:0_RP0# hw-module location all reload

power up/down node

RP/LC/FC reload

sysadmin-vm:0_RP0# hw-module location x/y reload

Rack Reload

sysadmin-vm:0_RP0# reload rack 0

SW Reload

admin

LC

reload admin location 0/0 or reload location 0/0 admin

restart sysadmin VM

RP

reload admin location 0/RP0 or  reload location 0/RP0 admin                                     

shutdown

exec mode

hw-module location all shutdown

hw-module location 0/0 shutdown 

hw-module location 0/RP0 shutdown

hw-module location 0/FC0 shutdown 

To recover: reload LC/RP/RSP/FC

admin config mode

hw-module shutdown location 0/0

hw-module shutdown location 0/RP0

hw-module shutdown location 0/FC0

To recover:

no hw-module shutdown location 0/0

no hw-module shutdown location 0/RP0

no hw-module shutdown location 0/FC0

SW Reload

restart XR VM

XR

 

 

LC

reload location 0/0/CPU0 

RP

reload location 0/RP0/CPU0 

All nodes

reload location all

RP switchover

redundancy switchover 

 

1552 Views
4 Comments

With release 6.1.2 of IOS-XR, 64-bit eXR is now available on CCO. With a ton of drivers and forward-looking features and functionality, eXR is the next generation of IOS-XR that runs on virtualized environment with underlying 64 bit Linux kernel.

The key capabilities of IOS XR 64-bit OS includes:
• Telemetry — push towards smarter visibility of the network by streaming data to a configured receiver
  for analysis and troubleshooting purposes
• Application Hosting — leverage hosting of third-party applications in container environment
• Data Models — automate configurations that belong to multiple routers across the network
• Flexible Packaging — easy routine upgrades and maintenance with modularized Redhat Packet    Manager (RPM) packages

Migration to eXR is achieved easily and the CCO documentation for Migration from 32-bit QNX kernel XR to 64-bit Linux kernel XR is now available.

https://www.cisco.com/c/en/us/td/docs/routers/asr9000/migration/guide/b-migration-to-ios-xr-64-bit.html

Happy Migrations...

41 Views
0 Comments

show congestion-control statistics { gtpcmgr | a11mgr | hamgr | l2tpmgr }
show resources session
show subscriber configuration username subscriber-name
show subscribers aaa-configuration username subscriber_name 
show subscribers all
show session counters pcf-summary 
show session progress
show session progress pcf all 
show session subsystem facility aaamgr all
show session subsystem facility famgr all
show session subsystem facility a11mgr all
show session subsystem facility l2tpdemux all
show session subsystem facility l2tpmgr all
show session subsystem facility sessmgr all
show session disconnect-reasons
show session recovery status [verbose] 

PPP

show ppp
show ppp username subscriber-name
show ppp counters username subscriber-name
show ppp full username subscriber-name
show ppp statistics pdsn-service 
show ppp statistics pdsn-service service_name 
show rp
show rp full username subscriber_name 
show rp statistics pdsn-service 
show rp statistics pdsn-service service_name 
show mipfa full username subscriber_name 
show mipfa statistics fa-service service_name 
show mipfa counters 

Radius

show radius authentication servers radius group group_name detail 
show radius accounting servers radius group group_name detail 
show radius counters all 
show radius counters summary 

L2TP

show l2tp sessions 
show l2tp session full username subscriber_name 
show l2tp statistics lac-service service_name 
show l2tp tunnels all
show l2tp full s
show crypto ipsec security-associations statistics 

Security 

show crypto isakmp keys 
show crypto statistics 

51 Views
0 Comments

Commands Reference
context <egress ctx>
show ip pool
show ip pool wide
show ip pool groups See group stats
show ip pool group-name <group name>
show subscribers data-rate high ip-pool <ip poolname> See if there's a problem with one way traffic on a specific ip pool
show task resources facility vpnmgr all ip pool runs under vpnmgr task
show ip pool summary wide
show ip pool pool-name <pool name>
show subscribers summary ip-pool <pool name>
show subscribers summary ip-pool <pool name> idle-time > 1800
show ip pool public
show ip pool static
show ip pool private

107 Views
0 Comments

show snmp trap history verbose
show l2tp statistics pdsnclosedrp-service CRP
show l2tp statistics lac-service XXXX
show l2tp statistics peer-address <ip address of peer>
show l2tp tunnels all
clear l2tp statistics
show l2tp full stat
logging filter active facility l2tp-control level warning
logging active
no logging active 
show session subsystem facility l2tpdemux all
show session subsystem facility l2tpmgr all
show congestion-control statistics { gtpcmgr | a11mgr | hamgr | l2tpmgr }