Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 

Service Providers Blogs

79 Views
0 Comments

Before going further ...

This document describes "BFD over Logical Bundle" (BLB) implementation on NCS5500 platforms.
BLB is supported on these platforms since IOS XR 6.3.2.
For older platforms (like ASR9K or CRS) running IOS XR, BLB has been supported since 4.3.0.
Unless noted, details in this document are applicable across all platforms running IOS XR.

There are many documentations already about XR implementation of BLB or BFD in general, this document took some info from them while also adding new data about specific NCS5500 implementation.

 

This is also a living document, the document will be updated if we have new information.

A particularly good general reference is as follows:
https://supportforums.cisco.com/t5/service-providers-documents/bfd-support-on-cisco-asr9000/ta-p/3153191

 

 

 

A very high level overview of BLB

In the context of routing, purpose of BFD is to detect communication failure between two routers faster than what is supported by routing protocols detection timers.
BFD detects the failure by monitoring incoming BFD control packets from neighbor router.
If a number of packets are lost in transmission for whatever reason and thus not received by the monitoring router, the monitoring router will bring down routing session to the neighbor router.

BFD detects failure by monitoring incoming BFD control packets from neighbor router.BFD detects failure by monitoring incoming BFD control packets from neighbor router.






Keep in mind that BFD will bring down the routing session, but it will be up to the routing protocols to bring up the routing session again (i.e. BFD is only responsible to bring down, not to bring up routing sessions).
So it is possible that the following scenario might happen:

  • BFD misses BFD control packets from a specific neighbor.
  • BFD session goes down, down BFD session brings down routing session (say, OSPF) to that neighbor.
  • After a while, OSPF for some reason manages to recover on its own (maybe the problem only affects BFD packets and not OSPF packets).
  • OSPF session to the neighbor will recover.
  • But BFD session will still be down since it still experiences missing BFD packets.
    So now we have OSPF up and BFD down.
    This outcome is counter-intuitive but expected.

BLB is a BFD implementation on bundle interface.
Since bundle interface has multiple member links, we need a somewhat more complex implementation of BFD when it runs on bundle interface, as compared with when it runs on regular physical interface.



BFD implementation on bundle interface:
BVLAN VS BOB VS BLB


There are three different BFD over bundle interface implementations on IOS XR platform:

 

  1. BVLAN ("BFD over VLAN over bundle")

    Note:

    Supported since IOS-XR 3.3, withdrawn and then supported again on 3.8.2.
    Now has reached end of development and thus not advised to be implemented anymore.
    BFD session can run only on VLAN subinterfaces (e.g. be1.1), not on main bundle interface (e.g. be1).

  2. BoB ("BFD over Bundle")

    Also known as:


    MicroBFD

    Note:

    Supported since IOS-XR 4.0.1 for ASR9K.

    Operation:

    Each bundle member link runs its own BFD session.

    In the above figure, we will have 4 total BFD sessions running (1 session per member link).In the above figure, we will have 4 total BFD sessions running (1 session per member link).

                




    BFD Client:

    bundlemgr, i.e. down BFD session on a specific member link can potentially bring down the whole bundle interface (say when down member link would make number of available links to fall below required minimum).
    Down bundle interface will in turn bring down routing session.

  3. BLB ("BFD over Logical Bundle")

    Also known as:

    Logical bundle
        
    Note:

    Supported since IOS-XR 4.3.0 for ASR9K, since 6.3.2 for NCS5500.
    Replaces BVLAN implementation.
    Does not support echo mode (since BLB relies on bfd multipath implementation).
    BFD session can run on main bundle interface (e.g. be1) as well as on VLAN subinterface (e.g. be1.1).
    For ASR9K, BoB and BLB coexistence can be configured via "bfd bundle coexistence bob-blb <>" config, but this is not supported yet for NCS5500.
                
    Operation:

    On NCS5500 platforms, BFD is hardware offloaded, meaning processing of the BFD packets will mostly be done in LC NPU.
    LC CPU will process BFD packets only during BFD initialization process.
    No BFD packets are ever sent to RP.
    This is different than default operation of ASR9K platform in which RP and LC will work together to support BFD sessions.
                
    User will configure a specific LC to host BFD sessions, which doesn't need to be the same LC on which bundle member links reside.
    So for example, it's possible that bundle member links are on LC slot 2 and slot 3, while the BFD sessions are actually hosted by LC on slot 5.
                
    The host LC will:
    - Send BFD packets into bundle by querying FIB for the list of next hops for the BFD session destination address.
    This is done according to load-balance algorithm, thus different sessions may use different member links of the bundle.
    - Receive BFD packets from bundle via internal path.

    At any time, Tx packets for a particular BFD session will be on one single bundle member link.
    Rx packets, on the other hand, can be received on any bundle member link.

    When LC NPU doesn't receive BFD packets in timely manner, it will generate protection packet towards LC CPU.
    LC CPU will then bring BFD sessions down and notify routing protocol clients.
                
    Same as other XR platforms, BFD packets will use UDP with destination port of 3784. Source port might differ though.

    Here's an example of BLB operation:

    blb_1.pngblb_2.png
    In the figure above, be1 has 2 member links: link1 on LC1 and link2 on LC2.
    LC1 is configured as BFD host LC.
    We configure 4 VLAN subinterfaces: be1.1, be1.2, be1.3, and be1.4.

    BFD session X is running for be1.1 (say, to serve OSPF on be1.1).
    BFD session Y is running for be1.2 (say, to serve ISIS on be1.2).

    Based on load-balance algorithm for the next hops of the BFD session destination address:
    - BFD packets for BFD session X will be transmitted on link1
    - BFD packets for BFD session Y will be transmitted on link2

    Incoming BFD packets for any session can be received on any link (link1 or link2).

    BFD Client:
            
    Routing protocols (single-hop BGP, multi-hop BGP, ISIS, OSPF, static as of IOS-XR 6.3.2), i.e. down BFD brings down routing session.
    Detection of physical bundle member link failure is done via ifmgr and/or LACP informing bundlemgr, which doesn't have anything to do with BFD.
                
    In case of member link failure (that happens to host a specific BLB session), bundlemgr will update the load-balance tables and transmit the BFD packets using different member link, which means a failure of member link will NOT bring down the BLB session.
                
    When it comes to BLB implementation, the only time bundle will bring down BLB sessions is when the whole bundle goes down.
    This is because BLB then won't be able to transmit BFD packets on any bundle member link.

 

Sample Configurations

BVLAN

configure BFD under each desired protocol.

 

-----------------------------------------------------------------------------

interface Bundle-Ether2.1
 ipv4 address 200.1.1.1 255.255.255.0
 encapsulation dot1q 1
!

router static
 address-family ipv4 unicast
  202.158.3.13/32 200.1.1.2 bfd fast-detect minimum-interval 300 multiplier 3
 !
!
router isis cybi
 interface Bundle-Ether2.1
   bfd minimum-interval 300
   bfd multiplier 3
   bfd fast-detect ipv4
   address-family ipv4 unicast
   !
 !
!
router ospf cybi
 area 0
  interface Bundle-Ether2.1
    bfd fast-detect
    bfd minimum-interval 300
    bfd multiplier 3
  !
 !
!
router bgp 4787
 neighbor 202.158.1.1
  remote-as 4787
  address-family ipv4 unicast
   route-policy PASS-ALL in
   route-policy PASS-ALL out
  !
  bfd fast-detect
  bfd multiplier 3
  bfd minimum-interval 300
 !
!

 

-----------------------------------------------------------------------------



BoB

No need for BFD config under protocol since routing protocol is NOT client to BoB.
BoB is configured on main bundle interface (i.e. be1), NOT on VLAN subinterface (i.e. be1.1).
Only ietf mode is supported for NCS5500 platform.

 

-----------------------------------------------------------------------------


interface be1
 bfd mode ietf
 bfd address-family ipv4 destination 202.158.2.1
 bfd address-family ipv4 fast-detect

 

-----------------------------------------------------------------------------



BLB


Configure BFD under each desired protocol (exactly the same with BVLAN).
Configure multipath capability under BFD since BFD needs to use multiple paths (i.e. bundle member links) to reach BFD neighbor.

assuming we pick LC on slot 6 to host BFD:

 

-----------------------------------------------------------------------------


bfd
 multipath include location 0/6/CPU0

 

-----------------------------------------------------------------------------


 Multiple LCs can be configured for load sharing / redundancy purpose:

 

-----------------------------------------------------------------------------


bfd
 multipath include location 0/6/CPU0
 multipath include location 0/2/CPU0

...

-----------------------------------------------------------------------------


There is no specific algorithm to pick the host LC for particular BFD session.
At any point in time, any configured LC that have sufficient resource (in terms of PPS, etc) can host any new BFD session.
Whenever a host LC can no longer support these session type and PPS etc, any new BFD session will be created in next LC in list.
Whenever a host LC restarts, its hosted BFD sessions will be brought down and recreated in next host LC in list.

 


Supported Scale and Timers

How many BFD sessions can run on one time depends on multiple factors as follow:

  1. How many clients served by the BFD sessions.
    The less BFD clients to support, the more BFD sessions can be run.
    For example:
    We can run more BFD sessions if they serve only OSPF clients, instead of the whole OSPF, ISIS, BGP, and static route as clients.

  2. How aggressive the BFD timers are.
    The less aggressive the timers are, the more BFD sessions can be run.
    For example:
    We can run more BFD sessions if they're configured with 300ms*3 timers, instead of 150ms*3.

On NCS5500 platforms, supported scale is as follows:

 

Per LC:

500 mixed BFD sessions (multihop, singlehop on regular interface, BLB, BoB).

 

BLB per LC:

125 sessions for NCS-5501 platform, 250 sessions for other NCS-5500 family.

During test, each of this session has OSPF, ISIS, and BGP as client.

Note that modular NCS-5500 platforms like NCS-5508 support multiple LC, while "pizza boxes" platforms like NCS-5501 is considered a single LC as a whole (LC 0/0/CPU0).

 

BLB per chassis:

Modular platforms (NCS-5508, NCS-5516 etc) : 4000 sessions

"Pizza boxes" platforms (NCS-5501 etc) : 125 sessions


On NCS5500 platforms, recommended timer is as follows:

minimum of 300ms with multiplier 3.
Value more aggressive (i.e. less) than 300ms can be configured but not advised.

"show bfd summary" command will give you the info about supported BFD PPS and sessions per chassis.

 

 


BLB and NSR

RP switch over will not tear down existing BLB sessions when NSR is configured under each desired routing protocols.

 


Caveats

Same with ASR9K, only BFD async mode is supported for BLB, echo mode is not supported.
In fact, echo mode is not supported at all with NCS5500 as of 6.3.2 release.

Specific to NCS5500, the following clients are not supported.

  1. IPv6 (e.g. OSPFv3).
  2. MPLS protocols like RSVP-TE and LDP.

Since BFD processing is hardware offloaded, BFD packet counters will not increment when we issue certain show bfd commands like "show bfd session detail".
This is expected since regular BFD CLI command will derive the counter from LC CPU, not from LC NPU.



Sample Scenario

Topology

 

NCS-5508 "potat"
|
|
|Bundle-Ether2.1 : 1 BLB session with OSPF and static route as client
|Bundle-Ether2.2 : 1 BLB session with ISIS and BGP as client
|
|
NCS-5501 "birin"



Router configurations
(only relevant config is shown)

NCS-5508 "potat"

bfd
 multipath include location 0/6/CPU0
!

interface Bundle-Ether2.1
 ipv4 address 202.158.1.1 255.255.255.0
 encapsulation dot1q 1
!

interface Bundle-Ether2.2
 ipv4 address 202.158.2.1 255.255.255.0
 encapsulation dot1q 2
!

router static
 address-family ipv4 unicast
  202.158.3.13/32 202.158.1.2 bfd fast-detect minimum-interval 300 multiplier 3
 !
!
router isis cybi
 nsr
 interface Bundle-Ether2.2
   bfd minimum-interval 300
   bfd multiplier 3
   bfd fast-detect ipv4
   address-family ipv4 unicast
   !
 !
!
router ospf cybi
 nsr
 area 0
  interface Bundle-Ether2.1
    bfd fast-detect
    bfd minimum-interval 300
    bfd multiplier 3
  !
 !
!
router bgp 4787
 nsr
 neighbor 202.158.2.2
  remote-as 4787
  address-family ipv4 unicast
   route-policy PASS-ALL in
   route-policy PASS-ALL out
  !
  update-source Bundle-Ether2.2
  bfd fast-detect
  bfd multiplier 3
  bfd minimum-interval 300
 !
!

NCS-5501 "birin"

bfd
 multipath include location 0/0/CPU0
!

interface Bundle-Ether2.1
 ipv4 address 202.158.1.2 255.255.255.0
 encapsulation dot1q 1
!

interface Bundle-Ether2.2
 ipv4 address 202.158.2.2 255.255.255.0
 encapsulation dot1q 2
!

router static
 address-family ipv4 unicast
  202.158.3.14/32 202.158.1.1 bfd fast-detect minimum-interval 300 multiplier 3
 !
!
router isis cybi
 interface Bundle-Ether2.2
   bfd minimum-interval 300
   bfd multiplier 3
   bfd fast-detect ipv4
   address-family ipv4 unicast
   !
 !
!
router ospf cybi
 area 0
  interface Bundle-Ether2.1
    bfd fast-detect
    bfd minimum-interval 300
    bfd multiplier 3
  !
 !
!
router bgp 4787
 neighbor 202.158.2.1
  remote-as 4787
  address-family ipv4 unicast
   route-policy PASS-ALL in
   route-policy PASS-ALL out
  !
  update-source Bundle-Ether2.2
  bfd fast-detect
  bfd multiplier 3
  bfd minimum-interval 300
 !
!


Show commands

(from NCS-5508 "potat")

RP/0/RP0/CPU0:potat#sh bfd session
Interface           Dest Addr           Local det time(int*mult)      State     
                                    Echo             Async   H/W   NPU     
------------------- --------------- ---------------- ---------------- ----------
BE2.1               202.158.1.2       0s(0s*0)         900ms(300ms*3)   UP        
                                                             Yes   0/6/CPU0       
BE2.2               202.158.2.2       0s(0s*0)         900ms(300ms*3)   UP        
                                                             Yes   0/6/CPU0    



RP/0/RP0/CPU0:potat#sh bfd session detail interface be2.1
I/f: Bundle-Ether2.1, Location: 0/6/CPU0
Dest: 202.158.1.2
Src: 202.158.1.1
 State: UP for 0d:21h:35m:54s, number of times UP: 1
 Session type: SW/V4/SH/BL
Received parameters:
 Version: 1, desired tx interval: 300 ms, required rx interval: 300 ms
 Required echo rx interval: 0 ms, multiplier: 3, diag: None
 My discr: 12584150, your discr: 845, state UP, D/F/P/C/A: 0/0/0/1/0
Transmitted parameters:
 Version: 1, desired tx interval: 300 ms, required rx interval: 300 ms
 Required echo rx interval: 0 ms, multiplier: 3, diag: None
 My discr: 845, your discr: 12584150, state UP, D/F/P/C/A: 0/1/0/1/0
Timer Values:
 Local negotiated async tx interval: 300 ms
 Remote negotiated async tx interval: 300 ms
 Desired echo tx interval: 0 s, local negotiated echo tx interval: 0 ms
 Echo detection time: 0 ms(0 ms*3), async detection time: 900 ms(300 ms*3)
Label:
 Internal label: 64119/0xfa77
Local Stats:
 Intervals between async packets:
   Tx: Number of intervals=3, min=160 ms, max=726 ms, avg=385 ms
       Last packet transmitted 77754 s ago
   Rx: Number of intervals=4, min=100 ms, max=270 ms, avg=183 ms
       Last packet received 77753 s ago
 Intervals between echo packets:
   Tx: Number of intervals=0, min=0 s, max=0 s, avg=0 s
       Last packet transmitted 0 s ago
   Rx: Number of intervals=0, min=0 s, max=0 s, avg=0 s
       Last packet received 0 s ago
 Latency of echo packets (time between tx and rx):
   Number of packets: 0, min=0 ms, max=0 ms, avg=0 ms
MP download state: BFD_MP_DOWNLOAD_ACK
State change time: Dec 14 18:38:06.721
Session owner information:
                            Desired               Adjusted
  Client               Interval   Multiplier Interval   Multiplier
  -------------------- --------------------- ---------------------
  ospf-cybi            300 ms     3          300 ms     3         
  ipv4_static          300 ms     3          300 ms     3         

H/W Offload Info:
 H/W Offload capability : Y, Hosted NPU     : 0/6/CPU0
 Async Offloaded        : Y, Echo Offloaded : N
 Async rx/tx            : 5/4

Platform Info:
NPU ID: 0
Async RTC ID        : 1          Echo RTC ID        : 0
Async Feature Mask  : 0x0        Echo Feature Mask  : 0x0
Async Session ID    : 0x34d      Echo Session ID    : 0x0
Async Tx Key        : 0x34d  Echo Tx Key        : 0x0
Async Tx Stats addr : 0x0   Echo Tx Stats addr : 0x0
Async Rx Stats addr : 0x0   Echo Rx Stats addr : 0x0



RP/0/RP0/CPU0:potat#sh bfd session detail destination 202.158.2.2
I/f: Bundle-Ether2.2, Location: 0/6/CPU0
Dest: 202.158.1.2
Src: 202.158.1.1
 State: UP for 0d:21h:39m:36s, number of times UP: 1
 Session type: SW/V4/SH/BL
Received parameters:
 Version: 1, desired tx interval: 300 ms, required rx interval: 300 ms
 Required echo rx interval: 0 ms, multiplier: 3, diag: None
 My discr: 12584129, your discr: 824, state UP, D/F/P/C/A: 0/0/0/1/0
Transmitted parameters:
 Version: 1, desired tx interval: 300 ms, required rx interval: 300 ms
 Required echo rx interval: 0 ms, multiplier: 3, diag: None
 My discr: 824, your discr: 12584129, state UP, D/F/P/C/A: 0/1/0/1/0
Timer Values:
 Local negotiated async tx interval: 300 ms
 Remote negotiated async tx interval: 300 ms
 Desired echo tx interval: 0 s, local negotiated echo tx interval: 0 ms
 Echo detection time: 0 ms(0 ms*3), async detection time: 900 ms(300 ms*3)
Label:
 Internal label: 64098/0xfa62
Local Stats:
 Intervals between async packets:
   Tx: Number of intervals=3, min=160 ms, max=616 ms, avg=383 ms
       Last packet transmitted 77975 s ago
   Rx: Number of intervals=4, min=100 ms, max=374 ms, avg=209 ms
       Last packet received 77975 s ago
 Intervals between echo packets:
   Tx: Number of intervals=0, min=0 s, max=0 s, avg=0 s
       Last packet transmitted 0 s ago
   Rx: Number of intervals=0, min=0 s, max=0 s, avg=0 s
       Last packet received 0 s ago
 Latency of echo packets (time between tx and rx):
   Number of packets: 0, min=0 ms, max=0 ms, avg=0 ms
MP download state: BFD_MP_DOWNLOAD_ACK
State change time: Dec 14 18:38:06.721
Session owner information:
                            Desired               Adjusted
  Client               Interval   Multiplier Interval   Multiplier
  -------------------- --------------------- ---------------------
  isis-cybi            300 ms     3          300 ms     3         
  bgp-default          300 ms     3          300 ms     3         

H/W Offload Info:
 H/W Offload capability : Y, Hosted NPU     : 0/6/CPU0
 Async Offloaded        : Y, Echo Offloaded : N
 Async rx/tx            : 5/4

Platform Info:
NPU ID: 0
Async RTC ID        : 1          Echo RTC ID        : 0
Async Feature Mask  : 0x0        Echo Feature Mask  : 0x0
Async Session ID    : 0x338      Echo Session ID    : 0x0
Async Tx Key        : 0x338  Echo Tx Key        : 0x0
Async Tx Stats addr : 0x0   Echo Tx Stats addr : 0x0
Async Rx Stats addr : 0x0   Echo Rx Stats addr : 0x0



Logs to provide to Cisco TAC for BLB related issues on NCS5500 platform

 

As BLB runs between 2 routers, we need to provide logs from both routers.

Gather the following set of logs from each router:

  • Replace "NAME_OF_ROUTER"with the name of your router.
  • Replace "HOSTLC" with the location of the LC, e.g: 00cpu0 for 0/0/CPU0.

 

  1. Timestamp when the problem occurs (e.g. 16:25:15.095 GMT-7 Fri Dec 15 2017), the more exact, the better.
    It's best if the timestamp can be copied from a specific line of "show log" output.

  2. show tech routing bfd file harddisk:/NAME_OF_ROUTER_sh_tech_routing_bfd

  3. show tech bfdhwoff location <host LC> file harddisk:/NAME_OF_ROUTER_sh_tech_bfdhwoff_loca_HOSTLC
    (if you have multiple host LCs, grab show tech from all of them)

  4. show dpa trace location <host LC>  | file harddisk:/NAME_OF_ROUTER_show_dpa_trace_location_HOSTLC.txt
    (if you have multiple host LCs, grab output from all of them)

  5. show controller fia trace all location <host LC> | file harddisk:/NAME_OF_ROUTER_show_controller_fia_trace_all_location_HOSTLC.txt
    (if you have multiple host LCs, grab output from all of them)

  6. show bfd session detail | file harddisk:/NAME_OF_ROUTER_show_bfd_sess_det.txt

  7. show log | file harddisk:/NAME_OF_ROUTER_show_log.txt
    (showing the events when the problem occurs)



84 Views
0 Comments

Quick note on the issues with support forums we are all experiencing

Read more...

183 Views
0 Comments

Read about the new features and fixes for CSM 3.5!!

 

Read more...

153 Views
0 Comments

Dear Sir

 

I am planning to use IPSEC over MPLS. Is there any sort of dependency on PE router or on Provider for configuring the IPSEC/GETVPN between CPE routers. 

 

Please answer. I am waiting.

Read more...

1594 Views
2 Comments

Cisco Software Manager (CSM) v2.0 is not able to retrieve platforms and releases information from Cisco.com.

Problem: Cisco.com no longer accepts HTTP protocol when querying platforms and releases information.  All queries must use the secure HTTPS protocol.

Symptom: When CSM v2.0 is open, it displays the following error message 

‘Unable to retrieve platforms and releases information from Cisco.com.  Close this Tab, check the Internet connection, and re-open the Tab.’

Workaround:  Use a text editor to include this entry in csm.ini (Windows) or .csmrc (UNIX or MAC) file

smu.meta_file_urls=https://www.cisco.com/web/Cisco_IOS_XR_Software/SMUMetaFile

 

On Windows: The csm.ini file resides in the c:\Users\<username> or c:\Documents and Settings\<username> directory

On UNIX/MAC: The .csmrc file resides in the user home directory (i.e. cd ~)

 

NOTE: CSM v2.0 (Java client application) is an End-of-Life product.  There is no planned development.  Customers are advised to migrate to CSM Server v3.4. CSM Server is a web application with centralized user authentication and database which provides end-to-end software management for IOS-XR devices.  CSM v3.4 can be downloaded at

https://software.cisco.com/download/release.html?mdfid=284012813&softwareid=284777134&release=3.4&relind=AVAILABLE&rellifecycle=&reltype=latest

Once download, unzip the file and follow the install_guide.pdf under the csm directory for installation instructions

or send us a note and we can provide a pre-installed OVA.

CSM Server Youtube Videos: 

1)       Overview

https://youtu.be/isxN08x-mr4

 

2)       Getting Started

https://www.youtube.com/watch?v=omdpr3uP_b4

Please send question to cs-software-manager@cisco.com regarding CSM Server.

xander & CSM team!

59 Views
1 Comment

Hi,

I am studying BGP and came across the concept of iBGP full mesh.

If I understand correctly, the need to use iBGP full mesh is to let the internal routers within the Autonomous System (AS) learn the best BGP routes.

Also, there's a BGP rule which states that:

"When a router learns routes from an iBGP peer, that router does not advertise those same routes to another iBGP peer"

Applying the above information to the topology , my question is why would I need iBGP peering between routers R1 and R3 when there would be no exchange of BGP routing information at all between the two?

810 Views
0 Comments

Up until 6.1.2, IOS-XR sshv2 supports only CBC ciphers (aes128-cbc,aes192-cbc,aes256-cbc,3des-cbcaes128-cbc,aes192-cbc,aes256-cbc,3des-cbc). That is, if a client were to request a CTR cipher (for e.g.: ssh -c aes128-ctr -l dpullat 1.1.1.2), IOS-XR will close the connection with:

RP/0/RSP0/CPU0:Feb 21 14:37:24.551 : SSHD_[65823]: %SECURITY-SSHD-6-INFO_GENERAL : Enc name is NULL: client aes128-ctr server aes128-cbc,aes192-cbc,aes256-cbc,3des-cbc

CBC ciphers have been well known for their security vulnerability:

SSH CBC vulnerability

As part of this effort to disable CBC ciphers and enable only CTR ciphers for SSHv2 on IOS-XR, from release 6.1.2 onwards, all CBC ciphers are disabled or not supported on IOS-XR. Only CTR ciphers are supported from 6.1.2 and up. This change was brought in by CSCvb53125.

Next, IOS-XR will have the capability to configure a specific CTR cipher to use, for customers who wish to strictly enforce a particular one. This is targeted for an upcoming release.

339 Views
0 Comments

With release 6.1.3, IOS-XR 64-bit (eXR) brings in the feature of Flexible Packaging (aka Golden ISO or GISO).

What is it:

Cisco releases IOS-XR 64-bit software shipped with a mini ISO which contains mandatory IOS-XR packages for a given platform, set of optional packages as RPMs and software patches, SMUs.

In response to Customer demands on more flexible ways to manage software on the router, Golden ISO was developed as a customised ISO which customers and field teams can build offline out of the mini ISO by using Cisco Released Golden ISO build script (written in python). The IOS-XR flexible packaging install infrastructure facilitates this feature from release 6.1.3 onwards on its 64-bit versions.

When the system is booted up with Golden ISO, additional SMUs/optional packages present in Golden ISO will be auto-installed. IOS-XR configuration, if present in Golden ISO, will be auto-applied. The router, once booted with a GISO, is ready for running traffic.

Golden ISO is not an image released by Cisco. The customer can create their own Golden ISO using Cisco released Golden ISO creation tool, Cisco released mini ISO. One can also move from one GISO to another within the same release. And, we are integrating GISO with CSM.

Where to find more info:

Cisco ASR 9000 Series Aggregation Services Router Flexible Packaging Configuration Guide

377 Views
0 Comments

Introduction

  • Virtual Machines (VMs) are running on all RP’s and LC’s
  • 1x SysAdmin VM and 1x XR VM (Default-SDR) per node
  • From the Active XR VM, connect to the SysAdmin VM by typing ‘admin’
  • For troubleshooting purposes it is possible to ‘attach’ to remote VM’s
  • From SysAdmin VM: attach location x/y
  • From XR VM: attach location x/y/cpu0

 

How does it look on a node

  • Sysadmin (also known as Calvados) offers admin plane separation from XR for better fault isolation
  • Linux Kernel for scale and performance
  • Better manageability – CLI/XML/SNMP/Netconf

Show platform in XR

RP/0/RP0/CPU0:eXR-LER1#show platform

Node              Type                       State             Config state

--------------------------------------------------------------------------------

0/RP0/CPU0        A99-RP2-SE(Active)         IOS XR RUN        NSHUT

0/RP1/CPU0        A99-RP2-SE(Standby)        IOS XR RUN        NSHUT

0/FT0             ASR-9922-FAN-V2            OPERATIONAL       NSHUT

0/FT1             ASR-9922-FAN-V2            OPERATIONAL       NSHUT

0/FT2             ASR-9922-FAN-V2            OPERATIONAL       NSHUT

0/FT3             ASR-9922-FAN-V2            OPERATIONAL       NSHUT

0/0/CPU0          A9K-8X100GE-SE             IOS XR RUN        NSHUT

0/11/CPU0         A99-12x100GE               IOS XR RUN        NSHUT

0/FC0             A99-SFC2                   OPERATIONAL       NSHUT

0/FC1             A99-SFC2                   OPERATIONAL       NSHUT

0/FC2             A99-SFC2                   OPERATIONAL       NSHUT

0/FC3             A99-SFC2                   OPERATIONAL       NSHUT

0/FC4             A99-SFC2                   OPERATIONAL       NSHUT

0/PT0             A9K-AC-PEM-V2              OPERATIONAL       NSHUT

0/PT1             A9K-AC-PEM-V2              OPERATIONAL       NSHUT

 

State, IOS XR RUN above for RP/LC indicates XR VM is up and IOS XR is running. For FC/FT/PT, the XR VM state is indicated as OPERATIONAL for it to be functional.

Show platform in Sysadmin

sysadmin-vm:0_RP0# show platform

Wed Nov  30 18:14:27.693 UTC

Location  Card Type               HW State      SW State      Config State 

----------------------------------------------------------------------------

0/0       A9K-8X100GE-SE          OPERATIONAL   OPERATIONAL   NSHUT        

0/11      A99-12X100GE            OPERATIONAL   OPERATIONAL   NSHUT        

0/RP0     A99-RP2-SE              OPERATIONAL   OPERATIONAL   NSHUT        

0/RP1     A99-RP2-SE              OPERATIONAL   OPERATIONAL   NSHUT        

0/FC0     A99-SFC2                OPERATIONAL   N/A           NSHUT        

0/FC1     A99-SFC2                OPERATIONAL   N/A           NSHUT        

0/FC2     A99-SFC2                OPERATIONAL   N/A           NSHUT        

0/FC3     A99-SFC2                OPERATIONAL   N/A           NSHUT        

0/FC4     A99-SFC2                OPERATIONAL   N/A           NSHUT        

0/FT0     ASR-9922-FAN-V2         OPERATIONAL   N/A           NSHUT        

0/FT1     ASR-9922-FAN-V2         OPERATIONAL   N/A           NSHUT        

0/FT2     ASR-9922-FAN-V2         OPERATIONAL   N/A           NSHUT        

0/FT3     ASR-9922-FAN-V2         OPERATIONAL   N/A           NSHUT        

0/PT0     A9K-AC-PEM-V2           OPERATIONAL   N/A           NSHUT        

0/PT1     A9K-AC-PEM-V2           OPERATIONAL   N/A           NSHUT        

 

HW state (OPERATIONAL) above indicates powered up state and SW state (OPERATIONAL) shows Sysadmin VM to be up and running.  

 

Managing VM reload and shutdown across nodes

Reload Type

VM Admin/XR

Test case

CLI

HW Reload

admin

Chassis reload

sysadmin-vm:0_RP0# hw-module location all reload

power up/down node

RP/LC/FC reload

sysadmin-vm:0_RP0# hw-module location x/y reload

Rack Reload

sysadmin-vm:0_RP0# reload rack 0

SW Reload

admin

LC

reload admin location 0/0 or reload location 0/0 admin

restart sysadmin VM

RP

reload admin location 0/RP0 or  reload location 0/RP0 admin                                     

shutdown

exec mode

hw-module location all shutdown

hw-module location 0/0 shutdown 

hw-module location 0/RP0 shutdown

hw-module location 0/FC0 shutdown 

To recover: reload LC/RP/RSP/FC

admin config mode

hw-module shutdown location 0/0

hw-module shutdown location 0/RP0

hw-module shutdown location 0/FC0

To recover:

no hw-module shutdown location 0/0

no hw-module shutdown location 0/RP0

no hw-module shutdown location 0/FC0

SW Reload

restart XR VM

XR

 

 

LC

reload location 0/0/CPU0 

RP

reload location 0/RP0/CPU0 

All nodes

reload location all

RP switchover

redundancy switchover 

 

709 Views
4 Comments

With release 6.1.2 of IOS-XR, 64-bit eXR is now available on CCO. With a ton of drivers and forward-looking features and functionality, eXR is the next generation of IOS-XR that runs on virtualized environment with underlying 64 bit Linux kernel.

The key capabilities of IOS XR 64-bit OS includes:
• Telemetry — push towards smarter visibility of the network by streaming data to a configured receiver
  for analysis and troubleshooting purposes
• Application Hosting — leverage hosting of third-party applications in container environment
• Data Models — automate configurations that belong to multiple routers across the network
• Flexible Packaging — easy routine upgrades and maintenance with modularized Redhat Packet    Manager (RPM) packages

Migration to eXR is achieved easily and the CCO documentation for Migration from 32-bit QNX kernel XR to 64-bit Linux kernel XR is now available.

https://www.cisco.com/c/en/us/td/docs/routers/asr9000/migration/guide/b-migration-to-ios-xr-64-bit.html

Happy Migrations...

18 Views
0 Comments

show congestion-control statistics { gtpcmgr | a11mgr | hamgr | l2tpmgr }
show resources session
show subscriber configuration username subscriber-name
show subscribers aaa-configuration username subscriber_name 
show subscribers all
show session counters pcf-summary 
show session progress
show session progress pcf all 
show session subsystem facility aaamgr all
show session subsystem facility famgr all
show session subsystem facility a11mgr all
show session subsystem facility l2tpdemux all
show session subsystem facility l2tpmgr all
show session subsystem facility sessmgr all
show session disconnect-reasons
show session recovery status [verbose] 

PPP

show ppp
show ppp username subscriber-name
show ppp counters username subscriber-name
show ppp full username subscriber-name
show ppp statistics pdsn-service 
show ppp statistics pdsn-service service_name 
show rp
show rp full username subscriber_name 
show rp statistics pdsn-service 
show rp statistics pdsn-service service_name 
show mipfa full username subscriber_name 
show mipfa statistics fa-service service_name 
show mipfa counters 

Radius

show radius authentication servers radius group group_name detail 
show radius accounting servers radius group group_name detail 
show radius counters all 
show radius counters summary 

L2TP

show l2tp sessions 
show l2tp session full username subscriber_name 
show l2tp statistics lac-service service_name 
show l2tp tunnels all
show l2tp full s
show crypto ipsec security-associations statistics 

Security 

show crypto isakmp keys 
show crypto statistics 

29 Views
0 Comments

Commands Reference
context <egress ctx>
show ip pool
show ip pool wide
show ip pool groups See group stats
show ip pool group-name <group name>
show subscribers data-rate high ip-pool <ip poolname> See if there's a problem with one way traffic on a specific ip pool
show task resources facility vpnmgr all ip pool runs under vpnmgr task
show ip pool summary wide
show ip pool pool-name <pool name>
show subscribers summary ip-pool <pool name>
show subscribers summary ip-pool <pool name> idle-time > 1800
show ip pool public
show ip pool static
show ip pool private

35 Views
0 Comments

show snmp trap history verbose
show l2tp statistics pdsnclosedrp-service CRP
show l2tp statistics lac-service XXXX
show l2tp statistics peer-address <ip address of peer>
show l2tp tunnels all
clear l2tp statistics
show l2tp full stat
logging filter active facility l2tp-control level warning
logging active
no logging active 
show session subsystem facility l2tpdemux all
show session subsystem facility l2tpmgr all
show congestion-control statistics { gtpcmgr | a11mgr | hamgr | l2tpmgr }

39 Views
0 Comments

Context {UPSin | XGWin}
show crypto map summary
show crypto ipsec security-associations summary
show crypto managers summary
show crypto managers detail
show task resources facility ipsecmgr all
show crypto statistics
show crypto statistics ikev2
show sub summary
show session recovery status verbose
show npu utilization info card <x> verbose
show npu stats debug all-pacs
show crypto ikev1 security-associations summary
show crypto group summary
crypto-group <group name> activate[primary / secondary]

30 Views
0 Comments

show context
show task resources facility vpnmgr all
context {PGWin/HSGWin/XGWin}  , go inside the context where DNS is defined
show dns-client cache client  {PGW-DNS or HSGW-DNS} 
clear dns-client cache cache
show dns-client stat client {PGW-DNS OR HSGW-DNS}
dns query client-name {{PGW-DNS or HSGW-DNS }query-type AAAA  query-name {query name}
show config context PGWin / XGWin/HSGWin
show config context PGWin / XGWin / HSGWin verbose
mon protocol {70 - DNS Client}

31 Views
0 Comments

Command Reference
show active-charging service all
show active-charging analyzer statistics
show active-charging flows summary
show active-charging session summary
show active-charging session full username <username>
show active-charging firewall statistics username <username>
show active-charging firewall statistics callid <callid> verbose
show active-charging file-space-usage  Usage should be under its limit
show cdr file-space-usage Should see no “File rotation failures” or “Files/Records deleted”
show active-charging nat statistics
show active-charging flows summary ip-address <node ip> Check a certain server or node for traffic flow (TCP connections). Output may have a cli output bug (shows flows for the whole chassis - use below cmd).
show active-charging flows ip-address X.X.X.X  | grep Total Check a certain server or node for traffic flow (TCP connections)
show cdr statistics | grep -i -E "deleted|fail" Check for any file deleted or failures

48 Views
0 Comments

show snmp trap history verbose| grep -i AAA
show radius counters all
monitor subscriber
show sub full
show radius clients status
show srp mon all
show radius (auth | account) server detail
ThreshAAAAuthFail
ThreshClearAAAAuthFail
AAAAuthSrvReachable/Unreachable
SRPAAAUnreachable/Reachable
show mipfa stat
show mipha stat
show ppp stat
ping
radius test (Auth | account)
show task resources facility aaamgr
show session subsystem facility aaamgr
show npu flow record min-flowid <aaa min flowid> max-flowid <aaa max flowid>
show radius info instance
show sub aaa-config
show session disconnect-reasons
logging filter runtime facility <aaamgr | aaa-client | radius-auth | radius-acct> level <warning | unusual | info | trace | debug>
radius test probe authentication server X.X.X.X port yyy username test password test

PS: Multiple iterations of above commands should suffice the troubleshooting process. Also "logging facility" logs should be taken with caution on the production logs as this could negative effect on CPU load and should be activated on the basis of cisco support advice only ( preferably in the low traffic time )

    

     

153 Views
0 Comments

Diameter   

Commands 

Reference

General Proxy/Non-Proxy 

Show snmp trap history verbose  | grep -i  diam
show logs | grep -i diam
show diameter peers full all   | grep -i -E "peers|Total"
show diameter peers full all | grep -i -E "Host|State|peers|Total" Show the TCP state for all peers with hostname
show diameter statistics  
show diameter statistics debug-info     
show diameter endpoints all   Show all diameter peers
show diameter peers all                            
show diameter peers full all    
show diameter peers full debug 
show diameter statistics proxy  
show diameter statistics proxy debug-info    
show diameter message-queue counters outbound
show diameter message-queue counters inbound
show diameter endpoints all  
show diameter dynamic-dictionary all contents  
show diameter endpoints all debug-info 

Routing 

show diameter route status
show diameter route status full
show diameter route table wide
show diameter route table debug-info    
show diameter route table wide debug-info

Accounting and Authentication

show diameter peers full endpoint PGW-S6B          Show just PGW-S6B peers
show diameter peers full endpoint PGW-S6B | grep -i -E "Host|State|peers|Total" PGW-S6B only
show diameter aaa-statistics  
show diameter aaa-statistics all
show diameter authentication servers  
show diameter accounting servers  
show diameter aaa-statistics debug-info    
show diameter accounting servers  debubg-info    
show diameter authentication servers debug-info    

Gx: Policy and Charging Control Application

show diameter peers full endpoint PGW-GX           Show just PGW-GX peers
show diameter peers full endpoint PGW-GX | grep -i -E "Host|State|peers|Total" PGW-GX only
show ims-authorization sessions {all | apn | callid | full | ggsn-only | summary | ims-auth-service | imsi | ip-address}
show ims-authorization servers ims-auth-service <svc-name>
show ims-authorization service [all {verbose} | name <svc-name>| summary | statistics {all {verbose}| verbose | name <svc-name> {verbose}}]
show ims-authorization service statistics   
show ims-authorization policy-control statistics { debug-info | server [ip-address <ipv4-addr> {port <port-value>} | name <server-name>] | service <svc-name> {debug-info}}
show ims-authorization policy-control statistics 
show session disconnect-reasons verbose | grep ims-auth
logging filter active facility ims-authorizatn (case by case basis)
loggin active 

◦Gy: Credit control Application

show diameter peers full endpoint HA-GY          Show just HA-GY peers
show diameter peers full endpoint PGW-GY         Show just PGW-GY peers
show active-charging credit-control statistics { group <> | {server name <> | ip-address <> } }
show active-charging credit-control statistics { group <> | {server name | ip-address } <> } debug-info
show active-charging sessions full { all | msid <> | imsi <> }
show active-charging credit-control session-states
show active-charging service name <service-name>
show session disconnect-reasons | grep failed-auth-with-charging-svc
show diameter route table endpoint HA-GY | grep ^S   Command will display the static peers for HA/3G
show diameter route table endpoint HA-GY | grep ^D   Command will display the dynamic peers HA/3G
show diameter peers full endpoint PGW-GY         Show just PGW-GY peers
show diameter route table endpoint PGW-GY | grep ^S  command will display the static peers for PGW/4G
show diameter route table endpoint PGW-GY | grep ^D command will display the dynamic peers for PGW/4G
show diameter peers full endpoint HA-GY | grep -i -E "Host|State|peers|Total" HA-GY only
show diameter peers full endpoint PGW-GY | grep -i -E "Host|State|peers|Total" PGW-GY only

879 Views
15 Comments

The Cisco CLI Analyzer is a smart SSH and telnet client with internal TAC tools and knowledge integrated.

Version 2.0.0 released on March 17th 2016 now supports IOS-XR.

You can download it from:

https://cway.cisco.com/go/sa/

There are 2 main features for IOS-XR:

  • Contextual Help & Highlighting (CHH). Some show command outputs are highlighted and can be clicked to get more information about that counter. For instance for IOS-XR:  process names in the "show processes" commands, some counters in "show interface" (MTU, input drops...),  NP counters on the ASR9000,  PSE statistics counters on the CRS and NCS6000, DDTS id in "show install" commands (to get the headline of the DDTS when a SMU is installed) etc.
  • Tools. System Diagnostics is the only tool currently available for IOS-XR. It parses the output of some show commands against the Cisco TAC knowledge and reports alerts and recommendations when some known issues are detected. Please note that for IOS-XR, it will take a few minutes to collect all commands, especially on systems with a large scale.

Here is a screenshot from the tool:

In the top left pane, you can see the "show process blocked" command highlighted by the CHH with one process being clicked to get the description of what the process does.

The top right pane appears when clicking on Tools and we can see the "System Diagnostics" tool being listed.

After clicking on "System Diagnostics", alerts are displayed in the bottom pane and they can be expanded to display more information.

Please provide feedback on the tool under this page or by clicking on the feedback button in the top right corner of the tool. This is a Beta tool so we're looking to improve it in future releases. Any counter that you'd like to see documented ? Any idea for a new tool ? Thank You for helping us improve this tool.

83 Views
0 Comments

folks! with the intese focus on XR usability and simplified operations, over the last year and a half when it was kicked off many improvements have been made already. You may have seen the (already dated) blog entry here Usability functionality read out status so far.

Following up to this I thought it was time to have a better structure around collecting requirements from you and to see what others request, and better rank them as to what is a user priority to address.

We'll have special development exercises and time during which such high ranked requirements can be implemented.

To get started, log on to CISCO WISH which is specially set up for our purpose to get all the ideas in and rank (like) them so we from cisco dev can set the right priorities what to implement!

cheers!!

xander

731 Views
3 Comments

The following is a recommendation for running 5.3.1 with tomahawk linecards.

The recommended set supersedes a number of posted or in the process of posting smu's. Since they are superseded by this set, there is no need to run them and to simplify the smu's for 531, have been removed.


Recommended set:
asr9k-px-5.3.1.CSCuw65671.pie and txt
asr9k-px-5.3.1.CSCux07434.pie and txt
asr9k-px-5.3.1.CSCuw94704.pie and txt
asr9k-px-5.3.1.CSCux07751.pie and txt
asr9k-px-5.3.1.CSCuv61090.pie and txt
asr9k-px-5.3.1.CSCuw15926.pie and txt
 

Supersede list for: asr9k-px-5.3.1.CSCuw65671.pie is full supersede SMU for the list of  SMUs given below


asr9k-px-5.3.1.CSCuw01277 full
asr9k-px-5.3.1.CSCuw87933 full
asr9k-px-5.3.1.CSCuu64661 full
asr9k-px-5.3.1.CSCuw86192 full
asr9k-px-5.3.1.CSCuu06295 full
asr9k-px-5.3.1.CSCuv14560 full
asr9k-px-5.3.1.CSCuv27376 full
asr9k-px-5.3.1.CSCuw76864 full
asr9k-px-5.3.1.CSCuu01343 full
asr9k-px-5.3.1.CSCuv09734 full
asr9k-px-5.3.1.CSCuw49153 full
asr9k-px-5.3.1.CSCur86301 full
asr9k-px-5.3.1.CSCuw98906 full
asr9k-px-5.3.1.CSCuu20483 full
 

Supersede list for: asr9k-px-5.3.1.CSCux07434.pie is full supersede SMU for the list of  SMUs given below


asr9k-px-5.3.1.CSCuw01521 full
asr9k-px-5.3.1.CSCuw80112 full
asr9k-px-5.3.1.CSCuw02269 full
asr9k-px-5.3.1.CSCuw37832 full
asr9k-px-5.3.1.CSCuv45353 full
asr9k-px-5.3.1.CSCut84328 full
asr9k-px-5.3.1.CSCut69754 full
asr9k-px-5.3.1.CSCuu68447 full
asr9k-px-5.3.1.CSCuu16063 full
asr9k-px-5.3.1.CSCuv98171 full
asr9k-px-5.3.1.CSCuw63554 full
asr9k-px-5.3.1.CSCut98928 full
asr9k-px-5.3.1.CSCuw32289 full
asr9k-px-5.3.1.CSCus81652 full
 

98 Views
0 Comments

Folks, wanted to give a heads up that when you have a PPC based RP in your ASR9000, that is either an RSP2 or ASR9001 then downgrading from 513 to an earlier release, it may leave the system in a troublesome state not allowing smu installation at some point.

This issue is introduced by CSCug31240.

This ddts, integrated in XR513 removed all x86 binaries to conserve disk space on the RSP2.

Now when the device is downgraded to a prior release, e.g. 434, this image does contain the x86 binaries. Although this code is not used on the PPC RP, it does cause API version conflicts inside the system due to that downgrade.

The recovery for this is a turboboot on the desired release <513 if it is running 513+.

We are looking at workaround solutions so that if you have to downgrade from 513, that this will go a little smoother without a turboboot. These options are all investigated.

For the moment, just the warning to be aware of this situation.

This pertains to RSP2 and ASR9001 only. Trident, Typhoon, Tomahawk and RSP440/880 are not affected by this.

cheers

xander

176 Views
0 Comments

The release dates for XR 5.3.3 which is the new EMR (Extended Maintenance Release) for the XR53 train has changed from end of december to the second half of January.

The reason for the change is to get more testing done and integrate more fixes.

We wanted to inform you of that change ahead of time for planning purposes. Hopefully this change doesn't affect your deployment schedule.

xander

251 Views
0 Comments

Hi Folks, as per earlier ask it was mentioned if we could give a read out on the current usability improvements made so far.

This is the list that we have been implementing. There are many more to come in future releases like 533 and forward in XR6 also we'll continue to integrate helpful things.

Note that the "first release" where it says 533 and 60/61 etc is target for the moment.

 

Feature nameDDTS idFirst shipping release
MPLS TE Autoroute Destination enhancementsCSCuf2630004.03.02
BGP multipath enhancement to use igp-metric for multipath selectionCSCuh1234804.03.02
Implement - show bgp [vrf <name>] scaleCSCud4425905.01.00
Lack ability to set IP Precedence TOS or DSCP for outgoing syslog pktsCSCtg8759605.01.01
Syslog with the exact link id in LSP Failure scenariosCSCtl1220205.01.01
Pre-socket IPP setting for TelnetCSCtz2424305.01.01
Place last the "inheritance" keyboard when using "show run <submode>" commandsCSCuc7423705.01.01
Include MPLS-TE tunnel signalled name in all logging output messagesCSCue1885905.01.01
"show mpls traffic-eng" output must include signaled name in command outputCSCue1886205.01.01
Provide commit failure details as part of commit execution outputCSCue3151505.01.01
MPLS-TE auto-bandwidth syslog message output enhancementsCSCue3154405.01.01
Enable command chaining over SSH sessionCSCue3330405.01.01
FTP should close socket after keep expiry or hard idle timeoutCSCug3477705.01.01
Unable to override parameters back to default values when they have been modified via flex-cli config groupsCSCuh0713105.01.01
XML interface for NP debug countersCSCuj2637405.01.01
Need add SCP support for ASR9KCSCuj5725605.01.01
Shortcut name for bundle-ether to BECSCuh0452605.01.03
RSVP-TE logging enhancementsCSCuh0770005.01.03
Add support for TE tunnel interfaces to "monitor interface" commandCSCuj6694705.01.03
DNS resolve - resolve IP addresses to Host names on "show mpls traffic-eng" commandCSCuj6904205.01.03
"show inventory" command truncates plugable optic serial numberCSCum1253305.01.03
SCP doesn't show a progress barCSCum4886605.01.03
XML TE attribute brokenCSCun9329605.01.03
TCP slow transferCSCuo2588705.01.03
IPv6 ND should print Interface name when Duplicate Address is configuredCSCuo5702905.01.03
Lower the severity from critical to warning in PFM for non-cisco SFPCSCuo7103405.01.03
TE tunnel display using interface's descriptionCSCuo9794905.01.03
Auto-backup TE tunnel signalled name configuration knobCSCup1633005.01.03
Decode RRO Flags into Strings in show mpls traf tunnelsCSCup5320605.01.03
XML for all transit LSPs transiting a specific interface onlyCSCup7230505.01.03
SNMP MPLS TE: Need to populate auto-bw snmp data for tunnel indexCSCuq1256705.01.03
Display of configuration differences as part of pre-commit commandCSCub9665305.02.00
Prevent destructive edits of route-policyCSCul9662805.02.00
Command equivalent to shell utility of GNU/linux grep -ACSCsy5207005.02.02
Add ACL/selective match criteria to LPTS configuration profilesCSCui2889705.02.02
sh config failed should highlight the config causing semantic error.CSCuo3039605.02.02
XML query to fetch a summary of transit LSP'sCSCup7232305.02.02
BGP Per-Neighbor Prefix Advertisement CounterCSCup7363605.02.04
Enhance output of "show mpls traffic-eng topology srlg"CSCup7877905.02.04
Ingress netflow shows TE tunnel instead of physical/bundlle egress interfaceCSCuq4159805.02.04
Need ability to throttle FRR Ready MessageCSCus2899405.02.04
feature to suppress FRR-Ready syslogCSCut2129205.02.04
RESMON enhancments for NG-XR 5.2.5CSCue4108105.02.05
egrep -A not working on post 5.2.2,5.2.3, and 5.3.0CSCut4856705.02.05
util egrep command count options does not workCSCuu9709405.02.05
commit show-error should remain in config submode after failureCSCur0068905.02.21
Need to remain in config submode for show commit changes diffCSCur0780605.02.21
EXEC cmd to modify auto-bw bw request immediately w/specific bw amountCSCul0095005.03.00
Set rsvp reserved bandwidth flood thresholds w/ single percentage valueCSCuo0180405.03.00
Enable ISIS announcement of maximum link metric for all NLRIsCSCuo1057105.03.00
"show configuration failed load" should point out where the syntax error isCSCup5570505.03.00
Need a command to show all interfaces configured for Netflow in asr9kCSCtc8272805.03.01
Replace pattern in config terminalCSCte8134505.03.01
XR: Remove "license <type> location all" CLI for slot based licensesCSCty1776605.03.01
To support tech-support for ntpCSCty7882105.03.01
BGP session flaps when ignore-connected-check is configured / unconfiguredCSCub6168205.03.01
Show tech support should be made available for RMCSCub8136205.03.01
SECURITY-login-6-AUTHEN_SUCCESS needs upper case login log standardsCSCub9667105.03.01
Support of predefined "nd-na" and "nd-ns" keywords in XR IPv6 ICMP ACLCSCue1900705.03.01
Non-interactive EXEC commandsCSCue3327405.03.01
Add xml support for "show controllers tengige/gige/hundredGigE phy"CSCue4677405.03.01
IOS-XR - location keyword should be removed from install operationsCSCuh8717705.03.01
XML support for "show hw-module fpd location all"CSCuj1555305.03.01
show qos for bundle should report info for all membersCSCul8162205.03.01
ASR9K Fabric VOQ Serviceability CLI improvementsCSCun0821805.03.01
No support for wildcards for SCPCSCun3987905.03.01
"sh controllers TenGigE * phy" needs interface name in the outputCSCuo0175005.03.01
Support for error counters in "monitor interface Bundle-Ether *"CSCuo1566405.03.01
Alias repeat=count and df-bit=donotfrag in ping commandCSCuq1044005.03.01
Need confirmation for copy configuration to runCSCus0551505.03.01
Rollback needs to load the config and then we should commitCSCus0826105.03.01
Feature request to Admin shut down ospf via protocol shutdown CLICSCus1229305.03.01
incorrect behavior for few commands when non-interactive is enabledCSCut2760205.03.01
BGP Graceful ShutdownCSCsl0848405.03.02
Support Interface Range command for configurationCSCsm7699005.03.02
Not able to modify RPL and delete prefix-set in a single commitCSCti5022705.03.02
TE-MIB slow when polling to nonexistent LSPsCSCtz2222405.03.02
Selective configuration replace without use of no commandsCSCue3158405.03.02
Syslog infrastructure enhancement for XR - Configurable UDP portCSCug0822605.03.02
Apply-groups priority inheritenceCSCui1581305.03.02
XML support for "show processes memory"CSCui8293305.03.02
LPTS XML supportCSCuj1073705.03.02
Disk/Storage Utilization through XMLCSCuj1074705.03.02
Last link flapped time in "show interface" for CLI and XMLCSCul2025105.03.02
TCP Establishment DSCP/Prec markeing for NTPCSCul5230305.03.02
Logging discriminators - Ability to create different user-defined log files based on keywordsCSCuo4910705.03.02
Increase last 10 preemption events limit in “ show mpls traffic preemption log” to 2000CSCur2215605.03.02
Misleading timeout message at bootup of ASR9922 with dual RPs in 5.1.3CSCur7439505.03.02
Need shortcut name for pseudowire ether to PECSCur9117905.03.02
Provide option to limit ltrace shmem usage with scaling optionCSCus3918805.03.02
OSPF: show command option to display default/non-default vrfCSCus8775805.03.02
Configured VRF names missing from help output in certain ping / traceroute commandsCSCus8823905.03.02
VRF aware IP-MIB Implementation for IOS-XRCSCus8955205.03.02
SNMP should allow configuring the same community for v4 and v6 with ACLCSCus8970905.03.02
Initiate switchover upon wdsysmon detecting minor/severe/critical  levelCSCut3053705.03.02
Eliminate warning present in output of "show hw-module fpd"CSCut5903405.03.02
Flexible location of "vrf" keyword in CLI PING / TRACEROUTE commands.CSCut6789405.03.02
entSensorMeasuredEntity support on asr9kCSCut8117205.03.02
Configured VRF names missing from help output in SHOW IPV4/IPV6 VRF cmdsCSCut8122305.03.02
SVS FB 513: Netconf interface fails to startCSCut8400305.03.02
FPD auto-upgrade on newly inserted linecardsCSCut9570805.03.02
SSTE:  Enhance Netconf debugabilityCSCuu2644105.03.02
provide show command to display list of wdsysmon OOR aware processesCSCuu3710105.03.02
ICMP Time Exceeded Msg Not Generated Correctly per RFC4884 and RFC1812CSCuu6860705.03.02
Umbrella DDTS for interface-range issuesCSCuu8560005.03.02
ICMP Time Exc Msg Not Generated Correctly per RFC4884 and RFC1812 - IPv6CSCuv0443505.03.02
Improper inherited config after deletion and addition of flex-cli groupCSCuv5572105.03.02
need standard task group with only read permissions for all tasksCSCuj9748005.03.03
Syslog Logging local-file destination with discriminatorsCSCuu2523105.03.03
Support only force downgrade and Modify UP_REV syslog, show cli for VKG_CARD_TYPE_LC_AVSMCSCuu8829005.03.03
Configuration rollback to specific Commit ID in EXEC modeCSCuv3938905.03.03
New option for Configuration RollbackCSCuv3944805.03.03

 

1471 Views
0 Comments

MPLS VPN Services are one of the most common services provided by Service Providers. Because of its wide and huge deployments, there is a need to have deep troubleshooting knowledge to fix the issues that occur in MPLS VPN deployment.

Today, we shall see how to troubleshoot Basic Layer-3 MPLS VPN Services. For this, lets consider the below topology:

 

Configuration

Config on PE1:
==============
ip vrf ABC
 rd 1:1
 route-target export 1:1
 route-target import 1:1
!
interface Loopback0
 ip address 1.1.1.1 255.255.255.255
!
interface Ethernet0/0
 ip vrf forwarding ABC
 ip address 192.168.10.1 255.255.255.252
! 
interface Ethernet1/0
 ip address 12.12.12.1 255.255.255.252
 ip ospf network point-to-point
 mpls ip
!
router ospf 100
 router-id 1.1.1.1
 network 1.1.1.1 0.0.0.0 area 0
 network 12.12.12.1 0.0.0.0 area 0
!
router bgp 100
 bgp router-id 1.1.1.1
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 3.3.3.3 remote-as 100
 neighbor 3.3.3.3 update-source Loopback0
 !        
 address-family ipv4
 exit-address-family
 !        
 address-family vpnv4
  neighbor 3.3.3.3 activate
  neighbor 3.3.3.3 send-community both
 exit-address-family
 !        
 address-family ipv4 vrf ABC
  redistribute connected
  neighbor 192.168.10.2 remote-as 65535
  neighbor 192.168.10.2 activate
 exit-address-family
!

Config on P-1:
==============
interface Loopback0
 ip address 2.2.2.2 255.255.255.255
!
interface Ethernet0/0
 ip address 23.23.23.1 255.255.255.252
 ip ospf network point-to-point
 mpls ip
!
interface Ethernet1/0
 ip address 12.12.12.2 255.255.255.252
 ip ospf network point-to-point
 mpls ip
!
router ospf 100
 router-id 2.2.2.2
 network 2.2.2.2 0.0.0.0 area 0
 network 12.12.12.2 0.0.0.0 area 0
 network 23.23.23.1 0.0.0.0 area 0
! 

Config PE2:
===========
ip vrf ABC
 rd 100:1
 route-target export 1:1
 route-target import 1:1
!
interface Loopback0
 ip address 3.3.3.3 255.255.255.255
!
interface Ethernet0/0
 ip address 23.23.23.2 255.255.255.252
 ip ospf network point-to-point
 mpls ip
!
interface Ethernet1/0
 ip vrf forwarding ABC
 ip address 192.168.20.1 255.255.255.252
!
router ospf 100
 router-id 3.3.3.3
 network 3.3.3.3 0.0.0.0 area 0
 network 23.23.23.2 0.0.0.0 area 0
!
router bgp 100
 bgp router-id 3.3.3.3
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 1.1.1.1 remote-as 100
 neighbor 1.1.1.1 update-source Loopback0
 !        
 address-family ipv4
 exit-address-family
 !        
 address-family vpnv4
  neighbor 1.1.1.1 activate
  neighbor 1.1.1.1 send-community both
 exit-address-family
 !        
 address-family ipv4 vrf ABC
  redistribute connected
  neighbor 192.168.20.2 remote-as 65000
  neighbor 192.168.20.2 activate
 exit-address-family
!

Config on CE1:
=============​
interface Loopback0
 ip address 100.1.1.1 255.255.255.255
!
interface Ethernet0/0
 ip address 192.168.10.2 255.255.255.252
!
router bgp 65535
 bgp router-id 100.1.1.1
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 192.168.10.1 remote-as 100
 !
 address-family ipv4
  network 100.1.1.1 mask 255.255.255.255
  neighbor 192.168.10.1 activate
 exit-address-family
!

Config on CE2:
==============
interface Loopback0
 ip address 100.2.2.1 255.255.255.255
!
interface Ethernet1/0
 ip address 192.168.20.2 255.255.255.252
!
router bgp 65000
 bgp router-id 100.2.2.1
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor 192.168.20.1 remote-as 100
 !
 address-family ipv4
  network 100.2.2.1 mask 255.255.255.255
  neighbor 192.168.20.1 activate
 exit-address-family
!

 

Verification

Output on CE1:
==============
CE1#show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

      100.0.0.0/32 is subnetted, 2 subnets
C        100.1.1.1 is directly connected, Loopback0
B        100.2.2.1 [20/0] via 192.168.10.1, 00:42:16
      192.168.10.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.10.0/30 is directly connected, Ethernet0/0
L        192.168.10.2/32 is directly connected, Ethernet0/0
      192.168.20.0/30 is subnetted, 1 subnets
B        192.168.20.0 [20/0] via 192.168.10.1, 00:23:50

CE1#ping 100.2.2.1 source lo0
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 100.2.2.1, timeout is 2 seconds:
Packet sent with a source address of 100.1.1.1 
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
CE1#

The above output shows that the routing table on CE1 has the routes from remote CE router CE2. Also, we can see that there is end to end reachability between CE1 and CE2 router loopbacks. Further looking at the output on PE1 (below), the VRF routing table of PE1 (VRF ABC) is populated with necessary prefixes.

PE1#show ip route vrf ABC

Routing Table: ABC
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

      100.0.0.0/32 is subnetted, 2 subnets
B        100.1.1.1 [20/0] via 192.168.10.2, 01:06:49
B        100.2.2.1 [200/0] via 3.3.3.3, 00:44:36
      192.168.10.0/24 is variably subnetted, 2 subnets, 2 masks
C        192.168.10.0/30 is directly connected, Ethernet0/0
L        192.168.10.1/32 is directly connected, Ethernet0/0
      192.168.20.0/30 is subnetted, 1 subnets
B        192.168.20.0 [200/0] via 3.3.3.3, 00:26:15
PE1#

 

Troubleshooting

Step1: The first step in troubleshooting MPLS VPN setup is to verify the LSP path between PE to PE. This can be done using two methods

1. MPLS PING

If the MPLS ping goes through from PE to PE loopback, then it would confirm that the LSP (Label Switched Path) is complete and there is no problem with it.

PE1# ping mpls ipv4 3.3.3.3 255.255.255.255        
Sending 5, 72-byte MPLS Echos to 3.3.3.3/32, 
     timeout is 2 seconds, send interval is 0 msec:

Codes: '!' - success, 'Q' - request not sent, '.' - timeout,
  'L' - labeled output interface, 'B' - unlabeled output interface, 
  'D' - DS Map mismatch, 'F' - no FEC mapping, 'f' - FEC mismatch,
  'M' - malformed request, 'm' - unsupported tlvs, 'N' - no label entry, 
  'P' - no rx intf label prot, 'p' - premature termination of LSP, 
  'R' - transit router, 'I' - unknown upstream index,
  'l' - Label switched with FEC change, 'd' - see DDMAP for return code,
  'X' - unknown return code, 'x' - return code 0

Type escape sequence to abort.
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
 Total Time Elapsed 3 ms

PE1#

2. PE to PE VRF PING

You can also perform a PE to PE to the VRF's IP Address:

PE1# ping vrf ABC 192.168.20.1 source 192.168.10.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.20.1, timeout is 2 seconds:
Packet sent with a source address of 192.168.10.1 
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/1 ms
PE1#

This would again confirm the same thing that the LSP is complete in the Service Provider Core.

Step2: Verifying the VPN and IGP label along the path. For this, the first step is check the VPN label. 

1. Verifying VPN Label:

PE1#show mpls forwarding vrf ABC
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
19         No Label   100.1.1.1/32[V]  2136          Et0/0      192.168.10.2
20         No Label   192.168.10.0/30[V]   \
                                       570           aggregate/ABC 
PE1#

The local label value shown above i.e. 19 and 20 are nothing but the VPN labels. show mpls forwarding vrf command helps with checking the VPN label. There is another way to check the VPN labels and that is using BGP. Since BGP allocates the VPN label, you can verify the VPN labels for the prefixes (local as well as remote).

PE1# show ip bgp vpnv4 all 100.1.1.1
BGP routing table entry for 1:1:100.1.1.1/32, version 2
Paths: (1 available, best #1, table ABC)
  Advertised to update-groups:
     2         
  Refresh Epoch 1
  65535
    192.168.10.2 (via vrf ABC) from 192.168.10.2 (100.1.1.1)
      Origin IGP, metric 0, localpref 100, valid, external, best
      Extended Community: RT:1:1
      mpls labels in/out 19/nolabel
      rx pathid: 0, tx pathid: 0x0
PE1#

All the local as well as remove VPN labels can be verified using the above command. There is another easy way to look at the VPN labels:

PE1# show ip bgp vpnv4 all labels
   Network          Next Hop      In label/Out label
Route Distinguisher: 1:1 (ABC)
   100.1.1.1/32     192.168.10.2    19/nolabel
   100.2.2.1/32     3.3.3.3         nolabel/19
   192.168.10.0/30  0.0.0.0         20/nolabel(ABC)
   192.168.20.0/30  3.3.3.3         nolabel/20
Route Distinguisher: 100:1
   100.2.2.1/32     3.3.3.3         nolabel/19
   192.168.20.0/30  3.3.3.3         nolabel/20

PE1#

In the above output, the label values highlighted on the left are the local labels (In Label) where as label values highlighted in the right (Out Label) are the remote prefix VPN label. You can filter the labels based on the RD value as well using "show ip bgp vpnv4 rd <rd_value> labels".

2. Verifying IGP labels in the path

For for the traffic destined to the remote prefix learnt from remote PE - PE2, we can check the vrf Routing table and then check the IGP labels for the same along the path:

PE1# show ip route vrf ABC 100.2.2.1 

Routing Table: ABC
Routing entry for 100.2.2.1/32
  Known via "bgp 100", distance 200, metric 0
  Tag 65000, type internal
  Last update from 3.3.3.3 01:40:38 ago
  Routing Descriptor Blocks:
  * 3.3.3.3 (default), from 3.3.3.3, 01:40:38 ago
      Route metric is 0, traffic share count is 1
      AS Hops 1
      Route tag 65000
      MPLS label: 19
      MPLS Flags: MPLS Required
PE1#

PE1# show mpls forwarding 3.3.3.3
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
16         17         3.3.3.3/32       0             Et1/0      12.12.12.2  
PE1#

In above output, the next-hop for prefix learnt (100.2.2.1/32) from remote CE router CE2 is 3.3.3.3 which is the loopback IP of PE2 router. The LFIB (Label Forwarding Information Base) for 3.3.3.3 shows that outgoing label as 17. This label 17 is learnt from the P-1 router where it should be the local label.

P-1# show mpls forwarding 3.3.3.3
Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    
Label      Label      or Tunnel Id     Switched      interface              
17         Pop Label  3.3.3.3/32       476193        Et0/0      23.23.23.2  
P-1#

Since the prefix is present on PE2 router itself, it will send a Pop label (implicit-null label) to P-1 router which is seen as the outgoing label on P-1 router for the PE2 loopback.

Step 3. Verifying CEF

On Cisco platform, CEF takes care of forwarding. Thus, its important to verify that the CEF is programmed correctly.

PE1# show ip cef vrf ABC 100.2.2.1 det
100.2.2.1/32, epoch 0, flags [rib defined all labels]
  recursive via 3.3.3.3 label 19
    nexthop 12.12.12.2 Ethernet1/0 label 17
PE1#

In the cef output for the remote prefix, we can see that label 19 is the VPN label where as 17 is the IGP label and we can also see the respective next-hop value. We can also check the Service Provider core for the IGP labels and forwarding information using the above method.

Step 4. Traceroute

Traceroute can be used for checking the labelling and figure where a possibly problem could be.

PE1#traceroute vrf ABC 100.2.2.1 numeric 
Type escape sequence to abort.
Tracing the route to 100.2.2.1
VRF info: (vrf in name/id, vrf out name/id)
  1 12.12.12.2 [MPLS: Labels 17/19 Exp 0] 1 msec 1 msec 0 msec
  2 192.168.20.1 [MPLS: Label 19 Exp 0] 0 msec 0 msec 1 msec
  3 192.168.20.2 0 msec *  1 msec
PE1#

We can check the LSP path as well using the trace from PE to PE:

PE1# trace mpls ipv4 3.3.3.3 255.255.255.255
Tracing MPLS Label Switched Path to 3.3.3.3/32, timeout is 2 seconds

Codes: '!' - success, 'Q' - request not sent, '.' - timeout,
  'L' - labeled output interface, 'B' - unlabeled output interface, 
  'D' - DS Map mismatch, 'F' - no FEC mapping, 'f' - FEC mismatch,
  'M' - malformed request, 'm' - unsupported tlvs, 'N' - no label entry, 
  'P' - no rx intf label prot, 'p' - premature termination of LSP, 
  'R' - transit router, 'I' - unknown upstream index,
  'l' - Label switched with FEC change, 'd' - see DDMAP for return code,
  'X' - unknown return code, 'x' - return code 0

Type escape sequence to abort.
  0 12.12.12.1 MRU 1500 [Labels: 17 Exp: 0]
L 1 12.12.12.2 MRU 1500 [Labels: implicit-null Exp: 0] 1 ms
! 2 23.23.23.2 1 ms
PE1#

Hope the above steps helps you troubleshoot Basic MPLS VPN deployments. Even for complex MPLS VPN deployments like Inter-AS MPLS VPN or CSC (Carrier Supporting Carrier), these steps can serve useful.

254 Views
0 Comments

What are core files?

A core file is created when a program terminates unexpectedly, due to a bug, or a violation of the operating system's or hardware's protection mechanisms. The operating system kills the program and creates a core file.

 

When does it get generated on ASR5x00?

A core file gets generated on the ASR5x00 under two circumstances listed below.

  1. A process crashes generating a core.
  2. A core is forced to be generated using cli.

 

How is it useful for TAC/BU ?

After the OS kills the program/process and creates the core file, it can be used by the programmers to figure out what went wrong. It contains a detailed description of the state that the program/process when it died/crashed. This information is used by the engineering team  to re-create the issue in the local lab to find the issue and fix it or figure out the cause for the program/process to crash.

 

What is the configuration for  getting a good core file ?

In the unlikely even of a software crash, the system stores information that could be useful in determining the reason for the crash. This information can be maintained in system memory or it can be transferred and stored on a network server.

 

Crash logs record all possible information pertaining to a software crash (full core dump). Due to their size, they can not be stored in system memory. Therefore, these logs are only generated if the system is configured with a Universal Resource Locator (URL) pointing to a local device or a network server where the log can be stored.

 

Whenever a crash occurs, the following crash information is stored:

1. The event record is stored in /flash/crashlog2 file (the crash log).

2. The associated minicore, NPU or kernel dump file is stored in the /flash/crsh2 directory.

3. A full core dump is stored in a user configured directory.

Important: The crashlog2 file along with associated minicore, NPU and kernel dumps are automatically

synchronized across redundant management cards (SMC, MIO/UMIO). Full core dumps are not synchronized across

management cards.

The following behaviors apply to the crash logging process.

· When a crash event arrives on an active management card, the event record is stored in its crashlog2 file along

with the minicore, NPU, or kernel dump file in /flash/crsh2. The crash event and dump file are also

automatically stored in the same locations on the standby management card.

· When a crash log entry is deleted via CLI command, it is deleted on both the active and standby management

cards.

· When a management card is added or replaced, active and standby cards will automatically synchronize crash

logs and dump files.

· When a crash event is received and the crash log file is full, the oldest entry in the crash log and its related dump

file will be replaced with the latest arrived event and dump file on both management cards. Information for a

maximum of 120 crash events can be stored on management cards.

· Duplicate crash events bump the count of hits in the existing record and update the new record with the old crash

record. Additions to the count use the timestamp for the first time the event happened.

 

Crash log files (full core dumps) are written with unique names as they occur to the specified location. The name format

is crash-card-cpu-time-core. Where card is the card slot, cpu is the number of the CPU on the card, and time is the

Portable Operating System Interface (POSIX) timestamp in hexadecimal notation.

 

Use the following example to configure a software crash log destination in the Global Configuration mode:

configure

 crash enable [ encrypted ] url crash_url

 end

 

[local]lab# show configuration|grep -i crash

  crash enable encrypted url +A0mkhq8vovt9r839m92n24ujeve3cvcauqg6n3od1n8zaed1czuwg3jqh2ocwct8t92w1dktko63ief1z3

ib8g595xaj2ukt1u9mpscd5

[local]lab#

 

How to force a core file ?

 

A core can be forced to be generated by using the below command after getting into the hidden mode.  Please contact the Cisco Support team to get assistance to generate the core file.

 

Below is an example.

[local]lab# show crash list  

==         ====         =======  ========== =========== ================

#          Time         Process   Card/CPU/     SW         HW_SER_NUM

                                     PID      VERSION   VPO / Crash Card

==         ====         =======  ========== =========== ================

 

1  2015-May-21+12:47:54 vpnmgr   01/0/03360 15.0(51852) NA                     >>> Generated crash

 

Total Crashes : 1

 

Impact of forcing a core file ?

There is no business impact of forcing a core. By forcing a core we are getting a snapshot of the current state of the system process.

 

Caveats and solution

Core files created from a crash on an ASR5000/ASR5500 sometimes become corrupted due to the size of the core file being created.

By default the maximum size of a core file is 1024 megabytes (1 Gigabyte) in ASR5000 and 2048 MB in ASR5500.  This means that when the core file is being created in the location specified, the creation of the core file will be terminated if the core file size is larger than 1024 Megabytes and 2048 MB on the ASR5000 and ASR5500 respectively.  This can be changed with a simple hidden configuration. Please contact Cisco Support for the related configuration. This configuration basically allows the operator to increase the maximum size of the core file to 2048 MB and 4096MB respectively in the ASR5000 and ASR5500 respectively.

 

When core files are written to lets say an EMS server, upon creation, core files are automatically gzipped, but are written without the .gz extension.  On the EMS server, after a set period of time, the core files are moved to an archived directory, where the core file is gzipped again, but this time has the .gz extension appended.

 

How to check the sanity of a core file?

 

Testing of core files can be facilitated in a simple manner.  Your testing is based upon whether the core file that is supplied to you is in a .gz format, or without the .gz format. 

 

Procedure

In this instance the core file has no .gz extension, but as stated, was compressed when written to the EMS server.

 

$ ls -ltr

-rwx------+ 1 Administrators Domain Users 6967918 Mar  2 08:51 crash-02-01-53133543-core <crash-card-cpu-time-core>

 

 

1)  Copy the core to a temp directory for posterity.

 

2)  Rename the core file adding a .gz extension to the file.

 

$ mv <crash-card-cpu-time-core> <crash-card-cpu-time-core>.gz

 

 

3) gunzip (unzip) the core file.  If no errors are seen, the core file should not be corrupted, providing there is no internal corruption.

 

$ gunzip <crash-card-cpu-time-core>.gz

(no error returned)

 

-------------------------

 

Using a file that is known to be corrupted the following is seen.

 

$ ls -lt

-rwx------+ 1 Administrators Domain Users 98467840 Feb 27 12:43 <crash-card-cpu-time-core>.gz

 

$ gunzip <crash-card-cpu-time-core>.gz

gzip: <crash-card-cpu-time-core>.gz:                     unexpected end of file  <<<<<<

 

If you have an error returned when gunzipping the core file, it is corrupted.

 

-------------------------

 

If the core file has a .gz extension, you will first have to gunzip the file, rename it adding on the .gz extension again, and then gunzip it a second time for testing of corruption.

 

Core File

 

$ ls -lt

-rwx------+ 1 Administrators Domain Users 94981317 Feb 27 12:43 <crash-card-cpu-time-core>.gz

 

First gunzip:

 

$ gunzip <crash-card-cpu-time-core>.gz

 

 

Rename the file adding on the .gz extension

 

$ mv <crash-card-cpu-time-core <crash-card-cpu-time-core>.gz

 

 

View the core file

 

$ ls -ltr

-rwx------+ 1 Administrators Domain Users 98467840 Feb 27 12:43 <crash-card-cpu-time-core>.gz

 

 

Gunzip the file second time

$ gunzip <crash-card-cpu-time-core>.gz

gzip: <crash-card-cpu-time-core>.gz:              unexpected end of file  <<<<<

199 Views
0 Comments

Today, companies are recognizing that effective collaboration is critical to future business success. Many think that if they just buy collaboration technology, they will become collaborative companies. But, that's not true. Business companies need to develop their collaborative capabilities before implementing collaboration tools.

If companies want to build collaboration, they need to see beyond activity streams at getting the basics right. Here are the five principles for effective collaboration that companies should follow:

1. Focus on Achieving Business Results: Collaboration should focus on where it can have the biggest impact on achieving business results. These are those parts of the business where decisions are complex and those decisions have far-reaching results. Collaboration should not be forced into every discussion, decision and dialogue if it is not warranted to get the desired results.

2. Collaboration is a Capability: This means that it is a complementary combination of processes, systems, business rules, data flows, staff responsibilities, organizational roles, etc. When you bring the right combination of these components together, only then a company create the ability to collaborate. If you want collaboration is to increase, when you increase each of the components, increase each of the others accordingly.

3. Manage Complex Trade-offs: Collaboration is not always about moving towards one universally best solution. Still, sometimes complex trade-offs need to be made. And for that, you require empowered and informed decision makers. The collaboration provides the tools for the right decision maker to get the right information at the right time.

4. Promote High Standards for Discussion, Dialogue and Information Sharing: Staff should be encouraged to adopt the highest standards of dialogue, discussion and information sharing with other staff members. These are life skills that will help staff at home as well as at work.

5. Promote Personal Accountability: Collaboration only works when individuals within the company act collaboratively. Therefore, it should be the responsibility of each individual to support collaborative working wherever it is appropriate. In a company, if individuals act collaboratively then work teams will also act collaboratively. And if work teams act collaboratively, organizations will automatically become more collaborative and achieve better business results.

In the starting, we mentioned that buying collaboration technology won’t automatically make a company collaborative. But, this doesn't mean that technology doesn't have a role to play in effective collaboration. Technology enables discussions between people all over the world, opens dialogue among communities of interest and fosters real-time information sharing.

 

196 Views
0 Comments

Acting on your feedback we created an install cheat sheet, please bookmark and ask questions:

 

https://supportforums.cisco.com/document/12440491/ios-xr-install-upgrade-cheat-sheet

 

Regards,

Eddie.