cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
12783
Views
15
Helpful
41
Replies

Ask the Expert: Cisco Nexus 2000, 5000, and 6000 Series Switches

ciscomoderator
Community Manager
Community Manager

Read the biowith Cisco Expert Vinayak Sudame

Welcome to the Cisco Support Community Ask the Expert conversation. This is an opportunity to learn and ask questions how to configure and troubleshoot the Cisco Nexus 2000, 5000 and 6000 Series Switches with Cisco subject matter expert Vinayak Sudame. You can ask any question on configuration, troubleshooting, features, design and Fiber Channel over Ethernet (FCoE).

Vinayak Sudame is a Technical Lead in Data Center Switching Support Team within Cisco's Technical Services in RTP, North Carolina. His current responsibilities include but are not limited to Troubleshooting Technical support problems and Escalations in the areas of Nexus 5000, Nexus 2000, FCoE. Vinayak is also involved in developing technical content for Cisco Internal as well as external. eg, Nexus 5000 Troubleshooting Guide (CCO), Nexus 5000 portal (partners), etc. This involves cross team collaboration and working with multiple different teams within Cisco. Vinayak has also contributed to training account teams and partners in CAE (Customer Assurance Engineering) bootcamp dealing with Nexus 5000 technologies. In the past, Vinayak's responsibilities included supporting MDS platform (Fiber Channel Technologies) and work with EMC support on Escalated MDS cases. Vinayak was the Subject Matter Expert for Santap Technologies before moving to Nexus 5000 support. Vinayak holds a Masters in Electrical Engineering with Specialization in Networking from Wichita State University, Kansas. He also holds Cisco Certification CCIE (#20672) in Routing and Switching.

Remember to use the rating system to let Vinayak know if you have received an adequate response.

Vinayak might not be able to answer each question due to the volume expected during this event. Remember that you can continue the conversation on the  Data Center sub-community, Other Data Center Topics discussion forum shortly after the event.

This event last through Friday July 12, 2013. Visit the community often to view responses to youe questions of other community members.

41 Replies 41

Sonugnair_2
Level 1
Level 1

Hi Vnayak,

We  have two Nexus 5548 switches along with a coupe of dual homed FEXes.  All vlan, portchannel, interface configurations were done in 'conf sync'  mode.

We  continued like this and faced no problems. However recently i found out  that after a change, i tried to verify and commit, however there is an  error saying that the peer is not reachable. But i am able to ping both  the Nexus switches from each others console via the appropiate vrfs. I  noticed that on switch 1 i am able to see the cfs peers, but on switch 2  on giving the command following is the output:-

5548_02# show cfs peers

CFS Discovery is in Progress ..Please wait

Could not get response. The network topology may be under going change.

Please try after about 30 seconds

5548_02#

{{ Above output takes some time to display}}

I  am not able to find much information about this. I disabled CFSoIP and  re enabled it. Still same result. I am actually stuck here. The other  things to do would be delete the switch profile and add back again or  may be reload the switch.

Any help would be appreciated.

The NX OS is:-

BIOS:      version 3.6.0

loader:    version N/A

kickstart: version 5.2(1)N1(2a)

system:    version 5.2(1)N1(2a)

power-seq: Module 1: version v1.0

Module 3: version v2.0

On switch 1 peers are showing up as normal:-

5548_01# sh cfs peer

Physical Fabric

-------------------------------------------------------------------------

Switch WWN              IP Address

-------------------------------------------------------------------------

20:00:54:7f:ee:b7:c2:80 172.16.1.53                           [Local]

                         5548_01

20:00:54:7f:ee:b6:99:c0 172.16.1.51

20:00:54:7f:ee:b6:9c:00 172.16.1.52

20:00:54:7f:ee:b6:3f:80 172.16.1.54

Total number of entries = 4

Any feed back would be appreciated.

Thanks in advance!!.

Regards

Hello Sonu

I have few initial questions on your query.

- Is switch 2 vpc primary or secondary ?

- Are these switches configured for FC/FCoE ?

- Were these switches upgraded non-disruptively in the past ?

- How are the mgmt ports physically connected ? Are they back-to-back or connecting to other 5k/2k switch ?

- On Switch 2 are you able to perform 'show run', 'config t' and other tasks without any problems ?

Can you send output of

show cfs lock

show system internal csm info trace

show cfs internal ethernet-peer database

- 'show cfs peers' output on Switch 2 was it working correctly before this issue happened? Do you know ?

- I have seen before that sometimes it would take time for that command output to be displayed but in our case Switch 2 could be having issues syncing with Switch1.

Thanks, Vinayak

Hi Vinayak,

Thanks for the reply.

As of now i do not have access to the device, but i will answer your queries.

Is switch 2 vpc primary or secondary ?

Vpc secondary

-Are these switches configured for FC/FCoE ?

No

Were these switches upgraded non-disruptively in the past ?

Yes, the upgrade went smoothly

On Switch 2 are you able to perform 'show run', 'config t' and other tasks without any problems?

Yes

How are the mgmt ports physically connected ? Are they back-to-back or connecting to other 5k/2k switch ?

The management ports of both Nexus 5548 switches are connected to a 3750X stack switch in an appropriate vlan

Since i do not have access to switches, i am not in a position to send the outputs but from memory the "show cfs lock" shows no output i.e. no pending locks

'show cfs peers' output on Switch 2 was it working correctly before this issue happened? Do you know ?

Yes "show cfs peers" command was giving proper output before.

Everything was working properly. Ping from one switch to another to peer IP is working properly via appropriate vrf.

I will send the output of the other commands as soon as i get them.

We have about 40 live servers connected to these 5548/FEXes.

Regards.

Hi Vinayak,

Output of "show cfs internal ethernet-peer database"

Switch 1

ETH Fabric

-------------------------------------------------------------------------

Switch WWN              logical-if_index

-------------------------------------------------------------------------

20:00:54:7f:ee:b7:c2:80 [Local]

20:00:54:7f:ee:b6:3f:80 16000005

Total number of entries = 2

Switch 2

ETH Fabric

-------------------------------------------------------------------------

Switch WWN              logical-if_index

-------------------------------------------------------------------------

20:00:54:7f:ee:b6:3f:80 [Local]

20:00:54:7f:ee:b7:c2:80 16000005

Total number of entries = 2

Output of "show system internal csm info trace"

Switch 1 in which "show cfs peers" show proper output

Mon Jul  1 05:46:19.145339  (CSM_T) csm_sp_buf_cmd_tbl_expand_range(8604): No range command in buf_cmd_tbl.

Mon Jul  1 05:46:19.145280  (CSM_T) csm_set_sync_status(6257): Peer RT status PSSed

Mon Jul  1 05:46:19.145188  (CSM_T) csm_sp_handle_local_verify_commit(4291):

Mon Jul  1 05:46:19.145131  csm_continue_verify_ac[597]: peer is not reachable over CFS so continuing with local verify/commit

Mon Jul  1 05:46:19.145071  csm_tl_lock(766): Peer information not found for IP address: '172.16.1.54'

Mon Jul  1 05:46:19.145011  csm_tl_lock(737):

Mon Jul  1 05:46:19.144955  (CSM_EV) csm_sp_build_tl_lock_req_n_send(941): sending lock-request for CONF_SYNC_TL_SESSION_TYPE_VERIFY subtype 0 to Peer ip = (172.16.1.54)

Mon Jul  1 05:46:19.143819  (CSM_T) csm_copy_image_and_internal_versions(788): sw_img_ver: 5.2(1)N1(2a), int_rev: 1

Mon Jul  1 05:46:19.143761  (CSM_T) csm_sp_get_peer_sync_rev(329): found the peer with address=172.16.1.54 and sync_rev=78

Mon Jul  1 05:46:19.143699  (CSM_T) csm_sp_get_peer_sync_rev(315):

Mon Jul  1 05:46:19.143641  (CSM_EV) csm_sp_build_tl_lock_req_n_send(838): Entered fn

Mon Jul  1 05:46:19.143582  (CSM_T) csm_set_sync_status(6257): Peer RT status PSSed

Switch 2 in which "show cfs peers" does not show proper output

Mon Jul  1 06:13:11.885354  (CSM_ERR) csm_pss_cmd_tree_walk_cb(2057): Parent command not found for cmd switchport mode trunk, cmd pseq 77 seq 482

Mon Jul  1 06:13:11.884992  (CSM_ERR) csm_pss_cmd_tree_walk_cb(2057): Parent command not found for cmd channel-group 51 mode active, cmd pseq 357 seq 369

Mon Jul  1 06:13:11.884932  (CSM_ERR) csm_pss_cmd_tree_walk_cb(2057): Parent command not found for cmd switchport trunk allowed vlan 2, 11, cmd pseq 357 seq 368

Mon Jul  1 06:13:11.884872  (CSM_ERR) csm_pss_cmd_tree_walk_cb(2057): Parent command not found for cmd switchport mode trunk, cmd pseq 357 seq 367

Mon Jul  1 06:13:11.884811  (CSM_ERR) csm_pss_cmd_tree_walk_cb(2057): Parent command not found for cmd description process_vpc, cmd pseq 357 seq 366

Mon Jul  1 06:13:11.884750  (CSM_ERR) csm_pss_cmd_tree_walk_cb(2057): Parent command not found for cmd channel-group 51 mode active, cmd pseq 352 seq 365

Mon Jul  1 06:13:11.884690  (CSM_ERR) csm_pss_cmd_tree_walk_cb(2057): Parent command not found for cmd switchport trunk allowed vlan 2, 11, cmd pseq 352 seq 364

Mon Jul  1 06:13:11.884630  (CSM_ERR) csm_pss_cmd_tree_walk_cb(2057): Parent command not found for cmd switchport mode trunk, cmd pseq 352 seq 363

Mon Jul  1 06:13:11.884568  (CSM_ERR) csm_pss_cmd_tree_walk_cb(2057): Parent command not found for cmd description process_vpc, cmd pseq 352 seq 362

Mon Jul  1 06:13:11.884207  (CSM_EV) csm_sp_acfg_gen_handler(3011):  Preparing config into /tmp/csm_sp_acfg_1733916569.txt

Mon Jul  1 06:13:11.878695  csm_get_locked_ssn_ctxt[539]: Lock not yet taken.

Mon Jul  1 06:13:11.878638  (CSM_EV) csm_sp_acfg_gen_handler(2937): Recieved sp acfg merge request for type: running cfg

Mon Jul  1 06:12:29.527840  (CSM_T) csm_pss_del_seq_tbl(1989): Freeing seq tbl data

Mon Jul  1 06:12:29.513255  (CSM_T) csm_sp_acfg_gen_handler(3106): Done acfg file write

Mon Jul  1 06:12:29.513179  (CSM_EV) csm_sp_acfg_gen_handler(3011):  Preparing config into /tmp/csm_sp_acfg_1733911262.txt

Mon Jul  1 06:12:29.508859  csm_get_locked_ssn_ctxt[539]: Lock not yet taken.

Mon Jul  1 06:12:29.508803  (CSM_EV) csm_sp_acfg_gen_handler(2937): Recieved sp acfg merge request for type: running cfg

Mon Jul  1 05:53:17.651236  Collecting peer info

Mon Jul  1 05:53:17.651181  Failed to get the argumentvalue for 'ip-address'

Mon Jul  1 05:40:59.262736  DB Unlocked Successfully

Mon Jul  1 05:40:59.262654  Unlocking DB, Lock Owner Details:Client:1 ID:1

Mon Jul  1 05:40:59.262570  (CSM_T) csm_sp_del_buf_cmd(1713): Deleting comand with Id = 1

Mon Jul  1 05:40:59.262513  DB Lock Successful by Client:1 ID:1

Mon Jul  1 05:40:59.262435  Recieved lock request by Client:1 ID:1

Mon Jul  1 05:40:41.741224  ssnmgr_ssn_handle_create_get: Session FSM already present, ID:1

Mon Jul  1 05:40:41.741167  ssnmgr_handle_mgmt_request: Create/Get request received for session[process_n5kprof]

show cfs lock gives no output.

Just to further clarify, we have 4 5548UP switches in the same management vlan. 2 switches are in one location lets say location A and they are CFS peers and are working fine.

These two switches which are having problem are in location B. All the switches are in the same vlan. Essentially the all CFS multicast messages will be seen by all 5548 switches as they are in the same vlan. I am assuming that this might not create any problems as we specify the peers in the respective configurations. Or do we have to change the CFSoIPv4 multicast addresses in location B or may be configure a different region.

Regards.

Thanks, will analyze this and get back to you. In the meantime can you also paste output of

'show switch-profile status'

after you make the change and send it to me?

Thanks, Vinayak

Hi Vinayak,

Switch 1 in whcih show cfs peers are showing

Profile-Revision: 85

Session-type: Commit

Session-subtype: -

Peer-triggered: No

Profile-status: Sync Success

Local information:

----------------

Status: Commit Success

Error(s):

Peer information:

----------------

IP-address: 172.16.1.54

Sync-status: Not yet merged

Merge Flags: pending_merge:1 rcv_merge:1 pend_validate:0

Status: Peer not reachable

Error(s):

Profile-Revision: 75

Session-type: Initial-Exchange

Session-subtype: Init-Exchange-All

Peer-triggered: No

Profile-status: -

Switch 2 in which show cfs peers are not shown

I tried a copy run start in the switch 2 but it got stuck and following output taken after that.

Local information:

----------------

Status: Verify Failure

Error(s): CFS send failure: [Failure: Lock failed]

Peer information:

----------------

IP-address: 172.16.1.53

Sync-status: Not yet merged

Merge Flags: pending_merge:1 rcv_merge:1 pend_validate:0

Status: -

Error(s):

Regards

Sonu

1) Do you have a large number of switches on the mgmt network ?

2) Have you used cfs ipv4 mcast-address config on the peer switches

Multicast address configuration will help in making sure only the 2 peer switches are in the CFS domain for config-sync.

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/configuration/guide/cli/cfs.html

3) Can you remove "cfs ipv4 distribute" config and re-config on the switch where the peer is not seen and check if this helps to resolve the issue?

Thanks, Vinayak

Hi Vinayak,

I am refering to the thread:https://supportforums.cisco.com/message/3980732#3980732.

Question:

I am planning an ISSU upgrade on my nexus 7000. On this chassis there are also Nexus 2000 connected. Since they don't support vpc, all fexes are connected to a single chassis. (but different card).

The chassis will upgrade without interruption, but what about the fexes? They will also need  the new NX-OS. Do they reboot after or during the upgrade? Or is it without interruption like the 7000 chassis?

Kindly let me know your opinion on this pleasE/

Thanks,

Fari.

Hi Fari

If the 7k chassis support ISSU, then we should be able to do hitless upgrade on the 2k as well. During ISSU it should not reload your fexes.

Thanks, Vinayak

pokwan
Level 1
Level 1

Hi Vinayak,

If I have the following configuration on the router at the moment, how do you migrate the configuration to the nexus 5000?

int vlan 10

ip address 172.31.10.3 255.255.255.0

int vlan 20

ip address 172.31.20.3 255.255.255.0

int vlan 30

ip address 10.10.30.3 255.255.255.0

router rip

version 2

passive-interface default

no passive-interface vlan20

network 10.0.0.0

network 172.31.0.0

no auto-summary

redistribute static route-map test

route-map test permit 10

match ip address 10

ip access-list 10

permit 0.0.0.0

deny any

TIA

PF

Hi PF

You should definitely go through our Nexus 5000 Unicast Configuration Guide on Cisco.com as it will assist you in understanding how to configure RIP on Nexus switch and how to configure for route redistribution. Also for Route-Maps we have "Configuring Route Policy Manager" guide which can assist you in configuring for Route-Maps.

"Configuring RIP"

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/unicast/521_N11/l3_rip.html

"Configuring Route Policy Manager"

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/unicast/521_N11/l3_rpm.html

Vinayak,

Thanks very much.  The link provides you with the pieces of config for the RIP.  What I need is an example of the RIP configuration by putting the required commands together for a working RIP.  If I have the following configuration below how is that being achieved?  An example will gives a better understanding of the RIP configuration on the Nexus and helps to consolidate the information/understanding I have.

TIA

PF

I also have a ethernet link between 2 data centres.  For example

Config for DC  B

int gi1/1

desc link to DC A

switchport access vlan 10

int vlan 10

ip address 172.31.10.3 255.255.255.0

int vlan 20

ip address 172.31.20.3 255.255.255.0

int vlan 30

ip address 10.10.30.3 255.255.255.0

router rip

version 2

passive-interface default

no passive-interface vlan20

network 10.0.0.0

network 172.31.0.0

no auto-summary

redistribute static route-map test

route-map test permit 10

match ip address 10

ip access-list 10

permit 0.0.0.0

deny any

Ok I can assist you with building an RIP example config based on what you already have.

The document I provided before would have helped you gather the below information which would have been useful to build a configuration which I pasted below.

- passive interface command is configured under vlan SVI.

- no support for automatic route summarization.

- no network statements definition under router rip.

- redistribute and route-map commands are similar.

Based on the above, following is the config.

int e1/1

desc link to DC A

switchport access vlan 10

feature rip

feature interface-vlan

int vlan 10

ip address 172.31.10.3 255.255.255.0

ip rip passive-interface

int vlan 20

ip address 172.31.20.3 255.255.255.0

int vlan 30

ip address 10.10.30.3 255.255.255.0

ip rip passive-interface

router rip abc

address-family ipv4 unicast

redistribute static route-map test

route-map test permit 10

match ip address 10

ip access-list 10

permit ip any any

You can configure similarly for other site.

Hope this helps,

Thanks,

Vinayak

Vinayak,

Thanks so much for your reply.  It is very much appreciated.

2 more questions.

1.  Don't you need " ip router ip abc" defined on the interface?  For exmaple

int vlan 10

  ip address 172.31.10.3 255.255.255.0

  ip rip passive-interface

  ip router rip abc <-- ???

int vlan 20

...

ip router rip abc

2.  If the  Nexus 5500 and 2000 are used (no other switches), 5500 as core switches,  can I joint the 2 mgmt 0 interfaces on both the 5500 (peer switches)  together?

Thanks again.

PF

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: