Solved: Cisco 6248 FI to Brocade 6510 SAN switch Link shows Down

praveenkumar.embramadathil1 · ‎07-15-2016

Hi All,

Its new installation of Cisco UCS with Brocade SAN switch 6510.

I have followed the Cisco UCS to Brocade Connectivity Guide.

Even though the on both Switch and FI side the Link is not coming up. No lights are glowing.

When I checked the switchshow @Brocade 6510, the ports are showing No_Light status. Where in the other ports it shows the WWN number of my Storage device.

At FI, when i click FC ports, its showing Over all Status - link Down.

Anyone face this issue.?

Am i missing some configuration steps?

Regards,

Praveen

Walter Dey · ‎07-15-2016

This must be either a trivial or a very complex problem ! I would rather think trivial.

The Brocade is a 16G FC switch; the negotiation down to 8G seems to work (storage subsystem).

Maybe you could try to set the speed on the Brocade port fix to 8G !

Or something with the cables is wrong ? send / receive crossed ??

Are the SFP's MMF short wave ? patch cables as well ?

Can you connect cable of port eg. 31 to 33, 32 to 34 and see if this works ?

View solution in original post

Walter Dey · ‎07-15-2016

Hi Praveen

- is the UCS FI in FC end host mode ?

- did you configure FC ports on the UCS FI ?

- do you have a FC 2/4/8 G SFP

- is the FC port : FC uplink port ?

- did you configure VSAN's (I hope you are not using default VSAN 1, but create 2 different VSAN's on FI-A and FI-B, e.g 20 and 30; and then put the FC ports in the apropriate VSAN.

Walter.

PS have a look at

http://www.cisco.com/c/dam/en/us/products/collateral/servers-unified-computing/ucs-b-series-blade-servers/whitepaper_C07-730016.pdf

praveenkumar.embramadathil1 · ‎07-15-2016

Hi Walter,

Thanks for the quick reply.

Please find my answers as below.

- is the UCS FI in FC end host mode ? - Yes. Attached screen shot.

- did you configure FC ports on the UCS FI ? - Yes, Port 31 & 32 are FC ports

- do you have a FC 2/4/8 G SFP - 8G SFP

- is the FC port : FC uplink port ? - Yes, its FC Uplink

- did you configure VSAN's (I hope you are not using default VSAN 1, but create 2 different VSAN's on FI-A and FI-B, e.g 20 and 30; and then put the FC ports in the apropriate VSAN.

- Yes, 2 different VSAN configured - Attached screen shots.

Thanks for your help,

Praveen

Walter Dey · ‎07-15-2016

Connect via ssh to each FI, then

connect nx-os

show npv status ...... plus any other available show npv commands

praveenkumar.embramadathil1 · ‎07-15-2016

Please find the details

#show npv status

npiv is enabled

disruptive load balancing is disabled

External Interfaces:
====================
Interface: fc1/31, State: Pre-Initialized
Interface: fc1/32, State: Pre-Initialized

Number of External Interfaces: 2

Server Interfaces:
==================
Interface: vfc703, VSAN: 20, State: Waiting for External Interface
Interface: vfc739, VSAN: 5, State: Waiting for External Interface
Interface: vfc751, VSAN: 5, State: Waiting for External Interface
Interface: vfc763, VSAN: 5, State: Waiting for External Interface

Number of Server Interfaces: 4

# show npv flogi-table
No flogi sessions found.

show npv internal info
NPV Globals:
============
NPV mode: ENABLED
Switch-Name: CISLABFI-A
Switch Mgmt IP Address: 192.168.47.31
proxy logo Retries: 3
Internal FLOGI max timeout Retries: 3
NS Registration max timeout Retries: 5
Disruptive Load Balance: Disabled
Load Balancing Timer (8), id: 00, active: FALSE time remaining: 0
Server Interface Block Timer is OFF
Server Interface Block Timer(10), id: 00, active: FALSE time remaining: 0
Recent External Interface Up Time: Sun Jul 3 18:06:28 2016

Switch WWN: 20:00:8c:60:4f:c8:df:c0
Port group information
=======================

External Interface Info:
========================

Interface Information:
ifindex: fc1/31, Port VSAN: 20, Internal FLOGI VSAN: 0, fcid: 0x000000
Internal FLOGI Done: False, In Progress: False
FSM current state: NPIVP_EXT_IF_ST_PREINIT_DONE
fcf_mac: 8c60:4fc8:dfe6
Pinned Server Intf Count: 0

Port Parameters:
Operational port mode: NP(0x4000), Logical Ifindex: (0x0)
Rx B2B Credits: 0, Multiplier: 0, Buff Size: 0
Tx B2B Credits: 0, Multiplier: 0, Buff Size: 0, bbscn: 0
bbscn_capable: FALSE bbscn_max: 0, port_bbscn: 0

Virtual Fabric Parameters:
VF Tagging Enabled: FALSE, Core supports VF: FALSE, Port Mode: NP
Physical Login Done: FALSE Port Channel: FALSE

Timer & Retry Information:
Busy Timer (1), id: 00, active: FALSE time remaining: 0
Fail Retry Timer (7), id: 00, active: FALSE time remaining: 0

Statistics:
flogi retry count : 0
ns registration retry count : 0
Others:
NPIV not enabled on core sysloged: FALSE
VSAN mis-match on core sysloged: FALSE
FC Delete Response Received: FALSE

VSAN 20:

Timers:
FDISC Response Timer (2), id: 00, active: FALSE time remaining: 0
Error Clear Timer (6), id: 07, active: TRUE time remaining: 408

Statistics:
number of LS_RJTs rcvd : 0
LS_RJT timestamp : Thu Jan 1 00:00:00 1970

retry count : 0
pending queue size : 0

FLOGIs on this VSAN:

Interface Information:
ifindex: fc1/32, Port VSAN: 20, Internal FLOGI VSAN: 0, fcid: 0x000000
Internal FLOGI Done: False, In Progress: False
FSM current state: NPIVP_EXT_IF_ST_PREINIT_DONE
fcf_mac: 8c60:4fc8:dfe7
Pinned Server Intf Count: 0

Port Parameters:
Operational port mode: NP(0x4000), Logical Ifindex: (0x0)
Rx B2B Credits: 0, Multiplier: 0, Buff Size: 0
Tx B2B Credits: 0, Multiplier: 0, Buff Size: 0, bbscn: 0
bbscn_capable: FALSE bbscn_max: 0, port_bbscn: 0

Virtual Fabric Parameters:
VF Tagging Enabled: FALSE, Core supports VF: FALSE, Port Mode: NP
Physical Login Done: FALSE Port Channel: FALSE

Timer & Retry Information:
Busy Timer (1), id: 00, active: FALSE time remaining: 0
Fail Retry Timer (7), id: 00, active: FALSE time remaining: 0

Statistics:
flogi retry count : 0
ns registration retry count : 0
Others:
NPIV not enabled on core sysloged: FALSE
VSAN mis-match on core sysloged: FALSE
FC Delete Response Received: FALSE

VSAN 20:

Timers:
FDISC Response Timer (2), id: 00, active: FALSE time remaining: 0
Error Clear Timer (6), id: 07, active: TRUE time remaining: 408

Statistics:
number of LS_RJTs rcvd : 0
LS_RJT timestamp : Thu Jan 1 00:00:00 1970

retry count : 0
pending queue size : 0

FLOGIs on this VSAN:

Server Interface Info:
======================

Interface Information:
ifindex: vfc703, VSAN: 20, NPIV requested: FALSE, lcp init done: FALSE
Selected External Interface:
FSM current state: NPIVP_SVR_IF_ST_WAITING_EXTERNAL_INTERFACE

Port Parameters:
rxbbcredit: 0 rxbufsize: 0
txbbcredit: 0 txbufsize: 0 txbbscn: 0
bbscn_capable: FALSE bbscn_max: 0, port_bbscn: 0
is_fip: FALSE

Statistics:

Traffic Map: not configured

Interface Information:
ifindex: vfc739, VSAN: 5, NPIV requested: FALSE, lcp init done: FALSE
Selected External Interface:
FSM current state: NPIVP_SVR_IF_ST_WAITING_EXTERNAL_INTERFACE

Port Parameters:
rxbbcredit: 0 rxbufsize: 0
txbbcredit: 0 txbufsize: 0 txbbscn: 0
bbscn_capable: FALSE bbscn_max: 0, port_bbscn: 0
is_fip: FALSE

Statistics:

Traffic Map: not configured

#

Walter Dey · ‎07-15-2016

Hi Praveen

The link between UCS FI and Brocade is down !

Please provide the output of the following commands on the Brocade

switchshow

and of the ports connecting the UCS FI

portshow
portcfgshow

praveenkumar.embramadathil1 · ‎07-15-2016

Hi Walter,

I have connected the FI A Port 31 & 32 are connected to Brocade 6510 Port 34 and 35.

One more point, to note, this Brocade switch has 3 Context. I am using Context 2 for my UCS connection.

Please find the details.

#switchshow
switchName: switch_2
switchType: 109.1
switchState: Online
switchMode: Native
switchRole: Principal
switchDomain: 2
switchId: fffc02
switchWwn: 10:00:50:eb:1a:65:09:42
zoning: OFF
switchBeacon: OFF
FC Router: OFF
Allow XISL Use: ON
LS Attributes: [FID: 2, Base Switch: No, Default Switch: No, Address Mode 0]

Index Port Address Media Speed State Proto
==================================================
32 32 020000 id N8 Online FC F-Port 50:06:0e:80:13:35:25:03
33 33 020100 id N8 Online FC F-Port 50:06:0e:80:13:35:25:23
34 34 020200 id N16 No_Light FC
35 35 020300 id N16 No_Light FC

#portshow 34
portIndex: 34
portName: port34
portHealth: No Fabric Watch License

Authentication: None
portDisableReason: None
portCFlags: 0x1
portFlags: 0x1 PRESENT U_PORT
LocalSwcFlags: 0x0
portType: 24.0
POD Port: Port is licensed
portState: 2 Offline
Protocol: FC
portPhys: 4 No_Light portScn: 0
port generation number: 0
state transition count: 0

portId: 020200
portIfId: 43020025
portWwn: 20:22:50:eb:1a:65:09:42
portWwn of device(s) connected:

Distance: normal
portSpeed: N16Gbps

FEC: Inactive
Credit Recovery: Inactive
LE domain: 0
FC Fastwrite: OFF
Interrupts: 0 Link_failure: 0 Frjt: 0
Unknown: 0 Loss_of_sync: 0 Fbsy: 0
Lli: 10 Loss_of_sig: 0
Proc_rqrd: 0 Protocol_err: 0
Timed_out: 0 Invalid_word: 0
Rx_flushed: 0 Invalid_crc: 0
Tx_unavail: 0 Delim_err: 0
Free_buffer: 0 Address_err: 0
Overrun: 0 Lr_in: 0
Suspended: 0 Lr_out: 0
Parity_err: 0 Ols_in: 0
2_parity_err: 0 Ols_out: 0
CMI_bus_err: 0

#portcfgshow 34
Area Number: 2
Octet Speed Combo: 1(16G|8G|4G|2G)
Speed Level: AUTO(SW)
AL_PA Offset 13: OFF
Trunk Port ON
Long Distance OFF
VC Link Init OFF
Locked L_Port OFF
Locked G_Port OFF
Disabled E_Port OFF
Locked E_Port OFF
ISL R_RDY Mode OFF
RSCN Suppressed OFF
Persistent Disable OFF
LOS TOV enable OFF
NPIV capability ON
QOS Port AE
Port Auto Disable: OFF
Rate Limit OFF
EX Port OFF
Mirror Port OFF
SIM Port OFF
Credit Recovery ON
F_Port Buffers OFF
E_Port Credits OFF
Fault Delay: 0(R_A_TOV)
NPIV PP Limit: 126
CSCTL mode: OFF
D-Port mode: OFF
D-Port over DWDM: OFF
Compression: OFF
Encryption: OFF
FEC: ON

Walter Dey · ‎07-15-2016

This is what I see

#switchshow
switchName: switch_2
switchType: 109.1
switchState: Online
switchMode: Native
switchRole: Principal
switchDomain: 2
switchId: fffc02
switchWwn: 10:00:50:eb:1a:65:09:42
zoning: OFF

portState: 2 Offline

portSpeed: N16Gbps
Octet Speed Combo: 1(16G|8G|4G|2G)
Speed Level: AUTO(SW)

NPIV capability ON

Question: did you enable port 34, 35 ?

portenable 34,35

Are the 2 devices logged in on port 32/33 storage subsystems ?

Zoning OFF on the switch is very strange ?

Btw. do you see any information in the log file on the UCS system ?

praveenkumar.embramadathil1 · ‎07-15-2016

Hi Walter,

Yes, I had enabled the Ports earlier also. Again i did the portenable and same status.

The other 2 devices showing the WWN are Storage ports.

Since, I have not seen the UCS ports, Zoning is not done.

Which logs has to check on UCS ?

switch_2:FID2:admin> portenable 34
switch_2:FID2:admin> portenable 35
switch_2:FID2:admin> switchshow
switchName: switch_2
switchType: 109.1
switchState: Online
switchMode: Native
switchRole: Principal
switchDomain: 2
switchId: fffc02
switchWwn: 10:00:50:eb:1a:65:09:42
zoning: OFF
switchBeacon: OFF
FC Router: OFF
Allow XISL Use: ON
LS Attributes: [FID: 2, Base Switch: No, Default Switch: No, Address Mode 0]

Index Port Address Media Speed State Proto
==================================================
32 32 020000 id N8 Online FC F-Port 50:06:0e:80:13:35:25:03
33 33 020100 id N8 Online FC F-Port 50:06:0e:80:13:35:25:23
34 34 020200 id N16 No_Light FC
35 35 020300 id N16 No_Light FC
switch_2:FID2:admin>

Walter Dey · ‎07-15-2016

This must be either a trivial or a very complex problem ! I would rather think trivial.

The Brocade is a 16G FC switch; the negotiation down to 8G seems to work (storage subsystem).

Maybe you could try to set the speed on the Brocade port fix to 8G !

Or something with the cables is wrong ? send / receive crossed ??

Are the SFP's MMF short wave ? patch cables as well ?

Can you connect cable of port eg. 31 to 33, 32 to 34 and see if this works ?

praveenkumar.embramadathil1 · ‎07-18-2016

Hi Walter,

Yes, it was problem with SFP.

The SFP which I had in FI, was LW and at the Brocade was SW.

Luckily I had 2 old (4Gbps ) SW SFP and used in FI. now the connection is showing UP and I can see the WWN in portshow.

Thanks a lot for your support and Guidance.

Regards,

Praveen