07-15-2016 02:06 AM - edited 03-01-2019 12:49 PM
Hi All,
Its new installation of Cisco UCS with Brocade SAN switch 6510.
I have followed the Cisco UCS to Brocade Connectivity Guide.
Even though the on both Switch and FI side the Link is not coming up. No lights are glowing.
When I checked the switchshow @Brocade 6510, the ports are showing No_Light status. Where in the other ports it shows the WWN number of my Storage device.
At FI, when i click FC ports, its showing Over all Status - link Down.
Anyone face this issue.?
Am i missing some configuration steps?
Regards,
Praveen
Solved! Go to Solution.
07-15-2016 06:12 AM
This must be either a trivial or a very complex problem ! I would rather think trivial.
The Brocade is a 16G FC switch; the negotiation down to 8G seems to work (storage subsystem).
Maybe you could try to set the speed on the Brocade port fix to 8G !
Or something with the cables is wrong ? send / receive crossed ??
Are the SFP's MMF short wave ? patch cables as well ?
Can you connect cable of port eg. 31 to 33, 32 to 34 and see if this works ?
07-15-2016 02:30 AM
Hi Praveen
- is the UCS FI in FC end host mode ?
- did you configure FC ports on the UCS FI ?
- do you have a FC 2/4/8 G SFP
- is the FC port : FC uplink port ?
- did you configure VSAN's (I hope you are not using default VSAN 1, but create 2 different VSAN's on FI-A and FI-B, e.g 20 and 30; and then put the FC ports in the apropriate VSAN.
Walter.
PS have a look at
http://www.cisco.com/c/dam/en/us/products/collateral/servers-unified-computing/ucs-b-series-blade-servers/whitepaper_C07-730016.pdf
07-15-2016 02:35 AM
Hi Walter,
Thanks for the quick reply.
Please find my answers as below.
- is the UCS FI in FC end host mode ? - Yes. Attached screen shot.
- did you configure FC ports on the UCS FI ? - Yes, Port 31 & 32 are FC ports
- do you have a FC 2/4/8 G SFP - 8G SFP
- is the FC port : FC uplink port ? - Yes, its FC Uplink
- did you configure VSAN's (I hope you are not using default VSAN 1, but create 2 different VSAN's on FI-A and FI-B, e.g 20 and 30; and then put the FC ports in the apropriate VSAN.
- Yes, 2 different VSAN configured - Attached screen shots.
Thanks for your help,
Praveen
07-15-2016 02:47 AM
Connect via ssh to each FI, then
connect nx-os
show npv status ...... plus any other available show npv commands
07-15-2016 03:32 AM
Please find the details
#show npv status
npiv is enabled
disruptive load balancing is disabled
External Interfaces:
====================
Interface: fc1/31, State: Pre-Initialized
Interface: fc1/32, State: Pre-Initialized
Number of External Interfaces: 2
Server Interfaces:
==================
Interface: vfc703, VSAN: 20, State: Waiting for External Interface
Interface: vfc739, VSAN: 5, State: Waiting for External Interface
Interface: vfc751, VSAN: 5, State: Waiting for External Interface
Interface: vfc763, VSAN: 5, State: Waiting for External Interface
Number of Server Interfaces: 4
# show npv flogi-table
No flogi sessions found.
show npv internal info
NPV Globals:
============
NPV mode: ENABLED
Switch-Name: CISLABFI-A
Switch Mgmt IP Address: 192.168.47.31
proxy logo Retries: 3
Internal FLOGI max timeout Retries: 3
NS Registration max timeout Retries: 5
Disruptive Load Balance: Disabled
Load Balancing Timer (8), id: 00, active: FALSE time remaining: 0
Server Interface Block Timer is OFF
Server Interface Block Timer(10), id: 00, active: FALSE time remaining: 0
Recent External Interface Up Time: Sun Jul 3 18:06:28 2016
Switch WWN: 20:00:8c:60:4f:c8:df:c0
Port group information
=======================
External Interface Info:
========================
Interface Information:
ifindex: fc1/31, Port VSAN: 20, Internal FLOGI VSAN: 0, fcid: 0x000000
Internal FLOGI Done: False, In Progress: False
FSM current state: NPIVP_EXT_IF_ST_PREINIT_DONE
fcf_mac: 8c60:4fc8:dfe6
Pinned Server Intf Count: 0
Port Parameters:
Operational port mode: NP(0x4000), Logical Ifindex: (0x0)
Rx B2B Credits: 0, Multiplier: 0, Buff Size: 0
Tx B2B Credits: 0, Multiplier: 0, Buff Size: 0, bbscn: 0
bbscn_capable: FALSE bbscn_max: 0, port_bbscn: 0
Virtual Fabric Parameters:
VF Tagging Enabled: FALSE, Core supports VF: FALSE, Port Mode: NP
Physical Login Done: FALSE Port Channel: FALSE
Timer & Retry Information:
Busy Timer (1), id: 00, active: FALSE time remaining: 0
Fail Retry Timer (7), id: 00, active: FALSE time remaining: 0
Statistics:
flogi retry count : 0
ns registration retry count : 0
Others:
NPIV not enabled on core sysloged: FALSE
VSAN mis-match on core sysloged: FALSE
FC Delete Response Received: FALSE
VSAN 20:
Timers:
FDISC Response Timer (2), id: 00, active: FALSE time remaining: 0
Error Clear Timer (6), id: 07, active: TRUE time remaining: 408
Statistics:
number of LS_RJTs rcvd : 0
LS_RJT timestamp : Thu Jan 1 00:00:00 1970
retry count : 0
pending queue size : 0
FLOGIs on this VSAN:
Interface Information:
ifindex: fc1/32, Port VSAN: 20, Internal FLOGI VSAN: 0, fcid: 0x000000
Internal FLOGI Done: False, In Progress: False
FSM current state: NPIVP_EXT_IF_ST_PREINIT_DONE
fcf_mac: 8c60:4fc8:dfe7
Pinned Server Intf Count: 0
Port Parameters:
Operational port mode: NP(0x4000), Logical Ifindex: (0x0)
Rx B2B Credits: 0, Multiplier: 0, Buff Size: 0
Tx B2B Credits: 0, Multiplier: 0, Buff Size: 0, bbscn: 0
bbscn_capable: FALSE bbscn_max: 0, port_bbscn: 0
Virtual Fabric Parameters:
VF Tagging Enabled: FALSE, Core supports VF: FALSE, Port Mode: NP
Physical Login Done: FALSE Port Channel: FALSE
Timer & Retry Information:
Busy Timer (1), id: 00, active: FALSE time remaining: 0
Fail Retry Timer (7), id: 00, active: FALSE time remaining: 0
Statistics:
flogi retry count : 0
ns registration retry count : 0
Others:
NPIV not enabled on core sysloged: FALSE
VSAN mis-match on core sysloged: FALSE
FC Delete Response Received: FALSE
VSAN 20:
Timers:
FDISC Response Timer (2), id: 00, active: FALSE time remaining: 0
Error Clear Timer (6), id: 07, active: TRUE time remaining: 408
Statistics:
number of LS_RJTs rcvd : 0
LS_RJT timestamp : Thu Jan 1 00:00:00 1970
retry count : 0
pending queue size : 0
FLOGIs on this VSAN:
Server Interface Info:
======================
Interface Information:
ifindex: vfc703, VSAN: 20, NPIV requested: FALSE, lcp init done: FALSE
Selected External Interface:
FSM current state: NPIVP_SVR_IF_ST_WAITING_EXTERNAL_INTERFACE
Port Parameters:
rxbbcredit: 0 rxbufsize: 0
txbbcredit: 0 txbufsize: 0 txbbscn: 0
bbscn_capable: FALSE bbscn_max: 0, port_bbscn: 0
is_fip: FALSE
Statistics:
Traffic Map: not configured
Interface Information:
ifindex: vfc739, VSAN: 5, NPIV requested: FALSE, lcp init done: FALSE
Selected External Interface:
FSM current state: NPIVP_SVR_IF_ST_WAITING_EXTERNAL_INTERFACE
Port Parameters:
rxbbcredit: 0 rxbufsize: 0
txbbcredit: 0 txbufsize: 0 txbbscn: 0
bbscn_capable: FALSE bbscn_max: 0, port_bbscn: 0
is_fip: FALSE
Statistics:
Traffic Map: not configured
#
07-15-2016 04:10 AM
Hi Praveen
The link between UCS FI and Brocade is down !
Please provide the output of the following commands on the Brocade
switchshow
and of the ports connecting the UCS FI
portshow
portcfgshow
07-15-2016 04:20 AM
Hi Walter,
I have connected the FI A Port 31 & 32 are connected to Brocade 6510 Port 34 and 35.
One more point, to note, this Brocade switch has 3 Context. I am using Context 2 for my UCS connection.
Please find the details.
#switchshow
switchName: switch_2
switchType: 109.1
switchState: Online
switchMode: Native
switchRole: Principal
switchDomain: 2
switchId: fffc02
switchWwn: 10:00:50:eb:1a:65:09:42
zoning: OFF
switchBeacon: OFF
FC Router: OFF
Allow XISL Use: ON
LS Attributes: [FID: 2, Base Switch: No, Default Switch: No, Address Mode 0]
Index Port Address Media Speed State Proto
==================================================
32 32 020000 id N8 Online FC F-Port 50:06:0e:80:13:35:25:03
33 33 020100 id N8 Online FC F-Port 50:06:0e:80:13:35:25:23
34 34 020200 id N16 No_Light FC
35 35 020300 id N16 No_Light FC
#portshow 34
portIndex: 34
portName: port34
portHealth: No Fabric Watch License
Authentication: None
portDisableReason: None
portCFlags: 0x1
portFlags: 0x1 PRESENT U_PORT
LocalSwcFlags: 0x0
portType: 24.0
POD Port: Port is licensed
portState: 2 Offline
Protocol: FC
portPhys: 4 No_Light portScn: 0
port generation number: 0
state transition count: 0
portId: 020200
portIfId: 43020025
portWwn: 20:22:50:eb:1a:65:09:42
portWwn of device(s) connected:
Distance: normal
portSpeed: N16Gbps
FEC: Inactive
Credit Recovery: Inactive
LE domain: 0
FC Fastwrite: OFF
Interrupts: 0 Link_failure: 0 Frjt: 0
Unknown: 0 Loss_of_sync: 0 Fbsy: 0
Lli: 10 Loss_of_sig: 0
Proc_rqrd: 0 Protocol_err: 0
Timed_out: 0 Invalid_word: 0
Rx_flushed: 0 Invalid_crc: 0
Tx_unavail: 0 Delim_err: 0
Free_buffer: 0 Address_err: 0
Overrun: 0 Lr_in: 0
Suspended: 0 Lr_out: 0
Parity_err: 0 Ols_in: 0
2_parity_err: 0 Ols_out: 0
CMI_bus_err: 0
#portcfgshow 34
Area Number: 2
Octet Speed Combo: 1(16G|8G|4G|2G)
Speed Level: AUTO(SW)
AL_PA Offset 13: OFF
Trunk Port ON
Long Distance OFF
VC Link Init OFF
Locked L_Port OFF
Locked G_Port OFF
Disabled E_Port OFF
Locked E_Port OFF
ISL R_RDY Mode OFF
RSCN Suppressed OFF
Persistent Disable OFF
LOS TOV enable OFF
NPIV capability ON
QOS Port AE
Port Auto Disable: OFF
Rate Limit OFF
EX Port OFF
Mirror Port OFF
SIM Port OFF
Credit Recovery ON
F_Port Buffers OFF
E_Port Credits OFF
Fault Delay: 0(R_A_TOV)
NPIV PP Limit: 126
CSCTL mode: OFF
D-Port mode: OFF
D-Port over DWDM: OFF
Compression: OFF
Encryption: OFF
FEC: ON
07-15-2016 05:12 AM
This is what I see
#switchshow
switchName: switch_2
switchType: 109.1
switchState: Online
switchMode: Native
switchRole: Principal
switchDomain: 2
switchId: fffc02
switchWwn: 10:00:50:eb:1a:65:09:42
zoning: OFF
portState: 2 Offline
portSpeed: N16Gbps
Octet Speed Combo: 1(16G|8G|4G|2G)
Speed Level: AUTO(SW)
NPIV capability ON
Question: did you enable port 34, 35 ?
portenable 34,35
Are the 2 devices logged in on port 32/33 storage subsystems ?
Zoning OFF on the switch is very strange ?
Btw. do you see any information in the log file on the UCS system ?
07-15-2016 06:00 AM
Hi Walter,
Yes, I had enabled the Ports earlier also. Again i did the portenable and same status.
The other 2 devices showing the WWN are Storage ports.
Since, I have not seen the UCS ports, Zoning is not done.
Which logs has to check on UCS ?
switch_2:FID2:admin> portenable 34
switch_2:FID2:admin> portenable 35
switch_2:FID2:admin> switchshow
switchName: switch_2
switchType: 109.1
switchState: Online
switchMode: Native
switchRole: Principal
switchDomain: 2
switchId: fffc02
switchWwn: 10:00:50:eb:1a:65:09:42
zoning: OFF
switchBeacon: OFF
FC Router: OFF
Allow XISL Use: ON
LS Attributes: [FID: 2, Base Switch: No, Default Switch: No, Address Mode 0]
Index Port Address Media Speed State Proto
==================================================
32 32 020000 id N8 Online FC F-Port 50:06:0e:80:13:35:25:03
33 33 020100 id N8 Online FC F-Port 50:06:0e:80:13:35:25:23
34 34 020200 id N16 No_Light FC
35 35 020300 id N16 No_Light FC
switch_2:FID2:admin>
07-15-2016 06:12 AM
This must be either a trivial or a very complex problem ! I would rather think trivial.
The Brocade is a 16G FC switch; the negotiation down to 8G seems to work (storage subsystem).
Maybe you could try to set the speed on the Brocade port fix to 8G !
Or something with the cables is wrong ? send / receive crossed ??
Are the SFP's MMF short wave ? patch cables as well ?
Can you connect cable of port eg. 31 to 33, 32 to 34 and see if this works ?
07-18-2016 03:16 AM
Hi Walter,
Yes, it was problem with SFP.
The SFP which I had in FI, was LW and at the Brocade was SW.
Luckily I had 2 old (4Gbps ) SW SFP and used in FI. now the connection is showing UP and I can see the WWN in portshow.
Thanks a lot for your support and Guidance.
Regards,
Praveen
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: