Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Coming this summer: Cisco Support Community and Cisco Communities are merging. Learn more

Data Center Blogs

33 Views
0 Comments

A place to content about technology, industry trends, and Cisco'slatest endeavors.

21 Views
0 Comments

I while ago I wrote a 6 parts blog about my first FlexPod deployment. I noticed I have never shared it on this forum. So for those who are interested, please refer to the following links.

I hope you enjoy reading it. I must add, since I switched my job I haven't done anything with a Cisco FlexPod for a while :( But I am still very excited about it :)

151 Views
2 Comments

The purpose of this BLOG is to discuss Questions, Concerns, and Issues with the Callhome Feature functionality in the ACI Fabrics. Since this is Feature focused we can monitor your feedback and push for improvements and enhancements if requested. Also, this forum allows you to ask questions and hopefully we can assist you and your questions.

Thank you again for participating in the Cisco Support Forum!

T.

65 Views
0 Comments

The purpose of this BLOG is to discuss Questions, Concerns, and Issues with the DNS Feature functionality in the ACI Fabrics. Since this is Feature focused we can monitor your feedback and push for improvements and enhancements if requested. Also, this forum allows you to ask questions and hopefully we can assist you and your questions.

Thank you again for participating in the Cisco Support Forum!

T.

78 Views
1 Comment

The purpose of this BLOG is to discuss Questions, Concerns, and Issues with the NTP Feature functionality in the ACI Fabrics. Since this is Feature focused we can monitor your feedback and push for improvements and enhancements if requested. Also, this forum allows you to ask questions and hopefully we can assist you and your questions.

Thank you again for participating in the Cisco Support Forum!

T.

663 Views
10 Comments

The purpose of this BLOG is to discuss Questions, Concerns, and Issues with the Syslog Feature functionality in the ACI Fabrics. Since this is Feature focused we can monitor your feedback and push for improvements and enhancements if requested. Also, this forum allows you to ask questions and hopefully we can assist you and your questions.

Thank you again for participating in the Cisco Support Forum!

T.

1262 Views
1 Comment

The purpose of this BLOG is to discuss Questions, Concerns, and Issues with the SNMP Feature functionality in the ACI Fabrics. Since this is Feature focused we can monitor your feedback and push for improvements and enhancements if requested. Also, this forum allows you to ask questions and hopefully we can assist you and your questions.

Attached to the blog is the latest Technote that I have developed for this feature in ACI. Some troubleshooting tips may require "root" access. This requires Cisco TAC assistance but I wanted to still document the process so that you understand the different ways to troubleshoot issues regarding to this feature.

Thank you again for participating in the Cisco Support Forum!

T.

361 Views
0 Comments

C-Series Integrated Server // Troubleshooting and Resolving Service Profile Association Failure When Incorrect Server Information is Given

This article will walk you through how to resolve a F0327 error on a UCSM managed rack server where there is a failure to associate and the customer has given you incorrect information as to what server they were trying to associate.

 

Problem

 

A customer called in with a problem associating a service profile to a server. They receive the following error when attempting to associate service profile STR_38 with a server.

 

Error Code: F0327

Cause: Configuration Failure

Description: Service profile STR_38 configuration failed due to insufficient-resources, connection-placement, insufficient-vnic-capacity.

 

I gathered the necessary UCSM logs and the customer told me they were trying to associate server 57 which was incorrect information. When I could see Server 57 was already associated with a different service profile, I didn't want to bother the customer with asking more questions and tracked it down myself so we could resolve the issue faster.

 

Solution

 

Customer errors from attempted service profile association of service profile STR_38:

 

Capture.jpg

 

Once you have UCSM logs, you can open the UCSM and expand the A side and B side zip files. Once this is done, open the SAM_TECHSUPPORT file on whichever unzipped A or B side has it open it in a text editor like Notepad++ (mac/pc) or TextWrangler (mac). Since we know the service profile name (in this case STR_38), search for the following phrase and search for all occurrences of it.

 

"STR_38 failed"

 

You will be jumped to the following minor errors and warnings telling you the following:

 

  • Both when and why it failed.
  • What portion of the service profile actually failed that would lead to insufficient resources (although the error already points to possible vnic issues)
  • What server it was trying to associate.

 

Severity: Minor
Code: F0169
Last Transition Time: 2016-07-07T11:43:04.671
ID: 3727100
Status: None
Description: Eth vNIC vNIC1, service profile STR_38 failed to apply configuration
Affected Object: org-root/org-STR_DDS/ls-STR_38/ether-vNIC1
Name: Vnic Ether Config Failed
Cause: Configuration Failed
Type: Configuration
Acknowledged: No
Occurrences: 1
Creation Time: 2016-07-07T11:43:04.671
Original Severity: Minor
Previous Severity: Minor
Highest Severity: Minor

Severity: Minor
Code: F0169
Last Transition Time: 2016-07-07T11:43:04.670
ID: 3727099
Status: None
Description: Eth vNIC vNIC0, service profile STR_38 failed to apply configuration
Affected Object: org-root/org-STR_DDS/ls-STR_38/ether-vNIC0
Name: Vnic Ether Config Failed
Cause: Configuration Failed
Type: Configuration
Acknowledged: No
Occurrences: 1
Creation Time: 2016-07-07T11:43:04.670
Original Severity: Minor
Previous Severity: Minor
Highest Severity: Minor

Severity: Major
Code: F0327
Last Transition Time: 2016-07-07T11:43:04.669
ID: 3727098
Status: None
Description: Service profile STR_38 configuration failed due to insufficient-resources,connection-placement,insufficient-vnic-capacity
Affected Object: org-root/org-STR_DDS/ls-STR_38
Name: Ls Server Config Failure
Cause: Configuration Failure
Type: Server
Acknowledged: No
Occurrences: 1
Creation Time: 2016-07-07T11:43:04.669
Original Severity: Major
Previous Severity: Major
Highest Severity: Major

Severity: Minor
Code: F0689
Last Transition Time: 2016-07-07T11:43:04.668
ID: 3727096
Status: None
Description: Assignment of service profile STR_38 to server sys/rack-unit-47 failed <=== Rack Server 47
Affected Object: org-root/org-STR_DDS/ls-STR_38/pn
Name: Ls Compute Binding Assignment Requirements Not Met
Cause: Assignment Failed
Type: Server
Acknowledged: No
Occurrences: 1
Creation Time: 2016-07-07T11:43:04.668
Original Severity: Minor
Previous Severity: Minor
Highest Severity: Minor

You can see this failed on the VNIC portion.

 

 

Using this information you can now search the same SAM_TECHSUPPORT file to look for all occurrences of Server 47.

 

The first hit on the list will take you to inventory listing of Server 47 here that will include the type of VIC (where your NICs are located). The VIC info is the last line of the inventory output.

 

Server 47:
    Name:
    Model: UCSC-C240-M4SX
    Acknowledged Vendor: Cisco Systems Inc
    Acknowledged Serial (SN): XXXXXXXX
    Acknowledged Revision: 0
    Acknowledged Product Name: Cisco UCS C240 M4SX
    Acknowledged PID: UCSC-C240-M4SX
    Acknowledged VID: 0
    Acknowledged Memory (MB): 131072
    Acknowledged Effective Memory (MB): 131072
    Acknowledged Cores: 16
    Acknowledged Adapters: 1
    Acknowledged Burned-In UUID: f98ebd24-5287-4d80-adf0-d04cdb359dd4
    Acknowledged Dynamic UUID: f98ebd24-5287-4d80-adf0-d04cdb359dd4

 

<======Information redacted to save space==================>

 

Adapter:

        Adapter PID          Vendor            Serial       Overall Status
        ------- ------------ ----------------- ------------ --------------
              1 UCSC-MLOM-CSC-02
                             Cisco Systems Inc FCH19447AQR  N/A

 

UCSC-MLOM-CSC-02 is a VIC 1227

 

The second hit on the list for Server 47 will take you to show you the connectivity.

 

Server 47:
    Conn Path: A,B
    Conn Status: A
    Managing Instance: A
    Availability: Available
    Admin State: In Service
    Overall Status: Unassociated
    Oper Qualifier: N/A
    Discovery: Complete
    Current Task:
    Check Point: Discovered

 

You will notice that the Conn Path is set for A,B but the Conn Status is only A. Instead of jumping to conclusions I checked other rack servers in the same area on the same log and all are set to A/B.

 

Server 45:
    Conn Path: A,B
    Conn Status: A,B
    Managing Instance: A
    Availability: Unavailable
    Admin State: In Service
    Overall Status: Ok
    Oper Qualifier: N/A
    Discovery: Complete
    Current Task:
    Check Point: Discovered

Server 46:
    Conn Path: A,B
    Conn Status: A,B
    Managing Instance: B
    Availability: Unavailable
    Admin State: In Service
    Overall Status: Ok
    Oper Qualifier: N/A
    Discovery: Complete
    Current Task:
    Check Point: Discovered

 

To triple check this a little further you can look for "scope server 47" on the same SAM_TECHSUPPORT file and the second hit will take you to the adapter details which will also reflect incorrect connectivity.

 

`scope server 47`
`scope adapter 1`
`show expand detail`

Adapter:
    Id: 1
    Product Name: Cisco UCS MLOM 1227
    PID: UCSC-MLOM-CSC-02
    VID: V00
    Vendor: Cisco Systems Inc
    Serial: XXXXXXXX
    Revision: 0
    Mfg Date: 2015-10-29T01:00:00.000
    Slot: MLOM
    Overall Status: N/A
    Conn Path: A
    Conn Status: A
    Managing Instance: A
    Product Description: PCI Express Dual Port Cisco Converged Network Adapter Supporting 10Gb Ethernet and 10Gb Fibre Channel

    Ext Interface:
        Mezz Id: 1
        Id: 1
        Port Channel:
        Mac: 58:AC:78:5C:BF:2C
        If Type: Physical
        Link Status: Up
        Oper State: Link Up
        Oper State Desc:
        Purpose: General
        Side: Left
        Slot Id: 1
        Fabric: NONE
        Peer Port Id: 0
        Peer Slot Id: 0
        Peer Chassis Id: N/A
        Current Task:

        Mezz Id: 1
        Id: 2
        Port Channel:
        Mac: 58:AC:78:5C:BF:2D
        If Type: Physical
        Link Status: Up
        Oper State: Link Up
        Oper State Desc:
        Purpose: General
        Side: Right
        Slot Id: 1
        Fabric: A
        Peer Port Id: 27
        Peer Slot Id: 1
        Peer Chassis Id: 3
        Current Task:

 

The giveaway there is only one side connectivity is that one side says Fabric A and the other side says NONE.

 

 

I know some of you are probably saying "Why not just go look at the VIF paths when you see the VNIC errors and also the main error on UCSM stating possible VNIC configuration?" The customer was looking at the wrong server and saw all VIF paths up when I asked him to look. This led us here.

 

You can see the same in the SAM_TECHSUPPORT for server 57 here. Shows both paths.

 

`scope server 57`
`scope adapter 1`
`show expand detail`

Adapter:
    Id: 1
    Product Name: Cisco UCS MLOM 1227
    PID: UCSC-MLOM-CSC-02
    VID: V00
    Vendor: Cisco Systems Inc
    Serial: XXXXXXX
    Revision: 0
    Mfg Date: 2015-09-11T01:00:00.000
    Slot: MLOM
    Overall Status: N/A
    Conn Path: A,B
    Conn Status: A,B
    Managing Instance: A
    Product Description: PCI Express Dual Port Cisco Converged Network Adapter Supporting 10Gb Ethernet and 10Gb Fibre Channel

    Ext Interface:
        Mezz Id: 1
        Id: 1
        Port Channel:
        Mac: E4:AA:5D:42:3E:93
        If Type: Physical
        Link Status: Up
        Oper State: Link Up
        Oper State Desc:
        Purpose: General
        Side: Left
        Slot Id: 1
        Fabric: B
        Peer Port Id: 25
        Peer Slot Id: 1
        Peer Chassis Id: 4
        Current Task:

        Mezz Id: 1
        Id: 2
        Port Channel:
        Mac: E4:AA:5D:42:3E:94
        If Type: Physical
        Link Status: Up
        Oper State: Link Up
        Oper State Desc:
        Purpose: General
        Side: Right
        Slot Id: 1
        Fabric: A
        Peer Port Id: 25
        Peer Slot Id: 1
        Peer Chassis Id: 3
        Current Task:

You can see how the customer telling you and looking at the wrong server on UCSM can lead you to the wrong path and some head scratching.

 

So once I had the right information, I asked the customer to check the VIF paths tab after selecting server 47 and sure enough only the A path was up causing the association of the service profile that wanted/needed two paths to fail with insufficient resources. Once they corrected the B path side connectivity issues, the service profile associated just fine.

 

I hope this will not only help you in this type of situation but I also hope that it's a good reference on how to check what causes a service profile to fail association and check connectivity that exists on server hardware.

 

Thanks!

 

Tray

1154 Views
0 Comments

Q: What is Cisco Unified Computing System?
Ans: Cisco Unified Computing System (UCS) unites physical and virtual computational and network resources into a single system.
 UCS is a set of pre-integrated data center components that comprises blade servers, adapters,
 fabric interconnects, and extenders that are integrated under a common embedded management system.
 This approach results in far fewer system components than comparable data center platforms.

Q. what is Unified ports
Ans: Unified ports are ports on the 6200 series fabric interconnect that can be configured to carry either Ethernet or Fibre Channel traffic.
  These ports are not reserved. They cannot be used by a Cisco UCS domain until you configure them.

Q. Is server port can be configured on expansion and fixed module
Ans: Server port cannot be configured on expansion module but can be done on fixed module

Q. Is uplink can be configured on fixed and expansion module
Ans: uplink can be configured on fixed and expansion modle

Q. How to upload firmware/upgrading install in FI
Ans: Equipment-->Firmware management-->Installed firmware-->Download firmware
 1.chassis-->I/O modules and blades (CIMC,BIOS,Adapters)
 2.FI --> kernel and system
 3.Rack mount server--> Servers (CIMC,BIOS,Adapters)

Q. How to configure CIMC in C-series.
Ans: Reboot the server and press F8 which prompts to the CIMC configuration where config nic redundancy, IP config and password for CIMC and press F10 for save and reload the server.

Q. How to download techsupport files
Ans: Admin-->ALL-->Faults,Events and Audit logs-->R-click on Tech support files--> select create and download tech support files(UCSM,UCSM-mgmt,Chassis, FEX,Rackserver)

Q. How chassis discovery done on the FI
Ans: Equipment-->Policies-->global policies-->Chassis discovery policy(1,2,4,8,max)

Q. what is Scrub policy and its types
Ans: scrub policy is applied when there is a diassociation in service profile there should not be going any changes made on the hardware
 it has DISK and BIOS settings, No for to remain the settings and YES for erase.

Q. what is maintenance policy and how it works
Ans: Maintenance policy is done for any hardware changes or any upgradation done on the hardware for reloading the device.
 They are of three types
 1.User Ack,
 2.Timer
 3.Immediate

Q. what is service profile template and its types
Ans: Initial template and Updating Template
 Initial: no change can be done after creating or linked or bind to service profile
 Updating: changes can be done to updating template even it is bind to service profile for any changes to template

Q. How to configure backup in a FI
Ans: Admin tab-->select All(left plane)-->select backup configuration(right plane)

Q. How to import configuration on FI
Ans: Admin tab-->select All(left plane)-->select Import configuration(right plane)

Q. Steps involved in configuring port-channel in FI.
Ans:
R-click on FI-->select connect to LAN manager-->select Create Port Channel-->Select Fabric A/B-->type port channel name-->add ports to Right plane

Q. Command to check the System uptime, NX-OS version in FI
Ans:
 FI-A(nxos)# show system uptime
 System start time:          Fri Jul 24 01:07:24 2015
 System uptime:              235 days, 9 hours, 53 minutes, 18 seconds
 Kernel uptime:              235 days, 10 hours, 55 minutes, 51 seconds
 Active supervisor uptime:   235 days, 9 hours, 53 minutes, 18 seconds

FI-A(nxos)# show Version

Q. where to configure CIMC ip for blades in FI
Ans:
 1.None—No management IP address is assigned to the service profile. The management IP address is set based on the CIMC management IP address settings on the server.
 2.Static—A static management IP address is assigned to the service profile, based on the information entered in this area.
 3.Pooled—A management IP address is assigned to the service profile from the management IP address pool.

Step 1    In the Navigation pane, click the Equipment tab.
Step 2    On the Equipment tab, expand Equipment > Chassis > Chassis Number > Servers.
Step 3    Click the server for which you want to configure an IP address.
Step 4    In the Work pane, click the Inventory tab.
Step 5    Click the CIMC subtab.
Step 6    In the Actions area, click Create/Modify Static Management IP


Q. how to change FI primary to sub-ordinate and viceversa
Ans:
 Process to failover
 1. Login to the primary 6120 via the UCS cluster IP address.
 2. Verify the current primary 6120 in the cluster.
 3. Enter local management on the primary via “connect local-mgmt” command.
 4. Issue the “cluster lead x” command to make the subordinate switch become the primary. Replace “x” with the correct switch letter.
 5. Verify that the role has changed on the previously subordinate switch by SSH’ing into it and issuing the “show cluster state” command.
 The switch show now show up as the primary.

 1. from FI-B to FI-A
  UCS-B# connect local-mgmt b
  UCS-B(local-mgmt)# cluster lead a

 2. from FI-A to FI-B
  UCS-A# connect local-mgmt b
  UCS-B(local-mgmt)# cluster lead b

Q. Modes in FI
Ans:
  adapter     Mezzanine Adapter
   cimc        Cisco Integrated Management Controller
   clp         Connect to DMTF CLP
   iom         IO Module
   local-mgmt  Connect to Local Management CLI
   nxos        Connect to NXOS CLI
 Scope     Connect to Chassis/Blade/IOM

Q: What is UCS Express?

Ans: UCS express is compact, all-in-one computing and networking system targeted at branch offices.
 UCS express is build on the concept of lean branch office which reduces the branch-office infrastructure footprint,
 equipment and operating costs, and management complexity.
Main components of UCS express are ISR Generation2 series router and Services Ready Engine (SRE) multipurpose x86 blade servers.
Cisco ISR G2 functions as a blade-server enclosure for SRE blade servers, which in themselves run branch office applications like DNS, DHCP, etc.
The system comes with VMware vSphere Hypervisor for virtualization and CIMC Express for blade server management.


Q. difference between expansion module and fixed module.
Ans: Fixed module are the ports which comes in the device
     Expansion Module are the ports which can be extended via expansion slots available on the device which may be FC or FCOE ports.

Q. what is UUID pool, UUID pool is for for blades
Ans: The UUID is a 128-bit number (32 hex digits, 16 groups of 2 hex digits). It is supposed to uniquely identify a component worldwide.
UUID pool UUID is a 128-bit number assigned to every compute node on a network to identify the compute node globally.

Q. Pre-requisites for upgrading FI
Ans:
 Review the Release Notes.
 Review the relevant Hardware and Software Interoperability Matrix to ensure the operating systems on all servers
 Back up the configuration into an All Configuration backup file.
 For a cluster configuration, verify that the high availability status of the fabric interconnects shows that both are up and running.
 For a standalone configuration, verify that the Overall Status of the fabric interconnect is Operable.
 Verify that the data path is up and running.
 Verify that all servers, I/O modules, and adapters are fully functional.
 Verify that the Cisco UCS domain does not include any critical or major faults.
 Verify that all servers have been discovered.

Q. What is callhomed and its use?
Ans: Call Home provides an email-based notification for critical system policies.
 The Call Home feature can deliver alert messages containing information about diagnostics and environmental faults and events.
 The Call Home feature can deliver alerts to multiple recipients, referred to as Call Home destination profiles.

Q. what is Service profile and its use
Ans: A service profile is a software definition of a server and its LAN and SAN network connectivity, in other words, a service profile defines a single server and its storage and networking characteristics. Service profiles are stored in the Cisco UCS 6100 Series Fabric Interconnects.

Q: What are UCS E-series servers?
Ans: UCS E-Series Server modules are next-generation, power-optimized, x86, Intel® Xeon® 64-bit blade servers designed to be deployed in Cisco Integrated Services Routers Generation 2 (ISR G2).

Q. What is a vPC and its use?
Ans: A virtual PortChannel (vPC) allows links that are physically connected to two different Cisco Nexus™ 5000 Series devices to appear as a single PortChannel to a third device. The third device can be a Cisco Nexus 2000 Series Fabric Extender or a switch, server, or any other networking device.

Q. How password reset can  be done on FI
Ans: If you have forgotten the password, you can set a new password. Connect to the console of UCS 6100 series fabric interconnect.
 Reload the fabric interconnect and when it boots up hit ctrl+L or ctrl+shift+R to go to loader prompt then boot the kickstart image;
 if you have two fabric interconnects connected in HA then first reload the subordinate fabric interconnect and bring it in loader prompt
 and then reload the primary and bring it in loader prompt, after this load kickstart image on primary.
 Configure the admin password using "admin-password" command by going to config terminal mode.
 Now load the system image on the fabric interconnect or if you have HA load system image on the primary fabric interconnect.
 The password is now set to the new password you have just configured; if you have HA then now you can load the kickstart and system images
 on the subordinate fabric interconnect.

Q: What is the maximum number of user sessions supported on UCS?
Ans: The maximum number of users that can be created on UCS is 48, this includes any kind of user.
 The maximum number of GUI sessions (accessing UCS manager via http) supported is 256.
 The maximum number of CLI sessions (telnet and SSH combined) supported is 32. Remember that CLI and GUI sessions are treated as separate;
 hence you can have at the max 256 GUI sessions and 32 CLI sessions at the same time.

Q. How to configure FI
Ans:
 Step 1    Connect to the console port. 
 Step 2    Power on the fabric interconnect. You will see the power on self-test messages as the fabric interconnect boots.
 Step 3    At the installation method prompt, enter gui. 
 Step 4    If the system cannot access a DHCP server, you are prompted to enter the following information:
  IP address for the management port on the fabric interconnect
  Subnet mask for the management port on the fabric interconnect
  IP address for the default gateway assigned to the fabric interconnect
 Step 5    Copy the web link from the prompt into a supported web browser and go to the Cisco UCS Manager GUI launch page. 
 Step 6    On the Cisco UCS Manager GUI launch page, select Express Setup. 
 Step 7    On the Express Setup page, select Initial Setup and click Submit. 
 Step 8    In the Cluster and Fabric Setup Area, select the Standalone Mode option. 
 Step 9    In the System Setup Area, enter the details of System Name field,Admin Password field,Confirm Admin Password field,Mgmt IP Address field,Mgmt IP Netmask field,Default Gateway field,DNS Server IP field,Domain Name field.
 Step 10   Click Submit.  

Q: From where to download OS patches and drivers?
Ans: You should download OS patches from the respective vendors. Drivers that are not part of OS and supplied by Cisco can be downloaded from cisco.com.

Q: Can I use a Nexus 5000 as a UCS 6100 fabric interconnect?
Ans: No, Nexus 5000 is a general purpose L2 switch and UCS 6100 series is an integral part of the unified computing system. The software that runs on the UCS 6100 is different from the Nexus 5000, as the UCS 6100 has both control and management planes.
 

126 Views
0 Comments

При анализе подключения Cisco UCS к сети важно понимать, каким образом пакеты передаются непосредственно между элементами UCS. По причине специфического поведения Cisco UCS Fabric Interconnect по сравнению с обычными коммутаторами и с Cisco Nexus поиск возможных ошибок конфигурации и сетевых неполадок будет несколько отличаться от стандартного набора процедур на классическом оборудовании уровня доступа.

Ключевые термины в рамках данной статьи

  1. Fabric Interconnect (FI) - коммутаторы, используемые одновременно как сетевые устройства при передаче LAN и SAN трафика, так и как средство управления сетевыми и серверными настройками Cisco UCS с помощью Cisco UCS Manager. Коммутаторы FI не обмениваются трафиком серверов напрямую - соединения между FI необходимы для синхронизации параметров Active и Subordinate процессов UCS Manager.
  2. I/O Module (IOM/FEX) - модули расширения, устанавливаемые в UCS 5108 chassis для обеспечения связности между сетевыми адаптерами серверов и FI. Два модуля IOM подключаются без пересечения к двум FI для создания двух независимых путей для прохождения трафика от сетевых адаптеров.
  3. vNIC - виртуальные сетевые адаптеры, презентуемые операционной системе, установленной на сервере. Количество данных адаптеров будет определяться типом физического сетевого адаптера в сервере и количеством физических соединений между модулями IOM и коммутаторами FI.
  4. vEth - виртуальный Ethernet интерфейс на FI, который создается для каждого vNIC  (на одном из двух FI, A или B, если не была выбрана опция Fabric Failover, и на обоих, если опция применена). Физические интерфейсы FI, обращенные в сторону модулей IOM выступают в качестве высокоскоростных каналов приема трафика множества карт vNIC. FI "передает" трафик на соответствующий vEth после прочтения метки VNTag, которая устанавливается во фреймы при передаче между IOM и FI.

Cisco UCS Fabric Interconnect являются связующим звеном между серверами и остальным сетевым оборудованием центра обработки данных, а также систем хранения данных в некоторых вариантах подключения. Однако, имея внешнее сходство с коммутаторами Cisco Nexus, FI по умолчанию передает данные между своими интерфейсами не как традиционный коммутатор. Такой режим работы получил название End-host mode (EHM), и его рекомендуется использовать в большинстве вариантов подключения Cisco UCS. В режиме EHM передача фреймов от сетевых адаптеров серверов через FI на исходящие интерфейсы идет путем процедуры, называемой пиннинг (Pinning). Каждому виртуальному сетевому адаптеру при создании назначается путь через один из двух FI (Fabric A или Fabric B). Трафик данного vNIC направляется на соответствующий выбранному пути модуль IOM, от которого по одному из физических каналов поток достигает FI.

Для пересылки фрейма на соответствующий Uplink интерфейс коммутатор FI обращается к политике пиннинга для карты vNIC, от которой данный трафик поступил. В правиле пиннинга либо задан конкретный интерфейс коммутатора FI (Static pinning), либо правило определено как динамическое. В последнем случае FI выбирает один из доступных Uplink интерфейсов, на котором разрешен к передаче соответствующий VLAN (Dynamic pinning). При осуществлении пересылки никаких изменений, за исключением изъятия VNTag, с фреймом не производится.

Вышестоящее оборудование, например, коммутатор, работающий в классическом режиме, воспринимает фреймы как поступающие непосредственно от сетевых адаптеров серверов и записывает их MAC-адреса в таблицу коммутации.

Ответный трафик в сторону конкретного vNIC должен вернуться на тот же интерфейс FI, через который политика пиннинга отправила оригинальные фреймы. Нарушившие это правило фреймы дропаются.

Преимущество подобного принципа пересылки фреймов состоит в том, что EHM исключает необходимость использования STP на участке от FI до вышестоящих коммутаторов, при этом такие механизмы, как Port-Channel, по-прежнему можно использовать. Так как FI всегда направляют трафик в строго определенные Uplink интерфейсы, а также благодаря реализованному на коммутаторах FI механизму устранения повторов и зацикливаний фреймов (Deja Vu check), появление петель коммутации исключается.

При необходимости FI может быть переведен в режим классической коммутации (Switching mode), но в таком случае становится обязательным использование STP для устранения возможных петель коммутации и правила пиннинга более не применяются.

 

Поиск пути прохождения трафика по MAC-адресу

В случае возникновения проблем с производительностью UCS или при поиске неполадок на уровне сети одной из частых задач на стадии сбора предварительной информации становится поиск пути, по которому трафик от конкретного MAC-адреса (vNIC) идет через модули IOM и коммутаторы FI на вышестоящее оборудование.
 

Предположим ситуацию, в которой необходимо установить местоположение (шасси и слот с сервером) MAC-адреса 00:11:22:33:44:55, а также путь, по которому трафик от данного MAC-адреса поступает на вышестоящее оборудование. Процедура разбивается на несколько этапов, последовательность которых может варьироваться.

Необходимо определить:

  1. FI, обслуживающий данный MAC-адрес (vNIC).

    Cluster# connect nxos a
    Cluster-A(nxos)# show mac-address address 0011.2233.4455
    VLAN          MAC Address                Type             Age         Port
    -------------+--------------------------+--------------+-------------+----------
    100           0011.2233.4455             dynamic        10            veth901

    Cluster# connect nxos b
    Cluster-B(nxos)# show mac-address address 0011.2233.4455
    Total MAC Addresses: 0

  2. Местонахождение сервера с данной картой vNIC (номер шасси и слот с сервером)

    Cluster-A(nxos)# show interface veth901
    vethernet901 is up
        Bound Interface is Ethernet1/1/1

    В данном выводе присутствует запись о неком интерфейсе Ehternet1/1/1, номер которого на самом деле указывает позицию сервера (первая цифра – номер шасси, третья – номер слота с нужным сервером).

  3. Uplink на FI, через который трафик этого vNIC покидает UCS.

    Cluster-A(nxos)# show pinning interface veth901

    SIF Interface        Sticky       Pinned Border interface
    --------------------+------------+--------------------------------
    veth901              No           Eth1/5

  4. Интерфейс FI, на который этот трафик поступает от модуля IOM.
    Cluster-A(nxos)# show fex <chassis#> detail
    ……
    ..
    Fex Port                    State             Fabric Port
    Eth1/1/1                    Up                 Eth1/11
    ………
  5. Uplink интерфейс IOM-а.
    Cluster-A(nxos)# show interface fex-fabric
               Fabric          Fabric             Fex
    Fex        Port            Port State         Uplink
    1          Eth1/11         Active             1
  6. Интерфейс модуля IOM в сторону карты vNIC. Опционально можно также вывести диаграмму подключения в GUI и CLI UCS Manager.

    Cluster# connect iom <chassis#>
    fex-1# show platform software woodside sts
    (для FI 62xx и IOM 220x)

    Вывод данной команды отобразит в CLI диаграмму портов IOM в зависимости от его модели с расположением серверов согласно их слотам в шасси и соответствующие им внутренние интерфейсы модуля IOM.

Отобразим на рисунке найденные интерфейсы

В следующей статье по UCS будут рассмотрены полезные команды для поиска неполадок на пути прохождения трафика и рекомендации по избеганию распространенных проблем.

---

Антон Погребняк
инструктор-консультант учебного центра Fast Lane в России

 

36 Views
0 Comments

При анализе подключения Cisco UCS к сети важно понимать, каким образом пакеты передаются непосредственно между элементами UCS. По причине специфического поведения Cisco UCS Fabric Interconnect по сравнению с обычными коммутаторами и с Cisco Nexus поиск возможных ошибок конфигурации и сетевых неполадок будет несколько отличаться от стандартного набора процедур на классическом оборудовании уровня доступа.

Ключевые термины в рамках данной статьи

  1. Fabric Interconnect (FI) - коммутаторы, используемые одновременно как сетевые устройства при передаче LAN и SAN трафика, так и как средство управления сетевыми и серверными настройками Cisco UCS с помощью Cisco UCS Manager. Коммутаторы FI не обмениваются трафиком серверов напрямую - соединения между FI необходимы для синхронизации параметров Active и Subordinate процессов UCS Manager.
  2. I/O Module (IOM/FEX) - модули расширения, устанавливаемые в UCS 5108 chassis для обеспечения связности между сетевыми адаптерами серверов и FI. Два модуля IOM подключаются без пересечения к двум FI для создания двух независимых путей для прохождения трафика от сетевых адаптеров.
  3. vNIC - виртуальные сетевые адаптеры, презентуемые операционной системе, установленной на сервере. Количество данных адаптеров будет определяться типом физического сетевого адаптера в сервере и количеством физических соединений между модулями IOM и коммутаторами FI.
  4. vEth - виртуальный Ethernet интерфейс на FI, который создается для каждого vNIC  (на одном из двух FI, A или B, если не была выбрана опция Fabric Failover, и на обоих, если опция применена). Физические интерфейсы FI, обращенные в сторону модулей IOM выступают в качестве высокоскоростных каналов приема трафика множества карт vNIC. FI "передает" трафик на соответствующий vEth после прочтения метки VNTag, которая устанавливается во фреймы при передаче между IOM и FI.

Cisco UCS Fabric Interconnect являются связующим звеном между серверами и остальным сетевым оборудованием центра обработки данных, а также систем хранения данных в некоторых вариантах подключения. Однако, имея внешнее сходство с коммутаторами Cisco Nexus, FI по умолчанию передает данные между своими интерфейсами не как традиционный коммутатор. Такой режим работы получил название End-host mode (EHM), и его рекомендуется использовать в большинстве вариантов подключения Cisco UCS. В режиме EHM передача фреймов от сетевых адаптеров серверов через FI на исходящие интерфейсы идет путем процедуры, называемой пиннинг (Pinning). Каждому виртуальному сетевому адаптеру при создании назначается путь через один из двух FI (Fabric A или Fabric B). Трафик данного vNIC направляется на соответствующий выбранному пути модуль IOM, от которого по одному из физических каналов поток достигает FI.

Для пересылки фрейма на соответствующий Uplink интерфейс коммутатор FI обращается к политике пиннинга для карты vNIC, от которой данный трафик поступил. В правиле пиннинга либо задан конкретный интерфейс коммутатора FI (Static pinning), либо правило определено как динамическое. В последнем случае FI выбирает один из доступных Uplink интерфейсов, на котором разрешен к передаче соответствующий VLAN (Dynamic pinning). При осуществлении пересылки никаких изменений, за исключением изъятия VNTag, с фреймом не производится.

Вышестоящее оборудование, например, коммутатор, работающий в классическом режиме, воспринимает фреймы как поступающие непосредственно от сетевых адаптеров серверов и записывает их MAC-адреса в таблицу коммутации.

Ответный трафик в сторону конкретного vNIC должен вернуться на тот же интерфейс FI, через который политика пиннинга отправила оригинальные фреймы. Нарушившие это правило фреймы дропаются.

Преимущество подобного принципа пересылки фреймов состоит в том, что EHM исключает необходимость использования STP на участке от FI до вышестоящих коммутаторов, при этом такие механизмы, как Port-Channel, по-прежнему можно использовать. Так как FI всегда направляют трафик в строго определенные Uplink интерфейсы, а также благодаря реализованному на коммутаторах FI механизму устранения повторов и зацикливаний фреймов (Deja Vu check), появление петель коммутации исключается.

При необходимости FI может быть переведен в режим классической коммутации (Switching mode), но в таком случае становится обязательным использование STP для устранения возможных петель коммутации и правила пиннинга более не применяются.

 

Поиск пути прохождения трафика по MAC-адресу

В случае возникновения проблем с производительностью UCS или при поиске неполадок на уровне сети одной из частых задач на стадии сбора предварительной информации становится поиск пути, по которому трафик от конкретного MAC-адреса (vNIC) идет через модули IOM и коммутаторы FI на вышестоящее оборудование.
 

Предположим ситуацию, в которой необходимо установить местоположение (шасси и слот с сервером) MAC-адреса 00:11:22:33:44:55, а также путь, по которому трафик от данного MAC-адреса поступает на вышестоящее оборудование. Процедура разбивается на несколько этапов, последовательность которых может варьироваться.

Необходимо определить:

  1. FI, обслуживающий данный MAC-адрес (vNIC).

    Cluster# connect nxos a
    Cluster-A(nxos)# show mac-address address 0011.2233.4455
    VLAN          MAC Address                Type             Age         Port
    -------------+--------------------------+--------------+-------------+----------
    100           0011.2233.4455             dynamic        10            veth901

    Cluster# connect nxos b
    Cluster-B(nxos)# show mac-address address 0011.2233.4455
    Total MAC Addresses: 0

  2. Местонахождение сервера с данной картой vNIC (номер шасси и слот с сервером)

    Cluster-A(nxos)# show interface veth901
    vethernet901 is up
        Bound Interface is Ethernet1/1/1

    В данном выводе присутствует запись о неком интерфейсе Ehternet1/1/1, номер которого на самом деле указывает позицию сервера (первая цифра – номер шасси, третья – номер слота с нужным сервером).

  3. Uplink на FI, через который трафик этого vNIC покидает UCS.

    Cluster-A(nxos)# show pinning interface veth901

    SIF Interface        Sticky       Pinned Border interface
    --------------------+------------+--------------------------------
    veth901              No           Eth1/5

  4. Интерфейс FI, на который этот трафик поступает от модуля IOM.
    Cluster-A(nxos)# show fex <chassis#> detail
    ……
    ..
    Fex Port                    State             Fabric Port
    Eth1/1/1                    Up                 Eth1/11
    ………
  5. Uplink интерфейс IOM-а.
    Cluster-A(nxos)# show interface fex-fabric
               Fabric          Fabric             Fex
    Fex        Port            Port State         Uplink
    1          Eth1/11         Active             1
  6. Интерфейс модуля IOM в сторону карты vNIC. Опционально можно также вывести диаграмму подключения в GUI и CLI UCS Manager.

    Cluster# connect iom <chassis#>
    fex-1# show platform software woodside sts
    (для FI 62xx и IOM 220x)

    Вывод данной команды отобразит в CLI диаграмму портов IOM в зависимости от его модели с расположением серверов согласно их слотам в шасси и соответствующие им внутренние интерфейсы модуля IOM.

Отобразим на рисунке найденные интерфейсы

В следующей статье по UCS будут рассмотрены полезные команды для поиска неполадок на пути прохождения трафика и рекомендации по избеганию распространенных проблем.

---

Антон Погребняк
инструктор-консультант учебного центра Fast Lane в России

 

664 Views
1 Comment

UCS - Re-numbering C-series integrated servers in UCSM when matching stale id numbers exist in UCS

 

Sometimes customers forget to decommission C-series integrated servers before doing an RMA. Many times this leads to an issue with stale duplicate ID numbers in UCS. For example, you have 3 rack servers numbered 1, 2, and 3 (C-series integrated) on UCSM. Server 3 has issues that needs an RMA to resolve. The FE or end customer doesn't decommission the server in UCSM and replaces it. When they integrate server 3 back into UCSM, it now shows up as server 4 instead of the id they wanted of server 3. This is because the act of yanking the server from UCSM without decommissioning left the id in the UCS db so server 3 is already allocated and will not be re-used. In my case, the customer had replaced the server 3 a couple times leading the server to ultimately show up as server 5 instead of 3. If you try to renumber the C-series using the normal means below in the link, it will say that id was already allocated and fail to save the change.

 

See the "Renumbering a Rack Mount Server" section on this doc:

 

http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/sw/gui/config/guide/2-2/b_UCSM_GUI_Configuration_Guide_2_2.pdf

 

If you see this error when renumbering the C-series integrated server on the decommissioned tab on UCSM, there is probably a stale entry on the UCS. You can also verify the current entries in the UCS db on an FI by running the below command.

 

UCS01-A# show config all | grep "recommission server"
 recommission server "Cisco Systems Inc" N20-B6625-1 FCH152372M8 N/A
 recommission server "Cisco Systems Inc" N20-B6625-1 FCH154571AR N/A
 recommission server "Cisco Systems Inc" N20-B6625-1 FCH154671LX N/A
 recommission server "Cisco Systems Inc" N20-B6625-1 FCH1545719M N/A
 recommission server "Cisco Systems Inc" N20-B6625-1 QCI1545A434 N/A
 recommission server "Cisco Systems Inc" UCSB-B200-M3 FCH164874ZW N/A
 recommission server "Cisco Systems Inc" UCSC-BASE-M2-C460 QCI1538A4ZH N/A
 recommission server "Cisco Systems Inc" UCSC-BASE-M2-C460 QCI1538A4ZT 3
 recommission server "Cisco Systems Inc" UCSC-BASE-M2-C460 QCI1543AAGP 1
 recommission server "Cisco Systems Inc" UCSC-BASE-M2-C460 QCI1543AAH4 4
 recommission server "Cisco Systems Inc" UCSC-BASE-M2-C460 QCI1641A73V 4
 recommission server "Cisco Systems Inc" UCSC-BASE-M2-C460 QCI1722A0HM 2

 

Here you can clearly see the 3 id entry as well as two 4 entries. This is why, after replacing the C-series and recommissioning it on UCSM, it shows up as 5 (3 and 4 already exist as stale entries that aren't associated with any real servers).


Typically the only way around this is to get a escalation open and get a DEV to help you delete the entry out of the FI. If you don't have the time to do all that, you can do the following work around. You can just make the existing 3 and 4 entries something really high up that they will never hit to free up the id you want. In my case we changed id 3, and the double id 4 entries to 100-102. Here is how you accomplish this.

 

recommission server "Cisco Systems Inc" UCSC-BASE-M2-C460 QCI1538A4ZT 100

recommission server "Cisco Systems Inc" UCSC-BASE-M2-C460 QCI1543AAH4 101
recommission server "Cisco Systems Inc" UCSC-BASE-M2-C460 QCI1641A73V 102

 

This then freed up id 3 for the existing server 5 so I could renumber it in the UCSM. So, once we had this all changed, I was able to go into the UCSM GUI per the doc above and go to the decommission tab, select 5, change it to 3, check the recommission box, click save changes and it took it. The server came up with id of 3. Now the output shows the proper id's with the proper servers.

 

UCS01-A# show config all | grep "recommission server"
recommission server "Cisco Systems Inc" N20-B6625-1 FCH152372M8 N/A
recommission server "Cisco Systems Inc" N20-B6625-1 FCH154571AR N/A
recommission server "Cisco Systems Inc" N20-B6625-1 FCH154671LX N/A
recommission server "Cisco Systems Inc" N20-B6625-1 FCH1545719M N/A
recommission server "Cisco Systems Inc" N20-B6625-1 QCI1545A434 N/A
recommission server "Cisco Systems Inc" UCSB-B200-M3 FCH164874ZW N/A
recommission server "Cisco Systems Inc" UCSC-BASE-M2-C460 QCI1538A4ZH N/A
recommission server "Cisco Systems Inc" UCSC-BASE-M2-C460 QCI1538A4ZT 100
recommission server "Cisco Systems Inc" UCSC-BASE-M2-C460 QCI1543AAGP 1
recommission server "Cisco Systems Inc" UCSC-BASE-M2-C460 QCI1543AAH4 101
recommission server "Cisco Systems Inc" UCSC-BASE-M2-C460 QCI1641A73V 102
recommission server "Cisco Systems Inc" UCSC-BASE-M2-C460 QCI1722A0HM 2
recommission server "Cisco Systems Inc" UCSC-BASE-M2-C460 QCI1743A25L 3

 

This helped get the customer where they needed to be and also avoid an escalation that wasn't necessarily needed. The customer will never hit the 100-102 id numbers.

220 Views
0 Comments

Cisco Unified Computing System (UCS) Mini Architecture, Troubleshooting and Upgrade
(Live Webcast May 12, 2015 at 10:00 am Pacific Time / 1:00 pm Eastern Time)

Please join us on Tuesday, May 12, 2015 at 10 a.m. PDT San Francisco (UTC -7 hours) for this live, interactive event. This corresponds to 1pm EDT New York (UTC -4 hours), 6 p.m. London (BST UTC +1.

The Cisco Unified Computing System™ is a next-generation data center platform that unites computing, networking, storage access, and virtualization resources into a cohesive system designed to reduce total cost of ownership (TCO) and increase business agility.

The system integrates a low-latency, lossless 10 Gigabit Ethernet unified network fabric with enterprise-class, x86-architecture servers. The system is an integrated, scalable, multichassis platform in which all resources participate in a unified management domain.

In the upcoming session we will cover the UCS Mini architecture, troubleshooting and upgrade procedure.

Featured Speakers

Cisco Experts

Avinash Shukla is a customer support engineer from the Server Virtualization team at the Cisco Technical Assistance Center in Bangalore, India.  He has more than 7 years of total experience. He has worked on a wide range of Cisco data center products such as Cisco UCS, Cisco Nexus 1000V, Cisco Invicta and Cisco Collaboration technology. He also holds a CCIE in Data Center & Voice


Cisco ExpertManas Kumar Nath is a customer support engineer from the Server Virtualization team at the Cisco Technical Assistance Center in Bangalore, India. He has more than 9 years of experience. He has worked on a wide range of Cisco data center products such as Cisco UCS, Cisco Nexus 1000V, Nexus 1110. He also holds VCP5 and RHCE certifications.



Cisco ExpertVignesh Kumar is a customer support engineer from the Server Virtualization team at the Cisco Technical Assistance Center in Bangalore, India. He has more than 9 years of experience. He has worked on a wide range of Cisco data center products such as Cisco UCS, Cisco Nexus 1000V and Cisco Collaboration technology. He also holds a CCIE in Data Center & Voice.

Need more information? Have more questions? Find more experts and topics by visiting https://supportforums.cisco.com/expert-corner/events

48 Views
0 Comments

Transforming Cisco switches into highly scalable zero latency load balancers! Cisco Intelligent Traffic Director (ITD) does exactly that and much more! Read More

Cisco ITD  has been selected as Best of Interop 2015 finalist in the Data Center Category. Finalists were chosen from nine categories including Data Center, Mobility, SDN and Cloud Technologies.

BOI15_Finalist_728x90-550x68
Cisco Intelligent Traffic Director (ITD)  is an innovative solution to bridge the performance gap between a multi-terabit switch and gigabit servers and appliances. It is a hardware based multi-terabit layer 4 load-balancing, traffic steering and clustering solution on the Nexus 9K,7K,6K,5K series of switches. Customers do not have to buy any new hardware. It works on all the existing hardware, linecards and supervisors.