Hi, I have redundant 11503s (sg0730106 standard feature) configured with VIP redundancy, accepting incoming SSL requests, then talking to backend ssl servers. The CSS are deployed in a "one-armed" network - packets go in and out the same interface. The two CSS are connected to two different switches, which are connected to the network backbone. The servers are connected to a third switch and must also talk across the backbone...CSS VIP and servers are in the same broadcast domain.
The CSS keepalives (type ssl, port 443) to the servers are regularly failing, and i am seeing a lot of state transitions.
CSS02# sho service summary
Service Name State Conn Weight Avg State
quantum_ssl01 Alive 3 1 2 60
quantum_ssl02 Alive 0 1 2 54
quantum_ssl_client Alive 6 1 2 0
This coincides with a spike in CPU. The problem is that at this time, i am able to connect to the servers directly via ping and ssl without any issues.
CSS02# sho sys cpu
Chassis CPU Utilizations
Name Slot Sub CPU%
CSS5-SCM-2GE F0 1 1 91%
CSS5-SSL-K9 D0 2 1 0%
The cpu on my ssl module rarely moves off 0%. I getting very slow ping respone times from the CSS VIP and interface when these cpu spike occur.
Output from the CPU HOG command shows top talkers (fmapmsg...
Checking CPU Hog
TID Name Milliseconds
--- ---- ------------
0x8de4e330 OndmLTickTxTask 0
0x8e10ef40 tDcacheUpd 0
0x8a82edf0 fmapmsg 63
0x8dfea5e0 tImmRx 1
0x8dfe5350 ImmGetAgent 0
All connected Switch ports look fine....we are not servicing many connections at all.
Any advice/help would be greatly appreciated. Is my CSS and server deployment not localised enough?
Thanks for the reply Giles. The CSSs's and the servers exist in a large network with a /16 subnet. The output of CPU HOG is attached when problem occurs:
I changed the backup CSS to send ICMP Keepalives. These also fail at the same time as the SSL KA on the primary.
The second CSS displays almost identical CPU HOG output.
The other point is that I am getting a lot of state changes on my secondary CSS virtual router. I am not preempting. The master CSS reports no failure and no state change. The backup CSS reports many state changes, 2 at a time, and remains the backup. Failure reason Preempted. These state changes also coincide with the high cpu and state changes to the services.
Introduction This article will help you understand the steps on how to
download the UCS licenses from the Cisco Systems website and then
installing it on the UCS. The redacted (blue lines) just covers up
certain numbers for privacy please do not take them...
Introduction This article will help you understand and educate the
customer on how to clear their "expired licenses"
(license-graceperiod-expired) from their UCS-M. If a customer just
purchased a license and needs a step by step guide on how to download
==================== VIC FNIC driver does not support Virtual Volumes (
second level LUN ID ) An enhancement request has been created to track
this feature - CSCux64473 UPDATE - 12-14-2016 We made some traction on
the enhancement request - The Fix is in t...