Complete System Lockup

philbenton_ironport · ‎10-03-2008

Hello,

I've had the benefit of using Ironport servers for years and we recently decommissioned our C60/600 servers and replaced them with C350s. Just today I noticed one of our C350s was non-responsive and one of my guys at our datacenter checked the server out physically. The LCD panel was blue and no error message, and the NICs both had green lights, but I couldn't connect to the machine on either the public or private IPs.

The machine was rebooted, and I checked the system logs, error logs and status logs and found nothing but a large "gap" between the time that the server froze and the time it was rebooted.

I've opened a ticket with Ironport, but figured I'd see if I could get some input from my fellow admins here on the board.

Has anyone come across this happening with their c350 appliance?

Here's some detail on the system itself:

Current Version
===============
Model: C350
Version: 5.5.1-015
Build Date: 2008-04-23
Install Date: 2008-07-16 13:40:09
Serial #: XXXXXXXXXXXX-XXXXXXX
BIOS: 2.0.4I
RAID: 1.00.01-0088, MT23, 1.02-007
RAID Status: Optimal
RAID Type: 1
BMC: 1.27

Any input is appreciated.

-Phil

Douglas Hardison · ‎10-09-2008

Hi,

In this particular case, opening a ticket is the best option. There's not much that will lock up an IronPort :)

The customer support engineer will likely want access to logs and possibly the unit itself, but there's a very high like likelihood that they will be able to determine the root cause of the lockup.

-whardison

david.shoesmith · ‎10-10-2008

Hi,
Did you have the Console cable connected at that time? I always find having the Console Cable connected and watch the boot sequence through a telnet session is a good way of picking up where the Ironport hangs.
I had a problem where I had a USB KVM connected to the Ironport and it would not boot. Once I watched the boot sequence I could see it hanging at a particular point in the *Linux load screen. (Exact point escapes me right now)
Once I removed the USB KVM it would boot fine. I don't think it likes having a USB device connected to it :/
Might be too late after the fact though :)

Just a thought,

Bogor

meyd45_ironport · ‎10-10-2008

If the kernel panics the IronPort is still pingable but you will get connection refused from server ports (e.g. 22, 25, 443). No option other than to power cycle.

meyd45_ironport · ‎10-10-2008

Re-reading your post. You are running an older version. There is/was a bug where the kernel could get wedged - usually on busy machines.

The machines would be pingable, TCP sessions would connect, but you would not get any banners from the daemon on the port.

Again no option other than to power cycle.