HI All, we have a Cisco 3750 stack with 5 switches.
It is made up of 2 x 3750 48 non POE (WS-3750-48TS) and 3 x 3750 POE with 4 SFP (WS-3750-48P). One of the POE is the master. We keep getting the error message (from the console) %ILPOWER-3-CONTROLLER_ERR Controller number 1 through 11, accessing failed. The master switch can not then issue power to any POE devices,the other two can continue to offer ILPOWER. I have rebooted the stack and it resolves the issue temporarily. On stack we are running 12.2(25)SEE3(c3750-ipbase-mz-122.25.SEE3.bin). Does anyone have any ideas please. I looked on the internet and the Cisco offering is to issue SHUT and NO SHUT command to the affected infterfaces, but this did not help.
From your rather detailed description I get the impression that you have been struggeling with this for a while already. I have also checked the error message and it seems to me this is an IOS problem.
On the download center I noticed that 12.2(25)SEE4(ED) is available and in my opinion you should try to upgrade the switches to this release.
we are installing a VoIP phone system and I have spent ages trying to fix it. I have restarted the stack again and for the last 4 hours there have been no problems at all, power can be issued on any inline power port (the interface settings are set to Auto for inline power). I think that for the moment everything is ok and I will just get everything up and running and look at upgrading the IOS next weekend, I am correct in thinking I can just use the Network Assistant to perform the upgrade?
What are you thoughts!! Thanks for the reply already,
Personally I prefer the command line for upgrading. Actually, I have always avoided using any GUI for configuring Cisco devices. Once you have found your way around in the CLI it all becomes easier because you have nearly the same syntax on all devices.
So I am unable to tell you if it is possible to use NetAss for this as I have no experience with it.
Please consult CCO about the upgrade procedure as you will have to upgrade the entire stack. No big deal, you only need to supply the image once but you have to follow the correct steps.
Ok, best to know the recommended route, I had to manually xmodem the IOS on one of the switches as it had become corrupt during the stack upgrade! i feel quite happy using the CLI but when upgrading the IOS on the entire stack i am a bit cautious!! Do you think the stack will continue to issue inline power know since I have had no problems since restarting the stack?i guess the IOS could have gone a bit wrong during boot!!
Appearently this conviction is based on inside information? I was not 100% sure that it was a hardware problem although the possibility has crossed my mind.
Is there a bugID for this one?
As the error can be resolved by reloading, an IOS issue looks just as likely and should be adressed first.
I initially thought it was a hardware problem, however it only occurs on the master switch in the stack, the other 2 x poe switches work ok, I rebooted the stack and powered on a different POE switch first to make it the master, this had the same problem, so either I have 3 x faulty POE switches or it might be an IOS bug? However at the moment everything is working fine,
I don't understand your reasoning, the controller number does not change after a reboot or by selecting another master.
Are you always getting the same error with the same controller number ? If so, it's pointing to the same switch regardless its status.
How often the problem occurs ?
Can you run on 2 switches (isolating the faulty switch) and see if you can reproduce the error ?
Often the customer would update the IOS and the problem goes away for few days and then return. It's not a persistent but a chronic problem.
If the OP wants to upgrade the IOS and see how it goes, it's fine with me. However, I'm not guaranteeing the problem will go away.
I rebuilt the stack without the switch that initially had the problem. The new master switch had the same problem. It is very strange. Do you think it could be a faulty batch of switches then? it just seems unlikely that there would be multiple hardware failures, especially when they have all issued POE individually (i.e. when stacked and not stacked).
Upgrade the IOS and let us know if the problem goes away. If the problem remains, do what I suggested before.
If they are all faulty, the cause could've been related to your environment such as:
or temperature condition.
It may as well be a bad batch....
Ok thanks fr your help, I will look at upgrading the IOS, what version would you think i should use? My company makes UPS and we have redundant PDU's, Cross Switch and dual UPS with generator backup so hopefully was not a power failure that has caused the issue!! Everything is still fine at the moment so fingers crossed until I can upgrade the IOS.
12.2.25-SEE4 would be the next logical path. Keep in mind, I haven't had the chance to formally do a bug scrub. If you want to be 100% sure this IOS is safe for your environment, I recommend opening a TAC case.
I have also since restarting the stack (and having no problems) enabled the debug ilpower event and debug ippower controller and cannot see any problem. Maybe this is a config problem, but all ports are set to auto for ilpower.