Solved: Call Manager 6.1 network issues

Steve Davis · ‎08-12-2010

I have an issue with my network continuously dropping calls or phones freezing. Its states CM down Features Disabled. We lose calls, phones are freezing intermittently.

Here's the hardware setup: MCS-7828-H3

Platform Type 7828H3

Serial MX282400CM

Active OS 6.1.5.10000-10

RTMT does show phones unregistering, but I am not seeing any devices lose connectivity except for some minimal packet loss to the Call Manager. No other errors reported in syslog for any Cisco device For example, pings sent to all major network devices have 0 packets lost over the weekend, but 250 out of 20,000 pings to the Call Manager server are lost. I have replaced cabling, patch cords and moved switch ports with no resolution.

CPU has never gone above 24%

Virtual Memory never above 30%

Active and Inactive partitions are at 15% disk space.

We have a 2nd NIC on this server but i'm unable to configure this 2nd NIC as a standalone connection for the Call Manager server, only as a failover for the 1st NIC. Does anyone know how to configure this 2nd NIC to host all IP services for these servers? Is there a way to replace the NIC on these servers (not onsite and haven't opened this box up yet) or do I have to replace the entire server?

Anyone have any other suggestions on where to take my troubleshooting?

Thanks for any and all advice!

Steve

Justin Brenton · ‎08-16-2010

Please see below, Your right in thinking this could be a defective nic, cable, I would also suggest looking into your STP configuration.

Error Message RTD-1-ADDR_FLAP [chars] relearning [dec] addrs per min

Explanation Normally, MAC addresses are learned once on a port. Occasionally, when a switched network reconfigures, due to either manual or STP reconfiguration, addresses learned on one port are relearned on a different port. However, if there is a port anywhere in the switched domain that is looped back to itself, addresses will jump back and forth between the real port and the port that is in the path to the looped back port. In this message, [chars] is the interface, and [dec] is the number of addresses being learnt.

Recommended Action Determine the real path (port) to the MAC address. Use the debug ethernet-controller addr privileged EXEC command to see the alternate path-port on which the address is being learned. Go to the switch attached to that port. Note that the show cdp neighbors command is useful in determining the next switch. Repeat this procedure until the port is found that is receiving what it is transmitting, and remove that port from the network.

Error Message RTD-1-LINK_FLAP [chars] link down/up [dec] times per min

Explanation This message means that an excessive number of link down-up events has been noticed on this interface: [chars] is the interface, and [dec] is the number of times the link goes up and down. This might be the result of reconfiguring the port, or it might mean a faulty device at the other end of the connection.

Recommended Action If someone is reconfiguring the interface or device at the other side of the interface, ignore this message. However, if no one is manipulating the interface or device at the other end of the interface, it is likely that the Ethernet transceiver at one end of the link is faulty and should be replaced.

View solution in original post

Paul Reck · ‎08-12-2010

Hi Steve,

the pings to the CUCM server aren't necessarily the best test here are the servers are designed to drop a certain number of ICMP packets as a way of limiting DoS attacks, so you may be seeing that here.

To use the 2nd NIC you could just pull the cable from the 1st NIC and let failover take place. Then see if the problem persists.

Do you have a security device, (ASA, FWSM, other firewall) between the phones and the CUCM server? You may need to check what ip inspection rules are in place.

are there any errors showing on the switch interfaces for the CUCM server (I'm assuming it's Business Edition from the platform).

regards,

Paul

Steve Davis · ‎08-13-2010

This is all on the same LAN, so there isn't a firewall between the phones and CM or the CM and voice gateway which is a 2811 router.

No errors on the switch interface the CM is connected to. I tried the failover but didn't seem to work properly. This is just enabled by entering, "set network failover enable" correct? After that I pulled the NIC and waited a few minutes and connectivity was not restored.

The only alert registered in RTMT is Number of registered phones in the cluster drops 10 Percent. Current monitored precanned object has decreased by 15 percent.

I've been through the entire network and cannot find any device dropping or trunks that have gone down...

Steve Davis

Steve Davis · ‎08-16-2010

In addition, on the sys logs the switch the Call Manager server is attached, I am seeing the following error in the syslog.

ADDR_FLAP: FastEthernet0/22 relearning 5 addrs per min.

Seeing this error consistently. Like I mentioned previously I have replaced switchports, cable and patch cords, essentially everything but the NIC on the server.

The port mentioned above is the port the Call Manager is connected to. There aren't any redundant links configured and spanning tree is enabled and working properly.

Any suggestions?

thanks

Steve Davis

Justin Brenton · ‎08-16-2010

Please see below, Your right in thinking this could be a defective nic, cable, I would also suggest looking into your STP configuration.

Error Message RTD-1-ADDR_FLAP [chars] relearning [dec] addrs per min

Explanation Normally, MAC addresses are learned once on a port. Occasionally, when a switched network reconfigures, due to either manual or STP reconfiguration, addresses learned on one port are relearned on a different port. However, if there is a port anywhere in the switched domain that is looped back to itself, addresses will jump back and forth between the real port and the port that is in the path to the looped back port. In this message, [chars] is the interface, and [dec] is the number of addresses being learnt.

Recommended Action Determine the real path (port) to the MAC address. Use the debug ethernet-controller addr privileged EXEC command to see the alternate path-port on which the address is being learned. Go to the switch attached to that port. Note that the show cdp neighbors command is useful in determining the next switch. Repeat this procedure until the port is found that is receiving what it is transmitting, and remove that port from the network.

Error Message RTD-1-LINK_FLAP [chars] link down/up [dec] times per min

Explanation This message means that an excessive number of link down-up events has been noticed on this interface: [chars] is the interface, and [dec] is the number of times the link goes up and down. This might be the result of reconfiguring the port, or it might mean a faulty device at the other end of the connection.

Recommended Action If someone is reconfiguring the interface or device at the other side of the interface, ignore this message. However, if no one is manipulating the interface or device at the other end of the interface, it is likely that the Ethernet transceiver at one end of the link is faulty and should be replaced.