Call Manager 6.1 network issues

Answered Question
Aug 12th, 2010

I have an issue with my network continuously dropping calls or phones freezing.  Its states CM down Features Disabled.  We lose calls, phones are freezing intermittently.

Here's the hardware setup:  MCS-7828-H3

Platform Type 7828H3

Serial MX282400CM

Active OS 6.1.5.10000-10

RTMT does show phones unregistering, but I am not seeing any devices lose connectivity except for some minimal packet loss to the Call Manager.  No other errors reported in syslog for any Cisco device For example, pings sent to all major network devices have 0 packets lost over the weekend, but 250 out of 20,000 pings to the Call Manager server are lost.  I have replaced cabling, patch cords and moved switch ports with no resolution.

CPU has never gone above 24%

Virtual Memory never above 30%

Active and Inactive partitions are at 15% disk space.

We have a 2nd NIC on this server but i'm unable to configure this 2nd NIC as a standalone connection for the Call Manager server, only as a failover for the 1st NIC.  Does anyone know how to configure this 2nd NIC to host all IP services for these servers?  Is there a way to replace the NIC on these servers (not onsite and haven't opened this box up yet) or do I have to replace the entire server?

Anyone have any other suggestions on where to take my troubleshooting?

Thanks for any and all advice!

Steve

I have this problem too.
0 votes
Correct Answer by Justin Brenton about 6 years 3 months ago

Please see below, Your right in thinking this could be a defective nic, cable,  I would also suggest looking into your STP configuration.

Error Message   RTD-1-ADDR_FLAP [chars] relearning [dec] addrs per min

Explanation   Normally,  MAC addresses are learned once on a port. Occasionally, when a switched  network reconfigures, due to either manual or STP reconfiguration,  addresses learned on one port are relearned on a different port.  However, if there is a port anywhere in the switched domain that is looped back to itself, addresses will jump back and forth  between the real port and the port that is in the path to the looped  back port. In this message, [chars] is the interface, and [dec] is the  number of addresses being learnt.

Recommended Action   Determine  the real path (port) to the MAC address. Use the debug  ethernet-controller addr privileged EXEC command to see the alternate  path-port on which the address is being learned. Go to the switch  attached to that port. Note that the show cdp neighbors command is  useful in determining the next switch. Repeat this procedure until the  port is found that is receiving what it is transmitting, and remove that  port from the network.

Error Message   RTD-1-LINK_FLAP [chars] link down/up [dec] times per min

Explanation   This  message means that an excessive number of link down-up events has been  noticed on this interface: [chars] is the interface, and [dec] is the  number of times the link goes up and down. This might be the result of  reconfiguring the port, or it might mean a faulty device at the other  end of the connection.

Recommended Action   If someone is  reconfiguring the interface or device at the other side of the  interface, ignore this message. However, if no one is manipulating the  interface or device at the other end of the interface, it is likely that  the Ethernet transceiver at one end of the link is faulty and should be  replaced.

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (2 ratings)
Loading.
Paul Reck Thu, 08/12/2010 - 22:17

Hi Steve,

the pings to the CUCM server aren't necessarily the best test here are the servers are designed to drop a certain number of ICMP packets as a way of limiting DoS attacks, so you may be seeing that here.

To use the 2nd NIC you could just pull the cable from the 1st NIC and let failover take place.  Then see if the problem persists. 

Do you have a security device, (ASA, FWSM, other firewall) between the phones and the CUCM server?  You may need to check what ip inspection rules are in place.

are there any errors showing on the switch interfaces for the CUCM server (I'm assuming it's Business Edition from the platform).

regards,

Paul

Steve Davis Fri, 08/13/2010 - 09:13

This is all on the same LAN, so there isn't a firewall between the phones and CM or the CM and voice gateway which is a 2811 router.

No errors on the switch interface the CM is connected to.  I tried the failover but didn't seem to work properly.  This is just enabled by entering, "set network failover enable" correct?  After that I pulled the NIC and waited a few minutes and connectivity was not restored.

The only alert registered in RTMT is Number of registered phones in the cluster drops 10 Percent. Current monitored precanned object has decreased by 15 percent.

I've been through the entire network and cannot find any device dropping or trunks that have gone down...

Steve Davis

Steve Davis Mon, 08/16/2010 - 09:55

In addition, on the sys logs the switch the Call Manager server is attached, I am seeing the following error in the syslog.

ADDR_FLAP: FastEthernet0/22 relearning 5 addrs per min.

Seeing this error consistently. Like I mentioned previously I have replaced switchports, cable and patch cords, essentially everything but the NIC on the server.

The port mentioned above is the port the Call Manager is connected to. There aren't any redundant links configured and spanning tree is enabled and working properly.

Any suggestions?

thanks

Steve Davis

Correct Answer
Justin Brenton Mon, 08/16/2010 - 11:36

Please see below, Your right in thinking this could be a defective nic, cable,  I would also suggest looking into your STP configuration.

Error Message   RTD-1-ADDR_FLAP [chars] relearning [dec] addrs per min

Explanation   Normally,  MAC addresses are learned once on a port. Occasionally, when a switched  network reconfigures, due to either manual or STP reconfiguration,  addresses learned on one port are relearned on a different port.  However, if there is a port anywhere in the switched domain that is looped back to itself, addresses will jump back and forth  between the real port and the port that is in the path to the looped  back port. In this message, [chars] is the interface, and [dec] is the  number of addresses being learnt.

Recommended Action   Determine  the real path (port) to the MAC address. Use the debug  ethernet-controller addr privileged EXEC command to see the alternate  path-port on which the address is being learned. Go to the switch  attached to that port. Note that the show cdp neighbors command is  useful in determining the next switch. Repeat this procedure until the  port is found that is receiving what it is transmitting, and remove that  port from the network.

Error Message   RTD-1-LINK_FLAP [chars] link down/up [dec] times per min

Explanation   This  message means that an excessive number of link down-up events has been  noticed on this interface: [chars] is the interface, and [dec] is the  number of times the link goes up and down. This might be the result of  reconfiguring the port, or it might mean a faulty device at the other  end of the connection.

Recommended Action   If someone is  reconfiguring the interface or device at the other side of the  interface, ignore this message. However, if no one is manipulating the  interface or device at the other end of the interface, it is likely that  the Ethernet transceiver at one end of the link is faulty and should be  replaced.

Actions

This Discussion