Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements
Webcast-Catalyst9k
New Member

Unable to ping until arp cache cleared

Hello Experts

I have 3 servers connected to Cisco Catalyst Switch  C2960 and this switch has uplink to one of the access switch and  ultimately this access switch connected to our 2 Core Switch

We are running HSRP and core switches has direct link between them.

Today  I encountered an issue these server are unreachable from vlans other  that its own.  I just cleared arp-cache and it started pinging.

Please can you help since this happening repeadetly.

Thanks

94 REPLIES

Unable to ping until arp cache cleared

Can you verify that your topology for this issue looks like this?

(3 Servers)<--->(2960)<--->(Access Switch)<---->(the two core switches)

How did you discover that you were unable to ping the servers from outside their local vlan?

New Member

Unable to ping until arp cache cleared

They medical application server and the users from other vlan suddenly were unable to access the application.

The situation is the issue repeatedly appearing and make it stable I kept continous ping from worksation ( other vlan) to these servers, once stop the ping the issue reoccur after 2-5 min.

Hall of Fame Super Blue

Unable to ping until arp cache cleared

What are your arp tables on the L3 switch and your mac address tables on each switch looking like ?

For the mac addresses you need to check each switch.

When you cannot ping the servers from a client can the servers ping  -

1) their default gateway

2) another L3 SVI IP address on the L3 switch

do you have any non standard configuration anywhere eg. port security/DAI/acls etc.

Jon

New Member

Unable to ping until arp cache cleared

Thanks for your help.

The servers cannot ping their default gateway i.e HSRP virutal IP but they can ping the L3 SVI IP address.

I have stopped continous ping to servers the issue should reappear again then I can note the mac addresses but I guess they were look like on access switch and core switches.

Hall of Fame Super Blue

Unable to ping until arp cache cleared

The servers cannot ping their default gateway i.e HSRP virutal IP but they can ping the L3 SVI IP address

That could well be the issue. Can you answer the following -

1) What is "show standby brief" showing on the core switches for that vlan ?

2) how is the access switch connected to the core switches ie. is it as John suggested -

servers -> 2960 -> access switch -> core switches

if so does the access switch connect to both core switches ?

3) is it all devices in the server vlan that cannot ping the VIP ?

4) if you look at the arp table on a server that doesn't work is there an entry for the HSRP VIP ?

we need to understand the topology and switch interconnects to be able to help.

Jon

New Member

Unable to ping until arp cache cleared

The problem has appeared again so I'll answer all possible question

1. What is "show standby brief" showing on the core switches for that vlan ?

Core1

Vl2         2   110  P Active   local           10.1.1.253      10.1.1.1

Core2

Vl2         2   95   P Standby  10.1.1.254      local           10.1.1.1

2. I have already posted on the above comment.

3.is it all devices in the server vlan that cannot ping the VIP ?

No. only these three servers

4.if you look at the arp table on a server that doesn't work is there an entry for the HSRP VIP ?

I can see arp entry on Core 1 but not on core 2

5.Are there any devices on the 2960 that are working ?

Yes there are other 4 servers connected and working

Hall of Fame Super Blue

Unable to ping until arp cache cleared

Can you also answer all the questions being asked eg. you were asked by John if that was the correct topology but we never got an answer.

Are there any redundant paths between switches or is it simply as John has drawn ?

Jon

New Member

Unable to ping until arp cache cleared

Hi,

This is the topology.

(3 Servers)<--->(2960)<--->(Access Switch)

                                          |               |  

                                          |               |

                                     Core 1         Core2

Actually 2960 is not our administration it is working as L2 and its trunked

Access Switch # sh int gi1/0/44 ( port connected to 2960

Operation_Room_SW_1#sh int gi1/0/44 switchport

Name: Gi1/0/44

Switchport: Enabled

Administrative Mode: dynamic auto

Operational Mode: trunk

Administrative Trunking Encapsulation: negotiate

Operational Trunking Encapsulation: dot1q

Negotiation of Trunking: On

Access Mode VLAN: 2

Trunking Native Mode VLAN: 1 (default)

Administrative Native VLAN tagging: enabled

Voice VLAN: none

Administrative private-vlan host-association: none

Administrative private-vlan mapping: none

Administrative private-vlan trunk native VLAN: none

Administrative private-vlan trunk Native VLAN tagging: enabled

Administrative private-vlan trunk encapsulation: dot1q

Administrative private-vlan trunk normal VLANs: none

Administrative private-vlan trunk associations: none

Administrative private-vlan trunk mappings: none

Operational private-vlan: none

Trunking VLANs Enabled: ALL

Pruning VLANs Enabled: 2-1001

Capture Mode Disabled

Capture VLANs Allowed: ALL

Protected: false

Unknown unicast blocked: disabled

Unknown multicast blocked: disabled

Appliance trust: none

Hall of Fame Super Blue

Unable to ping until arp cache cleared

Thanks for that.

Can you answer the other questions as well. We may well need other outputs depending on the answers.

Are there any devices on the 2960 that are working ?

Jon

New Member

Unable to ping until arp cache cleared

The problem has appeared again so I'll answer all possible question

1. What is "show standby brief" showing on the core switches for that vlan ?

Core1

Vl2         2   110  P Active   local           10.1.1.253      10.1.1.1

Core2

Vl2         2   95   P Standby  10.1.1.254      local           10.1.1.1

2. I have already posted on the above comment.

3.is it all devices in the server vlan that cannot ping the VIP ?

No. only these three servers

4.if you look at the arp table on a server that doesn't work is there an entry for the HSRP VIP ?

I can see arp entry on Core 1 but not on core 2

5.Are there any devices on the 2960 that are working ?

Yes there are other 4 servers connected and working

Unable to ping until arp cache cleared

The servers cannot ping their default gateway i.e HSRP virutal IP but they can ping the L3 SVI IP address

Can you paste the configuration of your HSRP for this vlan?

Also, since you can't ping the HSRP VIP, can you ping the actual physical IP Addresses of the routed ports on each router?

For example, I have network 192.168.1.0/24, and R1 has 192.168.1.2 and R2 has 192.168.1.3, you normally set your VIP to .3, In this vlan can you ping .2 and or .3?

Hall of Fame Super Blue

Unable to ping until arp cache cleared

Sorry to be bombarding you with questions but you said you cleared the arp cache. Was this on the servers or the core switches ?

Jon

New Member

Unable to ping until arp cache cleared

On core switches I have done anything on server since I don't have an access to it.

Hall of Fame Super Blue

Unable to ping until arp cache cleared

So you cannot get onto the servers in question ?

If not -

1) are the non working servers in the same vlan as the working servers ?

2) how do you know the servers can't ping the VIP, is someone else doing that for you ? If so can they do an "arp -a" on the server and look for the VIP entry ?

3) can you pick one of the non working servers and trace it's mac address from the core switch ie. on each switch up to the 2960 can you look in the mac address tables and make sure the mac for that server is going out of the right port.

Jon

Unable to ping until arp cache cleared

To go with Jon Marshall's recent post you can do that by doing the following.

1. Ping the IP Address

2. 'show arp | i IP-Address

3. 'show mac-address table | i IP-Address' *** Command for show mac address table may vary on the switch ***

This will show you the port it was learned from and you can do 'show cdp neighbor' and trace it form switch to switch that way.

New Member

Unable to ping until arp cache cleared

1.show arp | i IP-Address

Core Switch

Internet  10.1.1.17              16   0014.5ebc.7466  ARPA   Vlan2

2. show mac-address table | i IP-Address

no mac address found

3. Core 1

sh mac-address-table address 0014.5ebc.7466

2  0014.5ebc.7466   dynamic  Yes         35   Gi1/1 ( connected to access switch)

Access Switch

sh mac-address-table address 0014.5ebc.7466

2    0014.5ebc.7466    DYNAMIC     Gi1/0/44 ( connected to 2960 switch)

HSRP Config

Core 1

interface Vlan2
 ip address 10.1.1.254 255.255.255.0
 ip route-cache flow
 standby delay minimum 20 reload 25
 standby 1 ip 10.1.1.1
 standby 1 priority 110
 standby 1 preempt

Core 2

interface Vlan2
 ip address 10.1.1.253 255.255.255.0
 ip route-cache flow
 standby delay minimum 20 reload 25
 standby 2 ip 10.1.1.1
 standby 2 priority 95
 standby 2 preempt

Sh standby

Core 1

Vlan2 - Group 2
  Local state is Active, priority 110, may preempt
  Hellotime 3 sec, holdtime 10 sec
  Next hello sent in 1.143
  Virtual IP address is 10.1.1.1 configured
  Active router is local
  Standby router is 10.1.1.253 expires in 9.844
  Virtual mac address is 0000.0c07.ac02
  1 state changes, last state change 8w1d
  IP redundancy name is "hsrp-Vl2-2" (default)

Core 2

Vlan2 - Group 2
  Local state is Standby, priority 95, may preempt
  Hellotime 3 sec, holdtime 10 sec
  Next hello sent in 1.931
  Virtual IP address is 10.1.1.1 configured
  Active router is 10.1.1.254, priority 110 expires in 7.440
  Standby router is local
  43 state changes, last state change 8w1d
  IP redundancy name is "hsrp-Vl2-2" (default)
New Member

Unable to ping until arp cache cleared

1. are the non working servers in the same vlan as the working servers ?

No. they are in same vlan

2.how do you know the servers can't ping the VIP, is someone else doing  that for you ? If so can they do an "arp -a" on the server and look for  the VIP entry ?

Actually this is a medical server, the vendor logged in during the issue and discovered that they cannot ping the default gateway i.e VIP.

3. can you pick one of the non working servers and trace it's mac address  from the core switch ie. on each switch up to the 2960 can you look in  the mac address tables and make sure the mac for that server is going  out of the right port.

Yes they are coming on right port

Unable to ping until arp cache cleared

Can you post the HSRP config for this vlan from each router? Also can you post the results from 'show standby' with the specific HSRP number from each router?

Hall of Fame Super Blue

Unable to ping until arp cache cleared

When you cannot connect to the server ie. before you clear the arp cache on the core switch, is the mac address that is showing against the server IP the correct one or does it change after you clear the arp cache.

Jon

New Member

Unable to ping until arp cache cleared

It remains the same.

New Member

Unable to ping until arp cache cleared

One thing to be noted

2 servers showing same mac

0014.5ebc.7466

New Member

Unable to ping until arp cache cleared

2 IP addresses

10.1.1.15 and 17 show mac address entry in the mac table on core

Hall of Fame Super Blue

Unable to ping until arp cache cleared

Are these two of the servers not working ?

Jon

Hall of Fame Super Blue

Unable to ping until arp cache cleared

Okay, well that could well stop communication working ie. one server sends ping to VIP but the return packet is sent to the other server.

So i suspect you need to get that fixed.

Are both the servers showing the same mac address not working ?

Jon

New Member

Unable to ping until arp cache cleared

I have just communicated with server team and came to some information that

they have 2 servers 10.1.1.14 and 15 they are working as HA. and IP 17 is working like a virtual IP. and its working like active IP.

Unable to ping until arp cache cleared

I noticed something on your HSRP configuration.

HSRP Config

Core 1

interface Vlan2
 ip address 10.1.1.254 255.255.255.0
 ip route-cache flow
 standby delay minimum 20 reload 25
 standby 1 ip 10.1.1.1
 standby 1 priority 110
 standby 1 preempt

Core 2

interface Vlan2
 ip address 10.1.1.253 255.255.255.0
 ip route-cache flow
 standby delay minimum 20 reload 25
 standby 2 ip 10.1.1.1
 standby 2 priority 95
 standby 2 preempt

You have a priority of 110, if there is an issue, it wil go down by a default priority of 10, which will leave it at a priority of 100 which is higher than core2 HSRP config for this vlan.

New Member

Unable to ping until arp cache cleared

But the whole is not going down, the other servers and clients in the same vlan can ping the VIP.

New Member

Unable to ping until arp cache cleared

server 10.1.1.14 and 15 not working and 17 virtual on server not working as well.

Thanks

Hall of Fame Super Blue

Unable to ping until arp cache cleared

So if you generate a ping from one of those servers what should be the source IP ie. it's real IP or it's VIP ?

When you said you run a continous ping from a client which IP are you pinging ?

Jon

13367
Views
0
Helpful
94
Replies
CreatePlease to create content