Strange network problem

Unanswered Question
Sep 10th, 2010
User Badges:

We have in site A two Cisco ASA5510 Active/Passiv behind them their is two 3560 using HSRP for a number of VLANs

Behind the 3560 their is a number of access switches connected it could be procurve switches or cisco switches

We have VPN peer to peer connection with other ASA5505 ( Site B ) and behind the 3560 their is also a ISP cloud to

other locations ( Site C ).

My problem is that Servers sometimes disappear from the network especially for Site B ( tunnel permit IP any any ) but

also for Site C. But if i ping the host it appear again ( i lose first ping ) and then it works fine again.


Below is a 3560 config second 3560 is configured the same way. ASA inside has 10.177.190.1


wr t
Building configuration...

Current configuration : 5379 bytes
!
version 12.2
no service pad
service timestamps debug uptime
service timestamps log uptime
no service password-encryption
!
hostname SwitchA
!
boot-start-marker
boot-end-marker
!
enable secret 5 xxxxxxxxxxx
!
!
!
no aaa new-model
system mtu routing 1500
!
authentication mac-move permit
ip subnet-zero
ip routing
ip domain-name m.local
!
!
!
!
!
!
spanning-tree mode pvst
spanning-tree etherchannel guard misconfig
spanning-tree extend system-id
!
vlan internal allocation policy ascending
!
!
!
!
interface Loopback0
ip address 10.177.254.254 255.255.255.255
!
interface Port-channel1
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface FastEthernet0/1
switchport access vlan 190
switchport mode access
!
interface FastEthernet0/2
switchport access vlan 200
switchport mode access
!
interface FastEthernet0/3
switchport access vlan 177
switchport mode access
!
interface FastEthernet0/4
switchport access vlan 177
switchport mode access
!
interface FastEthernet0/5
switchport access vlan 177
switchport mode access
!
interface FastEthernet0/6
switchport access vlan 177
switchport mode access
!
interface FastEthernet0/7
switchport access vlan 176
switchport mode access
!
interface FastEthernet0/8
switchport access vlan 176
switchport mode access
speed 100
duplex full
!
interface FastEthernet0/9
!
interface FastEthernet0/10
switchport access vlan 179
switchport mode access
!
interface FastEthernet0/11
switchport access vlan 216
switchport mode access
!
interface FastEthernet0/12
!
interface FastEthernet0/13
!
interface FastEthernet0/14
!
interface FastEthernet0/15
!
interface FastEthernet0/16
!
interface FastEthernet0/17
!
interface FastEthernet0/18
!
interface FastEthernet0/19
!
interface FastEthernet0/20
!
interface FastEthernet0/21
!
interface FastEthernet0/22
!
interface FastEthernet0/23
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 1 mode on
!
interface FastEthernet0/24
switchport trunk encapsulation dot1q
switchport mode trunk
channel-group 1 mode on
!
interface GigabitEthernet0/1
description Dot1q-trunk to HP2510G
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
speed nonegotiate
!
interface GigabitEthernet0/2
!
interface Vlan1
no ip address
shutdown
!
interface Vlan170
ip address 10.177.170.2 255.255.255.0
standby delay reload 30
standby version 2
standby 170 ip 10.177.170.1
standby 170 priority 110
standby 170 preempt delay reload 50
standby 170 authentication md5 key-string xxxxx
standby 170 track FastEthernet0/1 50
!
interface Vlan176
ip address 10.177.176.16 255.255.255.0
standby delay reload 30
standby version 2
standby 176 ip 10.177.176.15
standby 176 priority 110
standby 176 preempt delay reload 50
standby 176 authentication md5 key-string xxxxx
standby 176 track FastEthernet0/1 50
!
interface Vlan177
ip address 10.177.177.3 255.255.255.0
standby delay reload 30
standby version 2
standby 177 ip 10.177.177.1
standby 177 priority 110
standby 177 preempt delay reload 50
standby 177 authentication md5 key-string xxxxx
standby 177 track FastEthernet0/1 50
!
interface Vlan179
description ISP Cloud to other locations
ip address 10.177.179.3 255.255.255.0
ip access-group 100 out
standby delay reload 30
standby version 2
standby 179 ip 10.177.179.1
standby 179 priority 110
standby 179 preempt delay reload 50
standby 179 authentication md5 key-string xxxxx
standby 179 track FastEthernet0/1 50
!
interface Vlan190
ip address 10.177.190.4 255.255.255.0
standby version 2
standby 190 ip 10.177.190.3
standby 190 priority 110
standby 190 preempt
standby 190 authentication md5 key-string xxxxx
standby 190 track FastEthernet0/1 50
!
interface Vlan200
ip address 10.177.200.2 255.255.255.0
standby delay reload 30
standby version 2
standby 200 ip 10.177.200.1
standby 200 priority 110
standby 200 preempt delay reload 50
standby 200 authentication md5 key-string xxxxx
standby 200 track FastEthernet0/1 50
!
interface Vlan216
description
no ip address
!
router ospf 1
log-adjacency-changes
network 10.0.0.0 0.255.255.255 area 0
default-information originate
!
ip classless
ip route 0.0.0.0 0.0.0.0 10.177.190.1
ip route 10.177.180.0 255.255.255.0 10.177.179.2
ip route 10.177.181.0 255.255.255.0 10.177.179.2
ip route 10.177.185.0 255.255.255.0 10.177.190.1
ip route 172.16.0.0 255.255.0.0 10.177.179.2
ip route 172.17.0.0 255.255.0.0 10.177.179.2
ip route 172.18.1.0 255.255.255.0 10.177.179.2
ip route 172.19.1.0 255.255.255.0 10.177.179.2
ip route 194.103.23.33 255.255.255.255 10.177.200.238
ip http server
!
!
ip sla enable reaction-alerts
access-list 100 permit ip any 172.17.0.0 0.0.255.255
access-list 100 permit ip any 172.16.0.0 0.0.255.255
access-list 100 permit ip any 172.19.1.0 0.0.0.255
access-list 100 permit ip any 172.18.1.0 0.0.0.255
access-list 100 permit ip any 10.177.0.0 0.0.255.255
access-list 100 permit icmp any any echo
access-list 100 permit icmp any any echo-reply
!
!
!
line con 0
exec-timeout 0 0
logging synchronous
line vty 0 4
exec-timeout 30 0
password xxxxx
logging synchronous
login
length 0
line vty 5 15
exec-timeout 30 0
logging synchronous
login
!
end

SwitchA#


Best Regards

Peter

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
ashok_boin Fri, 09/10/2010 - 02:09
User Badges:
  • Bronze, 100 points or more

Hi Peter,


Servers disappear suddenly: How did you come to know the servers disappeared? Any monitoring system and alarm?


Do you have ASA to connect to Site C through Internet cloud? Have you tried to bypass ASA and monitor for a site?


It might be peer to peer connection keepalive issue and might again trying to make the connectivity up once you start to ping.


Regards...

-Ashok.

peter.karlberg Sat, 09/11/2010 - 06:07
User Badges:

Hi Ashok, thanks for reply


The servers that disappear is behind the 3560 ( could be a Windows Server or AS400 ) but not connected to them.

We have no alarms on them yet but they do not disappear from all places in the network, it seems that it just is

remote sites. Site C is through a MPLS network not over internet and not through ASA.

Site B ( VPN through ASA ) has the tunnel up but cannot reach a webserver before using ping ( sometimes ).

i thought it was a ARP or firmware problem,so i upgrade both 3560 and ASA but it seems that the problem is the

same anyway. Most of the access switches is HP Procurve, could that be a problem ?


Regards

Peter

j-marenda Sat, 09/11/2010 - 14:36
User Badges:

Peter,


first of all, interconnecting hp procurve switches to Cisco switches

may^Wwill introduce spanning-tree protocol problems.


Cisco-Default is one spanning tree inside each vlan.

HP default is "off" which may result in loops or one global spanning tree.


Solution here is to not build redundancy or

use MST on both, with identical mst configuration (name, versionnumber, assignment of vlans to mst instances).

Migration to it means downtime.


The other problem may be that some sorts of hp switches (2626, 2650 for example) are not able

to have the same MAC in different VLANs. (if you have an older sun server with multiple network interfaces,

you will have the systems-MAC on all network interfaces.)

On those switches, you see mac-address change from one port and vlan to the next,

all the logfile full.

And of course you may see outages.

This also happens when you have a cluster with ONE vrrp group on differnet physical interfaces:

on all that ports you see the same VRRP-MAC derived from the group-number.

(this seems to be the way checkpoint works.)

Also here you will get lots of messages about mac-address move/flapping.


So please look at your syslog or local logging on the switches.


Hope this helps,


Juergen.

peter.karlberg Mon, 09/13/2010 - 05:07
User Badges:

Hi thanks for reply j-marenda.


We have not build redundant with HP switches so it should not be a STP problem.


I didnt know this "some sorts of hp switches (2626, 2650 for example) are not able

to have the same MAC in different VLANs."

I will analyze that but i think Servers just have NICs in one VLAN


We don't use VRRP just HSRP in 3560


Regards


Peter

Actions

This Discussion