×

Warning message

  • Cisco Support Forums is in Read Only mode while the site is being migrated.
  • Cisco Support Forums is in Read Only mode while the site is being migrated.

New 5525-x ASA Active/Standby pair Lacp port channel to HP - Application performance issues

Unanswered Question
Jul 3rd, 2013
User Badges:

Hello experts,


I'm experiencing performance problems with our main application after replacing our current firewalls with a new pair of 5525-x's. Basically i'm seeing 50-60ms RTT for our main business application on the old firewalls but when i bring the new ASA's into service the RTT becomes much more erratic (anywhere up to 10000ms response), leading to many complaints from users.


Basic setup is as follows.


Gb0/0 and Gb0/1 Port Channel1 to HP Procurve, Outside Traffic

Gb0/2 and Gb0/3 Port Channel2 to HP Procurve, Inside Vlan, DMZ Vlan and Application Vlan.


Each ASA connects to a seperate HP switch.


Ping from the asa to the gateway on the application vlan seems fine. Application traffic is Inside Vlan to Application Vlan.


I have disabled IPS to rule that out, struggling to see why i'm seeing such erratic traffic.


A little wireshark interrogation shows that we may be retransmitting packets but i'm not sure where to go from here.


Theres very little other traffice through the ASA at the moment, cpu and memory use is mimimal.


Versions are 9.1(2) on the ASA, 11.52 on the HP procurves..


Any ideas where to go on this?

Cheers

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
John Patrick Lopez Sat, 07/06/2013 - 11:11
User Badges:

Do you have a diagram of your setup? Config would also help.


Sent from Cisco Technical Support Android App

gilesinsurance Mon, 07/08/2013 - 02:47
User Badges:

Hi John, Heres a quick image of the setup along with the current config..


I have since removed the LACP trunks to rule those out and the problem still exists.





ASA Version 9.1(2)
!
hostname GIB-CISCOASA01
domain-name xxxxxxx.co.uk
enable password xxxxxxxxx encrypted
passwd xxxxxxxxx  encrypted
names
!
interface GigabitEthernet0/0
channel-group 1 mode passive
no nameif
no security-level
no ip address
!
interface GigabitEthernet0/1
channel-group 1 mode passive
no nameif
no security-level
no ip address
!
interface GigabitEthernet0/2
channel-group 2 mode passive
no nameif
no security-level
no ip address
!
interface GigabitEthernet0/3
channel-group 2 mode passive
no nameif
no security-level
no ip address
!
interface GigabitEthernet0/4
shutdown
no nameif
no security-level
no ip address
!
interface GigabitEthernet0/5
shutdown
no nameif
no security-level
no ip address
!
interface GigabitEthernet0/6
shutdown
no nameif
no security-level
no ip address
!
interface GigabitEthernet0/7
description LAN/STATE Failover Interface
!
interface Management0/0
management-only
nameif management
security-level 100
ip address 192.168.xxx.50 255.255.255.0 standby 192.168.xxx.51
!
interface Port-channel1
nameif Outside
security-level 0
ip address 2xx.xxx.xxx.4 255.255.255.240 standby 2xx.xxx.xxx.11
!
interface Port-channel2
no nameif
no security-level
no ip address
!
interface Port-channel2.10
vlan 10
nameif dmz
security-level 25
ip address 192.168.xxx.3 255.255.255.0 standby 192.168.xxx.4
!
interface Port-channel2.5
vlan 5
nameif Inside
security-level 100
ip address 192.168.xxx.7 255.255.255.0 standby 192.168.xxx.8
!
interface Port-channel2.725
vlan 725
nameif Wisecall
security-level 75
ip address 192.168.xxx.25 255.255.255.0 standby 192.168.xxx.26
!
interface Port-channel2.15
vlan 15
nameif Application
security-level 50
ip address 192.168.xxx.1 255.255.255.0 standby 192.168.xxx.2
!
boot system disk0:/asa912-smp-k8.bin
ftp mode passive

clock timezone GMT/BST 0
clock summer-time GMT/BDT recurring last Sun Mar 1:00 last Sun Oct 2:00
dns server-group DefaultDNS
domain-name xxxxxxx.co.uk
object network DMZ_DNAT
subnet 192.168.xxx.0 255.255.255.0
object network GIB-ADFS01
host 192.168.xxx.7
object network Total_Objects_XMP
host 2xx.xxx.xxx.11
description Total Objects London
object network GIB-CISCOASAIPS01
host 192.168.xxx.11
object network ML_Tower1
subnet 216.82.240.0 255.255.240.0
description ML_Tower
object network ML_Tower10
subnet 46.226.48.0 255.255.248.0
description ML_Tower
object network ML_Tower2
subnet 195.245.230.0 255.255.254.0
description ML_Tower
object network ML_Tower3
subnet 194.106.220.0 255.255.254.0
description ML_Tower
object network ML_Tower4
subnet 193.109.254.0 255.255.254.0
description ML_Tower
object network ML_Tower5
subnet 117.120.16.0 255.255.248.0
description ML_Tower
object network ML_Tower6
subnet 103.9.96.0 255.255.252.0
description ML_Tower
object network ML_Tower7
subnet 95.131.104.0 255.255.248.0
description ML_Tower
object network ML_Tower8
subnet 85.158.136.0 255.255.248.0
description ML_Tower
object network ML_Tower9
subnet 67.219.240.0 255.255.240.0
description ML_Tower
object network Star_Tower1
subnet 62.231.128.0 255.255.224.0
description Star_Tower
object network Star_Tower2
subnet 195.216.0.0 255.255.224.0

             
description Star_Tower
object network Star_Tower3
subnet 212.125.64.0 255.255.224.0
description Star_Tower
object network Application_Tower1
subnet 89.xxx.xxx.0 255.255.252.0
description Application_Tower
object network Application_Gateway
host 192.168.xxx.15
object network globalxmp.totalobjects.co.uk
host 168.63.107.240
description Production XMP in Azure
object network globalxmpuat.totalobjects.co.uk
host 168.63.107.116
description UAT XMP in Azure
object network Inside_Network
subnet 192.168.250.0 255.255.255.0
object network WNS_Network
subnet 10.XXX.XXX.0 255.255.240.0
object network GIB-VM-GW01
host 192.168.XXX.5
object network GIB-CISCOASAIPS02
host 192.168.XXX.4
object network GIB-EMAIL2-HT1

             
host 192.168.XXX.6
object network Autodiscover
host 192.168.XXX.32
object network Application_NAT
host 192.168.XXX.5
object network GIB-RDS-FARM05
range 192.168.XXX.98 192.168.XXX.102
object service rdp
service tcp destination eq 3389
object network WNS_Hosts
subnet 10.XXX.XXX.0 255.255.255.240
object-group service Total_Objects_XMP_HTTPS tcp
port-object eq 40443
object-group network ML_Towers
network-object object ML_Tower1
network-object object ML_Tower10
network-object object ML_Tower2
network-object object ML_Tower3
network-object object ML_Tower4
network-object object ML_Tower5
network-object object ML_Tower6
network-object object ML_Tower7
network-object object ML_Tower8
network-object object ML_Tower9

             
object-group network Star_Towers
network-object object Star_Tower1
network-object object Star_Tower2
network-object object Star_Tower3
object-group network Total_Objects_XMP_Group
description Total Objects XMP IPs accessing gib-adfs01
network-object object Total_Objects_XMP
network-object object globalxmp.totalobjects.co.uk
network-object object globalxmpuat.totalobjects.co.uk
object-group network Application_Towers
network-object object Application_Tower1
access-list IPS extended permit ip any any inactive
access-list Outside_access_in remark Inbound HTTPS traffic to ADFS for ManageEngine
access-list Outside_access_in extended permit tcp any object GIB-ADFS01 eq https
access-list Outside_access_in extended permit tcp object-group Total_Objects_XMP_Group object GIB-ADFS01 object-group Total_Objects_XMP_HTTPS
access-list Outside_access_in extended permit tcp any object Autodiscover eq https
access-list Outside_access_in extended permit tcp object-group ML_Towers object GIB-EMAIL2-HT1 eq smtp
access-list Outside_cryptomap extended permit ip 192.168.XXX.0 255.255.255.0 object WNS_Network
access-list Outside_cryptomap extended permit object rdp object WNS_Hosts object GIB-RDS-FARM05
pager lines 24
logging enable
logging asdm informational
logging from-address [email protected]
logging recipient-address [email protected] level errors

             
mtu management 1500
mtu Outside 1500
mtu dmz 1500
mtu Inside 1500
mtu Wisecall 1500
mtu Application 1500
failover
failover lan unit primary
failover lan interface Failover GigabitEthernet0/7
failover key *****
failover link Failover GigabitEthernet0/7
failover interface ip Failover 172.16.10.10 255.255.255.0 standby 172.16.10.20
monitor-interface dmz
monitor-interface Inside
monitor-interface Wisecall
monitor-interface Application
icmp unreachable rate-limit 1 burst-size 1
icmp deny any Outside
icmp permit any Application
asdm image disk0:/asdm-713.bin
no asdm history enable
arp timeout 14400
no arp permit-nonconnected
nat (Inside,Outside) source static Inside_Network Inside_Network destination static WNS_Network WNS_Network no-proxy-arp route-lookup

             
nat (Inside,Application) source static Inside_Network Application_NAT destination static Application_Tower1 Application_Tower1
!
object network DMZ_DNAT
nat (any,Outside) dynamic interface
object network GIB-ADFS01
nat (any,any) static 2XX.XXX.XXX.7
object network GIB-CISCOASAIPS01
nat (any,Outside) dynamic interface
object network GIB-VM-GW01
nat (any,Outside) dynamic interface
object network GIB-CISCOASAIPS02
nat (any,Outside) dynamic interface
object network GIB-EMAIL2-HT1
nat (dmz,Outside) static 2XX.XXX.XXX.6 service tcp smtp smtp
object network Autodiscover
nat (dmz,Outside) static 2XX.XXX.XXX.6 service tcp https https
access-group Outside_access_in in interface Outside
route Outside 0.0.0.0 0.0.0.0 2XX.XXX.XXX.1 1
route Application 89.XXX.XXX.0 255.255.252.0 192.168.XXX.15 1
timeout xlate 3:00:00
timeout pat-xlate 0:00:30
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 icmp 0:00:02
timeout sunrpc 0:10:00 h323 0:05:00 h225 1:00:00 mgcp 0:05:00 mgcp-pat 0:05:00

             
timeout sip 0:30:00 sip_media 0:02:00 sip-invite 0:03:00 sip-disconnect 0:02:00
timeout sip-provisional-media 0:02:00 uauth 0:05:00 absolute
timeout tcp-proxy-reassembly 0:01:00
timeout floating-conn 0:00:00
dynamic-access-policy-record DfltAccessPolicy
user-identity default-domain LOCAL
aaa authentication http console LOCAL
aaa authentication serial console LOCAL
aaa authentication ssh console LOCAL
aaa authentication telnet console LOCAL
http server enable
http 192.168.XXX.0 255.255.255.0 management
no snmp-server location
no snmp-server contact
snmp-server enable traps snmp authentication linkup linkdown coldstart warmstart
crypto ipsec ikev1 transform-set ESP-AES-128-SHA esp-aes esp-sha-hmac
crypto ipsec ikev1 transform-set ESP-AES-128-MD5 esp-aes esp-md5-hmac
crypto ipsec ikev1 transform-set ESP-AES-192-SHA esp-aes-192 esp-sha-hmac
crypto ipsec ikev1 transform-set ESP-AES-192-MD5 esp-aes-192 esp-md5-hmac
crypto ipsec ikev1 transform-set ESP-AES-256-SHA esp-aes-256 esp-sha-hmac
crypto ipsec ikev1 transform-set ESP-AES-256-MD5 esp-aes-256 esp-md5-hmac
crypto ipsec ikev1 transform-set ESP-AES-128-SHA-TRANS esp-aes esp-sha-hmac
crypto ipsec ikev1 transform-set ESP-AES-128-SHA-TRANS mode transport
crypto ipsec ikev1 transform-set ESP-AES-128-MD5-TRANS esp-aes esp-md5-hmac

             
crypto ipsec ikev1 transform-set ESP-AES-128-MD5-TRANS mode transport
crypto ipsec ikev1 transform-set ESP-AES-192-SHA-TRANS esp-aes-192 esp-sha-hmac
crypto ipsec ikev1 transform-set ESP-AES-192-SHA-TRANS mode transport
crypto ipsec ikev1 transform-set ESP-AES-192-MD5-TRANS esp-aes-192 esp-md5-hmac
crypto ipsec ikev1 transform-set ESP-AES-192-MD5-TRANS mode transport
crypto ipsec ikev1 transform-set ESP-AES-256-SHA-TRANS esp-aes-256 esp-sha-hmac
crypto ipsec ikev1 transform-set ESP-AES-256-SHA-TRANS mode transport
crypto ipsec ikev1 transform-set ESP-AES-256-MD5-TRANS esp-aes-256 esp-md5-hmac
crypto ipsec ikev1 transform-set ESP-AES-256-MD5-TRANS mode transport
crypto ipsec ikev1 transform-set ESP-3DES-SHA esp-3des esp-sha-hmac
crypto ipsec ikev1 transform-set ESP-3DES-MD5 esp-3des esp-md5-hmac
crypto ipsec ikev1 transform-set ESP-3DES-SHA-TRANS esp-3des esp-sha-hmac
crypto ipsec ikev1 transform-set ESP-3DES-SHA-TRANS mode transport
crypto ipsec ikev1 transform-set ESP-3DES-MD5-TRANS esp-3des esp-md5-hmac
crypto ipsec ikev1 transform-set ESP-3DES-MD5-TRANS mode transport
crypto ipsec ikev1 transform-set ESP-DES-SHA esp-des esp-sha-hmac
crypto ipsec ikev1 transform-set ESP-DES-MD5 esp-des esp-md5-hmac
crypto ipsec ikev1 transform-set ESP-DES-SHA-TRANS esp-des esp-sha-hmac
crypto ipsec ikev1 transform-set ESP-DES-SHA-TRANS mode transport
crypto ipsec ikev1 transform-set ESP-DES-MD5-TRANS esp-des esp-md5-hmac
crypto ipsec ikev1 transform-set ESP-DES-MD5-TRANS mode transport
crypto ipsec ikev2 ipsec-proposal AES256
protocol esp encryption aes-256
protocol esp integrity sha-1 md5

             
crypto ipsec ikev2 ipsec-proposal AES192
protocol esp encryption aes-192
protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal AES
protocol esp encryption aes
protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal 3DES
protocol esp encryption 3des
protocol esp integrity sha-1 md5
crypto ipsec ikev2 ipsec-proposal DES
protocol esp encryption des
protocol esp integrity sha-1 md5
crypto ipsec security-association pmtu-aging infinite
crypto map Outside_map 1 match address Outside_cryptomap
crypto map Outside_map 1 set peer 6X.XXX.XXX.82
crypto map Outside_map 1 set ikev1 transform-set ESP-AES-256-SHA
crypto map Outside_map 1 set ikev2 ipsec-proposal AES256
crypto map Outside_map interface Outside
crypto ca trustpoint ASDM_TrustPoint0
enrollment self
subject-name CN=ciscoasa
crl configure
crypto ca trustpoint _SmartCallHome_ServerCA
crl configure

             
crypto ca trustpool policy
crypto ca certificate chain _SmartCallHome_ServerCA
certificate ca 6ecc7aa5a7032009b8cebcf4e952d491
    6c2527b9 deb78458 c61f381e a4c4cb66
  quit
crypto ikev2 policy 1
encryption aes-256
integrity sha
group 5 2
prf sha
lifetime seconds 86400
crypto ikev2 policy 2
encryption aes-256
integrity sha
group 2
prf sha
lifetime seconds 3600
crypto ikev2 policy 10
encryption aes-192
integrity sha
group 5 2
prf sha
lifetime seconds 86400
crypto ikev2 policy 20
encryption aes

             
integrity sha
group 5 2
prf sha
lifetime seconds 86400
crypto ikev2 policy 30
encryption 3des
integrity sha
group 5 2
prf sha
lifetime seconds 86400
crypto ikev2 policy 40
encryption des
integrity sha
group 5 2
prf sha
lifetime seconds 86400
crypto ikev2 enable Outside
crypto ikev1 enable Outside
crypto ikev1 policy 10
authentication crack
encryption aes-256
hash sha
group 2
lifetime 86400

             
crypto ikev1 policy 20
authentication rsa-sig
encryption aes-256
hash sha
group 2
lifetime 86400
crypto ikev1 policy 30
authentication pre-share
encryption aes-256
hash sha
group 2
lifetime 86400
crypto ikev1 policy 40
authentication crack
encryption aes-192
hash sha
group 2
lifetime 86400
crypto ikev1 policy 50
authentication rsa-sig
encryption aes-192
hash sha
group 2
lifetime 86400

             
crypto ikev1 policy 60
authentication pre-share
encryption aes-192
hash sha
group 2
lifetime 86400
crypto ikev1 policy 70
authentication crack
encryption aes
hash sha
group 2
lifetime 86400
crypto ikev1 policy 80
authentication rsa-sig
encryption aes
hash sha
group 2
lifetime 86400
crypto ikev1 policy 90
authentication pre-share
encryption aes
hash sha
group 2
lifetime 86400

             
crypto ikev1 policy 100
authentication crack
encryption 3des
hash sha
group 2
lifetime 86400
crypto ikev1 policy 110
authentication rsa-sig
encryption 3des
hash sha
group 2
lifetime 86400
crypto ikev1 policy 120
authentication pre-share
encryption 3des
hash sha
group 2
lifetime 86400
crypto ikev1 policy 130
authentication crack
encryption des
hash sha
group 2
lifetime 86400

             
crypto ikev1 policy 140
authentication rsa-sig
encryption des
hash sha
group 2
lifetime 86400
crypto ikev1 policy 150
authentication pre-share
encryption des
hash sha
group 2
lifetime 86400
telnet timeout 5
ssh scopy enable
ssh 192.168.XXX.0 255.255.255.0 management
ssh timeout 60
ssh key-exchange group dh-group1-sha1
console timeout 10
threat-detection basic-threat
threat-detection statistics
threat-detection statistics tcp-intercept rate-interval 30 burst-rate 400 average-rate 200
ntp server 192.168.XXX.1 source Inside prefer
ssl encryption rc4-md5 rc4-sha1 aes128-sha1 aes256-sha1 3des-sha1
group-policy GroupPolicy_6X.XXX.XXX.82 internal

             
group-policy GroupPolicy_6X.XXX.XXX.82 attributes
vpn-tunnel-protocol ikev1 ikev2
username admin password XXXXXXXXXXXXSER encrypted privilege 15
tunnel-group 6X.XXX.XXX.XXX type ipsec-l2l
tunnel-group 6X.XXX.XXX.XXX general-attributes
default-group-policy GroupPolicy_6X.XXX.XXX.XXX
tunnel-group 6X.XXX.XXX.XXX ipsec-attributes
ikev1 pre-shared-key *****
ikev2 remote-authentication pre-shared-key *****
ikev2 local-authentication pre-shared-key *****
!
class-map IPS
match access-list IPS
!
!
policy-map type inspect dns preset_dns_map
parameters
  message-length maximum client auto
  message-length maximum 512
policy-map global_policy
class IPS
  ips inline fail-open sensor vs0
!
service-policy global_policy global

             
smtp-server 192.168.XXX.6 192.168.XXX.13
prompt hostname context
call-home reporting anonymous
hpm topN enable
Cryptochecksum:8827b4a8d8f1f7ad34f8f6a967d5b7eb

John Patrick Lopez Tue, 07/09/2013 - 10:26
User Badges:

Have you tried to totally disable the IPS inspection by removing the class-map and not just by deactivating the ACL?

Are all servers in application VLAN affected by the bad response time or just specific one?

Try to shutdown the secondary firewall to see if it's tryng to do something bad.

Lastly, use another physical interface dedicated for either inside or application. 10000ms seem to be so long for a response time and that's around 10 seconds. If it's 1000 then that's slow as well.


Sent from Cisco Technical Support Android App

gilesinsurance Wed, 07/10/2013 - 01:33
User Badges:

Hi John,


I havent tried to remove the class-map completely yet - i'll try this next.


I did wonder if the failover configuration could be somehow to blame, i will also try and break the HA next.


I have been running on the secondary firewall for the past few days to rule out interface/network cable/ switch issues and the problem still exists. I have a TAC open and support are busy analyzing the captures at the moment other than that they cant seen any obvious performance/configuration issue at the moment.

Actions

This Discussion

Related Content