IPSec Troubleshooting: Problem Scenarios Part 1

Blog

Jun 14, 2011 4:23 AM
Jun 14th, 2011

After incredible response on 1st Blog on IPSec important Debugging and logging” thought of coming up with this new blog on Ipsec troubleshooting and scenarios. I will cover this in 2 parts.

In this part I will be discussing the following problem scenarios----

  • IKE SA not established
  • IPSec SA’s not established
  • MTU/Fragmentation Issues

Problem Scenario 1:
No IKE SAs

If we are unable to establish IPSec tunnel from Branch location to  Hub location

Check for Routing. Ping the Branch (using HUB’s IKE endpoint)

HUB# ping ip 40.10.1.1 source 30.3.1.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 40.10.1.1, timeout is 2 seconds:
Packet sent with a source address of 30.3.1.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 1/1/4 ms

Check for IKE SA

HUB# sh crypto isakmp sa
IPv4 Crypto ISAKMP SA
dst             src             state          conn-id slot status
30.3.1.1        40.10.1.1       MM_NO_STATE          0    0 ACTIVE (deleted)

IPv6 Crypto ISAKMP SA

Use IKE Debugs to troubleshoot  [ debug crypto isakmp ]

Problem Scenario 1a:
No IKE SAs

ISAKMP:(0):Checking ISAKMP transform 1 against priority 10 policy
ISAKMP:      encryption 3DES-CBC
ISAKMP:      hash SHA
ISAKMP:      default group 2
ISAKMP:      auth pre-share
ISAKMP:      life type in seconds
ISAKMP:      life duration (basic) of 7200
ISAKMP:(0):Encryption algorithm offered does not match policy!
ISAKMP:(0):atts are not acceptable. Next payload is 0

ISAKMP:(0):Checking ISAKMP transform 1 against priority 65535 policy
ISAKMP:      encryption 3DES-CBC
ISAKMP:      hash SHA
ISAKMP:      default group 2
ISAKMP:      auth pre-share
ISAKMP:      life type in seconds
ISAKMP:      life duration (basic) of 7200
ISAKMP:(0):Encryption algorithm offered does not match policy!
ISAKMP:(0):atts are not acceptable. Next payload is 0

ISAKMP:(0):no offers accepted!
ISAKMP:(0): phase 1 SA policy not acceptable! (local 30.3.1.1 remote 40.10.1.1)

Check the IKE Policies


HUB# sh crypto isakmp policy

Global IKE policy
Protection suite of priority 10
         encryption algorithm:    AES - Advanced Encryption Standard (128 bit keys).
         hash algorithm:            Secure Hash Standard
         authentication method:  Pre-Shared Key
         Diffie-Hellman group:   #2 (1024 bit)
         lifetime:                        7200 seconds, no volume limit
Default protection suite
         encryption algorithm:   DES - Data Encryption Standard (56 bit keys).

                    1.jpg                                   2.jpg                                                                                                     

So once we change the encryption algorithm at spoke side to aes, phase 1 will come up.

Problem Scenario 1b:

No IKE SAs

ISAKMP:(1017): sending packet to 40.10.1.1 my_port 500 peer_port 500 (R) MM_KEY_EXCH
ISAKMP:(1017):Input = IKE_MESG_INTERNAL, IKE_PROCESS_COMPLETE
ISAKMP:(1017):Old State = IKE_R_MM3  New State = IKE_R_MM4

ISAKMP (0:1017): received packet from 40.10.1.1 dport 500 sport 500 Global (R) MM_KEY_EXCH
ISAKMP: reserved not zero on ID payload!
%CRYPTO-4-IKMP_BAD_MESSAGE: IKE message from 40.10.1.1 failed its sanity check or is malformed

4.jpg3.jpg

It means we have a mismatch in pre-shared key, on correcting it our IKE SA should come up.

HUB# sh cry isa sa
IPv4 Crypto ISAKMP SA
dst             src             state          conn-id slot status
30.3.1.1        40.10.1.1       QM_IDLE           1019    0 ACTIVE

Problem Scenario 2:
No IPSec SAs

If you notice that there is  no traffic is being received through the IPSec tunnel

IKE SAs exist, but no IPSec SAs

Check for IPSEC SA (look for inbound and outbound SPI’s)

HUB# sh crypto ipsec sa  peer 40.10.1.1

interface: GigabitEthernet0/1
     Crypto map tag: CMAP, local addr 30.3.1.1

    protected vrf: (none)
    local  ident (addr/mask/prot/port): (3.1.1.0/255.255.255.0/0/0)
    remote ident (addr/mask/prot/port): (4.1.1.0/255.255.255.0/0/0)
    current_peer 40.10.1.1 port 500
      PERMIT, flags={origin_is_acl,}
     #pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
     #pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0
     #pkts compressed: 0, #pkts decompressed: 0
     #pkts not compressed: 0, #pkts compr. failed: 0
     #pkts not decompressed: 0, #pkts decompress failed: 0
     #send errors 0, #recv errors 0

      local crypto endpt.: 30.3.1.1, remote crypto endpt.: 40.10.1.1
      path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/1
      current outbound spi: 0x0(0)

      inbound esp sas:
      inbound ah sas:


      outbound esp sas:
      outbound ah sas:

HUB#

Use IPSec Debugs to troubleshoot [ debug crypto ipsec ]

Problem Scenario 2a:
No IPSec SAs

ISAKMP (0:1022): received packet from 40.10.1.1 dport 500 sport 500 Global (R) QM_IDLE     

ISAKMP:(1022): processing SA payload. message ID = -549695704
ISAKMP:(1022):Checking IPSec proposal 1
ISAKMP: transform 1, ESP_3DES
ISAKMP:   attributes in transform:
ISAKMP:      encaps is 1 (Tunnel)
ISAKMP:      SA life type in seconds
ISAKMP:      SA life duration (basic) of 1800
ISAKMP:      SA life type in kilobytes
ISAKMP:      SA life duration (VPI) of  0x0 0x46 0x50 0x0
ISAKMP:      authenticator is HMAC-SHA
ISAKMP:(1022):atts are acceptable.

IPSEC(validate_proposal_request): proposal part #1,
   (key eng. msg.) INBOUND local= 30.3.1.1, remote= 40.10.1.1,
     local_proxy= 3.1.1.0/255.255.255.0/0/0 (type=4),
     remote_proxy= 4.1.1.0/255.255.255.0/0/0 (type=4),
     protocol= ESP, transform= NONE  (Tunnel),
     lifedur= 0s and 0kb,
     spi= 0x0(0), conn_id= 0, keysize= 0, flags= 0x0
Crypto mapdb : proxy_match
         src addr     : 3.1.1.0
         dst addr     : 4.1.1.0
         protocol     : 0
         src port     : 0
         dst port     : 0
IPSEC(ipsec_process_proposal): transform proposal not supported for identity:
    
{esp-3des esp-sha-hmac }

ISAKMP:(1022): IPSec policy invalidated proposal with error 256
ISAKMP:(1022): phase 2 SA policy not acceptable! (local 30.3.1.1 remote 40.10.1.1)

Check the IPSec Transform Sets

HUB# sh cry ips transform-set
Transform set TS: { esp-aes esp-sha-hmac  }
    will negotiate = { Tunnel,  },

6.jpg5.jpg

On Correcting encryption algorithm in tranform-set , tunnel should come up.

Problem Scenario 2b:
No IPSec SAs

Check the Crypto ACLs

HUB# sh access-list SPOKE-10-ACL
Extended IP access list SPOKE10-ACL
     10 permit ip 3.1.1.0 0.0.0.255 5.1.1.0 0.0.0.255
HUB#

7.jpg8.jpg

On Correcting crypto access-list , tunnel should come  up.

Problem Scenario 3:
Anti-Replay Issues

If you notice that some of the  applications are losing intermittent traffic, or that Voice quality  through tunnel is bad.

Check if the IPSec SA is showing anti-replay drops


HUB# sh cry ips sa peer 40.10.1.1 detail

interface: GigabitEthernet0/1
     Crypto map tag: CMAP, local addr 30.3.1.1

    protected vrf: (none)
    local  ident (addr/mask/prot/port): (3.1.1.0/255.255.255.0/0/0)
    remote ident (addr/mask/prot/port): (4.1.1.0/255.255.255.0/0/0)
    current_peer 40.10.1.1 port 500
      PERMIT, flags={origin_is_acl,}
     #pkts encaps: 2900, #pkts encrypt: 2900, #pkts digest: 2900
     #pkts decaps: 1909, #pkts decrypt: 1909, #pkts verify: 1909
     #pkts compressed: 0, #pkts decompressed: 0
     #pkts not compressed: 0, #pkts compr. failed: 0
     #pkts not decompressed: 0, #pkts decompress failed: 0
     #pkts no sa (send) 0, #pkts invalid sa (rcv) 0
     #pkts encaps failed (send) 0, #pkts decaps failed (rcv) 0
     #pkts invalid prot (recv) 0, #pkts verify failed: 0
     #pkts invalid identity (recv) 0, #pkts invalid len (rcv) 0
     #pkts replay rollover (send): 0, #pkts replay rollover (rcv) 0
     ##pkts replay failed (rcv): 1000
    #pkts internal err (send): 0, #pkts internal err (recv) 0

      local crypto endpt.: 30.3.1.1, remote crypto endpt.: 40.10.1.1
      path mtu 1500, ip mtu 1500, ip mtu idb GigabitEthernet0/1
      current outbound spi: 0xC37422AA(3279168170)

      inbound esp sas:
       spi: 0x135E76B1(324957873)
         transform: esp-3des esp-sha-hmac ,
         in use settings ={Tunnel, }
         conn id: 41, flow_id: SW:41, crypto map: CMAP
         sa timing: remaining key lifetime (k/sec): (4419198/860)
         IV size: 8 bytes
         replay detection support: Y
        Status: ACTIVE

Default IPSec Anti-Replay window is 64

Packets received outside the window are dropped

Re-ordering of packets could happen due to QoS on the encrypting router (Spoke) or in the Transit Network

In current Cisco IOS versions, the Anti-Replay window can be increased up to 1024, or diabled altogether


                 crypto ipsec security-association window-size

                 crypto ipsec security-association replay disable


Not recommended to disable anti-replay; first try to fix the QoS issue in the network or encrypting router; give better QoS to Voice traffic, or use crypto LLQ; then try to increase the anti-replay window size.

------------------------------------------------------------------------------------------------------------------------------

Hope this blog is helpful, I will cover some more scenarios in 2nd part of this blog in coming days.

Looking forward to your inputs and feedbacks.

Average Rating: 5 (5 ratings)

Comments

pavan.katakam Wed, 06/15/2011 - 07:19

Very nice blog for beginners :-)..I always prefer studying a scenario rather than a topic. Thank you sooo much Ankur. Thanks again...:-)

Hari_gedda Tue, 06/04/2013 - 19:21

Ankur, Greatly Thankful  for clearing all queries in a scenario based approach.

onyangoliech Tue, 06/18/2013 - 20:52

Thank you Ankur, this is a very good read. I am however presented with a situation where one of the routers is an IOS and so I can lo into it while the other is a smaller unit RV180 that I access using https. I'm using this reading to troubleshoot from the IOS accessible router.

trinhphanle Sat, 03/22/2014 - 09:35

Hi Athukaral

I got in trouble when try to troubleshoot VPN IPsec.

I turn on debug as your guide, however I see nothing in CLI, meant that I do not receive any information from that command.

I use Cisco 1811, version 12.4(24)T5. Did I miss something?

Thanks for your help a lot

 

trinhphanle Wed, 03/26/2014 - 08:10

Hi Athukral

Your blog is very useful. I understand what you read, but I have another trouble when tried to apply your experience to my VPN. I enable debug crypto for isakmp and ipsec but can not get the debug messages.

I donot know what happen, can you help me with the steps to work with debug tools?

Thanks and my router is Cisco 1811, Cisco IOS Software, C181X Software (C181X-ADVIPSERVICESK9-M), Version 12.4(6)T11, RELEASE SOFTWARE (fc2).

 

Thanks for your help a lot

Actions

Login or Register to take actions

This Blog

Posted June 14, 2011 at 4:23 AM
Stats:
Comments:13 Avg. Rating:5
Views:27363   
Shares:1
Categories: ASA
+

Related Content

Blogs Leaderboard