Optimal debug level for ASA ISAKMP / IPSEC on a Prod box

Sudeep Khuraijam · ‎02-23-2011

I have a question and a wish.

What is your favorite debug level for ISAKMP on ASAs running heavy load? Not lab, or do anything you want boxes. But on ASAs that is running multi Gig traffic? Whenever I am in environments where you control only your side of the VPN and not the customer/partner VPN GW, I find that

a) the debug messages on the ASA is not helpful unless you run a very deep debug levels. b) Deep debug levels are super verbose and may introduce packet latency/jitter. If you are in environments that are running voice/video/data streams, it is an issue. So I was curious what levels were folks comfortable with. What is their other tricks to dig down ISAKMP issues (when you have single sided view)?

As an example below, I got the following errors spewing to a customer gateway on level 127. You can tell it is failing phase 1 and never proceeds to secure a channel for phase 2. But not enough to tell a customer " you have this parameter wrong, change it to this". I don't know if debug 255 will actually give you that? I do know that it is profoundly verbose. Frankly I am too paranoid to run that level on any production ASA, given that it is already spewing thousands of syslog lines for archive.

Feb 23 12:22:22 [IKEv1]: IP = 192.168.250.250, IKE Initiator: New Phase 1, Intf ACCESS_DMZ_OUTSIDE, IKE Peer 192.168.250.250 local Proxy Address 172.17.250.224, remote Proxy Address 10.10.10.0, Crypto map (WEB_OUTSIDE_map)

Feb 23 12:22:22 [IKEv1 DEBUG]: IP = 192.168.250.250, constructing ISAKMP SA payload

Feb 23 12:22:22 [IKEv1 DEBUG]: IP = 192.168.250.250, constructing NAT-Traversal VID ver 02 payload

Feb 23 12:22:22 [IKEv1 DEBUG]: IP = 192.168.250.250, constructing NAT-Traversal VID ver 03 payload

Feb 23 12:22:22 [IKEv1 DEBUG]: IP = 192.168.250.250, constructing NAT-Traversal VID ver RFC payload

Feb 23 12:22:22 [IKEv1 DEBUG]: IP = 192.168.250.250, constructing Fragmentation VID + extended capabilities payload

Feb 23 12:22:22 [IKEv1]: IP = 192.168.250.250, IKE_DECODE SENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + VENDOR (13) + VENDOR (13) + NONE (0) total length : 168

Feb 23 12:22:22 [IKEv1]: IP = 192.168.250.250, IKE_DECODE RECEIVED Message (msgid=327b65da) with payloads : HDR + NOTIFY (11) + NONE (0) total length : 40

Feb 23 12:22:22 [IKEv1]: IP = 192.168.250.250, Received an un-encrypted NO_PROPOSAL_CHOSEN notify message, dropping

Feb 23 12:22:22 [IKEv1]: IP = 192.168.250.250, Information Exchange processing failed

Feb 23 12:22:27 [IKEv1 DEBUG]: Pitcher: received a key acquire message, spi 0x0

Feb 23 12:22:27 [IKEv1]: IP = 192.168.250.250, Queuing KEY-ACQUIRE messages to be processed when P1 SA is complete.

Feb 23 12:22:30 [IKEv1]: IP = 192.168.250.250, IKE_DECODE RESENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + VENDOR (13) + VENDOR (13) + NONE (0) total length : 168

Feb 23 12:22:32 [IKEv1 DEBUG]: Pitcher: received a key acquire message, spi 0x0

Feb 23 12:22:32 [IKEv1]: IP = 192.168.250.250, Queuing KEY-ACQUIRE messages to be processed when P1 SA is complete.

Feb 23 12:22:38 [IKEv1 DEBUG]: Pitcher: received a key acquire message, spi 0x0

Feb 23 12:22:38 [IKEv1]: IP = 192.168.250.250, Queuing KEY-ACQUIRE messages to be processed when P1 SA is complete.

Feb 23 12:22:38 [IKEv1]: IP = 192.168.250.250, IKE_DECODE RESENDING Message (msgid=0) with payloads : HDR + SA (1) + VENDOR (13) + VENDOR (13) + VENDOR (13) + VENDOR (13) + NONE (0) total length : 168

Now my wish is that the ASA had a level that simply printed the user variables, the different phases and messages associated with the actual negotiation. Not the function level message queues. ASA has introduced a zillion nice features and improvements over the years, but not in this module. Same thing for IPSEC but it is not nearly as important.

Patrick0711 · ‎02-23-2011

'debug crypto isakmp 254' will show you the exchanges packet-by-packet and will display the contents of the various payloads. If you're running 8.X code versions, you can do conditional debugging based on a specific peer IP.

In your case, you would gain little by increasing the debugging level as you can see the following notify message returned from the responder:

Feb 23 12:22:22 [IKEv1]: IP = 192.168.250.250, Received an un-encrypted NO_PROPOSAL_CHOSEN notify message, dropping

This tells me that the responder does not have a matching ISAKMP policy configured.

Debugging on the responding device is always more efficient and informational. I realize that you may not have access to the remote device so your best bet is to use the notify debug messages to identify where the problem is occurring and then you can compare your configuration parameters with the remote end.

Sudeep Khuraijam · ‎02-23-2011

Hi Patrick,

You are right but that's my point. Debug 254 is way too verbose for production ASA's cited above and it is almost a code trace routine. Way inefficient for field applicaiton.

Sorry I did not have a better example. And indeed the initiator does not know from the notify message why the SA was declined since there is only one notify type for all the SA variables. I wish the RFC put more message types.

But there must be an "optimal" (not perfect and not everything) debug level that gives you the IKE peer transactional messages and the SA variables, the phase transitions without too much code level nuances such as header construct, payload, nat or message queue logs.

Sudeep

Federico Coto Fajardo · ‎02-23-2011

Sudeep,

I personally use debug cry isa 127 and debug cry ipsec 127 for 99% of troubleshooting times.

Only very special cases a more detailed debug is needed 255... but with the downside that is more verbose.

Unfortunately I'm not sure if I can give you an ideal value for the debugs, personally I've find 127 more than enough for most times.

Federico.