vpn hardware error with packets bigger than 128 Byte payload

Unanswered Question
Jun 28th, 2010

anyone else having the same problem?

000085: Jun 21 18:09:12.635 CEST: %VPN_HW-1-PACKET_ERROR: slot: 0 Packet Encryption/Decryption error, Invalid Packet:srcadr=192.168.x.x,dstadr=172.x.x.x,size=120,handle=0x5801.....
Success rate is 0 percent (0/5)

legacy vpn

crypto map

transform-set esp-sha1 esp-aes-128

(hub)

2811 with aim-module AIM-VPN/EPII-PLUS

c2800nm-advipservicesk9-mz.124-8c.bin
c2800nm-advipservicesk9-mz.124-25c.bin

(spoke)

c1841-advipservicesk9-mz.124-10c.bin

on-board-vpn-chip

small packet passes

bigger packets hangs

Don't think of fragmentation, because we are still below the 576-size!

any hint is appreciated, William

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Marcin Latosiewicz Mon, 06/28/2010 - 06:38

William,

Please check if the problem persists with onboard accelarator.

I would be interested in:

"show crypt eli"

"show crypto engine accel stati"

Marcin

edit:

Ad, changing accelarator - please note that it will most likely need to re-establish the tunnels.

9w.boye Mon, 06/28/2010 - 07:42

changing from hardware-module to onboard to software ipsec brought no success

no crypto engine aim

no crypto engine onboard

 

with aim-module:

Show crypto engine accelerator stat

 

xxx#show crypto engine acc stat

Virtual Private Network (VPN) Module in slot : 0

Statistics for Hardware VPN Module since the last clear

of counters 285502 seconds ago

...

Errors:

ppq full errors : 0 ppq rx errors : 33

NR overflow : 0 pkts dropped : 33

Warnings:

sessions_expired : 0 packets_fragmented : 0

general: : 0

HSP details:

hsp_operations : 66775 hsp_sessions : 4

 

xxxx#ping 172.16.9.240 source loop0 size 200 repeat 1

Type escape sequence to abort.

Sending 1, 200-byte ICMP Echos to 172.16.9.240, timeout is 2 seconds:

Packet sent with a source address of x.x.x.x

000112: Jun 21 18:25:26.962 CEST: %VPN_HW-1-PACKET_ERROR: slot: 0 Packet Encryption/Decryption error, Invalid Packet:srcadr=192.168.x.x,dstadr=172.x.x.x size=120,handle=0x5801.

Success rate is 0 percent (0/1)

 

r-zhza1#show crypto engine acc stat

Virtual Private Network (VPN) Module in slot : 0

Statistics for Hardware VPN Module since the last clear

of counters 285543 seconds ago

...

Errors:

ppq full errors : 0 ppq rx errors : 34

..

Decrypt Failure : 0 Invalid Packet : 34

...

NR overflow : 0 pkts dropped : 34

Warnings:

sessions_expired : 0 packets_fragmented : 0

general: : 0

Marcin Latosiewicz Mon, 06/28/2010 - 08:27

Did you check that you changed crypto from AIM to software "show crypto ipsec sa | i flow" will tell you.

IF the problem persists on soft and hard crypto then it's time to check:

1) Other side - punt in from software to hardwar (or vice versa) - does it change anything?

2) ISP? Check capture on one side as compared to other side. (I would suggest SPAN if possible)

9w.boye Mon, 06/28/2010 - 09:12

Hello Marcin, I appreciate your efforts very much. But don't you think it is time for an internal answer?

After the first occurences - with the 25c-image on the hub - we made a test inside of our lab: with the same result.

Our second router had a bit older image: 8c. So we thought this might be the cause.

It turned out, that it was not.

What else was different? On the "working" hub-side we used as the transform-set esp-aes-256 AND comp-lzs

We wanted to change it - what makes sense - to esp-aes-128 and esp-sha1

Of course we cleared the crypto sas on both sides.

And if I see a hardware vpn error, severity 1, I am getting a bit courious: ping with size 99 works, ping with size 100 not.

Any ideas?

William

Marcin Latosiewicz Mon, 06/28/2010 - 10:20

William,

What would you exactly call "internal answer"?

This error appears mostly in case of those:

- Something mangling packet on the way (get capture on both end on physical interfaces)

- HW error - not likely since issue persist with and without HW encryption

- Misconfig/mismatch/not-supported config

- bug - not very likely.

That's why it was curious for me what happens with different encryption setting on both sides.

Marcin

9w.boye Mon, 06/28/2010 - 11:17

Hm, okay

- Something mangling packet on the way (get capture on both end on physical interfaces)

what would be mangling in the way in my testlab?

- HW error - not likely since issue persist with and without HW encryption

ok, if not hardware, then software?

- Misconfig/mismatch/not-supported config

well, why is it working with the other transform-set?

- bug - not very likely.

not very likely?

In one forum answer on such kind of issue, I found the answer to turn out cef and fast-switching ...

William

Marcin Latosiewicz Mon, 06/28/2010 - 11:53

Ah that's the info I missed.

Which transform sets work and which ones do not?

If you suspect a bug in CEF, disable it ... it's a test lab after all...

9w.boye Tue, 06/29/2010 - 00:16

There are enough infos on the table.

Hardware 1 error, internal information: handle= ...

accelerator output - what is the reason for the drop of this packets

anyone else with experiences or more information about possible incompatibilities between the mentioned devices/software?

turning off cef is a joke. I don't have a aim-module on one hand to process-switch on the outer interface.

William

Marcin Latosiewicz Tue, 06/29/2010 - 02:39

William,

If turning of CEF would help you would see different behavior when pinging FROM the device itself and through.

Which table are you refering to?

Marcin

Actions

This Discussion