Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. And see here for current known issues.

New Member

rv082 VPN remote desktop encryption error

We have two rv082 routers. One at our main oice and the other at a remote office. Both have the latest firmware installed. They are connected through a VPN tunnel. All our computers have WinXP on them. I often use remote desktop through the VPN from my laptop to my desktop computer at the main office with no problems.

Now, I installed Win7 on both my machines and now keep getting an error after a few seconds to a minute after I start a remote desktop session through the VPN. "Because of an error in data encryption, this session will end." If I try a remote desktop session when I am at the main office (not through the VPN), there is no problem. I am using the same computers as before. The only thing that has changed is the operating system.

Remote desktop between WinXP computers through the VPN still works fine. For now, I remote desktop to a Win2003 R2 server through the VPN and then remote desktop from the server to my Win7 machine, which is slow to say the least.

Both computers are connected to the network directly, not wirelessly. I tried turning off Jumbo packets on both machine, but that did not help.

This is driving me crazy, any ideas?

Everyone's tags (5)
1 ACCEPTED SOLUTION

Accepted Solutions
New Member

Re: rv082 VPN remote desktop encryption error

Setting the WAN1 interface to manual 1500 bytes just forces that WAN interface to operator at that specific MTU as opposed to allowing to to negotiate that automatically.

You asked whether or not there are any consequences for changing this setting and the answer really depends on what the correct MTU should be for your connection. Forcing it to this MTU may be causing you to fragment packets. You can run a test from your network to help determine what would be the most ideal setting for your connection.

Open up a command prompt on a machine on your LAN and get an external address to ping out on the internet. Run the following command where x.x.x.x is the external IP address.

ping -f -l 1500 x.x.x.x

This will send a ping with an packet size of 1500. If that MTU is not correct you will get a message that reads "Packet needs to be fragmented but DF set". If the MTU is fine then you will get your standard reply results. If you get the fragmentation error just keep testing by running the same command but lowering the packet size until you get the normal replies. I hope this helps and you may find that 1500 is the perfect MTU for your connection.

30 REPLIES
Cisco Employee

Re: rv082 VPN remote desktop encryption error

Just to make sure I fully understand; would this be correct?

Remote Site > [RDP] >> WinServer'03_R2 > [RDP] >> W7_Client (End Point)

So no issues with RDP session  to W2k3_R2, but when connected from the server to the W7 box you receive the error. Are connecting to a Windows domain? This sounds like there is a compatibility, or maybe authentication problem between the Server and W7. I do not have W7 readily available at the moment but will take a look. In vista, a more "secure" RDP session was introduced and that followed into W7. You may want to try to change the RDP setting to allow any type of connection (the least "secure") which would be the equivalent of XP RPD sessions. If that is already set, take a look at MS forums for more insight.

New Member

Re: rv082 VPN remote desktop encryption error

Alegalle,

Thanks for your reply. I am afraid that I did not explain myself very well. The problem only occurs when I try to go straight from one Win7 machine to the other Win7 machine through the VPN and then only after a few seconds. Some times it works even as long as a minute before I receive the encryption error.

W7_Remote Site > [RDP] >> VPN tunnel > [RDP] >> W7_Client (End Point) -  Error after a few seconds.

W7_Local Site > [RDP] >> LOCAL network > [RDP] >> W7_Client (End Point) - No error

W7_Remote Site > [RDP] >> VPN tunnel > [RDP] >> WinServer'03_R2 (End Point) - No error

W7_Remote Site > [RDP] >> VPN tunnel > [RDP] >> WinServer'03_R2  > [RDP] >> LOCAL network >> W7_Client (End Point) - No error

Both Win7 machines already have the RDP security setting set to 'less secure'. Connecting is not the problem. The seesion always ends abruplty because of the encryption error.

Since the error does not occur when I RDP between the two machines over a local network, but only when I RDP through the VPN, I know the error is caused by the VPN router.

New Member

Re: rv082 VPN remote desktop encryption error

have the exact same problem.

wrv200 at home. ipsec tunnelled to rv042 at work.

xp or vista pc at home can rdp via ipsec tunnel just fine to any of the xp machines at work.

set up a new win7 pc at the office.

- rdp to the win7 pc works from the xp machines at work (inside the lan, no tunnel involved).

- at home, i can rdp just fine to the win7 machine at work if i use port forwarding on the office router to bypass the ipsec tunnel. tested with both xp and vista pc's at home.

- however, at home i cannot rdp to the office win7 pc via the ipsec tunnel. it will log in just fine, but after a few seconds or a minute tops i get a "Because of an error in data encryption, this session will end." message and it boots me off.

the win7 pc will run rdp just fine and accepts requests even through port forwarding from the wan side, so whether it's a microsoft issue or a cisco issue, the ipsec tunnel definitely has something to do with the error.

Cisco Employee

Re: rv082 VPN remote desktop encryption error

Have not been able to replicate problem on W7 Ent. x64. Can you post phase1 & 2 configuration? Are the computers all part of a Domain, if so; is it a 2003 or 2008 Funtional Level domain?

Have you looked in event viewer  for any run time errors, IPSec service crash etc.? Any information that like this would be very useful.

Thank you.

New Member

Re: rv082 VPN remote desktop encryption error

have tried using different NICs, and also tried with another computer at work that has win7 installed. exact same error for all alternatives.

the computers are not part of a domain.

ipsec tunnel parameters:

IPSEC SETUP:
Keying Mode: IKE with Preshared key
Phase1 DH Group: Group5
Phase1 Encryption: 3DES
Phase1 Authentication: SHA1
Phase1 SA Life Time: 28800 seconds
Perfect Forward Secrecy: YES
Phase2 DH Group: Group5
Phase2 Encryption: 3DES
Phase2 Authentication: SHA1
Phase2 Life Time 3600 seconds

ADVANCED:
Aggressive Mode: YES
Compress (Support IP Payload Compression Protocol(IPComp)): No
Keep-Alive: NO
AH Hash Algorith: MD5
NetBIOS Broadcast: No
NAT Traversal: No
Dead Peer Detection: YES, Interval 10 seconds

Found this item in the event viewer:

System

  - Provider

   [ Name]  TermDD

  - EventID 56

   [ Qualifiers]  49162

   Level 2

   Task 0

   Keywords 0x80000000000000

  - TimeCreated

   [ SystemTime]  2009-12-21T02:10:13.693243200Z

   EventRecordID 2314

   Channel System

   Computer i5A

   Security

- EventData

   \Device\Termdd
   192.168.1.102
   0000040002002C000000000038000AC00000000038000AC00000000000000000000000000000000006000AD0


--------------------------------------------------------------------------------

Binary data:


In Words

0000: 00040000 002C0002 00000000 C00A0038
0008: 00000000 C00A0038 00000000 00000000
0010: 00000000 00000000 D00A0006


In Bytes

0000: 00 00 04 00 02 00 2C 00   ......,.
0008: 00 00 00 00 38 00 0A C0   ....8..À
0010: 00 00 00 00 38 00 0A C0   ....8..À
0018: 00 00 00 00 00 00 00 00   ........
0020: 00 00 00 00 00 00 00 00   ........
0028: 06 00 0A D0               ...Ð

Cisco Employee

Re: rv082 VPN remote desktop encryption error

Phase2 Encryption: 3DES
Phase2 Authentication: SHA1
======
ADVANCED:
======
AH Hash Algorith: MD5

Again I have not been at work so I have not had a chance to test your settings exactly; but one thing stood out very clearly. In the settings above it is better (typically) to set the ESP encryption/decryption to NULL when we are using AH in the tunnel.

Another thing to take a look at, is time. Make sure both computer's time is correct and you do not have any other errors that may pertain to authentication.

Give that a go and let us know if we are making progress.

New Member

Re: rv082 VPN remote desktop encryption error

Sorry- my earlier details were incomplete. under Advanced > AH Has Algorithm, MD5 is selected in the drop-down box,but that parameter *does not* have a check mark.

sorry i'm not familiar with how to set ESP to null (i am a complete novice at this), but I did try disabling "perfect forward secrecy" on both routers. the tunnel re-established just fine after that (able to rdp to the xp machines at work using lan ip address, as before), but i still get the exact same error when trying to rdp to the win7 machines. i have tried this from 3 different computers at my house (2 vista, 1 xp), trying to log into 2 different windows 7 computers at the office, and still the same error in every case.

and much thanks, btw, for helping me try to troubleshoot over the weekend.

Cisco Employee

Re: rv082 VPN remote desktop encryption error

OK, at this point I feel that the problem may be with the certificate on the W7 machine. We need to take a closer look at the event log, but from the event you posted it is very similar to issues I ran into with Vista. What happens is that the W7 client tries to hand out its Certificate for authentication and when that fails, the RDP session drops. If this is the case we should be able to see an event stating that W7 client ended the session, not the other way around. If you feel comfortable, follow these steps to remove the certificate for RDP on the W7 clients:

Start > Run > mmc.exe

from mmc console select > File > Add/remove snap in > Certificates >>>> New Window > "Computer Account" > "Local Computer">> Finish and then OK

Expand Certificates > Remote Destop > Certificates > There should be one cert there with your computer name on it. **IMPORTANT** Before you continue:

Make a system restore point before you delete the cert or just take it out and save it in a different place. Just a precaution!

Once you have removed the cert try again and see if the problem is resolved. Once more though, make sure the time on all computers are correct as any computer connecting to the W7/Vista machine will cause it to regenerate a cert and the problem will persist as long as the time is not correct!

EDIT:

Dont worry about the AH setting, if it is not being used just leave it as is. No need to add more complexity.

New Member

Re: rv082 VPN remote desktop encryption error

just tried your steps and still getting the same error. also pls remember that i am able to rdp into these 2 very same windows 7 machines at the office if:

1) the client is on the same lan (i.e. another office pc); or

2) if the client is connected from the house via an open port in the office router

i did double check the clocks on all the computers involved though.

Cisco Employee

Re: rv082 VPN remote desktop encryption error

Yeah, did realize that the problem was basically on the tunnel only; just all other symptoms seemed all too familiar. Again, take a look at event logs and post anything of interest. Also ensure logging is enabled on the RV and we will take a look at that as well. It would be a good idea to dump all events and logs to begin log capture from momment of tunnel connection through a few attempts of the RDP connection.

At this point it may be best if you call the supprt center and open a support ticket. I will continue to assist as much as possible; and will test this tomorrow.

866.606.1866 Small Business Center.

New Member

Re: rv082 VPN remote desktop encryption error

I also tried the above step of removing the certificate, but it did not change anything. It is a tunnel issue. As I mentioned in my original post, I too can RDP to the WIN7 machine from on the local network, and through the tunnel only for several seconds. I can RDP to WINXP and Win 2003 machines through the tunnel without issue. My settings look the same as jtejavanija.

Thanks for helping us.

Cisco Employee

Re: rv082 VPN remote desktop encryption error

OK, this is what I have:

RV082 (Remote) ==> RV016 (Local)

Remote site is my computer running XP SP3 that will connect to W7 x64 ent.

Tunnel Information:

Screen shot 2009-12-22 at 1.52.35 AM.png

Advanced Options:

Net Bios Broadcast

Keep Alive

Dead Peer Detection

I have had this connection up for the 5 hours, and most of the time is has stayed idle. All computers are "Workgroup", and there is a DHCP and DNS server on both sides. I have changed that behavior to just use the router for DHCP and DNS but there was no effect. I am trying really hard to replicate this issue but at the momment I have not been able to.

Make sure you have all updates from MS for W7, and the computer you are using to connect from. At this point, we will really need to take a look at your router, and clients to see if there are any problems there. If you can post logs, from the routers and clients that would be great. I would like to make sure the tunnel is stable and running correctly.


I will continue to leave this RDP connection running and wait for either a time out, or disconnect. I really feel that the problem is on the W7 client but it is just a suspicion and not ruling out the router yet.

New Member

Re: rv082 VPN remote desktop encryption error

i solved the problem. i replaced the brand new RV042 i just bought with a new RVS4000 and that did the trick. all the settings are the same as your test case, except NET BIOS broadcast is disabled in our setup. i didn't change a single setting on the WRV200 that is on the other end of the tunnel.

i've had rdp up for nearly an hour now. i'm even typing this post via rdp.

some notes:

RV042 had the latest firmware available on the cisco website as of this weekend

RVS4000 has a firmware build that is even newer than the one avaialable on the cisco site (i just received it today and it came that way)

a lot of internet chatter discusses how the RV042 is more reliable/stable than the RVS4000. but for us it turned out to be the opposite. i actually was prompted to do the replacement because i could not get quickvpn to work on the RV042, and i knew that it would work with the RVS4000 (we had one for many years and it finally died this month and i replaced it with the RV042 ... then i got the windows7 machine so we never tested win7 RDP w/ the old RVS4000).

either we had a defective RV042 unit, or there is something wrong with the model's hardware/firmware that is impeding RDP via IPSEC tunnel for win7.

THANK YOU again for your help in troubleshooting. i am a loyal linksys customer now because of this, despite my problems with the RV042.

New Member

Re: rv082 VPN remote desktop encryption error

I've been having the same problem.  Before I go buy the RVS4000, will you please tell me if you are using the Win7 native client, or are you using something else, like Shrew Soft.

New Member

Re: rv082 VPN remote desktop encryption error

not familiar with shrew soft. i'm on a gateway-to-gateway ipsec vpn connecting home and work, and i use my home xp and vista pc's to rdp into my work win7 pc. hope that answers your question.

Cisco Employee

Re: rv082 VPN remote desktop encryption error

That doesn't make any sense!



Really wish we had some logs to look at, as I have not been able to reproduce the problem or even come close to see an error. I will take a closer look at the RV042 firmware and will use that router for the IPSec tunnel, but really there are no big differences between the RV0xx routers.

I am glad you were able to correct the issue, but replacing the product is not what I was hoping for. Thank you very much for the input and I am sorry I was not able to find the problem but will continue to look at this issue. If you were able to open a case with Small Business Support could you please IM me the case number? I would like to take a closer look at the case notes and possibly speak with you about your configuration.

Thank you,

Alejandro

New Member

Re: rv082 VPN remote desktop encryption error

Okay.  Using Win7Pro, Shrew, and RV042, I am able to get to the point where "tunnel enabled."  I then try to open a remote desktop connection.  No answer from my office computer, which is also using Win7Pro and has been set to accept incoming remote desktop connection.  I then try to set up new connection to my office. The closest I can get is an error code saying DNS not responding or remote server not responding, but I don't have a remote server as such.  Here's the topology:

HomeComputer >> CableModem >> Internet >> DSLModem(bridged) >> RV042 >> Switch(Linksys) >> OfficeComputer(x5)

My particular office computer is set to listen to a particular port.  When I try to establish a remote desktop connection, I type in the IP address and the port number (e.g. 12.34.56.789:12345).  Still, I get no response.  Assuming all of the configurations are correct, should I get a new router, or do I need to dedicate one of the five office computer to being a server with Win Server 2008?

New Member

Re: rv082 VPN remote desktop encryption error

I think that pgorden's post is off topic. These posts have been about an encryption error that occurs after a remote desktop connection is made. Please start a new post.

Re: rv082 VPN remote desktop encryption error

Hi pgordon,

Check out this posting below as well.  Well worth a look at using the native IPSec client in windows 7.  Just wish I had a powerful enough PC that would accept  windows 7 so i could validate the procedure in the posting below;

https://www.myciscocommunity.com/docs/DOC-13592#cf

New Member

Re: rv082 VPN remote desktop encryption error

I am still having the same problem. I am using rv082 on both sides with the latest frimware (2.0.0.19-tm). My settings are the same as you describe. This is really driving me crazy!

Which logs would you like to see specifically?

I would like to mention that it does not happen right away. You need to create some traffic by opening windows through the remote desktop.

Cisco Employee

Re: rv082 VPN remote desktop encryption error

I would like to see the VPN, and Firewall logs; keep in mind that they be very lengthy and would have all your IP information and also ports being used for connections. If possible please call the Small Business center and open a case or send me an IM via your Cisco Community account.

SBSC: 1.866.606.1866 (USA)

New Member

Re: rv082 VPN remote desktop encryption error

I looked in the logs on the target machine and found something interesting:

"encountered a bad packet from xxx.xxx.xxx.xxx. Packet processing leads beyond packet length".

Googling the error lead me to the following post:

http://social.technet.microsoft.com/Forums/en-US/winservergen/thread/51d005e2-8ac4-4fa8-bbf7-f8c2e3f4dce4

I read the post and we don't have Broadcom network cards, so that can't be it. The cards are Intel on both sides. I disabled "Jumbo packets" and  "IPv4 Large Send Offload" on both, but it did not help. Besides I don't have any problems when I RDP locally.

Then he goes on to suggest enabling "Rate Control" on the  "Bandwidth Management" page. I tried it on both sides and it changed nothing.

Any ideas now?

Cisco Employee

Re: rv082 VPN remote desktop encryption error

From that line in  your log

"encountered a bad packet from xxx.xxx.xxx.xxx. Packet processing leads beyond packet length".

I would think that the problem is not due to jumbo frames or anyting like that. It seems like we are getting CRC errors. You mentioned that you turned off jumbo frames on your NIC, are they enabled on all your machines? Also is the router connected to a switch with jumbo frames enabled? In your log did you happen to notice anything saying "Policy Violation" or any logged events on either W7 machine (Application Event Logs or System Event Logs)?

New Member

Re: rv082 VPN remote desktop encryption error

The router is connected to a Cisco SR2024 Gigabit Switch.

New Member

Re: rv082 VPN remote desktop encryption error

I found something that seems to work for now!

Under the 'setup' tab of rvo82, I set the MTU size for WAN1 to 'Manual' 1500 bytes. Since then, I have not seen the encryption error once!

Does this settings have any other consequences?

New Member

Re: rv082 VPN remote desktop encryption error

Setting the WAN1 interface to manual 1500 bytes just forces that WAN interface to operator at that specific MTU as opposed to allowing to to negotiate that automatically.

You asked whether or not there are any consequences for changing this setting and the answer really depends on what the correct MTU should be for your connection. Forcing it to this MTU may be causing you to fragment packets. You can run a test from your network to help determine what would be the most ideal setting for your connection.

Open up a command prompt on a machine on your LAN and get an external address to ping out on the internet. Run the following command where x.x.x.x is the external IP address.

ping -f -l 1500 x.x.x.x

This will send a ping with an packet size of 1500. If that MTU is not correct you will get a message that reads "Packet needs to be fragmented but DF set". If the MTU is fine then you will get your standard reply results. If you get the fragmentation error just keep testing by running the same command but lowering the packet size until you get the normal replies. I hope this helps and you may find that 1500 is the perfect MTU for your connection.

New Member

Re: rv082 VPN remote desktop encryption error

Good idea. I played around with the ping using a fixed packet size and I found that 1472 was the right size for my connection. So that is the solution encrption error.

The target computer was receiving packets which were too large. This caused CRC errors which the RDP saw as encryption errors.

Under the 'setup' tab of the rv082 router, I set the MTU size for WAN1 to 'Manual' 1472 bytes. I did this on both routers. Since then I have not had one encryption error.

The correct packet size will depend on your internet connection. Use Mihagen's idea to check the correct packet size for each router: ping -f -l 1500 x.x.x.x

Thnaks to Alegalle and Mihagen.

Problem solved!!

New Member

Re: rv082 VPN remote desktop encryption error

i tried doing the ping test. on my home router it works just fine- highest mtu with normal ping results is 1430. changing this value allowed me to properly use quickvpn from home into my office- it didn't work when mtu was set to auto (this is with the ipsec tunnel disabled of course).

however, on my work router (rvs4000) at mtu 1473 it says it has to break up the packets. but at mtu 1472 it says timed out for all 4 attempts. (all mtu values lower than 1472 also say timed out, and all values higher than 1473 also says it has to break up the packets).

the rvs4000 at the office does seem to work fine right now though, with mtu at "auto" (the field says "1500" grayed out, in case that matters). i just don't know whether i need to manually configure it to have an optimized connection.

any advice would be appreciated.

Cisco Employee

Re: rv082 VPN remote desktop encryption error

As Michael stated you may have found the problem. In TCP if we send a packet that is malformed or has to be fragmented, the receiving end will take care of it by either requesting a new packet or if fragmented, waits until all fragments are received and then puts the data back together. Now if we add on top of that encryption for the VPN tunnel and Encryption for the RDP session and we are constantly having to resend packets, it is easy to see why the connection would drop. I do most of my work from a mac, and in the Mac version of Command Prompt we are able to get a really good idea as to what the packet size should be. Here is a little example I did from home: (cable connection)

>>> The options for "ping" in Mac are different than on Windows, but the test is the same. We don't see "Reply from" just the data is returned which is more useful in my opinion and shows more clearly what happens to the data. Take note of the colored numbers. <<<

LayOff2:~ agallego$ ping -s 1500 google.com
PING google.com (64.233.169.103): 1500 data bytes
1480 bytes from 64.233.169.103: icmp_seq=0 ttl=243 time=37.670 ms
wrong total length 1500 instead of 1528

Now I reduced the MTU to 1480

LayOff2:~ agallego$ ping -s 1480 google.com
PING google.com (64.233.169.106): 1480 data bytes
1480 bytes from 64.233.169.106: icmp_seq=0 ttl=243 time=33.656 ms
wrong total length 1500 instead of 1508

Now to 1478

LayOff2:~ agallego$ ping -s 1478 google.com
PING google.com (64.233.169.106): 1478 data bytes
1480 bytes from 64.233.169.106: icmp_seq=0 ttl=243 time=39.252 ms
wrong total length 1500 instead of 1506 <=== Still off by 6

Finally

LayOff2:~ agallego$ ping -s 1472 google.com <=== This is the Max I can transmit. Anything less will be just fine.
PING google.com (64.233.169.147): 1472 data bytes
1480 bytes from 64.233.169.147: icmp_seq=0 ttl=243 time=33.566 ms
1480 bytes from 64.233.169.147: icmp_seq=1 ttl=243 time=37.852 ms
1480 bytes from 64.233.169.147: icmp_seq=2 ttl=243 time=35.010 ms

12005
Views
0
Helpful
30
Replies
CreatePlease login to create content