Latest Microsoft Feb. 2015 patch breaks AnyConnect SMC

Unanswered Question
Feb 11th, 2015
User Badges:

Hi all,

 

I just wanted to give the community a heads up in regards to the latest February 2015 Microsoft patches.KB3023607 makes some AnyConnect clients give the "Failed to initialize connection subsystem" error.  You can fix this here:

http://christierney.com/2015/02/11/cisco-anyconnect-failed-to-initialize-connection-subsystem/

 

Also updated in the article:

This issue was introduced by KB# 3023607: Secure Channel cumulative update changes TLS protocol renegotiation and fallback behavior (https://support.microsoft.com/kb/3023607)

Included with Microsoft Security Bulletin MS15-009 – Critical Security Update for Internet Explorer (3034682)

This issue should also affect Windows 7 user with IE 11, but no reports of failure have been seen yet.”

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 4.2 (7 ratings)
Loading.
Vikingsoft Wed, 02/11/2015 - 14:04
User Badges:

FYI: on my Windows 8.1 system the christierney.com procedure was not sufficient to workaround the problem. I had to repeat the compatibility troubleshooter against "vpnagent.exe" before I could get VPN connections via my AnyConnect client.

("vpnagent.exe" is the local service that supports the client user interface.)

liambreathnach Thu, 02/12/2015 - 02:48
User Badges:

So, is there any info on which AnyConnect clients can work with KB3023607?

And if this is a bigger issue, does anyone know if Microsoft are working on a fix?

Thanks. Just wondering what to do about all my staff on Windows 8.1 who use AnyConnect.

Peter Davis Thu, 02/12/2015 - 09:23
User Badges:
  • Cisco Employee,

Cisco Tracking ID: CSCus89729

https://tools.cisco.com/bugsearch/bug/CSCus89729

Cisco opened a priority 1 case with Microsoft yesterday as soon as we found out about this issue. We are continuing to escalate this issue with Microsoft for a resolution timeframe. We recommend that all customers open their own cases with Microsoft since the ultimate fix will need to come from them. You can feel free to reference Cisco's case # which is 115021112390273 in order to expedite having your ticket properly triaged by their support team.

 

There are two potential workarounds until Microsoft provides a fix

1. Windows 8 compatibility mode for the app

2.Customers can uninstall the KB3023607 update from Microsoft. However, this will also remove any other security fixes provided by Microsoft as part of the update. This can be removed under:

Control Panel / Programs / Programs and Features, click "View installed updates” on the left and locate and uninstall the update labeled with KB3023607.  This update is not visible when you try to locate it through the Windows Update application’s history, but it is accessible via Control Panel.

S M85 Thu, 02/12/2015 - 07:11
User Badges:

Hi Peter,

 

I've also opened a TAC case. Is it possible to make the BUG tracking ID public for us? 

 

regards,
Sander

Peter Davis Fri, 02/13/2015 - 06:12
User Badges:
  • Cisco Employee,

This is a defect in the Microsoft 02/10/15 patch and not a bug in the AnyConnect software. Microsoft is aware of this and is working on a fix. There are two possible workarounds until a fix is available, the first is to use Windows 8 compatibility mode for the app, the other is to uninstall this specific KB article (you would also lose other security fixes associated with it, so proceed with caution on this option).

Dmitry Sobolevsky Fri, 02/13/2015 - 07:58
User Badges:

Does Cisco have official response on this issue yet? Also, it would be great to know which version of ASA OS affected users are running (not all versions and interim releases have all necessary SSL security fixes). Also, not every sub release of AnyConnect client is the same. We are not experiencing any issues with AnyConnect 4.0.00051and AnyConnect 3.1.05060 currently installed in environment with this patch installed. All the PCs are mix of Windows 8.1 Pro and/or Windows 7 with IE11 (of course). Our OS is 9.1.5.21, latest Interim release, and we have everything except TLSv1.0 disabled (no SSLv3.0 allowed). Configuration of AnyConnect policies can also play a role here (SSL vs DTLS vs IKEv2).  This issue is making the rounds over the Internet as a significant problem and is being brought up by my management - basically creating somewhat a concern. Clarification is necessary.

Peter Davis Fri, 02/13/2015 - 08:13
User Badges:
  • Cisco Employee,

The issue is not the ASA or AnyConnect, it is a defect in Microsoft's February 2015 (02/10/15) security patch which affects all AnyConnect users on Windows 8.1 and a subset (unclear what subset yet) of users on Windows 7 with IE11. This has nothing to do with TLS versions which are enabled or disabled. Microsoft is aware of the defect that they introduced and are actively working on a fix.

 

Our public statement on the topic and a couple of workarounds can be found in the Cisco bug search tool (link below) for authorized Cisco.com users or you can view an abbreviated statement on our social media Facebook page at www.facebook.com/anyconnect

https://tools.cisco.com/bugsearch/bug/CSCus89729

Dmitry Sobolevsky Fri, 02/13/2015 - 09:18
User Badges:

This article clearly only applies to Windows 8.1. On Windows 7 update 3023607 gets installed with update 3021952, and not with 3034682 (as referenced in the article). Also on my Windows 8.1 PCs KB3203607 shows as a separate updated, and not part of anything. The more I look at it, the more it looks like a corrupt update install behavior and not really a problem with update. All my updates, be that Windows 7 or Windows 8.1 are listed installed exactly in a manner described in MS article for MS15-009. Perhaps users affected got early versions of MS15-009 updates, while the rest of us got normal versions on 02/11. Cisco really needs to do better job on troubleshooting and documenting the issues.

 

Peter Davis Fri, 02/13/2015 - 09:52
User Badges:
  • Cisco Employee,

We are very sorry you are not pleased with our analysis. We worked very hard to be responsive to customers in evaluating and reporting on the situation as soon as we learned about it.

 

We can confirm (since we are working directly with Microsoft on this issue) that it is due to a bug in the Windows 8.1 patch KB3012982 (which gets wrapped under KB3203607) and not a corrupt update install. This patch was wrapped in with the MS15-009 update for Windows 8.1 users.

 

As far as the few reports we have had with issues on Windows 7 w/ IE11, we have removed any reference to this pending further investigation since Microsoft does not believe that their update should affect W7 users.

Drew T Sun, 02/15/2015 - 22:11
User Badges:

Peter, It's been three days now since we had an update. Is there any update on this? With the bad weather the East Coast USA is experiencing, I have a heap of my remote staff now not able to remotely connect to work from home, and it's causing headaches.

 

There's some reports that 4.x verisons of AnyConnect Mobility may not be affected, but I can't roll that out on a whim. 

Peter Davis Mon, 02/16/2015 - 04:51
User Badges:
  • Cisco Employee,

Hi Drew,

MS is still working on putting out a patch but they have not given us any timeframe as to when this will go out.  Since the fix will need to be released by Microsoft, my recommendation is to open up a direct case with them on this issue.  While not perfect, we did publish a couple of workarounds for this topic.

Mark H Tue, 02/17/2015 - 01:59
User Badges:

We have an active case open with Microsoft and they have stated they intend to resolve this issue in the March security updates.

 

In the meantime, they have released a 'FixIt' which is available at https://support2.microsoft.com/kb/3023607.

Pacerfan9_2 Tue, 02/17/2015 - 10:07
User Badges:

Windows 8 computers unable to connect after KB3023607 installed. I tried Microsoft's Fix It 51033 however that does not resolve the issue. VPN Client says "Lost connection to VPN service. Reattaching...." We authenticate using AD + certificate, could the certificate be causing an issue as well?  If I uninstall KB3023607 I am able to connect.

patoberli Tue, 02/17/2015 - 22:42
User Badges:
  • Bronze, 100 points or more

You did reboot after the installation? That is stated on the download page (at least a logout/login).

Here it works fine with the Fixit, but we use username/password without certificate.

Pacerfan9_2 Wed, 02/18/2015 - 06:22
User Badges:

Tried rebooting and two different computers, did not work with Fix It. Tested with AnyConnect version 3.1.05187 and also tried upgrading 3.1.06079.

danpomeroy Thu, 02/19/2015 - 11:47
User Badges:

Did you ever resolve your issues with this? We did by putting our exe's in Win7 compat mode. Simple and quick.

Peter Davis Wed, 02/18/2015 - 07:21
User Badges:
  • Cisco Employee,

This is the first report we have had that Microsoft's fixit did not work for someone as a workaround.  Please try to re-install KB3023607 and the fixit, restart your machine and if it does not work, please send us a Diagnostics Report (DART) from AnyConnect to [email protected]. What version of AnyConnect are you using? Perhaps it is an old version and you should be updating.

danpomeroy Wed, 02/18/2015 - 08:57
User Badges:

We are having the exact same issue. Hundreds of our customers are down. Tried the Microsoft Fix-It, terrible results. We have only had 3 successes with that. We are still uninstalling and hiding the update. Working our way through them but this is ridiculous.

 

I have a clean 8.1 vm and can re-create the Fix-It not working. Also have 8.1 on my home computer and can re-create it there too. I would LOVE to help if we can get something for our customers.

Peter Davis Wed, 02/18/2015 - 09:00
User Badges:
  • Cisco Employee,

Sorry to hear that. Please make sure you are using current AnyConnect releases of 3.1.x or 4.0.x to make sure you are not hitting an old bug.  Once doing that, if you are still hitting failures, please open a case with Microsoft for further troubleshooting.

danpomeroy Wed, 02/18/2015 - 09:11
User Badges:

Thank you,

 

we have tried every version of AnyConnect we have including the 3.1.06079.

 

We are opening a ticket with Microsoft.

Peter Davis Wed, 02/18/2015 - 09:14
User Badges:
  • Cisco Employee,
danpomeroy Wed, 02/18/2015 - 10:30
User Badges:

Peter,

 

I just sent you a Dart bundle. Had some of my guys test the newest AC and the Newest fix on a couple of our end-users also. Just to make sure. That's why it took me so long to respond. No luck.

Peter Davis Wed, 02/18/2015 - 10:34
User Badges:
  • Cisco Employee,

No problem. Please confirm you see Microsoft's "fixit" installed under Control Panel and that you have logged out/in or rebooted after installing it.  We are examining your logs now.

Peter Davis Wed, 02/18/2015 - 11:53
User Badges:
  • Cisco Employee,

In Dan's case, the Microsoft "fixit" does not cover the API usage, only the normal User Interface for AnyConnect. We have escalated this issue back to Microsoft to inform them of this gap. Unfortunately the "fixit" is a workaround a not a full fix for the OS regression, which Microsoft is planning for their March patch cycle (Microsoft's dates are subject to change).

 

We will also document this limitation with their fixit in our release notes.

 

Peter Davis Thu, 02/19/2015 - 06:52
User Badges:
  • Cisco Employee,

Microsoft has informed us that they will not be pushing out an updated fixit for customers leveraging the API. They recommend compatibility mode for both vpnagent.exe and vpnui.exe as a temporary workaround for these customers.

HKEY_LOCAL_MACHINE
SOFTWARE\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers
Valuename : C:\Program Files (x86)\Cisco\Cisco AnyConnect Secure Mobility Client\vpnui.exe  <--- also do the same for vpnagent.exe
Valuedata : ~ WIN7RTM

We were able to solve the issue (thus far) using the Microsoft FixIt and by adding the following registry entries:  Neither the FixIt nor the registry entries alone solved the issue.  These entries set the two programs to Windows 7 compatibility mode for all users.

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers
Valuename : C:\Program Files (x86)\Cisco\Cisco AnyConnect Secure Mobility Client\vpnui.exe
Valuedata : ~ WIN7RTM

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers
Valuename : C:\Program Files (x86)\Cisco\Cisco AnyConnect Secure Mobility Client\vpnagent.exe
Valuedata : ~ WIN7RTM

danpomeroy Thu, 02/19/2015 - 07:22
User Badges:

Peter,

 

I added the 2 reg entries to my computer, rebooted, and still can't connect. I verified that the shim is installed.

 

I attached a screenshot of my registry entries.

 

Any ideas?

Attachment: 
danpomeroy Thu, 02/19/2015 - 11:44
User Badges:

We have a work-around that is quick, for us.

 

We have put our apps exe's in Win 7 compatibility mode. That's it. No reboot/shim/anything else .required.

 

Thank you all for your help. Hopefully this helps someone else.

pwing0001 Fri, 02/13/2015 - 23:13
User Badges:

Thanks for these suggestions. I experienced the same problems on my Windows 8.1 machine. The compatibility workarounds were ineffective, but uninstalling the KB3023607 update restored normal function of Cisco Anyconnect.

 

Tyson Mock Wed, 02/11/2015 - 21:42
User Badges:

I'm having the same issue... "failed to initialize connection subsystem"

Anyconnect v3.0.05178

I have Win8.1 Pro.  Setting the vpnui.exe file to run using Windows 7 compatibility and it works again.

 

However, I will add that I also use the legacy Cisco VPN client v5.0.07.0440 was working fine prior to the update and now it fails with "Reason 442: Failed to Enable Virtual Adapter".  Not sure if there is a workaround for the legacy VPN client or if this is the final nail in its coffin.  Running the legacy VPN client in compatibility mode does not address this new issue.

dannyngo99 Thu, 02/12/2015 - 08:01
User Badges:

Thanks Todd for posting this info solution. Hopefully Microsoft will provide us a permanent workaround solution.

Marvin Rhoads Thu, 02/12/2015 - 12:51
User Badges:
  • Super Silver, 17500 points or more
  • Cisco Designated VIP,

    2017 Firewalling, Network Management, VPN

Interesting - I have the latest February 2015 patches on a Windows 7 system and my AnyConnect 4.0.00051 VPN module is working fine.

Peter Davis Thu, 02/12/2015 - 14:41
User Badges:
  • Cisco Employee,

From our analysis, the KB patch from Microsoft should only affect Windows 7 users with IE 11 (which is not there by default). It affects all Windows 8.1 users.

Marvin Rhoads Thu, 02/12/2015 - 15:00
User Badges:
  • Super Silver, 17500 points or more
  • Cisco Designated VIP,

    2017 Firewalling, Network Management, VPN

I have Windows 7 with IE 11 and the KB 3023607. Works fine, so it might not affect AnyConnect 4 or some other set of conditions I have?.

See screenshot below:


Peter Davis Thu, 02/12/2015 - 16:35
User Badges:
  • Cisco Employee,

Not sure. We had trouble reproducing this permutation in house but have had a few reports of it.

lynette.nelson1 Wed, 04/22/2015 - 10:35
User Badges:

I'm running into this issue now.  Windows 7 machine, IE 11.  If I'm connected via VPN (version 3.1.04072), IE 11 does not work at all.  It just sits and "spins" on any website I try.  However, I can access any sites through Chrome when using VPN.  If I disconnect from VPN, only then can I access sites via IE 11. 

I know this original post was about Windows 8.  Is there anything being looked at for this issue on Windows 7 with IE 11?

I just upgraded to IE 11 last week.  I had no problem with VPN connection when I had IE 9.

Oleg Volkov Fri, 02/13/2015 - 12:45
User Badges:
  • Почетные Знаки Сообщества,

    Лучшая публикация, Май 2015

Hello


Dear Sirs!
I find two trouble!


One trouble - I get the error "Failed to initialize connection subsystem"

It is resolved by the next step:

1. Go to folder with AnyConnect client, for 64bit OS, "C:\Program Files (x86)\Cisco\Cisco AnyConnect Secure Mobility Client"
For 32bit OS "C:\Program Files\Cisco\Cisco AnyConnect Secure Mobility Client"


2. Find the vpnui.exe file, right click -> properties, go to Compatibility tab, and select "Run this program in compatibility mode for: Windows 8"


Also, after update windows 8.1, I think, it is no longer work with ssl encryption rc4-sha1 !
When my config contain the ssl encryption rc4-sha1
I get the error:
"Failed to get configuration because AnyConnect cannot confirm it is connected to your secure gateway. Contact your system administrator".


After I change it to: ssl encryption aes128-sha1, AnyConnect client can connect to ASA.

Have a nice day!

 

Peter Davis Thu, 02/19/2015 - 06:52
User Badges:
  • Cisco Employee,

Microsoft has released a "fixit" to workaround the regression in their WIndows 8.1 KB3023607 02/10/15 patch. This is accessible by following the instructions at:

https://support.microsoft.com/kb/3023607

Once the fixit is installed, Cisco recommends you reboot (or logoff/logon) your PC as you need to fully restart the AnyConnect service (not just the User Interface), and not all users will have access to do so.

Microsoft's "fixit" covers the standard AnyConnect User Interface. It will not work for customers who are controlling AnyConnect via its API.

Note: The Fixit Microsoft has released is not a fix for the OS regression.

Microsoft has informed us that they will not be pushing out an updated fixit for customers leveraging the API. They recommend compatibility mode for both vpnagent.exe and vpnui.exe as a temporary workaround for these customers.

HKEY_LOCAL_MACHINE
SOFTWARE\Microsoft\Windows NT\CurrentVersion\AppCompatFlags\Layers
Valuename : C:\Program Files (x86)\Cisco\Cisco AnyConnect Secure Mobility Client\vpnui.exe  <--- also do the same for vpnagent.exe
Valuedata : ~ WIN7RTM

Microsoft is planning to release a Windows Update patch on 03/10/15 to correct the underlying issue. Microsoft's dates are subject to change.

 

t-rubio Thu, 02/19/2015 - 10:51
User Badges:

Thanks for the update Peter,  Do you by chance know if the patch that is tentatively slated for March will supersede the KB3023607 patch.  I believe it was a part of the Rollup for IE and just wondering how this will play in terms of Windows Updates.  Will the IE Cumulative update be revised to include the updated patch(KB3023607) or will MS just release a revised patch for KB3023607?  We don't want to be left unpatched after the dust settles on the issue and the new patch is released in what ever form it comes. 

Thanks again.

Peter Davis Thu, 02/19/2015 - 10:55
User Badges:
  • Cisco Employee,

We unfortunately do not have any details from Microsoft on how they will roll out this patch in March (subject to change). You may want to consider opening up a direct trouble ticket with them to see if they will supply you with this information.

Peter Davis Thu, 02/19/2015 - 11:48
User Badges:
  • Cisco Employee,

We reached out directly to Microsoft for you. They have stated that MS15-009 would need to be reapplied and that the fix due out in their March patch (date subject to change), only addresses this specific issue.

t-rubio Thu, 02/19/2015 - 11:53
User Badges:

Thanks so much for the clarification.  it much appreciated.

Peter Davis Tue, 03/10/2015 - 22:03
User Badges:
  • Cisco Employee,

Microsoft has released a permanent fix for this issue as part of their Windows 8.1 03/10/15 update.

 

See: http://support.microsoft.com/kb/3040335 for more details

 

The workarounds can be removed after applying the update and restarting your Windows 8.1 machine.

Actions

This Discussion