Solved: Software Error in VCS x8.1.1 Causing Call Drops

Matt Stevenson · ‎05-12-2014

Hi there,

Our org was stern in wanting to remediate the Heartbleed vulnerability so we quickly upgraded to x8.1.1 after it's release. Since then we've started to see a software error on VCS that we've never seen previously. I did some Googling and looking through Cisco docs but couldn't find much.

The error is as follows:

An unexpected software error was detected in app[454]: SIGABRT (tkill(2) or tgkill(2))

When the error manifests itself we see call and registration failures throughout our environment. Anyone seen this error before or have any insight? I have a TAC case opened but it is moving slowly.

We have two VCS-C's clustered and running this code and both VCS's receive a similar alarm. I can provide the entire error log if anyone is interested.

Thanks in advance,

~Matt

Matt Stevenson · ‎07-25-2014

Hi All,

Since someone recently upvoted this post I realized I hadn't replied with the answer. After working with TAC for a while a bug was found and here is the current fix. After following these steps the call drops stopped. Below is the bug and workaround. Not sure if this is fixed in x8.2 as we haven't had a chance to upgrade yet.

VCS crash - ACR: SdpMsg_MD_isSetToDefaultFormat Line: 2486

CSCuo42252
Description
Symptom:
Internal application error can cause a system restart.

Conditions:
This issue gets triggered when CUCM sends VCS malformed media lines without media formats on those lines.

e.g.
m=application 0 RTP/AVP

Note: correct media line would have a "0" at the end.

Workaround:
If this condition is encountered the VCS will restart without intervention. No user input is required.

Neighbour zones to CUCM can be edited so that they use a Custom zone profile rather than the CUCM zone profile. Then set this Custom zone to have all the same settings as a normal CUCM zone, except set the SIP/UDP IX Filer to off.

Page 155 of the Administrators Guide covers these zone profiles: http://www.cisco.com/c/dam/en/us/td/docs/telepresence/infrastructure/vcs/admin_guide/Cisco-VCS-Administrator-Guide-X8-1-1.pdf

Thanks,

~Matt

View solution in original post

Alok Jaiswal · ‎05-13-2014

Hi Matt,

we need to see the crash report on the VCS to check this issue. can you tell me the SR number and if you have the crash report attached there i can check the logs.

regards

Alok

Matt Stevenson · ‎05-14-2014

Hi Alok,

Thanks for reaching out. The crash report has been added to the case as of last night and I am preparing to upload full system snapshots within the hour.

SR - 630226543

Thanks,

~Matt

Bryan Birchfield · ‎12-16-2014

Matt,

We have been seeing the same issue with a bunch of our VCS Controls (X8.1.1). I happened across this workaround and will be implementing it shortly. However, has a permanent fix or patch been released by Cisco that you know of?

petrojosh · ‎12-16-2014

Matt,

We have the exact same issue in 8.2.1, single VCS C and E. We get the same error you do as well as this one. I would recommend staying at 8.1 or below for now since TAC says the bug we're hitting is not fixed until 8.5 with no ETA. You can mitigate the Heartbleed vulnerability with Firewall rules for now. We had zero issues on VCS 7.2, so i would stay put.

https://tools.cisco.com/bugsearch/bug/CSCup83131/

SR 632496063

shantanoo lonkar · ‎05-20-2015

I am getting this error on VCS 8.5.1 too.

An unexpected software error was detected in app[8214]: SIGABRT (tkill(2) or tgkill(2))

When cisco going to release the FIX for this?

Note:It was happening from VCS 6 release but still not fixed upto 8.5.1

Matt Stevenson · ‎07-25-2014

Hi All,

Since someone recently upvoted this post I realized I hadn't replied with the answer. After working with TAC for a while a bug was found and here is the current fix. After following these steps the call drops stopped. Below is the bug and workaround. Not sure if this is fixed in x8.2 as we haven't had a chance to upgrade yet.

VCS crash - ACR: SdpMsg_MD_isSetToDefaultFormat Line: 2486

CSCuo42252
Description
Symptom:
Internal application error can cause a system restart.

Conditions:
This issue gets triggered when CUCM sends VCS malformed media lines without media formats on those lines.

e.g.
m=application 0 RTP/AVP

Note: correct media line would have a "0" at the end.

Workaround:
If this condition is encountered the VCS will restart without intervention. No user input is required.

Neighbour zones to CUCM can be edited so that they use a Custom zone profile rather than the CUCM zone profile. Then set this Custom zone to have all the same settings as a normal CUCM zone, except set the SIP/UDP IX Filer to off.

Page 155 of the Administrators Guide covers these zone profiles: http://www.cisco.com/c/dam/en/us/td/docs/telepresence/infrastructure/vcs/admin_guide/Cisco-VCS-Administrator-Guide-X8-1-1.pdf

Thanks,

~Matt