Buy or Renew
Buy or Renew
Cisco + Splunk: It’s a new day for your data. Learn more.
 
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
cancel
Start a conversation
3108
Views
0
Helpful
11
Replies

UCCX RTMT Alert- CoreDumpFileFound

fenilantony
Level 1
Level 1

‎11-08-2014 08:16 PM - edited ‎03-14-2019 02:05 PM

I am getting a new alert CoreDumpFileFound on UCCX RTMT.

If any one facing same issue please share

The running Cisco Unified CCX  version is 10.5.1.11001-49

 At Tue Jun 02 09:43:32 AST 2015 on node 172.XX.XX.XX;

the following CoreDumpFileFound events generated:  TotalCoresFound : 1

CoreDetails : The following lists up to 6 cores dumped by corresponding applications.

Core1 : Cisco Desktop LDAP Monitor Service (core.19621.11.LdapMonSvr.1433227379)

AppID : Cisco Log Partition Monitoring Tool ClusterID :  NodeID : UCCX1 

TimeStamp : Tue Jun 02 09:43:01 AST 2015

1 person had this problem
Labels:
0 Helpful
11 Replies 11

fenilantony
Level 1
Level 1

‎06-02-2015 01:20 AM

I got same alerts on previous uCCX version also (8.X)

0 Helpful

fenilantony
Level 1
Level 1
In response to fenilantony

‎06-02-2015 01:21 AM

Here I updated my old post with latest alert

0 Helpful

neil wooloff
Level 1
Level 1
In response to fenilantony

‎08-10-2016 12:58 AM

Hi,

Sorry, my bad.

Passed all to Cisco TAC and they confirmed this Bug:

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCuu20569/?reffering_site=dumpcr

and implemented same workaround and all stabilised.

Needs root access to do.

Neil

0 Helpful

fenilantony
Level 1
Level 1

‎06-04-2015 12:46 AM

Following are my core dump file, let me know any points you may think of.

admin:utils core active list

      Size         Date            Core File Name
=================================================================
  38068 KB   2015-06-02 09:42:59   core.19621.11.LdapMonSvr.1433227379
  38068 KB   2015-03-17 14:45:17   core.4453.11.LdapMonSvr.1426592717
  38068 KB   2015-04-14 16:46:14   core.31823.11.LdapMonSvr.1429019174
  38316 KB   2015-04-28 10:13:14   core.11363.11.LdapMonSvr.1430205193
  52552 KB   2014-12-03 06:21:02   core.13320.6.FCVoIPMonSvr.1417576862
  37040 KB   2015-03-25 00:19:14   core.16844.11.LdapMonSvr.1427231954
  70028 KB   2014-12-03 07:38:28   core.13559.6.FCVoIPMonSvr.1417581508
  38068 KB   2015-02-24 22:19:10   core.2382.11.LdapMonSvr.1424805550
  38068 KB   2015-03-10 17:46:29   core.32509.11.LdapMonSvr.1425998789
  39476 KB   2015-03-31 14:14:48   core.25141.11.LdapMonSvr.1427800488
  38068 KB   2015-02-10 20:18:16   core.18079.11.LdapMonSvr.1423588696
  67712 KB   2014-12-03 07:23:39   core.20282.6.FCVoIPMonSvr.1417580619
  38068 KB   2015-05-19 14:45:14   core.1422.11.LdapMonSvr.1432035913
  39468 KB   2015-02-04 00:18:33   core.4269.11.LdapMonSvr.1422998312
  39428 KB   2015-05-12 08:43:58   core.18253.11.LdapMonSvr.1431409438
  38212 KB   2015-04-07 15:45:29   core.12913.11.LdapMonSvr.1428410729
  38068 KB   2015-02-17 14:45:34   core.15014.11.LdapMonSvr.1424173534
  38068 KB   2015-05-05 06:41:54   core.30786.11.LdapMonSvr.1430797314
  38068 KB   2015-05-26 13:44:40   core.30898.11.LdapMonSvr.1432637080


admin:utils core active analyze core.19621.11.LdapMonSvr.1433227379

This command may use a considerable amount of I/O
and running it may impact system performance.

It is highly recommended that this command be
run off-hours.

Continue (y/n)?y
Reading symbols from /opt/cisco/uccx/desktop/bin/LdapMonSvr...(no debugging symbols found)...done.
[New Thread 1515]
[New Thread 19621]
[New Thread 19622]
[New Thread 19623]
[New Thread 19624]
[New Thread 19629]
[New Thread 19631]
[New Thread 19632]
[New Thread 19633]
[New Thread 19638]
[New Thread 19639]
[New Thread 19641]
[New Thread 19643]
[New Thread 19645]
[New Thread 19649]
[New Thread 19651]
[New Thread 19654]
[New Thread 19655]
[New Thread 21816]
[New Thread 31190]
[New Thread 31188]
[New Thread 21771]
[New Thread 19659]
[New Thread 19646]
[New Thread 19644]
[New Thread 19642]
[New Thread 19640]
Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/librt.so.1
Reading symbols from /opt/cisco/uccx/desktop/lib/libsasl2.so.3...done.
Loaded symbols for /opt/cisco/uccx/desktop/lib/libsasl2.so.3
Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /usr/local/platform/lib/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/local/platform/lib/libz.so.1
Reading symbols from /opt/cisco/uccx/desktop/lib/libxerces-c.so.28...(no debugging symbols found)...done.
Loaded symbols for /opt/cisco/uccx/desktop/lib/libxerces-c.so.28
Reading symbols from /opt/cisco/uccx/desktop/lib/libcrypto.so.1.0.0...(no debugging symbols found)...done.
Loaded symbols for /opt/cisco/uccx/desktop/lib/libcrypto.so.1.0.0
Reading symbols from /opt/cisco/uccx/desktop/lib/libCadSplkStd.so...(no debugging symbols found)...done.
Loaded symbols for /opt/cisco/uccx/desktop/lib/libCadSplkStd.so
Reading symbols from /opt/cisco/uccx/desktop/lib/libSplkSecurity.so...(no debugging symbols found)...done.
Loaded symbols for /opt/cisco/uccx/desktop/lib/libSplkSecurity.so
Reading symbols from /opt/cisco/uccx/desktop/lib/libssl.so.1.0.0...(no debugging symbols found)...done.
Loaded symbols for /opt/cisco/uccx/desktop/lib/libssl.so.1.0.0
Reading symbols from /opt/cisco/uccx/desktop/lib/libSplkSocket.so...(no debugging symbols found)...done.
Loaded symbols for /opt/cisco/uccx/desktop/lib/libSplkSocket.so
Reading symbols from /opt/cisco/uccx/desktop/lib/libLDAPClientDLL.so...(no debugging symbols found)...done.
Loaded symbols for /opt/cisco/uccx/desktop/lib/libLDAPClientDLL.so
Reading symbols from /opt/cisco/uccx/desktop/lib/libCVDClient.so...(no debugging symbols found)...done.
Loaded symbols for /opt/cisco/uccx/desktop/lib/libCVDClient.so
Reading symbols from /opt/cisco/uccx/desktop/lib/libSplkStd.so...(no debugging symbols found)...done.
Loaded symbols for /opt/cisco/uccx/desktop/lib/libSplkStd.so
Reading symbols from /usr/lib/libstdc++.so.6...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libstdc++.so.6
Reading symbols from /lib/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libgcc_s.so.1
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/libresolv.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libresolv.so.2
Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libm.so.6
Reading symbols from /usr/lib/libelf.so.1...(no debugging symbols found)...done.
Loaded symbols for /usr/lib/libelf.so.1
Reading symbols from /opt/cisco/uccx/desktop/lib/libAlarmAppenderLib.so...(no debugging symbols found)...done.
Loaded symbols for /opt/cisco/uccx/desktop/lib/libAlarmAppenderLib.so
Reading symbols from /usr/local/platform/lib/libstlport.so...(no debugging symbols found)...done.
Loaded symbols for /usr/local/platform/lib/libstlport.so
Reading symbols from /usr/local/platform/lib/libACE.so...done.
Loaded symbols for /usr/local/platform/lib/libACE.so
Reading symbols from /opt/cisco/uccx/desktop/lib/libAlarmAppenderWrapper.so...done.
Loaded symbols for /opt/cisco/uccx/desktop/lib/libAlarmAppenderWrapper.so
Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_files.so.2
Core was generated by `/opt/cisco/uccx/desktop/bin/LdapMonSvr'.
Program terminated with signal 11, Segmentation fault.
#0  0x00b622e6 in std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >::~basic_string() () from /usr/lib/libstdc++.so.6

  ====================================
 backtrace - CUCM
 ===================================
 #0  0x00b622e6 in std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >::~basic_string() () from /usr/lib/libstdc++.so.6
#1  0x05bcf37b in splk_std::LogFunctionCall::~LogFunctionCall() () from /opt/cisco/uccx/desktop/lib/libSplkStd.so
#2  0x00fc460b in ldap_client::LDAPConnectionCheckerThread::ldapMonitor() () from /opt/cisco/uccx/desktop/lib/libLDAPClientDLL.so
#3  0x00fc1d34 in ldap_client::LDAPConnectionCheckerThread::run() () from /opt/cisco/uccx/desktop/lib/libLDAPClientDLL.so
#4  0x05c64cf1 in splk_std_internal::ThreadPimpl::thread_func(void*) () from /opt/cisco/uccx/desktop/lib/libSplkStd.so
#5  0x005edb39 in start_thread () from /lib/libpthread.so.0
#6  0x0180dd6e in clone () from /lib/libc.so.6
 ====================================
 info registers
 ====================================
 eax            0xfffffff4      -12
ecx            0xf7500030       -145752016
edx            0xffffffff       -1
ebx            0xb9fff4 12189684
esp            0xf56e8d40       0xf56e8d40
ebp            0xf56e8d68       0xf56e8d68
esi            0x2      2
edi            0x0      0
eip            0xb622e6 0xb622e6 <std::basic_string<wchar_t, std::char_traits<wchar_t>, std::allocator<wchar_t> >::~basic_string()+54>
eflags         0x210286 [ PF SF IF RF ID ]
cs             0x23     35
ss             0x2b     43
ds             0x2b     43
es             0x2b     43
fs             0x0      0
gs             0x63     99
 ====================================
 info threads
 ====================================
   27 Thread 0xf65f8b70 (LWP 19640)  0x00603430 in __kernel_vsyscall ()
  26 Thread 0xf63f6b70 (LWP 19642)  0x00603430 in __kernel_vsyscall ()
  25 Thread 0xf61f4b70 (LWP 19644)  0x00603430 in __kernel_vsyscall ()
  24 Thread 0xf5ff2b70 (LWP 19646)  0x00603430 in __kernel_vsyscall ()
  23 Thread 0xf5aedb70 (LWP 19659)  0x00603430 in __kernel_vsyscall ()
  22 Thread 0xf6dfeb70 (LWP 21771)  0x00603430 in __kernel_vsyscall ()
  21 Thread 0xf58ebb70 (LWP 31188)  0x00603430 in __kernel_vsyscall ()
  20 Thread 0xf57eab70 (LWP 31190)  0x00603430 in __kernel_vsyscall ()
  19 Thread 0xf59ecb70 (LWP 21816)  0x00603430 in __kernel_vsyscall ()
  18 Thread 0xf5beeb70 (LWP 19655)  0x00603430 in __kernel_vsyscall ()
  17 Thread 0xf5cefb70 (LWP 19654)  0x00603430 in __kernel_vsyscall ()
  16 Thread 0xf5df0b70 (LWP 19651)  0x00603430 in __kernel_vsyscall ()
  15 Thread 0xf5ef1b70 (LWP 19649)  0x00603430 in __kernel_vsyscall ()
  14 Thread 0xf60f3b70 (LWP 19645)  0x00603430 in __kernel_vsyscall ()
  13 Thread 0xf62f5b70 (LWP 19643)  0x00603430 in __kernel_vsyscall ()
  12 Thread 0xf64f7b70 (LWP 19641)  0x00603430 in __kernel_vsyscall ()
  11 Thread 0xf68f9b70 (LWP 19639)  0x00603430 in __kernel_vsyscall ()
  10 Thread 0xf69fab70 (LWP 19638)  0x00603430 in __kernel_vsyscall ()
  9 Thread 0xf6afbb70 (LWP 19633)  0x00603430 in __kernel_vsyscall ()
  8 Thread 0xf6bfcb70 (LWP 19632)  0x00603430 in __kernel_vsyscall ()
  7 Thread 0xf6cfdb70 (LWP 19631)  0x00603430 in __kernel_vsyscall ()
  6 Thread 0xf6effb70 (LWP 19629)  0x00603430 in __kernel_vsyscall ()
  5 Thread 0xf73feb70 (LWP 19624)  0x00603430 in __kernel_vsyscall ()
  4 Thread 0xf74ffb70 (LWP 19623)  0x00603430 in __kernel_vsyscall ()
  3 Thread 0xf7779b70 (LWP 19622)  0x00603430 in __kernel_vsyscall ()
  2 Thread 0xf777e6e0 (LWP 19621)  0x00603430 in __kernel_vsyscall ()
* 1 Thread 0xf56e9b70 (LWP 1515)  0x00b622e6 in std::basic_string<wchar_t, std::
char_traits<wchar_t>, std::allocator<wchar_t> >::~basic_string() () from /usr/lib/libstdc++.so.6

0 Helpful

neil wooloff
Level 1
Level 1

‎06-28-2016 08:20 AM

Hello,

Have same version and same issue:

admin:utils core active list

Size Date Core File Name
=================================================================
68848 KB 2016-02-18 07:28:13 core.12023.11.LRMServer.1455780492
38088 KB 2016-04-28 22:20:07 core.26346.11.LdapMonSvr.1461878406
38068 KB 2016-06-13 03:10:12 core.1149.11.LdapMonSvr.1465783812
38360 KB 2016-05-29 09:00:38 core.8822.11.LdapMonSvr.1464508837
38088 KB 2015-12-17 23:27:15 core.14456.11.LdapMonSvr.1450394835
48364 KB 2015-06-21 04:24:18 core.5342.11.FCVoIPMonSvr.1434857058
38088 KB 2016-01-14 20:16:53 core.7478.11.LdapMonSvr.1452802613
38092 KB 2015-06-26 04:05:37 core.13472.11.LdapMonSvr.1435287937
59916 KB 2016-05-07 00:50:31 core.588.11.LRMServer.1462578631
73164 KB 2015-12-31 20:13:54 core.30526.6.FCVoIPMonSvr.1451592827

Did you get to the bottom of it?

Rgrds,

neil

0 Helpful

Deepak Mehta
VIP Alumni
VIP Alumni
In response to neil wooloff

‎06-28-2016 08:39 AM

Hi Neil,

I believe you need to share these files with TAC and see if there is any memory leak or bug in the current version.You can share active analyse output with cisco

admin:utils core active analyze core.19621.11.LdapMonSvr.1433227379>>>>>>>>>>>>>>

I do see a bug for UCCX older version (9.X) with same sysmptoms.

https://bst.cloudapps.cisco.com/bugsearch/bug/CSCub91461/?referring_site=bugquickviewredir

0 Helpful

neil wooloff
Level 1
Level 1
In response to Deepak Mehta

‎06-28-2016 08:46 AM

I found 4 bugs and seems main one is not fixed

Looks like cisco jobbie

Thanks Deepak.

0 Helpful

fenilantony
Level 1
Level 1
In response to neil wooloff

‎06-28-2016 09:34 PM

Hi Neil,

Still we are getting same issue.And our running version is 10.5.1.11001-49.

I think this issue will resolve by an upgrade to 10.6.

0 Helpful

neil wooloff
Level 1
Level 1
In response to fenilantony

‎06-29-2016 12:50 AM

Thanks fenilantony

Am looking at asking Cisco TAC so can be sure - will update once I get any useful

Neil

0 Helpful

fenilantony
Level 1
Level 1
In response to neil wooloff

‎08-08-2016 02:17 AM

Hi Neil,

Any update on this?

0 Helpful

Larry Roberson
Level 1
Level 1
In response to neil wooloff

‎09-27-2017 08:01 AM

In July 2016 I upgraded UCCX to 10.6.. I have always been getting alerts on the LDAP stopping at night when the DRS runs. 

 

Now I see I am getting this core dump issues and the Document manger is in Partial on the UCCX SUB..

 At Wed Sep 27 07:53:36 PDT 2017 on node , the following CoreDumpFileFound events generated: 

TotalCoresFound : 1

CoreDetails : The following lists up to 6 cores dumped by corresponding applications.

Core1 : Cisco Desktop VoIP Monitor Service (core.4822.6.FCVoIPMonSvr.1506524004)

AppID : Cisco Log Partition Monitoring Tool ClusterID :

NodeID : uccx-sub

 TimeStamp : Wed Sep 27 07:53:29 PDT 2017

0 Helpful
Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Customers Also Viewed These Support Documents