Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. If you'd prefer to explore, try our test area to get started. And see here for current known issues.

New Member

CUCM 8.6.2 - LowAvailableVirtualMemory

Hi Guys,

My Customer have the Cluster with 6 CUCM server and the Publisher is presenting LowAvailableVirtualMemory. I read the post below, but, Not stay clear what is the solution or if that another solution. Any have Idea about this?


admin:show status

Host Name    : spoccm01
Date         : Thu Feb 6, 2014 11:20:40
Time Zone    : Brasilia Summer Time (America/Sao_Paulo)
Locale       : en_US.UTF-8
Product Ver  : 8.6.2.22900-9
OS Ver       : 5.0.0.0-2
License MAC  : 3440B59DD214

Uptime:
11:20:41 up 151 days, 16:39,  1 user,  load average: 0.47, 0.58, 0.55

CPU Idle:   74.50%  System:   01.50%    User:   23.50%
  IOWAIT:   00.00%     IRQ:   00.00%    Soft:   00.50%   Intr/sec: 1555.45

Memory Total:        3997572K
        Free:         194632K
        Used:        3802940K
      Cached:         631508K
      Shared:              0K
     Buffers:          10316K

                        Total            Free            Used
Disk/active         25197252K       12643252K       12298008K (50%)
Disk/inactive       24797380K       12383964K       12157424K (50%)
Disk/logging        87665856K       30192512K       53020128K (64%)


admin:

admin:show memory size

Total size of memory: 4.0 GB

admin:

https://supportforums.cisco.com/thread/2174443

Thanks,

Wilson

  • IP Telephony
1 ACCEPTED SOLUTION

Accepted Solutions
Cisco Employee

CUCM 8.6.2 - LowAvailableVirtualMemory

Hi Wlison,

Hope you are doing good.

Just wanted to check the status of virtual memory on the servers.

Regards,
Tirtha

Regards, Tirtha
12 REPLIES
Cisco Employee

CUCM 8.6.2 - LowAvailableVirtualMemory

Hi Wilson,

A few things to check first.

1) Is the alert being generated at some specific time during the day or is it random? if it is at some specific times then it could be due to scheduled processes like DRS Backup , CDR Load , MWI sync etc. Any random alerts could be due to call loops , high call volume or some bug on cucm.

2) Please check the output of following for any issues on the server

show status

utils diagnose test

utils core active list

HTH

Manish

New Member

CUCM 8.6.2 - LowAvailableVirtualMemory

Hi Manish,

Thank You for your contact. No, the alert is permanent not is random.

Follow the outputs that You mentioned:

admin:show status

Host Name    : spoccm01
Date         : Thu Feb 6, 2014 11:59:52
Time Zone    : Brasilia Summer Time (America/Sao_Paulo)
Locale       : en_US.UTF-8
Product Ver  : 8.6.2.22900-9
OS Ver       : 5.0.0.0-2
License MAC  : 3440B59DD214

Uptime:
11:59:53 up 151 days, 17:18,  1 user,  load average: 0.34, 0.45, 0.46

CPU Idle:   67.16%  System:   04.48%    User:   26.62%
  IOWAIT:   01.24%     IRQ:   00.00%    Soft:   00.50%   Intr/sec: 1546.00

Memory Total:        3997572K
        Free:         194612K
        Used:        3802960K
      Cached:         649620K
      Shared:              0K
     Buffers:          11052K

                        Total            Free            Used
Disk/active         25197252K       12642272K       12298988K (50%)
Disk/inactive       24797380K       12383964K       12157424K (50%)
Disk/logging        87665856K       30202956K       53009684K (64%)


admin:

admin:utils diagnose test

Log file: platform/log/diag1.log

Starting diagnostic test(s)
===========================
test - disk_space          : Passed (available: 12346 MB, used: 12011 MB)
skip - disk_files          : This module must be run directly and off hours
test - service_manager     : Passed
test - tomcat              : Passed
test - tomcat_deadlocks    : Passed
test - tomcat_keystore     : Passed
test - tomcat_connectors   : Passed
test - tomcat_threads      : Passed
test - tomcat_memory       : Passed
test - tomcat_sessions     : Failed - The following web applications have an unusually large number of active sessions: _root ccmcip.  Please collect all of the Tomcat logs for root cause analysis: file get activelog tomcat/logs/*
test - validate_network    : Reverse DNS lookup missmatch
test - raid                : Passed
test - system_info         : Passed (Collected system information in diagnostic log)
test - ntp_reachability    : Passed
test - ntp_clock_drift     : Passed
test - ntp_stratum         : Passed
skip - sdl_fragmentation   : This module must be run directly and off hours
skip - sdi_fragmentation   : This module must be run directly and off hours
test - ipv6_networking     : Passed

Diagnostics Completed

admin:


admin:utils core active list

      Size         Date            Core File Name
=================================================================
496676 KB   2013-09-07 07:40:01   core.11679.6.ccm.1378550392
admin:

Thanks again,

Regards,

Wilson

Cisco Employee

CUCM 8.6.2 - LowAvailableVirtualMemory

Hi Wilson,

A couple of things that you can try as of now.

1) restart tomcat service on Pub

2) Fix the DNS issue as indicated in the 'utils diagnose test'

let us know if the issue persists.

Manish

New Member

CUCM 8.6.2 - LowAvailableVirtualMemory

Hi Manish,

About the answer 1, the restart is necessary because this message:

Failed: The following web applications have an unusually large number
of active sessions?

About the collect requested:  file get activelog tomcat/logs/* , are there any impact? Where this files are saved?

About the answer 2, is about Reverse DNS lookup missmatch? Where and verify and fix this information?

Thanks,

Wilson

Cisco Employee

CUCM 8.6.2 - LowAvailableVirtualMemory

Hi Wilson,

As Manish said we we need to restart the Tomcat and fix of DNS will help.

They are not impacting and the location is updated in the file get command.

Reverse DNS needs to fixed on the DNS servers or you might point your CUCM server to the right DNS server.

HTH

Tirtha

Regards, Tirtha
New Member

CUCM 8.6.2 - LowAvailableVirtualMemory

Hi Tirtha,

Thanks a lot for your contact. I agree with Manish and You that is necessary restart the Tomcat service, it can be causing the problem. About the DNS, is a problem, but, I don´t believe that there are any relationship about lowavalaiblememoryvirtual. I did the test about DNS and not was possible identify where is the problem.

10.218.60.12 Subscriber / TAUCCM01
10.221.0.12 Subscriber / CASCCM01
172.17.32.10 Publisher / SPOCCM01
172.17.32.11 Subscriber / SPOCCM02
172.17.32.12 Subscriber / SPOCCM03
172.17.32.13 Subscriber / SPOCCM04


Microsoft Windows [versão 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. Todos os direitos reservados.

M:\>nslookup 172.17.32.10
Servidor:  srv10509.x.y.z.w
Address:  172.16.32.7

Nome:    spoccm01.a.b.c.d.e
Address:  172.17.32.10


M:\>nslookup 172.17.32.11
Servidor:  srv10509.x.y.z.w
Address:  172.16.32.7

Nome:    spoccm02.a.b.c.d.e
Address:  172.17.32.11


M:\>nslookup 172.17.32.12
Servidor:  srv10509.x.y.z.w
Address:  172.16.32.7

Nome:    spoccm03.x.y.z.w
Address:  172.17.32.12


M:\>nslookup 172.17.32.13
Servidor:  srv10509.x.y.z.w
Address:  172.16.32.7

Nome:    spoccm04.x.y.z.w
Address:  172.17.32.13


M:\>nslookup 10.221.0.12
Servidor:  srv10509.x.y.z.w
Address:  172.16.32.7

Nome:    casccm01.x.y.z.w
Address:  10.221.0.12


M:\>nslookup 10.218.60.12
Servidor:  srv10509.x.y.z.w
Address:  172.16.32.7

Nome:    tauccm01.x.y.z.w
Address:  10.218.60.12


M:\>nslookup 10.218.60.13
Servidor:  srv10509.x.y.z.w
Address:  172.16.32.7

Microsoft Windows [versão 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. Todos os direitos reservados.

M:\>nslookup 172.17.32.10
Servidor:  srv10509.x.y.z.w
Address:  172.16.32.7

Nome:    spoccm01.a.b.c.d.e
Address:  172.17.32.10


M:\>nslookup 172.17.32.11
Servidor:  srv10509.x.y.z.w
Address:  172.16.32.7

Nome:    spoccm02.a.b.c.d.e
Address:  172.17.32.11


M:\>nslookup 172.17.32.12
Servidor:  srv10509.x.y.z.w
Address:  172.16.32.7

Nome:    spoccm03.x.y.z.w
Address:  172.17.32.12


M:\>nslookup 172.17.32.13
Servidor:  srv10509.x.y.z.w
Address:  172.16.32.7

Nome:    spoccm04.x.y.z.w
Address:  172.17.32.13


M:\>nslookup 10.221.0.12
Servidor:  srv10509.x.y.z.w
Address:  172.16.32.7

Nome:    casccm01.x.y.z.w
Address:  10.221.0.12


M:\>nslookup 10.218.60.12
Servidor:  srv10509.x.y.z.w
Address:  172.16.32.7

Nome:    tauccm01.x.y.z.w
Address:  10.218.60.12


M:\>nslookup 10.218.60.13
Servidor:  srv10509.x.y.z.w
Address:  172.16.32.7

*** srv10509.x.y.z.w não encontrou 10.218.60.13: Non-existent do
main

M:\>ping spoccm01

Disparando spoccm01.a.b.c.d.e [172.17.32.10] com 32 bytes de dados:

Resposta de 172.17.32.10: bytes=32 tempo<1ms TTL=63
Resposta de 172.17.32.10: bytes=32 tempo<1ms TTL=63
Resposta de 172.17.32.10: bytes=32 tempo<1ms TTL=63
Resposta de 172.17.32.10: bytes=32 tempo<1ms TTL=63

Estatísticas do Ping para 172.17.32.10:
    Pacotes: Enviados = 4, Recebidos = 4, Perdidos = 0 (0% de
             perda),
Aproximar um número redondo de vezes em milissegundos:
    Mínimo = 0ms, Máximo = 0ms, Média = 0ms

M:\>

M:\>ping spoccm012
A solicitação ping não pôde encontrar o host spoccm012. Verifique o nome e tente

novamente.

M:\>ping spoccm02

Disparando spoccm02.a.b.c.d.e [172.17.32.11] com 32 bytes de dados:

Resposta de 172.17.32.11: bytes=32 tempo<1ms TTL=63
Resposta de 172.17.32.11: bytes=32 tempo=1ms TTL=63
Resposta de 172.17.32.11: bytes=32 tempo<1ms TTL=63
Resposta de 172.17.32.11: bytes=32 tempo<1ms TTL=63

Estatísticas do Ping para 172.17.32.11:
    Pacotes: Enviados = 4, Recebidos = 4, Perdidos = 0 (0% de
             perda),
Aproximar um número redondo de vezes em milissegundos:
    Mínimo = 0ms, Máximo = 1ms, Média = 0ms

M:\>ping spoccm03

Disparando spoccm03.x.y.z.w [172.17.32.12] com 32 bytes de dados
:
Resposta de 172.17.32.12: bytes=32 tempo<1ms TTL=63
Resposta de 172.17.32.12: bytes=32 tempo=4ms TTL=63
Resposta de 172.17.32.12: bytes=32 tempo<1ms TTL=63
Resposta de 172.17.32.12: bytes=32 tempo=2ms TTL=63

Estatísticas do Ping para 172.17.32.12:
    Pacotes: Enviados = 4, Recebidos = 4, Perdidos = 0 (0% de
             perda),
Aproximar um número redondo de vezes em milissegundos:
    Mínimo = 0ms, Máximo = 4ms, Média = 1ms

M:\>ping spoccm04

Disparando spoccm04.x.y.z.w [172.17.32.13] com 32 bytes de dados
:
Resposta de 172.17.32.13: bytes=32 tempo<1ms TTL=63
Resposta de 172.17.32.13: bytes=32 tempo<1ms TTL=63
Resposta de 172.17.32.13: bytes=32 tempo<1ms TTL=63
Resposta de 172.17.32.13: bytes=32 tempo<1ms TTL=63

Estatísticas do Ping para 172.17.32.13:
    Pacotes: Enviados = 4, Recebidos = 4, Perdidos = 0 (0% de
             perda),
Aproximar um número redondo de vezes em milissegundos:
    Mínimo = 0ms, Máximo = 0ms, Média = 0ms

M:\>ping tauccm01

Disparando tauccm01.x.y.z.w [10.218.60.12] com 32 bytes de dados
:
Resposta de 10.218.60.12: bytes=32 tempo=5ms TTL=56
Resposta de 10.218.60.12: bytes=32 tempo=6ms TTL=56
Resposta de 10.218.60.12: bytes=32 tempo=5ms TTL=56
Resposta de 10.218.60.12: bytes=32 tempo=5ms TTL=56

Estatísticas do Ping para 10.218.60.12:
    Pacotes: Enviados = 4, Recebidos = 4, Perdidos = 0 (0% de
             perda),
Aproximar um número redondo de vezes em milissegundos:
    Mínimo = 5ms, Máximo = 6ms, Média = 5ms

M:\>ping casccm01

Disparando casccm01.x.y.z.w [10.221.0.12] com 32 bytes de dados:

Resposta de 10.221.0.12: bytes=32 tempo=52ms TTL=56
Resposta de 10.221.0.12: bytes=32 tempo=41ms TTL=56
Resposta de 10.221.0.12: bytes=32 tempo=43ms TTL=56
Resposta de 10.221.0.12: bytes=32 tempo=44ms TTL=56

Estatísticas do Ping para 10.221.0.12:
    Pacotes: Enviados = 4, Recebidos = 4, Perdidos = 0 (0% de
             perda),
Aproximar um número redondo de vezes em milissegundos:
    Mínimo = 41ms, Máximo = 52ms, Média = 45ms

M:\>

Thanks,


Wilson

Cisco Employee

CUCM 8.6.2 - LowAvailableVirtualMemory

Hi Wilson,

The approach that we have taken here is proactive rather than re-active.

Pro-active aproach - restart suspected services looking at the symptom and configure the recommended settings.

Reactive approach - Get the output of some memory commands, get perfmon logs, look at the logs which service consuming high memory, then look at the respective service logs and ....

So here I cannot commit that Tomcat and DNS are causing the issues... but by correcting these two things we can think of anything else. Hope this helps in understanding why we are trying to fix DNS.

Now the output of lookup has to be done from CUCM servers and not from windows machine. The above output that you have shared is not going to help much to figure out DNS issue.

Let us restart the Tomcat for now and later if you see issue the issue is resolved then you are good to go

Regards,
Tirtha

Regards, Tirtha
New Member

CUCM 8.6.2 - LowAvailableVirtualMemory

Hi Tirtha,

I used the command utils network ping and didn´t found the problem about DNS. Are there are another way to verify this?

Thanks

Wilson

Cisco Employee

CUCM 8.6.2 - LowAvailableVirtualMemory

Hi Wilson,

These are the commands and you should be looking at external resolution not the local ones'.

DNS lookup commands

utils network host

utils network host

Regards,
Tirtha

Regards, Tirtha
563
Views
0
Helpful
12
Replies