Nexus Memory issues

Document

Mon, 11/07/2011 - 22:31

For general assessment of memory available, following two commands are used. From these command output you will be able to tell whether the platform memory utilization is normal or un-expected:

N7K-1-VDC2# show system resources

Load average: 1 minute: 0.11 5 minutes: 0.09 15 minutes: 0.14

Processes : 1241 total, 2 running

CPU states : 2.0% user, 3.4% kernel, 94.6% idle

Memory usage: 4115232K total, 3606556K used, 508676K free

Note: The “show system resources” command is used to display platform memory statistics.

For memory information per process for current VDC , following command is used:

N7k-3-VDC3# show processes memory

PID MemAlloc MemLimit MemUsed StackBase/Ptr Process

----- -------- ---------- ---------- ----------------- ----------------

4662 52756480 562929945 150167552 bfffdf00/bfffd970 netstack

MemAlloc: Data Segement size

MemLimit: Maximum memory process can use

MemUsed: Virtual Memory

For a detailed assessment of platform memory utilization, use:

N7k-3-VDC3# show system internal kernel meminfo

MemTotal: 4115232 kB -------This is the total amount of memory in the system (4 GB in Sup 1)

MemFree: 263684 kB

Buffers: 82400 kB

Cached: 1817788 kB -------The memory used by the page cache

ShmFS: 1533324 kB

Allowed: 1028808 Pages

Free: 65921 Pages

Available: 164026 Pages -------Amount of free memory in pages

SwapCached: 0 kB

Active: 2080320 kB

Inactive: 1433752 kB

HighTotal: 3338960 kB

HighFree: 4092 kB

LowTotal: 776272 kB

LowFree: 259592 kB

SwapTotal: 0 kB

SwapFree: 0 kB

Dirty: 0 kB

Writeback: 0 kB

AnonPages: 1613748 kB

Mapped: 456088 kB  ------Memory mapped into page tables

Slab: 142884 kB --------Rough indication of kernel memory consumption

SReclaimable: 25556 kB

SUnreclaim: 117328 kB

PageTables: 32756 kB

Note: One page of memory is equivalent of 4 KB of memory.

Using above commands you can determine if the utilization is high due to page cache, process holding memory or Kernel.

For further troubleshooting:

For Page cache:

If Cached is high, you should check the file system utilization and determine what kind of files are using the file system:

N7K-1-VDC2# show system internal flash

Mount-on                  1K-blocks      Used   Available   Use%  Filesystem

/                            409600     43008      367616     11   /dev/root

/proc                             0         0           0      0   proc

/sys                              0         0           0      0   none

/isan                        409600    269312      140288     66   none

/var/tmp                     307200       876      306324      1   none

/var/sysmgr                 1048576    999424       49152      96   none

/var/sysmgr/ftp              307200     24576      282624      8   none

/dev/shm                    1048576    412672      635904     40   none

/volatile                    204800         0      204800      0   none

/debug                         2048        16        2032      1   none

/dev/mqueue                       0         0           0      0   none

/mnt/cfg/0                    76099      5674       66496      8   /dev/hda5

/mnt/cfg/1                    75605      5674       66027      8   /dev/hda6

/bootflash                  1796768    629784     1075712     37   /dev/hda3

/var/sysmgr/startup-cfg      409600     27536      382064      7   none

/mnt/plog                     56192      3064       53128      6   /dev/mtdblock2

/dev/pts                          0         0           0      0   devpts

/mnt/pss                      38554      6682       29882     19   /dev/hda4

/slot0                      2026608         4     2026604      1   /dev/hdc1

/logflash                   7997912    219408     7372232      3   /dev/hde1

/bootflash_sup-remote       1767480   1121784      555912     67   127.1.1.6:/mnt/bootflash/

/logflash_sup-remote        7953616    554976     6994608      8   127.1.1.6:/mnt/logflash/

For displaying contents of these sub-folders, use:

show system internal dir full directory path

for example: show system internal dir /bootflash

And, to delete a specific file use:

filesys delete full file path

For Kernel:

Contact Cisco TAC

For user processes:

N7K# show system internal processes memory

PID TTY STAT TIME MAJFLT TRS RSS VSZ %MEM COMMAND

4727 ? Ss 00:00:00 0 1549 123248 132832 2.9 /isan/bin/pixm

4728 ? Ssl 00:00:00 0 408 78388 143104 1.8 /isan/bin/routing-sw/mrib -m 4

6662 ? Ssl 00:00:05 0 2762 64024 144396 1.5 /isan/bin/routing-sw/netstack /isan/etc/routing-sw/pm.cfg

4538 ? Ssl 00:00:00 0 2762 60448 211664 1.4 /isan/bin/routing-sw/netstack /isan/etc/routing-sw/pm.cfg

5865 ? Ssl 00:00:01 0 2762 60416 113320 1.4 /isan/bin/routing-sw/netstack /isan/etc/routing-sw/pm.cfg

6395 ? Ssl 00:00:00 0 2762 52008 105552 1.2 /isan/bin/routing-sw/netstack /isan/etc/routing-sw/pm.cfg

4271 ? Ssl 00:00:00 0 609 49812 61420 1.2 /isan/bin/routing-sw/urib

7879 ? Ssl 00:00:00 0 1909 44800 90508 1.0 /isan/bin/routing-sw/bgp -t 64000

5696 ? Ssl 00:00:17 0 337 44696 55252 1.0 /isan/bin/routing-sw/clis -cli /isan/etc/routing-sw/cli

5333 ? Ssl 00:00:14 0 337 44652 55208 1.0 /isan/bin/routing-sw/clis -cli /isan/etc/routing-sw/cli

4182 ? Ssl 00:00:15 0 337 44648 55204 1.0 /isan/bin/routing-sw/clis -cli /isan/etc/routing-sw/cli

6076 ? Ssl 00:00:14 0 337 44624 55284 1.0 /isan/bin/routing-sw/clis -cli /isan/etc/routing-sw/cli

6825 ? Ssl 00:00:00 0 1402 44576 84020 1.0 /isan/bin/routing-sw/pim -t

4268 ? Ssl 00:00:00 0 363 27132 38896 0.6 /isan/bin/routing-sw/u6rib

4732 ? Ssl 00:00:00 0 404 25220 65360 0.6 /isan/bin/routing-sw/m6rib

4726 ? S<s 00:00:00 0 144 25208 30188 0.6 /isan/bin/pixmc

Increase in the utilization for a specific process will indicate a problem. Troubleshooting process afterward depends on the culprit process.

Note: The above output when taken from any VDC will show content of all VDCs.

Memory threshold for releases prior to 4.2(4):

  • 70% Minor
  • 80% Critical
  • 90% Major

For 4.2(4) and later releases:

  • 85% Minor
  • 90% Critical
  • 95% Major

These thresholds can be configured by using command:

system memory-thresholds minor percentage severe percentage critical percentage

The command that can be used to check memory alert info is:

Show system internal memory-alert-log


Loading.

Actions

This Document

Related Content