2691 Router Buffer leak due to syslog

[mahesh18]

Hi all,

On my 2691 Router i see the buffer leak due to syslog

2691Router#                         sh buffers leak

  Header DataArea  Pool  Size Link  Enc    Flags      Input     Output User

650743C4  F200084 Small     0    0    0        0       None       None Init

65074748  F2001C4 Small     0    0    0        0       None       None Init

65465300  F6F8864 VeryB    40    0    0        0       None       None Init

65C97528 65C978B0 Syslo   144    0    0       10       None       None EEM ED Sy

65C97BD4 65C97F5C Syslo   123    0    0       10       None       None EEM ED Sy

65C98280 65C98608 Syslo   118    0    0       10       None       None EEM ED Sy

65C9892C 65C98CB4 Syslo   118    0    0       10       None       None EEM ED Sy

65C98FD8 65C99360 Syslo   123    0    0       10       None       None EEM ED Sy

65C99684 65C99A0C Syslo   147    0    0       10       None       None EEM ED Sy

65C99D30 65C9A0B8 Syslo   149    0    0       10       None       None EEM ED Sy

65C9A3DC 65C9A764 Syslo   144    0    0       10       None       None EEM ED Sy

65C9AA88 65C9AE10 Syslo   118    0    0       10       None       None EEM ED Sy

65C9B134 65C9B4BC Syslo   144    0    0       10       None       None EEM ED Sy

65C9B7E0 65C9BB68 Syslo   144    0    0       10       None       None EEM ED Sy

65C9BE8C 65C9C214 Syslo   118    0    0       10       None       None EEM ED Sy

65C9C538 65C9C8C0 Syslo   118    0    0       10       None       None EEM ED Sy

65C9CBE4 65C9CF6C Syslo   144    0    0       10       None       None EEM ED Sy

65C9D290 65C9D618 Syslo   108    0    0       10       None       None EEM ED Sy

65C9D93C 65C9DCC4 Syslo   112    0    0       10       None       None EEM ED Sy

65C9DFE8 65C9E370 Syslo    88    0    0       10       None       None EEM ED Sy

65C9E694 65C9EA1C Syslo   110    0    0       10       None       None EEM ED Sy

65C9ED40 65C9F0C8 Syslo    98    0    0       10       None       None EEM ED Sy

65C9F3EC 65C9F774 Syslo   119    0    0       10       None       None EEM ED Sy

65C9FA98 65C9FE20 Syslo   142    0    0       10       None       None EEM ED Sy

65CA0144 65CA04CC Syslo   138    0    0       10       None       None EEM ED Sy

65CA07F0 65CA0B78 Syslo   118    0    0       10       None       None EEM ED Sy

65CA0E9C 65CA1224 Syslo    98    0    0       10       None       None EEM ED Sy

65CA1548 65CA18D0 Syslo    98    0    0       10       None       None EEM ED Sy

65CA1BF4 65CA1F7C Syslo    98    0    0       10       None       None EEM ED Sy

65CA22A0 65CA2628 Syslo   118    0    0       10       None       None EEM ED Sy

65CA294C 65CA2CD4 Syslo    98    0    0       10       None       None EEM ED Sy

65CA2FF8 65CA3380 Syslo   118    0    0       10       None       None EEM ED Sy

65CA36A4 65CA3A2C Syslo    99    0    0       10       None       None EEM ED Sy

65CA3D50 65CA40D8 Syslo   118    0    0       10       None       None EEM ED Sy

65CA43FC 65CA4784 Syslo   117    0    0       10       None       None EEM ED Sy

Also

2691Router#sh buffers failures

Caller       Pool          Size      When

0x60247840  Small             60    08:24:05

0x60247840  Small             60    08:24:05

0x60247840  Small             60    08:24:05

0x60247840  Small             60    08:24:05

0x60247840  Small             60    08:24:05

0x60247840  Small             60    08:24:05

0x60247840  Small             60    08:24:05

0x60247840  Small             60    08:24:05

0x60247840  Small             60    08:24:05

0x60247840  Small             60    08:24:05

I know one way to free up buffers is to reload the router.

Is there any way i can fix the syslog buffers?i am using kiwi syslog server?

Thanks

Mahesh

[Reza Sharifi]

Hi Mahesh,

I don't believe there is a way to adjust the Syslog buffers.  One way to test is to reload the router and then reduce the amount of info you are sending to the Syslog sever so the router does not have to use as much buffer and see if you still have the same issue.

HTH

[mahesh18]

Hi Reza,

How can i reduce the amount of info to syslog?

I removed logging trap  and IP  of server.

For syslog i tried logging buffer 2

2691Router#sh log
Syslog logging: enabled (11 messages dropped, 0 messages rate-limited,
                0 flushes, 0 overruns, xml disabled, filtering disabled)
    Console logging: level debugging, 28 messages logged, xml disabled,
                     filtering disabled
    Monitor logging: level debugging, 0 messages logged, xml disabled,
                     filtering disabled
    Buffer logging: level critical, 2 messages logged, xml disabled, *************************************************
                    filtering disabled
    Logging Exception size (4096 bytes)
    Count and timestamp logging messages: disabled

No active filter modules.

    Trap logging: disabled

as above it only fixed the buffer logging not syslog logging?

How can i fix the syslog logging --info?

thanks

mahesh

[Richard Burts]

Mahesh

Probably the best way to reduce the amount of info to syslog is to change the severity level of what you send to the syslog server.

In the original post it looks as if the leak might be related to EEM. Are you running EEM scripts on this router? If so perhaps there is something that might be done in the EEM scripts to reduce or eliminate the leaks?

HTH

Rick

[mahesh18]

Hi Rick,

Thanks for reply.

Yes i was running script below

event manager applet High_CPU

action 0.1 syslog msg "CPU Utilization is high"

action 0.2 cli command "enable"

action 0.4 cli command "show log | append flash:CPU_Profile.txt"

action 0.5 cli command "show process cpu sorted | append"

action 0.6 cli command "show interfaces | append"

action 1.1 cli command "configure terminal"

action 1.2 cli command "profile xxxxxxx yyyyyyyyZ"

action 1.3 cli command "profile start"

action 2.3 syslog msg "Entering TCLSH"

action 2.4 cli command "tclsh"

action 2.5 cli command "after 240000"

action 2.6 cli command "exit"

action 2.9 syslog msg "Exiting TCLSH"

action 3.0 cli command "profile stop"

action 3.1 cli command "show profile terse | append flash:CPU_Profile.txt"

action 3.2 cli command "clear profile"

action 3.3 cli command "unprofile all"

action 4.1 syslog msg "Finished logging information to flash:CPU_Profile.txt...

"

action 4.2 cli command "end"

!

I removed the script and save config and reloaded the router but it again shows --

sh buffers leak

  Header DataArea  Pool  Size Link  Enc    Flags      Input     Output User

64E05B70  F200084 Small     0    0    0        0       None       None Init

64E05EF4  F2001C4 Small     0    0    0        0       None       None Init

651F6A80  F6F8864 VeryB    40    0    0        0       None       None Init

65A1DF30 65A1E2B8 Syslo   102    0    0       10       None       None EEM ED Sy

65A1E5DC 65A1E964 Syslo    85    0    0       10       None       None EEM ED Sy

65A1EC88 65A1F010 Syslo    85    0    0       10       None       None EEM ED Sy

65A1F334 65A1F6BC Syslo   101    0    0       10       None       None EEM ED Sy

65A1F9E0 65A1FD68 Syslo   178    0    0       10       None       None EEM ED Sy

65A2008C 65A20414 Syslo   179    0    0       10       None       None EEM ED Sy

65A20738 65A20AC0 Syslo   111    0    0       10       None       None EEM ED Sy

65A20DE4 65A2116C Syslo   113    0    0       10       None       None EEM ED Sy

65A21490 65A21818 Syslo   105    0    0       10       None       None EEM ED Sy

65A21B3C 65A21EC4 Syslo   100    0    0       10       None       None EEM ED Sy

65A221E8 65A22570 Syslo   111    0    0       10       None       None EEM ED Sy

65A22894 65A22C1C Syslo   111    0    0       10       None       None EEM ED Sy

65A22F40 65A232C8 Syslo    77    0    0       10       None       None EEM ED Sy

65A235EC 65A23974 Syslo   310    0    0       10       None       None EEM ED Sy

65A23C98 65A24020 Syslo   102    0    0       10       None       None EEM ED Sy

65A24344 65A246CC Syslo    68    0    0       10       None       None EEM ED Sy

65A249F0 65A24D78 Syslo   119    0    0       10       None       None EEM ED Sy

65A2509C 65A25424 Syslo   153    0    0       10       None       None EEM ED Sy

65A25748 65A25AD0 Syslo   118    0    0       10       None       None EEM ED Sy

65A25DF4 65A2617C Syslo   118    0    0       10       None       None EEM ED Sy

64FCFB74        0 FS He     0    0    1        0       None       None Init

Thanks

MAhesh

[Reza Sharifi]

Hi Mahesh,

Can you disable Syslog and see if this goes away.  Can you also post you syslog config?

HTH

[mahesh18]

Hi Reza,

Do you need this info--

2691Router#sh log
Syslog logging: enabled (11 messages dropped, 0 messages rate-limited,
                0 flushes, 0 overruns, xml disabled, filtering disabled)
    Console logging: disabled
    Monitor logging: level debugging, 0 messages logged, xml disabled,
                     filtering disabled
    Buffer logging: level debugging, 22 messages logged, xml disabled,
                    filtering disabled
    Logging Exception size (4096 bytes)
    Count and timestamp logging messages: enabled

No active filter modules.

    Trap logging: disabled

which command i should use to get the syslog config?

Thanks

Mahesh

[Reza Sharifi]

Hi Mahesh,

Try disabling syslog completely and see if the problem goes away. You can also enable it later.

also post sh run

HTH

[mahesh18]

Hi Reza,

I have disabled logging buffer   to the router.

also i have attached sh run from the router under first post.

i did sh log

691Router#sh log
Syslog logging: enabled (11 messages dropped, 0 messages rate-limited,***************************
                0 flushes, 0 overruns, xml disabled, filtering disabled)
    Console logging: disabled
    Monitor logging: level debugging, 0 messages logged, xml disabled,
                     filtering disabled
    Buffer logging: disabled, xml disabled,
                    filtering disabled
    Logging Exception Disabled
    Count and timestamp logging messages: enabled

No active filter modules.

    Trap logging: disabled

it still shows syslog logging enabled  -- how can i disable it?

Thanks

MAhesh

[Reza Sharifi]

Mahesh,

You config looks ok.  I don't see a Syslog server config in your sh run.

Is there a "sh buffers failures" you can try?

HTH

[mahesh18]

Hi Reza,

Thanks for reply

2691Router#                         sh buffers failures

Caller       Pool          Size      When

0x60247840  Small             60    01:50:24

0x60247840  Small             60    01:50:23

0x60247840  Small             60    01:50:23

0x60247840  Small             60    01:50:23

0x60247840  Small             60    01:50:23

0x60247840  Small             60    01:50:23

0x60247840  Small             60    01:50:23

0x60247840  Middle            60    01:50:23

0x60247840  Small             60    01:50:23

0x60247840  Middle            60    01:50:23

2691Router#

[Reza Sharifi]

Mahesh,

This is a memory issue. I would try a different version of IOS to see if you can clear this.

HTH

Reza

[mahesh parmar]

Hi Reza,

Thanks for reply.

Can i try this IOS.

c2691-advipservicesk9-mz.124-25d

I will try this and keep you posted.

Thanks

MAhesh

[mahesh18]

Hi Reza,

I got different IOS  c2691-advipservicesk9-mz.124-25d.

Its still same issue.

So if memory leak due to syslog is not big issue then i can ignore it for now?

Thanks

Mahesh

[Reza Sharifi · Accepted solution]

Hi Mahesh,

Thanks for the update.  I think, you can ignore it for now, but eventually you are going to run out of memory and have to reboot the router to refresh. If this is in production and before it start effecting your environment, you may want to replace it.

HTH