%FWSM-6-106028: Deny TCP (Connection marked for deletion) from src

Unanswered Question
Dec 2nd, 2009

I received the following error on the FWSM. the IP in question is for a particular server which is part of a group servers and all rules in FWSM apply to the group servers. The user however only have problems accessing this 1 server and not all the servers.

%FWSM-6-106028: Deny TCP (Connection marked for deletion) from src
ip-address/src-port to dst ip-address/dst-port flags flag on interface intf-name

A user which have the correct rights to access all servers is getting a generic "explorer cannot view this page" for this particular server.

I did do a search on the error on the cisco site but need a little bit help troubleshooting this. Any ideas.

Thanks

JNel

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Panos Kampanakis Wed, 12/02/2009 - 12:50

What is the flag of the packet deleted?

Usually this log shows up if we see FINs or RSTs and we are ready to close the conn and we see some more packets for that conn.

PK

JMC Nel Thu, 12/03/2009 - 06:33

sh local x.x.x.x

ipv4 local hosts:

local host: (x.x.x.x), tcp conn(s)/limit - 0/0, embryonic(s)/limit = 0/0 udp conn(s)/limit = 24/0

xlate(s):

global x.x.x.x local x.x.x.x

Kureli Sankar Wed, 12/02/2009 - 19:43

Pls. book mark this link below for fwsm documentation:

http://www.cisco.com/en/US/products/hw/modules/ps2706/ps4452/tsd_products_support_model_home.html

If you search for "system logs messges" in the above link it will take you to the syslog messages link where you can search for 106028 message.

http://www.cisco.com/en/US/docs/security/fwsm/fwsm40/system/message/logmsgs.html#wp1279924

106028

Error Message    %FWSM-6-106028: Deny TCP (Connection marked for deletion) from src 
ip-address/src-port to dst ip-address/dst-port flags flag on interface intf-name

Explanation   This syslog message is generated because a TCP port (source, destination, or both) was  reused to start a new connection within the TIME_WAIT period prescribed in the TCP RFC for an  existing connection. This message appears only for a short period of time (until the connection is  actually cleaned up), and only for new connections for SYN-only TCP packets whose flow has been  marked for deletion. Although reuse is not mandatory, it is recommended. If a VLAN is shared in  more than one context, then the VLAN number appears instead of the intf-name variable.

Recommended Action   Investigate why the client is reusing the port. After the connection is deleted in  the FWSM, if the same port is reused for a new connection, then the connection is established.

-KS

Giorgio Romano Thu, 05/05/2011 - 02:13

HI,

I'm having the same problem.

My proxies has the time_wait value set to 120 seconds.

How can I get from my fwsm 4.1 the time_wait value?

Is it possible to change it?

Thank you very much for any answer

giorgio

Giorgio Romano Sun, 05/08/2011 - 10:31

Hi,

an update.

I found in the genuine cisco doc something like this:

(http://www.cisco.com/en/US/docs/security/pix/pix61/command/reference/s.html#wp1026942)

"sysopt connection timewait

The sysopt connection timewait command is necessary for end host  applications whose default TCP terminating sequence is a simultaneous  close instead of the normal shutdown sequence (see RFC 793). In a  simultaneous close, both ends of the transaction initiate the closing  sequence, as opposed to the normal sequence where one end closes and the  other end acknowledges prior to initiating its own closing sequence.

The default behavior of the PIX Firewall is to track the normal shutdown  sequence and release the connection after two FINs and the  ACKnowledgment of the last FIN segment. This quick release heuristic  enables the PIX Firewall to sustain a high connection rate.

However with a simultaneous close, the quick release forces one side of  the connection to linger in the CLOSING state (see RFC 793). Many  sockets in the CLOSING state can degrade the performance of an end host.  For instance, some WinSock mainframe clients are known to exhibit this  behavior and degrade the performance of the mainframe server. Old  versions of HP/UX are also susceptible to this behavior. Enabling the  sysopt connection timewait command enables a quiet time window for the  abnormal close down sequence to complete.

The no sysopt connection timewait command disables the option, which is off by default.


Note Use  of the sysopt connection timewait command may impact PIX Firewall  performance especially with low memory configuration and highly dynamic  traffic pattern such as HTTP. "

but this doc is referred to cisco pix 6.1 FWOS version and I don't know if it is applicable to the fwsm version 4.1.(3).

Instead into the cisco fwsm 4.1 command reference I found this:

(http://www.cisco.com/en/US/docs/security/fwsm/fwsm41/command/reference/s6.html#wp2763407)

The "sysopt connection timewait" command is used but when I try to enter it in the FWSM CLI, the output I get is:

"FWSM(config)# sysopt connection timewait

                                            ^

ERROR: % Invalid input detected at '^' marker."

It seems not exist!

Where is the mistake?

Thank you very much!

giorgio

Kureli Sankar Sun, 05/08/2011 - 11:06

I beleive the mistake may have come from ASA command reference.

Varun is right that the command doesn't exist on the FWSM. But, the link that you posted is for the command "show run sysopt"

That section of the document may have come from the ASA guide. Thanks for pointing it out. We will file a documentation defect and have this corrected.

See if the half-closed timeout will help in this case.

http://www.cisco.com/en/US/docs/security/fwsm/fwsm40/command/reference/s1.html#wp2725339

Thanks,

KS

Giorgio Romano Sun, 05/08/2011 - 11:50

Hi,

thank you for your answer!

Yes, I posted link for the command "show run sysopt" but in the same argument there is (as related command) "sysopt connection timeout" command.

Anyway I think I didn't understand well.

The "half-closed timeout" connection you seggested to me is the same thing as "time_wait" described in RFC 973?

I have got a BlueCoat Proxies with the "time_wait" timeout set to 120 seconds.

On my FWSM 4.1.(3) the timeout conn half-closed is set to 10 minutes.

May be this difference to generate the error "%FWSM-6-106028"?

Thank you very much again!

giorgio

Kureli Sankar Sun, 05/08/2011 - 13:01

Yes, pls. follow this sample below and see if this helps. It should.

hostname(config)# access-list CONNS permit ip any host 10.1.1.1
hostname(config)# class-map conns
hostname(config-cmap)# match access-list CONNS
hostname(config-cmap)# policy-map conns
hostname(config-pmap)# class conns
hostname(config-pmap-c)# set connection timeout half-closed 0:20:0

They sysopt connection time-wait is supported on the ASA platform but, not int he FWSM.

Pls. follow the above MPF (Modular Policy Framework) method above to configure half-closed timeout selectively
for certain traffic.

-KS
Giorgio Romano Sun, 05/08/2011 - 15:12

Hi, thank you for your advise. Tomorrow I 'll try it and then I'm gonna report you the result.  Thank you  giorgiol

Giorgio Romano Mon, 05/09/2011 - 02:01

Hi,

I applied the following commands. the "interface-name" in the service-policy command is the interface where the error occurs. The access-list name "half-closed" doesn't match. Is it normal?

The error still occurs.

access-list half-closed extended permit ip any any

!

class-map half-closed

match access-list half-closed

!

policy-map half-closed

class half-closed

  set connection timeout half-closed 0:02:00

!
service-policy half-closed interface "interface-name"
Following the error messsages.
May 09 2011 08:38:02: %FWSM-6-106028: Deny TCP (Connection marked for Deletion) from 192.168.1.22/65266 to 122.167.64.13/443 flags SYN  on interface dmz_proxy
May 09 2011 08:38:03: %FWSM-6-106028: Deny TCP (Connection marked for Deletion) from 192.168.1.22/56257 to 122.170.3.155/443 flags SYN  on interface dmz_proxy
May 09 2011 08:38:03: %FWSM-6-106028: Deny TCP (Connection marked for Deletion) from 192.168.1.21/55900 to 83.211.26.2/443 flags SYN  on interface dmz_proxy
May 09 2011 08:38:04: %FWSM-6-106028: Deny TCP (Connection marked for Deletion) from 192.168.1.22/51982 to 122.167.152.52/443 flags SYN  on interface dmz_proxy
May 09 2011 08:38:04: %FWSM-6-106028: Deny TCP (Connection marked for Deletion) from 192.168.1.22/62148 to 122.162.237.149/443 flags SYN  on interface dmz_proxy
May 09 2011 08:38:04: %FWSM-6-106028: Deny TCP (Connection marked for Deletion) from 192.168.1.22/60103 to 122.168.48.16/443 flags SYN  on interface dmz_proxy
Thank you
giorgio
Kureli Sankar Mon, 05/09/2011 - 03:12

These logs show SYN flag set in the packets. So I am assuming this is because TCP port (source,

destination or both) was reused to start a new connection  within the
TIME_WAIT period prescribed in the TCP RFC (793) for an  existing connection.
Pls. check logs prior to these to see if the same hosts sent same source port and dest port to establish the
connection. If so, these logs are to be expected.


-KS
Giorgio Romano Mon, 05/09/2011 - 04:32

HI,

it is for this reason that I'm trying to find out the TIME_WAIT value set on the FWSM.

The logs prior are present.

The destination ports are always TCP 80 e TCP 443 (web server). The destination hosts are many servers on internet.

The source hosts are some bluecoat proxy. Thay have the TIME_WAIT value set to 120 seconds. May be this thing to create the error? Is there a command on FWSM to find out the TIME_WAIT value on the FWSM?

thank you, thank you very much

giorgio

Kureli Sankar Mon, 05/09/2011 - 05:23

Time_wait is set on the end hosts.

Here is an example:
Once a host sends a FIN, it moves to CLOSING state in the TCP
state-machine. Later, once it receives the ACK for this FIN, it moves to
CLOSED state. From this time to again open the same port is called the
TIME_WAIT period. This is defined by the RFC; however, this is also a
configurable item in some of the hosts. However, for fail-safe
mechanisms, there will be a timeout for the CLOSING state, after which
even if there is no ACK, the state-machine moves to the CLOSED state.

However, in case of the FWSM, once it receives a FIN in one direction,
the conn moves to the half-closed state. Once the ACK for this is
received, the conn is "marked-for-deletion". Later a separate thread
running in the FWSM, (which runs periodically), scans for conns marked
for deletion and permanently deletes it from the system. Similar to
end-hosts, if the ACK for the FIN is not received, we have a timeout
(for half-closed, which can be configured), after which even if there is
no ACK, we "mark" the connection for deletion and the aging thread takes
over as described.

What you configured via MPF is half-closed timeout.

-KS

Giorgio Romano Mon, 05/09/2011 - 22:53

Hi,

your explanation has been exhaustive.

But the error occurs again.

Now I try to change some settings on the proxies.

I'll keep you posted.

Thank you

giorgio

shankar1023 Mon, 12/05/2011 - 09:10

I got the same error in one of the fwsm. It got resolved after interchanging the ACL object groups.

Actions

This Discussion