Finding source of traffic - behind router

Answered Question
Feb 25th, 2010

I have an esoft internet gateway appliance that is producing this message :

Feb 25 00:00:21 System martian source 192.168.1.255 from 192.168.1.135, on dev br0
Feb 25 00:00:21 System ll header: 00:01:4e:01:7a:b4:00(Destination - E-soft App):1e:f7:ae:b6:c0:08:00(Source Router at 10.x.x.x)

Our network scheme is 10.x.x.x, with no 192.168.*.* subnets on the local lan.  However, for some reason the router behind this gateway is still forwarding traffic from 192.168.1.* to the gateway.  My router has 3 interfaces, 10.x.x.x; 10.x.x.x, and 72.x.x.x.  Not sure how traffic is even getting to the router, let alone being forwarded.

From the switch I can identify the the devices by the mac's using "show mac address-table", but I cannot determine the source of the traffic from the other side of the router.

How do I track down the source of this traffic?

I have this problem too.
0 votes
Correct Answer by Jon Marshall about 6 years 9 months ago

David

Apologies but i gave some misleading info. To log port numbers your acl must match on port numbers so can you make your acl -

access-list 151 permit tcp 192.168.1.0 0.0.0.255 any range 1 65535 log

access-list 151 permit udp 192.168.1.0 0.0.0.255 any range 1 65535 log

access-list 151 permit icmp 192.168.1.0 0.0.0.255 any log
access-list 151 permit ip any any

then apply this acl to the interface that connect to your MPLS network.

Note that the router will buffer the logs so you may only see one entry for multiple hits.

Jon

Correct Answer by Federico Coto F... about 6 years 9 months ago

So, the only two connections from the 3845 are the Gateway and the MPLS Switch.

We know that the internal network has no 192.168.x.x

What about the MPLS network?

You mentioned the 3845 has no route to 192.168.x.x, but it has a default gateway pointing to the Gateway (because you said that a traceroute to 192.168.x.x goes from the 3845 to the gateway).

Are those messages in the Gateway recent and constant messages?

Because as you describe it, there's no sign of 192.168.x.x inside the Gateway.

Somebody could be spoofing IPs and that's why it shows on the Gateway, but it seems the router does not see that.

You can try the following:

Since the Gateway claims that the 192.168.x.x comes from the 3845, you can do an ACL on the 3845:

access-list test permit ip host 192.168.0.0 0.0.255.255 any

access-list permit ip any any

Apply the ACL to the interface of the 3845 connected to the Gateway, so we'll see if the ACL shows hitcounts when this happen again.

This will prove if the path to 192.168.x.x is indeed passing through the 3845.

Federico.

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (2 ratings)
Loading.
Federico Coto F... Thu, 02/25/2010 - 09:14

Hi,

Can you PING the 192.168.1.135 address from the router?

Is this a Cisco Router? Can you traceroute that IP?

Check the routing table on the router to see if there's a route for 192.168.x.x

Federico.

davidjknapp Thu, 02/25/2010 - 10:22

192.168.1.0/24 is not in the routing table, a tracert takes me out to our internet gateway then drops the packet.  Router is a 3845 with c3845-advsecurityk9-mz.124-3i.bin

Federico Coto F... Thu, 02/25/2010 - 10:53

Problem is that is the gateway that is seeing this message not the router. Is the message telling that the

packet comes from the 3845?

If so, the MAC address corresponds to the Ethernet interface of the 3845 connected to the Gateway?

davidjknapp Thu, 02/25/2010 - 11:37

According to the gateway (eSoft Appliance), The sending MAC address matches the g0/0 (LAN) MAC of the 3845.  The gateway is on the same subnet as that interface (router is x.x.x.1, gateway is x.x.x.5).  So the orgin of the packets could be comming form the lan or accross the MPLS which is also attached to that router.

Correct Answer
Federico Coto F... Thu, 02/25/2010 - 11:52

So, the only two connections from the 3845 are the Gateway and the MPLS Switch.

We know that the internal network has no 192.168.x.x

What about the MPLS network?

You mentioned the 3845 has no route to 192.168.x.x, but it has a default gateway pointing to the Gateway (because you said that a traceroute to 192.168.x.x goes from the 3845 to the gateway).

Are those messages in the Gateway recent and constant messages?

Because as you describe it, there's no sign of 192.168.x.x inside the Gateway.

Somebody could be spoofing IPs and that's why it shows on the Gateway, but it seems the router does not see that.

You can try the following:

Since the Gateway claims that the 192.168.x.x comes from the 3845, you can do an ACL on the 3845:

access-list test permit ip host 192.168.0.0 0.0.255.255 any

access-list permit ip any any

Apply the ACL to the interface of the 3845 connected to the Gateway, so we'll see if the ACL shows hitcounts when this happen again.

This will prove if the path to 192.168.x.x is indeed passing through the 3845.

Federico.

davidjknapp Thu, 02/25/2010 - 12:08

OK... had to modify slightly..

interface GigabitEthernet0/0
description This is the local area connection
ip address 10.4.117.1 255.255.255.0
ip access-group 151 out
ip route-cache flow
duplex auto
speed auto
media-type rj45
negotiation auto
!
access-list 151 permit ip 192.168.1.0 0.0.0.255 any
access-list 151 permit ip any any
!

Here is the result after a few seconds of a "sho ip access-list" command


Extended IP access list 151
    10 permit ip 192.168.1.0 0.0.0.255 any (107 matches)
    20 permit ip any any (48573 matches)

Federico Coto F... Thu, 02/25/2010 - 12:16

This is very interesting...
You applied the ACL outbound to the interface directly connected to the Gateway correct?

The ACL shows that there are matches to traffic coming from 192.168.1.x

Now, we know that packets from 192.168.1.x are exiting out the interface of the 3845 to the Gateway, but through which interface is that traffic
coming?

Try implementing the same ACL on the inside interface or the interface connected to the MPLS switch to check which one is the traffic coming from.

Federico.

davidjknapp Thu, 02/25/2010 - 12:44

Good Call, the traffic is being generated from the S1/0, which is the MPLS hookup.  So, although we do not have a route to get back the the 192.168.1.x network, it is still routing out to the I-net gateway somehow.  I know I can put an access list on the serial interface blocking the traffic (below - I think).  How do I find out what ip the traffic is being routed from out on the MPLS network?  We have 18 sites.

access-list 101 permit 10.0.0.0 0.128.255.255 any

access-list 101 permit 172.16.0.0 0.240.255.255 any

Int S1/0

     ip access-group 101 in

Federico Coto F... Thu, 02/25/2010 - 12:51

As i understand the connection to the MPLS switch is through the 3845.

And the 3845 only has the internal network and a default gateway pointing to the esoft Gateway.

Is there also a connection between the esoft Gateway and the MPLS switch?

How do you communicate with your 18 sites through the MPLS cloud if the 3845 only has a default gateway to the esoft gateway and no other routes pointing anywhere else, and there are no other connections from your network to the MPLS cloud? Or this is not the case?

Federico.

davidjknapp Thu, 02/25/2010 - 13:03

The 3845 does have routes to the other networks, and the defaul gateway is to the esoft appliance. there is just no route to any 192.168.1.x networks.

The MPLS is terminated at the 3845 only, the eSoft Firewall is aattched to local lan and internet services provider only.

Below is routing table of 3845 for your pleasure....

     172.20.0.0/30 is subnetted, 2 subnets
S       172.20.243.16 [1/0] via 172.20.242.129
C       172.20.242.128 is directly connected, Serial1/0
S    192.168.26.0/24 [1/0] via 172.20.242.129
     10.0.0.0/8 is variably subnetted, 22 subnets, 2 masks
C       10.15.255.254/32 is directly connected, Loopback0
S       10.18.67.0/24 [1/0] via 172.20.242.129
S       10.4.124.0/24 [1/0] via 172.20.242.129
S       10.4.122.0/24 [1/0] via 172.20.242.129
S       10.4.123.0/24 [1/0] via 172.20.242.129
S       10.4.120.0/24 [1/0] via 172.20.242.129
S       10.4.121.0/24 [1/0] via 172.20.242.129
S       10.4.118.0/24 [1/0] via 172.20.242.129
S       10.4.119.0/24 [1/0] via 172.20.242.129
C       10.4.117.0/24 is directly connected, GigabitEthernet0/0
S       10.150.10.0/24 [1/0] via 172.20.242.129
S       10.150.11.0/24 [1/0] via 172.20.242.129
S       10.4.172.0/24 [1/0] via 172.20.242.129
S       10.4.173.0/24 [1/0] via 172.20.242.129
S       10.15.237.0/24 [1/0] via 172.20.242.129
S       10.15.235.0/24 [1/0] via 172.20.242.129
S       10.15.245.0/24 [1/0] via 172.20.242.129
S       10.15.247.0/24 [1/0] via 172.20.242.129
S       10.15.246.0/24 [1/0] via 172.20.242.129
S       10.15.248.0/24 [1/0] via 172.20.242.129
S       10.15.251.0/24 [1/0] via 172.20.242.129
S       10.15.250.0/24 [1/0] via 172.20.242.129
S    192.168.253.0/24 [1/0] via 172.20.242.129
S*   0.0.0.0/0 [1/0] via 10.4.117.5

Federico Coto F... Thu, 02/25/2010 - 13:12

Sounds like what is happening is the following:

192.168.1.x traffic is coming from the MPLS cloud.
Traffic reaches the 3845
Since the 3845 has no route to 192.168.1.x, it uses the default gateway 10.4.117.5 and that's why you see the log message in the esoft Gateway.

The question is:
Why is the MPLS cloud sending you traffic intended to 192.168.1.x?

Federico.

Jon Marshall Thu, 02/25/2010 - 13:39

David

It would help if we knew what port/protocol was being used. Can you add the "log" keyword temporarily to your acl for just the line permitting 192.168.1.0 ie.

access-list 151 permit ip 192.168.1.0 0.0.0.255 any log
access-list 151 permit ip any any

Jon

davidjknapp Thu, 02/25/2010 - 13:51

I did so, but do not know where to look at the log  a "show log" does not dsiplay.  Sorry, newbie showing through.

Jon Marshall Thu, 02/25/2010 - 13:58

David

No problem, have a look at this very short doc -

IOS logging

Basically -

1) enable logging if it is disabled ie. "logging on"

2) log to the buffer  "logging buffered"

"sh logging" should then show logs.

Jon

davidjknapp Thu, 02/25/2010 - 14:12

Still nothing comming through... hope I have relavent data below

I ran commonds below in Config t

logging on

logging buffered

then ran following:

Systems#Sho Conf

!

access-list 151 permit ip 192.168.1.0 0.0.0.255 any log
logging buffered 4096 debugging

Systems#sho logging
Syslog logging: enabled (11 messages dropped, 3 messages rate-limited,
                0 flushes, 0 overruns, xml disabled, filtering disabled)
    Console logging: level debugging, 75 messages logged, xml disabled,
                     filtering disabled
    Monitor logging: level debugging, 0 messages logged, xml disabled,
                     filtering disabled
    Buffer logging: level debugging, 1 messages logged, xml disabled,
                    filtering disabled
    Logging Exception size (4096 bytes)
    Count and timestamp logging messages: disabled
    Trap logging: level informational, 76 message lines logged

Log Buffer (4096 bytes):

*Feb 25 22:06:34.702: %SYS-5-CONFIG_I: Configured from console by djknapp on vty0 (10.4.117.110)

Correct Answer
Jon Marshall Thu, 02/25/2010 - 14:35

David

Apologies but i gave some misleading info. To log port numbers your acl must match on port numbers so can you make your acl -

access-list 151 permit tcp 192.168.1.0 0.0.0.255 any range 1 65535 log

access-list 151 permit udp 192.168.1.0 0.0.0.255 any range 1 65535 log

access-list 151 permit icmp 192.168.1.0 0.0.0.255 any log
access-list 151 permit ip any any

then apply this acl to the interface that connect to your MPLS network.

Note that the router will buffer the logs so you may only see one entry for multiple hits.

Jon

davidjknapp Thu, 02/25/2010 - 14:57

Got it:


*Feb 25 22:50:33.178: %SEC-6-IPACCESSLOGP: list 152 permitted udp 192.168.1.125(138) -> 192.168.1.255(138), 1 packet
*Feb 25 22:50:44.022: %SEC-6-IPACCESSLOGP: list 152 permitted udp 192.168.1.135(138) -> 192.168.1.255(138), 1 packet
*Feb 25 22:51:14.110: %SEC-6-IPACCESSLOGP: list 152 permitted udp 192.168.1.136(137) -> 192.168.1.255(137), 1 packet
*Feb 25 22:51:55.206: %SEC-6-IPACCESSLOGP: list 152 permitted udp 192.168.1.133(138) -> 192.168.1.255(138), 1 packet
*Feb 25 22:53:08.650: %SEC-6-IPACCESSLOGP: list 152 permitted udp 192.168.1.100(138) -> 192.168.1.255(138), 1 packet
*Feb 25 22:54:14.042: %SEC-6-IPACCESSLOGP: list 152 permitted udp 192.168.1.136(138) -> 192.168.1.255(138), 1 packet

Federico Coto F... Thu, 02/25/2010 - 15:01

Seems like regular NetBIOS traffic being broadcasted.

Still don't knoiw why is reaching your esoft Gateway or even the 3845.

Federico.

Jon Marshall Thu, 02/25/2010 - 15:04

As Federico says this is Windows netbios traffic. Netbios can be a very chatty protocol.

Do all your remote sites use a default-route to get to the site where the gateway is ?

Edit - the other thing to note is that this is a directed broadcast ie. 192.168.1.255. This could be caused by an IP helper address on one of your remote site routers.

Jon

davidjknapp Fri, 02/26/2010 - 06:00

I get how the packets are traversing, and yes the main site is the default gateway for all daughter sites.  My fear is a rouge router or WAP plugged in (linksys comes to mind) that is broadcasting unsecured and attached to our network.

Understanding I can use acl's at the hub or at all the sites to block the traffic, I would still very much like to find the source.  Is there a way of doing that?

Thanks for all the help!!

Jon Marshall Fri, 02/26/2010 - 06:11

How many daughter sites do you have ?

The next step would be to apply the acl you have in each daughter site to work out which one is originating the packets.

Jon

davidjknapp Fri, 02/26/2010 - 07:00

There are 17 Daughter sites.  Will take a while, but it would be best to impliment ACL's at each site anyway to drop any packets not meant for our network or from the network at that site.

Thanks a lot Both of you have been a great help.

Actions

This Discussion

Related Content