Need help with RIP issue- spokes not talking

Answered Question
Nov 25th, 2008

Hello,

I have a hub and spoke router setup, and for some reason, the edge routers are not able to communicate with each other.

R1 (10.0.1.1) is the hub and connects by frame relay to the remote sites (10.0.1.x).

Each remote site has an ethernet interface to the network 192.168.x.1. The hub router also has an ethernet interface to the 192 network (192.168.1.1).

I kind of dropped into this spot to look at router configs that had been done by someone I never met, so I don't know what I might have goofed up. We added a non-Cisco router to the network, so I had to change the routing protocol from EIGRP to RIP v2.

For instance, I SSH into 10.0.1.8 and into 10.0.1.20. Neither can ping the other, though they can still ping the hub. The router shows the route for the devices, though.

Here is the routing table for R8:

Gateway of last resort is 10.0.1.1 to network 0.0.0.0

R 192.168.12.0/24 [120/2] via 10.0.1.1, 00:00:14, Serial0/0

R 192.168.13.0/24 [120/2] via 10.0.1.1, 00:00:14, Serial0/0

R 192.168.14.0/24 [120/2] via 10.0.1.1, 00:00:14, Serial0/0

C 192.168.8.0/24 is directly connected, FastEthernet0/0

R 192.168.9.0/24 [120/2] via 10.0.1.1, 00:00:14, Serial0/0

R 192.168.10.0/24 [120/2] via 10.0.1.1, 00:00:14, Serial0/0

R 172.16.0.0/16 [120/1] via 10.0.1.1, 00:00:14, Serial0/0

R 192.168.11.0/24 [120/2] via 10.0.1.1, 00:00:14, Serial0/0

R 192.168.4.0/24 [120/2] via 10.0.1.1, 00:00:14, Serial0/0

R 192.168.21.0/24 [120/2] via 10.0.1.1, 00:00:14, Serial0/0

R 192.168.20.0/24 [120/2] via 10.0.1.1, 00:00:14, Serial0/0

R 192.168.5.0/24 [120/2] via 10.0.1.1, 00:00:14, Serial0/0

C 10.0.0.0/8 is directly connected, Serial0/0

S 192.168.23.0/24 [1/0] via 192.168.1.4

R 192.168.22.0/24 [120/2] via 10.0.1.1, 00:00:14, Serial0/0

R 192.168.7.0/24 [120/2] via 10.0.1.1, 00:00:14, Serial0/0

R 192.168.17.0/24 [120/2] via 10.0.1.1, 00:00:14, Serial0/0

R 192.168.16.0/24 [120/2] via 10.0.1.1, 00:00:14, Serial0/0

R 192.168.1.0/24 [120/1] via 10.0.1.1, 00:00:14, Serial0/0

R 192.168.19.0/24 [120/2] via 10.0.1.1, 00:00:14, Serial0/0

R 192.168.18.0/24 [120/2] via 10.0.1.1, 00:00:14, Serial0/0

R 192.168.3.0/24 [120/2] via 10.0.1.1, 00:00:14, Serial0/0

S* 0.0.0.0/0 [1/0] via 10.0.1.1

[1/0] via 192.168.1.1

And here is the routing table for R20:

Gateway of last resort is 10.0.1.1 to network 0.0.0.0

R 192.168.12.0/24 [120/2] via 10.0.1.1, 00:00:09, Serial0/0

R 192.168.13.0/24 [120/2] via 10.0.1.1, 00:00:09, Serial0/0

R 192.168.14.0/24 [120/2] via 10.0.1.1, 00:00:09, Serial0/0

R 192.168.8.0/24 [120/2] via 10.0.1.1, 00:00:09, Serial0/0

R 192.168.9.0/24 [120/2] via 10.0.1.1, 00:00:09, Serial0/0

R 192.168.10.0/24 [120/2] via 10.0.1.1, 00:00:09, Serial0/0

R 172.16.0.0/16 [120/1] via 10.0.1.1, 00:00:09, Serial0/0

R 192.168.11.0/24 [120/2] via 10.0.1.1, 00:00:09, Serial0/0

R 192.168.4.0/24 [120/2] via 10.0.1.1, 00:00:09, Serial0/0

R 192.168.21.0/24 [120/2] via 10.0.1.1, 00:00:09, Serial0/0

C 192.168.20.0/24 is directly connected, FastEthernet0/0

R 192.168.5.0/24 [120/2] via 10.0.1.1, 00:00:09, Serial0/0

C 10.0.0.0/8 is directly connected, Serial0/0

S 192.168.23.0/24 [1/0] via 192.168.1.4

R 192.168.22.0/24 [120/2] via 10.0.1.1, 00:00:09, Serial0/0

R 192.168.7.0/24 [120/2] via 10.0.1.1, 00:00:09, Serial0/0

R 192.168.17.0/24 [120/2] via 10.0.1.1, 00:00:09, Serial0/0

R 192.168.16.0/24 [120/2] via 10.0.1.1, 00:00:09, Serial0/0

R 192.168.1.0/24 [120/1] via 10.0.1.1, 00:00:09, Serial0/0

R 192.168.19.0/24 [120/2] via 10.0.1.1, 00:00:09, Serial0/0

R 192.168.18.0/24 [120/2] via 10.0.1.1, 00:00:09, Serial0/0

R 192.168.3.0/24 [120/2] via 10.0.1.1, 00:00:09, Serial0/0

S* 0.0.0.0/0 [1/0] via 10.0.1.1

[1/0] via 192.168.1.1

Any help is greatly appreciated!

Correct Answer by Richard Burts about 8 years 2 months ago

Sean

I have looked at the config that you posted. I do not see anything in it that would get around the issue of router to router ping without multiple Frame Relay map commands. I would assume that it has always not worked (and people just did not realize that it did not work). This would not impact the functionality of the network traffic going LAN to LAN but only impacts spoke router to spoke router traffic.

[edit] thank you for using the rating system to indicate that your issue was resolved (and thanks for the ratings). It makes the forum more useful when people can read about issues and can know that there were responses which did lead to a solution of the issue.

The forum is an excellent place to learn about Cisco networking. I encourage you to continue your participation in the forum.

HTH

Rick

Correct Answer by Giuseppe Larosa about 8 years 2 months ago

Hello Sean,

Rick is right:

each spoke has only one valid PVC that is the one towards the hub so the additional frame-relay map commands for other spoke ip address need to use the same DLCI they use to reach hub.

Sorry if we have been unclear in previous posts.

Hope to help

Giuseppe

Correct Answer by Richard Burts about 8 years 2 months ago

Sean

I am not clear when you say:"I added the statement frame-relay map ip 10.0.1.8 xx broadcast to the R12 serial interface" what xx is. It should be the same as what R12 has in its map for the hub router. From your statement I wonder if it is what the hub has in its map for the spoke. Similarly R8 should use the same DLCI for its map to R12 that it uses on its map for the hub.

I am not sure that I understand what you are saying about the extended ping. But until we get straight on the map statements it is hard to address other aspects of the problem.

Perhaps you could post the output of show frame-relay map from the 2 spoke routers (and perhaps also the output from the ping and extended ping).

HTH

Rick

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 4.6 (9 ratings)
Loading.
Giuseppe Larosa Tue, 11/25/2008 - 08:47

Hello Sean,

>> We added a non-Cisco router to the network, so I had to change the routing protocol from EIGRP to RIP v2.

I would have added RIPv2 in parallel to EIGRP and then later you could remove EIGRP but only after a careful testing.

I suppose you have attached the sh ip route of two spokes:

RIP looks like to work correctly:

are all IP subnets present in the table ?

the ip next-hop is the hub 10.0.1.1

This means you have disabled split-horizon on the hub.

Next step is to look at frame-relay mapping:

if you ping 192.168.20.1 from first spoke you have no result.

What if you perform an extended ping using source from 192.168.8.1 to 192.168.20.1 (for the two spokes I see from the connected) what is the result ?

If you can ping in this way the problem is that each side knows how to reach 10.1.1.1 on the NBMA but not to reach another spoke on the hub.

To solve this you would need additional frame-relay map statements for each spoke ip address on the FR NBMA all using the same DLCI that leads to the hub router.

What are your frame-relay statements ?

I see you are using the physical interface 0/0.

Hope to help

Giuseppe

Wabbit___ Tue, 11/25/2008 - 11:30

Thanks for the help Guiseppe!

I did have RIP running with EIGRP, but it seemed to be working fine because the data moves from edge to hub router as needed. The problem comes with moving data from one edge router to another, which doesn't happen often, but has surfaced as a problem.

All subnets that I have tried to connect to show up on the routing tables.

This is the entry for Split-Horizon on the hub interface S4/0:

no ip split-horizon eigrp 2

No I need to do the same on the hub for RIP?

On one of the edge routers, the no split-horizon entry is only on FA0/0 (the 192 network), but not anywhere else.

Is split-horizon a potential problem? Do I need to go through all of the routers and do a no ip split-horizon statement on every interface? Since it is a hub and spoke network, there is no chance of routing loops.

Do I really need to add frame-relay maps on every router? It doesn't seem to make sense, since the edges only frame map to the hub router, and that is what is frame mapped. After that, it should be a matter of IP routing.

Is there an issue with using the physical interface 0/0?

Thanks!

Giuseppe Larosa Tue, 11/25/2008 - 13:02

Hello Sean,

on the hub you have for sure a split-horizon command that applies to RIP otherwise it wouldn't propagate spoke to spoke the LAN IP subnets but the show ip route on spokes demonstrate the routes arrive.

it should be simply

no ip split-horizon

frame-relay map ip .... broadcast

it is necessary the broadcast option to propagate multicast traffic for example.

all the frame-relay mappings are not needed for LAN to LAN IP connectivity.

Auto-summary is disabled ?

ip classless

should be enabled by default allow to avoid to discard packets for unknown subnets on the same major network.

net 192.168.x.0/24 are already major networks and so it shouldn't be a problem even with auto-summary enabled and ip classless disabled.

There is any type of IP ACL applied on the HUB ?

Hope to help

Giuseppe

Wabbit___ Wed, 11/26/2008 - 05:07

Hi Guiseppe,

I added the no ip split horizon command (and the no ip split horizon eigrp 2, just for good measure, since there are still 2 routers on eirgp).

All the frame relay maps already include the broadcast command.

I found that auto-summary was disabled on the router eigrp section, but not in the RIP section, so I added that line.

It does say "ip classless" in the general section. There are no ACL's applied on the edge routers, but there are 3 on the hub:

access-list 190 permit tcp any any established

access-list 190 permit udp any any

access-list 190 permit icmp any any echo-reply

The access list 190, however, does not appear to be applied to any interface. I checked this by looking at show ip int, then scanning to the line that says outgoing (and inbound) access list is not set.

I also did the same things on the two edge routers I am testing this with.

I did all of that. Now just to see what is going on, I did a series of pings from the edge routers. None made it to the other edge routers (10.0.1.x), but it does make it to the hub router (10.0.1.1).

OK, here is where it REALLY gets weird! Pinging it's own interface (10.0.1.8, for instance) gets a failure, too!

Now I'm stumped! I'm looking through the configs for any security setting on this, but I'm not seeing it.

Help!

Thanks!

Richard Burts Sat, 11/29/2008 - 00:02

Sean

The reason that you can not ping your own Frame Relay interface on the spoke is not due to any security setting but is another issue with your Frame Relay maps (or lack of Frame Relay maps to be more specific).

To understand this lets start with a review of Frame Relay maps and what they do. A Frame Relay map establishes a mapping between a layer 3 address and a corresponding layer 2 address. Conceptually they are very similar to MAC addresses for Ethernet. So when a router is transmitting a packet over Frame Relay it knows what is the destination IP address and it must find the correct layer 2 address (what PVC is used to get to that IP address) to use in the packet to be able to correctly send the packet.

Now lets consider how that relates to your issues. Any spoke router does have a Frame Relay map to the hub router because they are directly connected. And so any spoke router can ping the hub router. By default any spoke router does not have Frame Relay maps to any other spoke router. So if a spoke router (say for example 10.0.1.8) attempts to ping another spoke router (say for example 10.0.1.12) it does not have a Frame Relay map for 10.0.1.12, so it does not know what PVC to use to get to 10.0.1.12 and the ping fails. If you want spoke routers to be able to ping other spoke router Frame Relay interfaces then you will need to configure Frame Relay maps on each of the spoke routers for every other spoke router. If you do not want to configure all these Frame Relay maps your network will operate ok - but you will not be able to ping spoke to spoke.

Consider what happens when a spoke router attempts to ping the Ethernet of some other spoke router. Its routing table has the other spoke Ethernet with a next hop of 10.0.1.1. And the router has a Frame Relay map for 10.0.1.1 and so the spoke is able to ping other spoke Ethernet (while it is not able to ping other spoke Frame Relay interface).

The function of the Frame Relay map is also the reason why your attempt to ping your own Frame Relay interface is failing. When 10.0.1.8 router attempts to ping 10.0.1.8 it looks for a Frame Relay map for 10.0.1.8 (it needs to know what PVC to use to get to 10.0.1.8). Since it does not have a Frame Relay map for 10.0.1.8 then the ping fails. If you want to ping your own Frame Relay interface then you must configure a Frame Relay map for your own interface address.

Note that all of these complications with Frame Relay maps are due to the use of multipoint Frame Relay at the hub. If the hub were configured with point to point subinterfaces none of these complications would be a problem. This is one of the reasons why point to point subinterfaces are generally preferred to multipoint for Frame Relay.

HTH

Rick

Wabbit___ Mon, 12/01/2008 - 05:44

I knew that you needed frame relay maps to access other frame relay routers, but I thought that since the edge router's route table shows that the next hop to an edge router is through the hub, and the hub is able to ping edge routers, then shouldn't the hub forward and return pings the same way?

Even if they are not, and this is my real problem, why can't I ping the inside interface (192.168.x.1) of one edge router to another? For example, R8 is trying to ping R12's inside interface (192.168.12.1). Here is an abbreviated route table for R8:

* * * * * * * * * * *

R 192.168.12.0/24 [120/2] via 10.0.1.1, 5d00h, Serial0/0

[120/2] via 10.0.1.12, 00:00:22, Serial0/0

C 192.168.8.0/24 is directly connected, FastEthernet0/0

R 192.168.1.0/24 [120/1] via 10.0.1.1, 00:00:22, Serial0/0

C 10.0.0.0/8 is directly connected, Serial0/0

S* 0.0.0.0/0 [1/0] via 10.0.1.1

[1/0] via 192.168.1.1

* * * * * * * * * * *

I can ping the edge router's internal interface (192.168.x.1) from the hub router, but I can't ping the same address from an edge router.

This is driving me nuts!

Any help is greatly appreciated!

Richard Burts Mon, 12/01/2008 - 06:55

Sean

How many times do Guiseppe and I need to say that the issue is Frame Relay maps before you will believe us???

If R8 is sending ping to R12 Ethernet then the ping packet destination address is 192.168.12.1 and the source address is 10.0.1.8. Since the destination address is in the routing table with a next hop of the hub router the ping packet is sent and arrives at R12. Now R12 is attempting to send a response. The response will go to 10.0.1.8. What PVC should R12 use to get to 10.0.1.8? Since there is no map on R12 which says what PVC to use to get to 10.0.1.8 then the response packet is dropped and the ping fails.

To fix this you need Frame Relay maps at each spoke for every other spoke!!!

HTH

Rick

Giuseppe Larosa Mon, 12/01/2008 - 07:02

Hello Sean,

>>

What if you perform an extended ping using source from 192.168.8.1 to 192.168.20.1 (for the two spokes I see from the connected) what is the result ?

If you can ping in this way the problem is that each side knows how to reach 10.1.1.1 on the NBMA but not to reach another spoke on the hub.

one very basic concept about ICMP and router :

if you don't specify a source address in the ping the router will use the nearest address to the destination that is the NBMA FR address.

If this happen the problem is on the return it is the icmp echo reply to 10.0.1.8 that cannot be routed back by the other edge

Hope to help

Giuseppe

Wabbit___ Mon, 12/01/2008 - 08:28

Hi Rick and Guiseppe,

Ok, first I added the statement "frame-relay map ip 10.0.1.8 xx broadcast" to the R12 serial interface. I did the same on the R8 interface, pointing to R12 and using the same DLCI's that are on the hub for those routers.

I then tried the extended ping and got some weird responses. 20% of the pings respond when using the internal ip, but 0% respond still when I do a normal ping.

When I do a show frame-relay map, it shows that the other edge router's dlci is "deleted".

Any ideas?

Correct Answer
Richard Burts Mon, 12/01/2008 - 08:40

Sean

I am not clear when you say:"I added the statement frame-relay map ip 10.0.1.8 xx broadcast to the R12 serial interface" what xx is. It should be the same as what R12 has in its map for the hub router. From your statement I wonder if it is what the hub has in its map for the spoke. Similarly R8 should use the same DLCI for its map to R12 that it uses on its map for the hub.

I am not sure that I understand what you are saying about the extended ping. But until we get straight on the map statements it is hard to address other aspects of the problem.

Perhaps you could post the output of show frame-relay map from the 2 spoke routers (and perhaps also the output from the ping and extended ping).

HTH

Rick

Correct Answer
Giuseppe Larosa Mon, 12/01/2008 - 10:31

Hello Sean,

Rick is right:

each spoke has only one valid PVC that is the one towards the hub so the additional frame-relay map commands for other spoke ip address need to use the same DLCI they use to reach hub.

Sorry if we have been unclear in previous posts.

Hope to help

Giuseppe

Wabbit___ Mon, 12/01/2008 - 10:36

Hi Rick and Guiseppe,

I used the "xx" as a generic for the DLCI, for instance 19. I did enter the DLCI incorrectly, giving it the same DLCI as that used by the hub. Thinking about it, I can see why it would need to be the next hop DLCI. I changed that and the pings are now successful.

The person (from another company) who complained about this says that the lack of connectivity between routers is new, since the time I added the new router and configured RIP. I suspect he never tried to ping router to router before this, but I will never get him to tell me the truth on that, I fear. But it leaves me wondering if this was, perhaps, somehow working before?

Looks like I am just about solved on this one, but am I to understand that using sub-interfaces on the hub router (but not necessarily on the edge routers) would allow this traffic without having to manually enter DLCI mappings on each edge router?

Thanks!

Richard Burts Mon, 12/01/2008 - 11:36

Sean

I am glad that you now have this pretty well sorted out and that pings are now working.

It is hard to know (based on what we know at this point) whether it worked under EIGRP or not. (but I am guessing that it was broken then as well) Would you have a config from a spoke router when they were running EIGRP that you could post? I am wondering if the routers were configured with EIGRP neighbor statements for the spokes if that might let the router get around the map issue.

Yes you understand correctly that if you use point to point subinterfaces on the hub router that it avoids the complexities of configuring Frame Relay maps on the spoke routers. (be aware that using point to point subinterfaces on the hub means that the link from hub to spoke will be a unique subnet for each spoke, where they are now all on a common subnet. so it would not be transparent to the spoke if you change it)

HTH

Rick

Wabbit___ Mon, 12/01/2008 - 12:10

Hi Rick,

Here is the config for R22 (minus some security info):

* * * * * * * * * * *

version 12.3

service timestamps debug datetime msec

service timestamps log datetime msec

service password-encryption

!

hostname R20

!

boot-start-marker

boot-end-marker

!

enable secret 5

enable password 7

!

username password 7

clock timezone ET -6

clock summer-time ET recurring 2 Sun Mar 2:00 1 Sun Nov 2:00

mmi polling-interval 60

no mmi auto-configure

no mmi pvc

mmi snmp-timeout 180

no aaa new-model

ip subnet-zero

!

ip domain name info.com

!

ip cef

ip audit po max-events 100

ip ssh time-out 60

ip ssh authentication-retries 2

no ftp-server write-enable

!

interface FastEthernet0/0

ip address 192.168.20.1 255.255.255.0

speed auto

full-duplex

!

interface Serial0/0

ip address 10.0.1.20 255.0.0.0

encapsulation frame-relay

frame-relay map ip 10.0.1.1 16 broadcast

frame-relay map ip 10.0.1.2 17 broadcast

frame-relay lmi-type ansi

!

router eigrp 2

network 10.0.0.0

network 192.168.20.0

auto-summary

no eigrp log-neighbor-changes

!

ip classless

ip route 192.168.22.0 255.255.255.0 10.0.1.1

ip route 192.168.23.0 255.255.255.0 192.168.1.4

no ip http server

no ip http secure-server

!

logging trap debugging

logging 172.16.200.6

!

snmp-server community password RO

snmp-server enable traps tty

snmp-server enable traps frame-relay

snmp-server enable traps frame-relay subif

banner motd _

State and Federal laws make it a crime to gain unauthorized access to

this computer system. System is for approved business purposes only.

Violators will be PROSECUTED!

_

!

line con 0

password 7

login local

line aux 0

line vty 0 4

password 7

login local

transport input ssh

!

!

end

* * * * * * * * * * *

I didn't see anything on the config about EIGRP neigbors. Is that possible?

I don't think the subinterfaces idea would work out for me, since I need all the sites to be on the same subnet.

Thanks!

Correct Answer
Richard Burts Mon, 12/01/2008 - 12:48

Sean

I have looked at the config that you posted. I do not see anything in it that would get around the issue of router to router ping without multiple Frame Relay map commands. I would assume that it has always not worked (and people just did not realize that it did not work). This would not impact the functionality of the network traffic going LAN to LAN but only impacts spoke router to spoke router traffic.

[edit] thank you for using the rating system to indicate that your issue was resolved (and thanks for the ratings). It makes the forum more useful when people can read about issues and can know that there were responses which did lead to a solution of the issue.

The forum is an excellent place to learn about Cisco networking. I encourage you to continue your participation in the forum.

HTH

Rick

Wabbit___ Mon, 12/01/2008 - 12:58

Rick and Guiseppe,

Thanks to both of you for the help and education!

Actions

This Discussion