EIGRP debug information

[martin_lx1980]

I have a simple topology to check how eigrp works.

R1(F0/0)---SW1---(F0/0)R2

Below is information from debug eigrp packet

……………………….

*Mar 1 02:20:32.135: EIGRP: Received Sequence TLV from 192.168.123.2

*Mar 1 02:20:32.135: 192.168.123.1

*Mar 1 02:20:32.139: address matched

*Mar 1 02:20:32.139: clearing CR-mode

*Mar 1 02:20:32.139: EIGRP: Received CR sequence TLV from 192.168.123.2, sequence 5

*Mar 1 02:20:32.139: EIGRP: Received UPDATE on FastEthernet0/0 nbr 192.168.123.2

*Mar 1 02:20:32.139: AS 1, Flags 0xA, Seq 5/0 idbQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/1, not in CR-mode, packet discarded

………………………….

*Mar 1 02:20:34.123: EIGRP: FastEthernet0/0 multicast flow blocking cleared

………………………….

• When does the sequence TLV use in a eigrp network? I only see sequence TLV in broadcast network.
  
• What CR-mode means? I know nothing other than that CR means Conditional Receive
  
• What “multicast flow blocking cleared” means? Does it about multicast flow timer?how does it work?
  

I am so confused with those for a long time and I could find little detail documents about those.

Thanks a lot.

Martin

[Peter Paluch]

Hi Martin,

What you are seeing is the EIGRP implementation of reliable routing information delivery over both multicast and unicast kicking into action. Explaining it will take some time

On multiaccess segments, EIGRP uses multicasts for effective delivery of routing information to all directly connected neighbors. A basic, although naive principle for reliability in this setup would be to send a multicast packet, then expect an explicit (unicast) acknowledgement from each neighbor, and then proceed to send the next multicast packet. However, the major drawback in this setup is that if a single neighbor is delayed, overloaded or with a lossy connection, it may take considerable time till we receive an acknowledgement from it. Until then, no further multicast messages may be sent, delaying the convergence - just because of this single "laggard" neighbor.

A more sophisticated approach would be to wait for acknowledgements only for a certain time period. If some neighbors do not respond within this timeframe, we will consider them "laggard" and requiring special treatment. Other neighbors that have acknowledged the multicast packet can continue receiving next multicast packet in sequence. This way, we split the directly connected neighbors into two groups - the "good" group that is capable of receiving multicast packets normally, and the "laggard" group that has obvious problems with responding in a timely fashion and which needs to be treated differently. Note that the "laggard" group is still on the same segment as the "good" group, and if we continue talking to the "good" group using multicasts, these multicast may also be received by the "laggard" group of routers. However, routers in the "laggard" group may have missed the previous multicast packet (and therefore have not responded to it), and if they received the subsequent multicast packet, they would be processing it out of sequence. Therefore, this approach requires that the "laggard" group does not process the possibly out-of-sequence multicast communication with the "good" group even if it is on the same segment,  until it is able to catch up again.

EIGRP does this using its so-called Conditional Receive mode (CR-mode). When a multicast packet is sent by our EIGRP router to its neighbors on a particular interface, a timer is started called the multicast flow timer. This timer is computed from the SRTT values of all neighbors on that interface. If a neighbor does not respond until the multicast flow timer expires, this neighbor is considered laggard. EIGRP will try to resend the message to this laggard neighbor via unicast. However, it must make sure that this laggard neighbor does not erroneously process the parallel multicast communication with "good" routers on the segment that simply continue the message exchange.

This is accomplished in EIGRP by our router multicasting a special sequenced Hello message that contains a list of laggard routers - the Sequence TLV. If a router is listed in this list of laggard routers, it is informed that it should ignore the subsequent multicast communication. Other routers that receive this special Hello packet and do not see themselves listed in the Sequence TLV know they are "good" receivers and automatically enter the Conditional Receive mode (they may receive the subsequent multicast communication under the condition they have not been identified as laggard in the Sequence TLV). Routers that do not receive this Hello packet at all (because of lossy link, for example) also do not enter the CR-mode. In other words, only those routers that received this sequenced Hello and did not find themselves in the Sequence TLV will enter the CR-mode. Routers that have found themselves in the Sequence TLV, or routers that have not received this sequenced Hello at all, will not enter the CR-mode.

After this splitting of all neighbors into "good" routers and putting them into CR-mode, and "laggard" routers, our router will continue the communication as follows:

• The communication with "good" routers will proceed via multicast. However, the multicast messages will be sent with the CR flag set. Messages with CR flag set are processed only by routers in CR-mode, which are the "good" routers.
• The communication with "laggard" routers will proceed via unicast. Our router will send individual messages to each laggard router in proper sequence and expect an acknowledgement, then proceed to the next message in appropriate order.

Eventually, after the network stabilizes, the laggard routers will catch up with the good routers, even though it takes more time and more concentrated effort of our router. The CR-mode then can be cancelled by our router sending a normal unsequenced Hello message, and the entire communication reverts back to fully multicast. The entire process may repeat in the future as necessary.

An example is probably called for Consider three routers on the same segment, A, B and C:

1. Router A sends an update with the Seq=123 onto the segment and is also ready to send update with Seq=124. Router A places both these updates on the transmission lists for neighbors B and C.
2. Router B receives this update and sends an acknowledgement. Router A will remove this update from the transmission list for neighbor B. However, it can not yet send update with Seq=124 because not all neighbors have yet acknowledged they got update with Seq=123.
3. Router C either misses this update, or the acknowledgement is lost. In any case, router A does not receive the acknowledgement in due time. Therefore, A considers C a laggard router. Router A constructs a Hello message that contains a Sequence TLV listing router C, and sends it onto the segment. Router B receives this Hello message and, not seeing itself in the Sequence TLV, puts itself into CR-mode. Router C will also receive this Hello but as it will find itself inside the Sequence TLV, it will not enter the CR-mode. The same would happen if router C lost this Hello packet altogether. Router A will be sending these sequenced Hello packets until all routers catch up and there is no more need for the CR-mode. Note that the update with Seq=123 and Seq=124 still remains on the transmission list for router C. For router B, the transmission list contains only the update with Seq=124.
4. Router A will now send update with Seq=124 as a multicast with CR bit set. Router B is in CR-mode and will process this packet along with sending an acknowedgement, upon which router A will remove this update from router B's transmission list. Router C will ignore this packet because it is not in CR-mode, and packets flagged with CR bit may be received only by routers in CR-mode.
5. Router A will try to send each message from the transmission list for the router C as unicast to the router C, starting with update Seq=123. Assuming that the router C receives and acknowledges it, router A will remove the update Seq=123 from the transmission list for C and send another update with Seq=124 also as unicast. After router C acknowledges this packet as well, the transmission list for routers B and C will be empty and so the router A has now made sure that the both routers are fully synchronized. It will therefore cancel the CR-mode by reverting to the normal unsequenced Hello messages.

More detailed information can be found in the U.S. patents 5519704 and 7512106 freely available via patents.google.com.

The basic principle is very simple, the explanation however may be quite confusing so please feel more than welcome to ask further! I've had my own share of troubles when trying to understand this

Best regards,

Peter

[Kishore Chennupati]

5 stars Peter

[martin_lx1980]

Hi peter

  Thank you very much for your vivid reply and I understand the process you describe clearly now.

  But I can not certify how this process works with real environment. A simple topology likes that

(192.168.10.1)Loopback0—R1—F0/0(192.168.12.1)---SW—(192.168.12.2)F0/0—R1—Loopback0(192.168.20.1)

I use debug eigrp packets to check the startup process and below is information from R1

*Mar 1 00:39:08.603: EIGRP: Sending HELLO on Loopback0

*Mar 1 00:39:08.603:   AS 1, Flags 0x0, Seq 0/0 idbQ 0/0 iidbQ un/rely 0/0

*Mar 1 00:39:08.607: EIGRP: Received HELLO on Loopback0 nbr 192.168.10.1

*Mar 1 00:39:08.611:   AS 1, Flags 0x0, Seq 0/0 idbQ 0/0

*Mar 1 00:39:08.611: EIGRP: Packet from ourselves ignored

*Mar 1 00:39:08.647: EIGRP: Sending HELLO on FastEthernet0/0

*Mar 1 00:39:08.651:   AS 1, Flags 0x0, Seq 0/0 idbQ 0/0 iidbQ un/rely 0/0

*Mar 1 00:39:08.667: EIGRP: Received HELLO on FastEthernet0/0 nbr 192.168.12.2

*Mar 1 00:39:08.667:   AS 1, Flags 0x0, Seq 0/0 idbQ 2/0

*Mar 1 00:39:08.671: %DUAL-5-NBRCHANGE: IP-EIGRP(0) 1: Neighbor 192.168.12.2 (FastEthernet0/0) is up: new adjacency

*Mar 1 00:39:08.671: EIGRP: Enqueueing UPDATE on FastEthernet0/0 nbr 192.168.12.2 iidbQ un/rely 0/1 peerQ un/rely 0/0

*Mar 1 00:39:08.671: EIGRP: Received UPDATE on FastEthernet0/0 nbr 192.168.12.2

*Mar 1 00:39:08.675:   AS 1, Flags 0x1, Seq 1/0 idbQ 2/0 iidbQ un/rely 0/1 peerQ un/rely 0/0

### first update received from R2######

*Mar 1 00:39:08.675: EIGRP: Requeued unicast on FastEthernet0/0

*Mar 1 00:39:08.679: EIGRP: Sending HELLO on FastEthernet0/0

*Mar 1 00:39:08.683:   AS 1, Flags 0x0, Seq 0/0 idbQ 2/0 iidbQ un/rely 0/0

*Mar 1 00:39:08.683: EIGRP: Received HELLO on FastEthernet0/0 nbr 192.168.12.2

*Mar 1 00:39:08.683:   AS 1, Flags 0x0, Seq 0/0 idbQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/1

*Mar 1 00:39:08.687: EIGRP: Received Sequence TLV from 192.168.12.2

*Mar 1 00:39:08.687:       192.168.12.1

*Mar 1 00:39:08.687:       address matched

*Mar 1 00:39:08.687:       clearing CR-mode

*Mar 1 00:39:08.687: EIGRP: Received CR sequence TLV from 192.168.12.2, sequence 2

### Receive Sequence TLV from R2######

*Mar 1 00:39:08.691: EIGRP: Received UPDATE on FastEthernet0/0 nbr 192.168.12.2

*Mar 1 00:39:08.691:   AS 1, Flags 0xA, Seq 2/0 idbQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/1, not in CR-mode, packet discarded

*Mar 1 00:39:08.695: EIGRP: Sending UPDATE on FastEthernet0/0 nbr 192.168.12.2

*Mar 1 00:39:08.695:   AS 1, Flags 0x1, Seq 1/1 idbQ 0/0 iidbQ un/rely 0/0 peerQ un/rely 0/1

*Mar 1 00:39:08.695: EIGRP: Enqueueing UPDATE on FastEthernet0/0 iidbQ un/rely 0/1 serno 1-2

*Mar 1 00:39:08.699: EIGRP: Building Sequence TLV

####Only 4ms later####

*Mar 1 00:39:08.699: EIGRP: Enqueueing UPDATE on FastEthernet0/0 nbr 192.168.12.2 iidbQ un/rely 0/0 peerQ un/rely 0/1 serno 1-2

*Mar 1 00:39:08.703: EIGRP: Sending HELLO with Sequence TLV on FastEthernet0/0, seq 2

*Mar 1 00:39:08.703: EIGRP: Sending HELLO on FastEthernet0/0

*Mar 1 00:39:08.703:   AS 1, Flags 0x0, Seq 0/0 idbQ 0/0 iidbQ un/rely 0/0

。。。。。。。。。。。。。。。。。。。。。。。。。。。。。

Note the bold words:

R1 receives update from R2 for the first time. And only 16ms(00:39:08.671--00:39:08.687) later R1 receives the sequence TLV listing R1 and is identified as laggard router.But the output of show ip eigrp interface from R2 show that multicast flow timer is 224ms。R2 should identify R1 as a laggard router after multicast flow timer is expired. Why R2 only waits 16ms before sending the sequence TLV. R1 has the same process

R2#sh ip eigrp int

IP-EIGRP interfaces for process 1

                       Xmit Queue   Mean   Pacing Time   Multicast   Pending

Interface       Peers Un/Reliable SRTT   Un/Reliable   Flow Timer   Routes

Lo0               0       0/0         0       0/1           0          0

Fa0/0             1       0/0       52       0/1         224           0

Martin

[Peter Paluch]

Hi Martin,

Ah, I did not know that you were speaking about the initial EIGRP synchronization.

During the initial EIGRP sync, a newly discovered neighbor is always considered laggard, as it is assumed that the synchronization with it will take some time, during which possibly new (multicast) messages have to be sent to already synchronized neighbors. Therefore, if our router detects a new neighbor, it automatically places it in the "laggard" group and performs separate unicast synchronization with it. This allows the sync with the neighbor to occur in parallel with normal multicast communication on the segment. After the newly discovered neighbor is brought on par with other already synchronized routers, it will stop being considered a laggard router. Hence, seeing a new neighbor being treated as a laggard router is actually intentional.

I would also like to correct a previous inaccuracy. I have stated that the CR-mode is cancelled by our router transmitting a normal unsequenced Hello. There is little information about it but it seems that I was not correct about this statement. To what I was able to understand further, the CR-mode is always relative to a multicast segment with a particular sequence number. This sequence number is included in the sequenced Hello along with the Sequence TLV. After this multicast packet with a CR flag is received and accepted by a router in CR-mode, its arrival should also represent the end of the CR-mode. If another multicast message is to be conditionally received, it must first be preceded with another Hello message containing the Sequence TLV and the sequence number of this next multicast message in question. In other words, the CR-mode always lasts only for a single multicast message whose sequence number was indicated in the preceding sequenced Hello.

My previous example should therefore be reworked as follows:

1. Router A sends an update with the Seq=123 onto the segment and  is also ready to send update with Seq=124. Router A places both these  updates on the transmission lists for neighbors B and C.
2. Router B  receives this update and sends an acknowledgement. Router A will remove  this update from the transmission list for neighbor B. However, it can  not yet send update with Seq=124 because not all neighbors have yet  acknowledged they got update with Seq=123.
3. Router C either  misses this update, or the acknowledgement is lost. In any case, router A  does not receive the acknowledgement in due time. Therefore, A  considers C a laggard router. Router A constructs a Hello message that  contains a Sequence TLV listing router C, sequence number 124, and sends it onto the segment.
4. Router B receives this Hello message and, not seeing itself in the  Sequence TLV, puts itself into CR-mode, expecting the multicast update with the Seq=124 as indicated in the sequenced Hello. Router C will also receive this  Hello but as it will find itself inside the Sequence TLV, it will not  enter the CR-mode. The same would happen if router C lost this Hello  packet altogether. Note that the update with Seq=123 and Seq=124 still remains on  the transmission list for router C. For router B, the transmission list  contains only the update with Seq=124.
5. Router A will now send  update with Seq=124 as a multicast with CR bit set. Router B is in  CR-mode and will process this packet along with sending an  acknowedgement, upon which router A will remove this update from router  B's transmission list. Router B will also exit the CR-mode at this point. Router C will ignore this packet because it is  not in CR-mode, and packets flagged with CR bit may be received only by  routers in CR-mode.
6. Router A will try to send each message from  the transmission list for the router C as unicast to the router C,  starting with update Seq=123. Assuming that the router C receives and  acknowledges it, router A will remove the update Seq=123 from the  transmission list for C and send another update with Seq=124 also as  unicast. After router C acknowledges this packet as well, the  transmission list for routers B and C will be empty and so the router A  has now made sure that the both routers are fully synchronized.

There is a gentleman here on CSC named Donald Slice who is intimately familiar with EIGRP internals - Don, if you are reading this, can you please provide a definitive confirmation or correction of what has been described here? Thank you!

Best regards,

Peter

[martin_lx1980]

Hi Peter

   According to your last two replies I make a further experiment with network topology below

(192.168.10.1)Loopback0—R1—F0/0(192.168.12.1)---SW—(192.168.12.2)F0/0—R2

                                            |

                                            |

                                            |

                                         (192.168.12.3)

                                            F0/0—R3

I use both debug information and a network protocol analyzer to check the process.

I set three routers’ hold interval as 100s. After three routers complete initial sync, I shut down F0/0 of R3. Bacause R3 connects R1 with a switch, R1 can not know neighbor R3 is down until hold timer is expired.

In that due I change delay value of loopback0 of R1 to trigger a update packet. R2 receives the update and send ACK to R1. R1 can not receive ACK from R3 and send unicast update continuously. I could not find hello packet originating from R1 that contains sequence TLV and next multicast sequence TLV.

I change relay value of loopback0 of R1 again. Before sending multicast update packet, R1 first send a hello packet including sequence TLV listing R3 and next multicast sequence TLV. R2 receives this hello packet and does not find itself in sequence TLV. So R2 enters CR-Mode.

*Mar 1 00:15:32.263: EIGRP: Received Sequence TLV from 192.168.12.1

*Mar 1 00:15:32.267:       192.168.12.3

*Mar 1 00:15:32.267:       setting CR-mode

*Mar 1 00:15:32.267: EIGRP: Received CR sequence TLV from 192.168.12.1, sequence 9

You can see that when R1 does not receive ACK from R3 for the first time R1 does not send hello message that contains sequence TLV and next multicast TLV. When next update should be sent for R1, R1 would send hello message that contains sequence TLV and next multicast TLV before sends multicast update to the segment.

So the process you describe in your example is in keep with my process. But I could not certify the process about when CR-Mode was cancelled(the step 5 of your example) because I could only find key words, “setting CR-Mode” or “clearing CR-Mode”, in the output of debug eigrp packets command when router receives the hello message that contains sequence TLV. Could I see more output about CR-Mode from other debug command?

I could find information about multicast flow timer from output of debug eigrp packets like, FastEthernet0/0 multicast flow blocking cleared when router receives ACK. But I want to see more, such as multicast flow timer is starting or multicast flow timer is expired. I think those information may help learner understand eigrp process deeply.

Best regards

Martin

[Peter Paluch]

Hi Martin,

What you described here is actually a very nice demonstration that the CR-mode kicks into action if our EIGRP router is waiting for an acknowledgement of a multicasted message, and at the same time, it already has a new multicast message ready to be sent. The sequenced Hello always contains the sequence number of the next multicast segment to be conditionally received by the "good" routers. If you do not have any further multicast message to send, there is no need to put the other "good" routers into CR-mode - note that the CR-mode is to prevent the laggard routers from mistakenly processing an out-of-sequence multicast message. However, if you do not intend to send another multicast message, you don't need to invoke the CR-mode.

But I could not certify the process about when CR-Mode was cancelled(the  step 5 of your example) because I could only find key words, “setting  CR-Mode” or “clearing CR-Mode”, in the output of debug eigrp packets  command when router receives the hello message that contains sequence  TLV. Could I see more output about CR-Mode from other debug command? 

I wish I knew, Martin. I am not sure. Trying to run every EIGRP debug command is tedious but it may help... but I can not guarantee that.

Best regards,

Peter

[dslice]

Peter,

Beautifully explained.  And you're right that we preceed each multicast packet that is to be ignored by the "laggard" group with a sequence TLV identifying the sequence number to ignore.  We clear CR mode on a receiver when he receives an update without the CR bit set (which should only happen after he has been caught up with any unicasted packets to make him catch up with the other routers on the segment.)

One other point that you nicely pointed out (but gave us a bit more credit than we're due).  At startup, we use CR-mode to bring up a new peer but many of us in development believe that this could be drastically improved in the single-peer environment.  Due to how we do the re-use of the peer and interface queues, we "pretend" a peer is laggard when he's the only other one on the subnet.  Seems a bit stupid to me.