Multilink issues with one T1

Unanswered Question
Mar 11th, 2010
User Badges:

We have a real doozy of a problem here. 2 T1's one side on a 2800 router, the other end on different DS3's on the same 7200 router. We have done this with other sites with no problems, but this one has us stumped. One T1 is missing the PPP keepalives, and causes the multilink to flap and not work. The T1 is testing 100% clean, both directions CSU to CSU. No errors at all, but every minute, it misses 5 keepalives and drops protocol, renegotiates, and comes back up. Right now I have it as a single PtP T1 with IP's at both ends, and I can slam it with max bytes forever, and only when the LCP drops do I lose packets, but again, no errors. When brought into the multilink though, massive errors, both on the T1 and the multilink.


Anyone give me an idea what to look for next? The only thing I can think of is a misprovisioned circuit, but there are no errors.

Here is the debug output

/* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;}

*Mar 11 16:04:40.440: Se0/3/0 PPP: Missed 5 keepalives, taking LCP down

*Mar 11 16:04:40.440: Se0/3/0 PPP: Sending Acct Event[Down] id[5B7F]

*Mar 11 16:04:40.440: Se0/3/0 LCP: State is Closed

*Mar 11 16:04:40.440: Se0/3/0 PPP: Phase is DOWN

*Mar 11 16:04:40.440: Se0/3/0 CDPCP: State is Closed

*Mar 11 16:04:40.440: Se0/3/0 IPCP: State is Closed

*Mar 11 16:04:40.444: Se0/3/0 PPP: Phase is ESTABLISHING, Passive Open

*Mar 11 16:04:40.444: Se0/3/0 LCP: State is Listen

*Mar 11 16:04:40.444: Se0/3/0 IPCP: Remove route to 10.100.106.17

*Mar 11 16:04:41.440: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/3/0, changed state to down

*Mar 11 16:04:42.444: Se0/3/0 PPP: Outbound cdp packet dropped

*Mar 11 16:04:42.456: Se0/3/0 LCP: Timeout: State Listen

*Mar 11 16:04:42.456: Se0/3/0 LCP: O CONFREQ [Listen] id 113 len 10

*Mar 11 16:04:42.456: Se0/3/0 LCP:    MagicNumber 0x26ED9FC2 (0x050626ED9FC2)

*Mar 11 16:04:42.920: Se0/3/0 LCP: I CONFREQ [REQsent] id 1 len 10

*Mar 11 16:04:42.920: Se0/3/0 LCP:    MagicNumber 0x36E8A91B (0x050636E8A91B)

*Mar 11 16:04:42.920: Se0/3/0 LCP: O CONFACK [REQsent] id 1 len 10

*Mar 11 16:04:42.920: Se0/3/0 LCP:    MagicNumber 0x36E8A91B (0x050636E8A91B)

*Mar 11 16:04:42.920: Se0/3/0 LCP: I CONFACK [ACKsent] id 113 len 10

*Mar 11 16:04:42.920: Se0/3/0 LCP:    MagicNumber 0x26ED9FC2 (0x050626ED9FC2)

*Mar 11 16:04:42.920: Se0/3/0 LCP: State is Open

*Mar 11 16:04:42.920: Se0/3/0 PPP: Phase is FORWARDING, Attempting Forward

*Mar 11 16:04:42.920: Se0/3/0 PPP: Phase is ESTABLISHING, Finish LCP

*Mar 11 16:04:42.924: Se0/3/0 PPP: Phase is UP

*Mar 11 16:04:42.924: Se0/3/0 IPCP: O CONFREQ [Closed] id 1 len 10

*Mar 11 16:04:42.924: Se0/3/0 IPCP:    Address 10.100.106.18 (0x03060A646A12)

*Mar 11 16:04:42.924: Se0/3/0 CDPCP: O CONFREQ [Closed] id 1 len 4

*Mar 11 16:04:42.924: Se0/3/0 PPP: Process pending ncp packets

*Mar 11 16:04:43.372: Se0/3/0 IPCP: I CONFREQ [REQsent] id 1 len 10

*Mar 11 16:04:43.372: Se0/3/0 IPCP:    Address 10.100.106.17 (0x03060A646A11)

*Mar 11 16:04:43.376: Se0/3/0 IPCP: O CONFACK [REQsent] id 1 len 10

*Mar 11 16:04:43.376: Se0/3/0 IPCP:    Address 10.100.106.17 (0x03060A646A11)

*Mar 11 16:04:43.376: Se0/3/0 IPCP: I CONFACK [ACKsent] id 1 len 10

*Mar 11 16:04:43.376: Se0/3/0 IPCP:    Address 10.100.106.18 (0x03060A646A12)

*Mar 11 16:04:43.376: Se0/3/0 IPCP: State is Open

*Mar 11 16:04:43.376: Se0/3/0 CDPCP: I CONFREQ [REQsent] id 1 len 4

*Mar 11 16:04:43.376: Se0/3/0 CDPCP: O CONFACK [REQsent] id 1 len 4

*Mar 11 16:04:43.380: Se0/3/0 IPCP: Install route to 10.100.106.17

*Mar 11 16:04:43.648: Se0/3/0 PPP: Outbound cdp packet dropped

*Mar 11 16:04:43.928: %LINEPROTO-5-UPDOWN: Line protocol on Interface Serial0/3/0, changed state to up

*Mar 11 16:04:43.928: Se0/3/0 PPP: Outbound cdp packet dropped

*Mar 11 16:04:44.920: Se0/3/0 CDPCP: Timeout: State ACKsent

*Mar 11 16:04:44.920: Se0/3/0 CDPCP: O CONFREQ [ACKsent] id 2 len 4

*Mar 11 16:04:44.920: Se0/3/0 CDPCP: I CONFACK [ACKsent] id 2 len 4

*Mar 11 16:04:44.920: Se0/3/0 CDPCP: State is Open

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
paolo bevilacqua Thu, 03/11/2010 - 08:40
User Badges:
  • Super Gold, 25000 points or more
  • Hall of Fame,

    Founding Member

Understand that you may checked 1000 times, but symptoms are of "no keepalives" at one side.


5 missed keepalives = 50 seconds

tahequivoice Thu, 03/11/2010 - 08:45
User Badges:

That part I get, but what would be blocking the keepalives?  The interfaces are configred all default, only IP and encapsulation are configured on both ends.

paolo bevilacqua Thu, 03/11/2010 - 08:52
User Badges:
  • Super Gold, 25000 points or more
  • Hall of Fame,

    Founding Member

Take debug at the other side, we want to see the keepalive being sent out to begin with.

tahequivoice Thu, 03/11/2010 - 08:55
User Badges:

I will need to do that in a maintenance window, it is our core router with hundreds of othe connections.

paolo bevilacqua Thu, 03/11/2010 - 09:03
User Badges:
  • Super Gold, 25000 points or more
  • Hall of Fame,

    Founding Member

May be a long shot, but the core may be suffering some data structure corruption and not sending keepalive even if supposed to.

Try switching encap to hdlc and back.

Actions

This Discussion