Troubleshooting T1

[amcsystemstech]

I have a point to point T1 that is acting up. Both sites are using Cisco 1721 wih T1 CSU/DSU wic cards. This T1 link has been running fine for about two years. Two weeks ago, the remote site started complaining of speed issues. Upon investigation, I noticed that their router's serial interface is receiving a high number of input, CRC, frame and abort errors.

Here is what has been done so far:

1) I temporarily moved the router next to the incoming T1 and connected it with a new patch cable. The serial interface still reported errors.

2) I replaced the router with a known working Cisco 1721. The serial interface on the replacement router showed the same errors.

3) I contacted the phone company to test the T1. They did and claim it is working fine.

4) I contacted Cisco. They had me perform loopback extended ping tests on the router. While in loopback, it received no errors and all ping tests succeeded.

5) With both the phone company and Cisco blaming the cabling that connects the CSU/DSU wic card to the incoming T1. I had a local cabling company come out and test the cable. They say the cable is good.

I hope the above makes sense. Anyway, Could it be possible that the host site (which does not receive any errors) could be the problem? I've done all the above at the host except have the cabling checked out. Any suggestions on what elese I should look at?

Thanks,

Output showing the errors from the remote sites router:

bond#clear counters

Clear "show interface" counters on all interfaces [confirm]

bond#sh interfaces serial0

Serial0 is up, line protocol is up

Hardware is PQUICC with Fractional T1 CSU/DSU

Internet address is x.x.x.x

MTU 1500 bytes, BW 1536 Kbit, DLY 20000 usec,

reliability 155/255, txload 1/255, rxload 1/255

Encapsulation HDLC, loopback not set

Keepalive set (10 sec)

Last input 00:00:00, output 00:00:00, output hang never

Last clearing of "show interface" counters 00:00:05

Input queue: 1/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue :0/40 (size/max)

5 minute input rate 2000 bits/sec, 4 packets/sec

5 minute output rate 1000 bits/sec, 4 packets/sec

45 packets input, 2003 bytes, 0 no buffer

Received 0 broadcasts, 0 runts, 0 giants, 0 throttles

857 input errors, 384 CRC, 398 frame, 0 overrun, 0 ignored, 75 abort

24 packets output, 1622 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 output buffer failures, 0 output buffers swapped out

0 carrier transitions

DCD=up DSR=up DTR=up RTS=up CTS=up

bond#sh interfaces serial0

Serial0 is up, line protocol is up

Hardware is PQUICC with Fractional T1 CSU/DSU

Internet address is x.x.x.x

MTU 1500 bytes, BW 1536 Kbit, DLY 20000 usec,

reliability 155/255, txload 1/255, rxload 1/255

Encapsulation HDLC, loopback not set

Keepalive set (10 sec)

Last input 00:00:00, output 00:00:00, output hang never

Last clearing of "show interface" counters 00:00:06

Input queue: 1/75/0/0 (size/max/drops/flushes); Total output drops: 0

Queueing strategy: fifo

Output queue :0/40 (size/max)

5 minute input rate 2000 bits/sec, 4 packets/sec

5 minute output rate 1000 bits/sec, 4 packets/sec

52 packets input, 2316 bytes, 0 no buffer

Received 0 broadcasts, 0 runts, 0 giants, 0 throttles

1160 input errors, 519 CRC, 522 frame, 0 overrun, 0 ignored, 119 abort

29 packets output, 2942 bytes, 0 underruns

0 output errors, 0 collisions, 0 interface resets

0 output buffer failures, 0 output buffers swapped out

0 carrier transitions

DCD=up DSR=up DTR=up RTS=up CTS=up

[Roberto Salazar]

Do a cable loopback but the software loopback that TAC had ask you to do is enough to really confirm that there is no hardware issue in the Cisco device. The CRC are packet being recieved with CRC errors, although they are at most time due to physical layer issue such as cable, connecting device's interface. If you suspect that the CRC is coming from the host, you can also do a cable loopback test on that site, the theory is if that router is sending packets with CRC errors, it should get it back when looped. If you do not see it, then it all points to the SP's cloud or devices along it's cloud.

Please rate helpful posts.

[amcsystemstech]

They actually had me do loopback using a cable. I just re-tested the host site and it works fine with the cable loopback. I'm going to re-test the remote site one more time.

The tech from the phone company hooked up his test equipment to the T1. He said that the could see the errors, but that they were not coming from the T1. He said the errors were most likely from the cabling or from the device.

Anyway, any suggestions on what type of testing I should request from the phone company?

Thank you,

[sean]

bosalaza is correct with the problem being in the upstream provider network. On the serial interface with the T1, the errors you are seeing are input, meaning coming to the router from the provider. If they were output, then I would suspect the router itself or something on the inside of your network. Unfortunately, when contacting the provider to have them test, the individual you get is normally a level 1 tech that can just open a ticket. The testing is done from an automated tester. All it does is run patterns of 1s and 0s for specified intevals on the line to see if there are errors. This has a tendancy to clear up 90% of the issues, which is slipping of the syncrhonization on the line. After the test is done, they didnt see any errors because the link is now synched again. You could have an issue with the provider smartjack, especially if they had to extend it past the normal demarc. I would keep hammering the provider, escalate the issue, and get somebody from the local lec out there to test the line and the smartjack. Hope this helps.

[amcsystemstech]

Thank you both for taking the time to respond. I've contacted the phone company for more testing. Hopefully the cause of the problem will be found this time.

Thanks,