UK BT ADSL2+ Annex A - Cisco 877 - Page 2

Ian Cowley · ‎10-10-2009

UK is currently going through a nationwide migration from ADSL, ADSL2 and ADSL2max to ADSL2+.

Most British Telecom (BT) MSANs are configured for Annex A, though we know that some LLU suppliers are using Annex M (with UK mask) which Cisco currently doesn't support.

Our issues are that all out Cisco 877s and there are 100s deployed that are subject to a line upgrade, or a new installs on ADSL2+ fail.

We have used the latest IOS that will runn on the older 128M DRAM units 12.4(15)T11 I think, and 12.4(24)T1 on newer units. We have also played with DSL Modem firmware 3.0.33 and more recently 4.0.015 (which incidently works very well on UK ADSL2 Annex A). None have helped.

The connectivity in UK is PPPoA

Symptoms:

It appears to be dependent on the amount of data through the DSL interface.

So a 24Mb/s connection will fail in hours, whereas a 9Mb/s will fail in days.

'sh int atm0' will show drops

'sh buffers' will show the middle and big buffers failing in particular.

Small packets such as ICMP and SSH will continue to work, so we can log on to the router(s) remotely.

But large packets such as those used for normal data will not pass the router reliably if at all.

Replace the router with a $100 item and the connection speed is normally better and there are no issues.

All these routers are similarly configured, have CBAC inspect code and a single IPSEC VPN.

MTU and MSS sizes are reduced to support IPSEC.

(though I hear the same problem from other UK deployments)

I have placed some info on the NetPro forum, seach for ADSL2+ and you should find it.

I have calls out to BT, large UK distributors and ISPs without any resolution or identification of the problem.

I think this is a significant issue given the value of Cisco DSL real estate in UK.

HELP!!!!

Ian Cowley

Christopher Adlam · ‎03-15-2010

I'm running 12.4.24(T2) on my 1841, didn't find any improvement with 4.0.018 or 3.0.33, so I'm running 4.0.015 right now.

Interesting note:

"show dsl int" gives me:

ATM0/0/0
Alcatel 20190 chipset information
                ATU-R (DS)                      ATU-C (US)
Modem Status:    Showtime (DMTDSL_SHOWTIME)
DSL Mode:        ITU G.992.5 (ADSL2+) Annex A
ITU STD NUM:     0x03                            0x2
Chip Vendor ID:  'STMI'                          'BDCM'
Chip Vendor Specific:  0x0000                    0x93B2
Chip Vendor Country:   0x0F                      0xB5
Modem Vendor ID: 'CSCO'                          'BDCM'
Modem Vendor Specific: 0x0000                    0x0000
Modem Vendor Country:  0xB5                      0xB5
Serial Number Near:    FOC12052QJQCISCO73993206
Serial Number Far:  Chip ID:     C196 (0)
DFE BOM:         DFE3.0 Annex A (1)
Capacity Used:   100%                            86%
Noise Margin:     9.0 dB                         13.5 dB
Output Power:    18.0 dBm                        12.5 dBm
Attenuation:     23.5 dB                         11.0 dB
FEC ES Errors:    1                              12368
ES Errors:        1                              28168
SES Errors:       1                              22366
LOSES Errors:     1                              22294
UES Errors:       0                              22324
Defect Status:   None                            None
Last Fail Code:  None
Watchdog Counter: 0x75
Watchdog Resets: 0
Selftest Result: 0x00
Subfunction:     0x00
Interrupts:      12445 (0 spurious)
PHY Access Err:  0
Activations:     2
LED Status:      OFF
LED On Time:     0
LED Off Time:    0
Init FW:         init_AMR-4.0.015.bin
Operation FW:    AMR-4.0.015.bin
FW Source:       external
FW Version:      4.0.15

                 DS Channel1      DS Channel0   US Channel1       US Channel0
Speed (kbps):             0            19130             0              1020
Cells:                    0          7672982             0         149488887
Reed-Solomon EC:          0            24490             0             39629
CRC Errors:               0            40643             0              6164
Header Errors:            0            61346             0              7276
Total BER:                0E-0           5152E-9
Leakage Average BER:      0E-0           4008E-9
Interleave Delay:         0               29             0                 0
                        ATU-R (DS)      ATU-C (US)
Bitswap:               enabled            enabled
Bitswap success:          0                   0
Bitswap failure:          0                   0

Whilst show int atm0/0/0 yields:

#show int atm0/0/0
ATM0/0/0 is up, line protocol is up
  Hardware is HWIC-DSLSAR (with Alcatel ADSL Module)
  MTU 4470 bytes, sub MTU 4470, BW 1020 Kbit/sec, DLY 500 usec,
     reliability 255/255, txload 88/255, rxload 8/255
  Encapsulation ATM, loopback not set
  Encapsulation(s): AAL5  AAL2, PVC mode
  23 maximum active VCs, 256 VCs per VP, 1 current VCCs
  VC Auto Creation Disabled.
  VC idle disconnect time: 300 seconds
  Last input 18:53:06, output 00:00:00, output hang never
  Last clearing of "show interface" counters 13:45:04
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 1113
  Queueing strategy: Per VC Queueing
  5 minute input rate 35000 bits/sec, 42 packets/sec
  5 minute output rate 353000 bits/sec, 70 packets/sec
     1651870 packets input, 264031407 bytes, 0 no buffer
     Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 356 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     2189990 packets output, 1968256989 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out

And show atm pvc 0/35 yields:

#show atm pvc 0/35
Description: N/A
ATM0/0/0.1: VCD: 1, VPI: 0, VCI: 35
UBR, PeakRate: 1020 (2406 cps)
AAL5-LLC/SNAP, etype:0x0, Flags: 0xC20, VCmode: 0x0, Encapsize: 12
OAM frequency: 0 second(s), OAM retry frequency: 1 second(s)
OAM up retry count: 3, OAM down retry count: 5
OAM END CC Activate retry count: 3, OAM END CC Deactivate retry count: 3
OAM END CC retry frequency: 30 second(s),
OAM SEGMENT CC Activate retry count: 3, OAM SEGMENT CC Deactivate retry count: 3
OAM SEGMENT CC retry frequency: 30 second(s),
OAM Loopback status: OAM Disabled
OAM VC Status: Not Managed
ILMI VC status: Not Managed
InARP frequency: 15 minutes(s)
InPkts: 1659167, OutPkts: 2199852, InBytes: 266316494, OutBytes: 1975095601
InPRoc: 10588, OutPRoc: 37105, Broadcasts: 0
InFast: 1648579, OutFast: 2147154, InAS: 0, OutAS: 0
InPktDrops: 0, OutPktDrops: 0
CrcErrors: 0, SarTimeOuts: 0, OverSizedSDUs: 0, LengthViolation: 0, CPIErrors: 0
Out CLP=1 Pkts: 0
OAM cells received: 0
F5 InEndloop: 0, F5 InSegloop: 0,
F5 InEndcc: 0, F5 InSegcc: 0, F5 InAIS: 0, F5 InRDI: 0
F4 InEndloop: 0, F4 InSegloop: 0, F4 InAIS: 0, F4 InRDI: 0
OAM cells sent: 0
F5 OutEndloop: 0, F5 OutSegloop: 0,
F5 OutEndcc: 0, F5 OutSegcc: 0, F5 OutAIS: 0, F5 OutRDI: 0
F4 OutEndloop: 0, F4 OutSegloop: 0, F4 OutRDI: 0
OAM cell drops: 0
Status: UP
PPPOE enabled. Current number of pppoe sessions: 1

Notice the MARKED difference in the # of CRC errors shown between the initial command, and the subsequent ones for each interface.....????

Ian Cowley · ‎03-15-2010

Brian

You're correct in that I can't generally go beyond 12.4(15)T10.

Upgrading the Flash and DRAM is just too expensive and it may be simpler/cheap to replace boxes with latest

Ianc

Ian Cowley · ‎03-15-2010

Also I've noted that all my issues todate have been on IFTN = Infineon (Huawei) Multi-Service Access Network equipment.

I notice Brian's is on TSTC = Texas Instruments

and Chris BDCM = Broadcom

IanC

Ian Cowley · ‎03-22-2010

I've heard that Cisco may be producing a new (v3?) 877.

Does anyonelse know more?

Christopher Adlam · ‎03-22-2010

Ian:

I would like to note that since I posted the above, and my switch to the 4.0.015 firmware, I have not had a single disconnect. So it appears to have fixed my issue.

James Perks · ‎04-20-2010

Hi Ian

I have been having been having pretty poor performance and stability issues with an 877 router on adsl2+. I am also connected to an Infineon msan.

I have a TAC call open regarding these issues which has resulted in the following bug

CSCtg20300 Bug Details
Performance and Stability issues with adsl2+ CISCO877-K9

I presume this bug is publically available to view and hopefully will result in a new firmware release.

I am on my third beta firmware which is looking promising so far.

James

Christopher Adlam · ‎04-21-2010

Good to hear that things are being worked on James. My issue was definitely resolved with 4.0.015.

-Chris

Ian Cowley · ‎04-21-2010

Chris

It's worth noting you are on Broadcom MSAN equipment, whereas myself, some of our clients and James are connecting to Infineon kit. For whatever reason this is where the incompatbility lies.

Christopher Adlam · ‎04-21-2010

Indeed. Which is what I thought 4.0.018 was supposed to resolve? But apparently isn't for you guys. Given that James has tried a few beta firmwares though, it does sound as though Cisco is actively working on the problem, which lends promise to the idea that it might actually be fixed sooner rather than later

Ian Cowley · ‎04-21-2010

likewise I though 4.0.018 would help and it is stable for me, though my line runs at 56dB attentuation so I only get 4Mb. Anyone on low attenuation lines and able to get in excess of 10Mb has problems.

Thankfully James has the ability to raise a TAC case.

Fingers X'd. If D-Link, Netgear and 2Wire can do it.....................

Christopher Adlam · ‎04-21-2010

Yes indeed...... I imagine Efficient and a few others have it working fine as well. It wasn't until 4.0.015 that my $500.00 frickin' HWIC was able to match the line speed and reliability of the $30.00 D-Link DSL-2320B it replaced. So I feel your pain.

James Perks · ‎07-25-2010

The TAC call CSCtg20300 has now been closed.

After much debugging via the dsl training log outputs and about 7 firmware revisions later I am now using ver 4.0.195.
This has stabilised the noise margin, reduced the large number of errors, and fixed the state where the link would not train up after a flap.

I am occasionally still seeing the atm0 int going down/up but this is generally from 23:00 through till 04:00 in the morning. I suspect it is caused by BT making changes in the exchange, though I have no proof of this.

The TAC guys also could not see any reason from the routers point of view why this was happening. i.e. After the new firmware there was no evidence that this was the routers fault

The firmware has been tweaked to work well with the MSAN I am connected to which I am told (via Zen ISP) is a Huwawei HWE/HM/CMSAN/2625. These tweaks are cummulative so the next public firmware release should have these changes.

I have just checked the Cisco site and the latest firmware for the 877 is 4.0.17. The TAC said they generally release new firmware either after a major problem or after they accumulate a number of smaller issues. So I guess if you want to try the new firmware you will need to contact Cisco TAC and request it. Whether that requires a support contract I am not sure.

The speed is also now as good as other cheaper routers that did not have these issues (Broadcom based) such as a Thomson 585 v7 and a 3com officeconnect adsl wireless 11g (3CRWDR200A-75)

Many thanks to Cisco Support

sh dsl int atm0
ATM0
Alcatel 20190 chipset information
                ATU-R (DS)                      ATU-C (US)
Modem Status:    Showtime (DMTDSL_SHOWTIME)
DSL Mode:        ITU G.992.5 (ADSL2+) Annex A
ITU STD NUM:     0x03                            0x2
Chip Vendor ID: 'STMI'                          'IFTN'
Chip Vendor Specific: 0x0000                    0x71B9
Chip Vendor Country:   0x0F                      0xB5
Modem Vendor ID: 'CSCO'                          '    '
Modem Vendor Specific: 0x0000                    0x0000
Modem Vendor Country: 0xB5                      0x00
Serial Number Near:    FCZ111820BS
Serial Number Far:
Modem VerChip ID:        C196 (0)
DFE BOM:         DFE3.0 Annex A (1)
Capacity Used:   98%                             99%
Noise Margin:    10.5 dB                          6.5 dB
Output Power:    20.0 dBm                        13.0 dBm
Attenuation:     25.5 dB                         11.0 dB
FEC ES Errors:   1512                             0
ES Errors:       62                               0
SES Errors:      61                               0
LOSES Errors:    61                               0
UES Errors:      58                              13
Defect Status:   None                            None
Last Fail Code: None
Watchdog Counter: 0x0B
Watchdog Resets: 0
Selftest Result: 0x00
Subfunction:     0x00
Interrupts:      42184 (0 spurious)
PHY Access Err: 0
Activations:     72
LED Status:      ON
LED On Time:     100
LED Off Time:    100
Init FW:         init_AMR_4.0.195.bin
Operation FW:    AMR-E-4.0.195.bin
FW Source:       external
FW Version:      4.0.195

                 DS Channel1      DS Channel0   US Channel1       US Channel0
Speed (kbps):             0            12952             0              1266
Cells:                    0         86854626             0         369594520
Reed-Solomon EC:          0                0             0                 0
CRC Errors:               0                4             0                 0
Header Errors:            0                4             0                 0
Total BER:                0E-0           1114E-12
Leakage Average BER:      0E-0           1384E-12
Interleave Delay:         0               42             0                18
                        ATU-R (DS)      ATU-C (US)
Bitswap:               enabled            enabled
Bitswap success:          0                   0
Bitswap failure:          0                   0

LOM Monitoring : Disabled

DMT Bits Per Bin
000: 0 0 0 0 0 0 4 7 9 A C C D D E F
010: F F F F F F F F F F F F F F E E
020: 0 2 3 3 5 7 5 7 8 9 9 9 A B B B
030: C C C D D D D D D D E E E E E E
040: E E 0 0 0 0 0 E E E E E E E E E
050: E D D D D D D D D 2 D D D D D D
060: D D D D D D D D D D C D D D 0 D
070: D D D C C D D C C C C C C 2 2 0
080: 2 2 C C C C C C C C C C C C C C
090: C C C C B C C C C C C C C C B C
0A0: B B C B B B C C B B B B B B B B
0B0: B B B B B B B B B B B B 0 B B 0
0C0: 5 A B B B B B B B B B B B B B B
0D0: B A A 7 A A B B B B B B B B B B
0E0: A B B B B B B A B A A A A A A A
0F0: A A 9 0 2 7 9 A A A A A A 9 A B
100: A A A A A A A 9 A A A A A A A A
110: A 9 9 9 8 5 4 5 8 9 9 A A A A A
120: A A 9 0 A A 9 A A A A A A A A A
130: A A A A A A A 8 A A 9 4 9 9 9 8
140: 9 9 A 9 9 9 9 9 9 9 7 9 9 9 5 9
150: 9 8 6 7 2 6 8 8 8 8 7 8 0 9 9 9
160: 8 8 9 8 9 9 8 5 8 8 8 8 8 8 8 8
170: 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8
180: 8 8 8 8 8 8 8 8 8 8 8 8 8 7 7 7
190: 7 7 7 7 7 7 7 7 7 7 7 7 7 7 0 0
1A0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1B0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1C0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1D0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1E0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1F0: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

DSL: Training log buffer capability is not enabled

Ian Cowley · ‎07-25-2010

Well done James and Cisco Support.

This has taken a very long time to resolve (posted Oct 2009)

Lets hope Cisco reallise its in their interest to post this file for public access!

IanC

Christopher Adlam · ‎07-26-2010

Wonderful to hear this is finally fixed!

gturner · ‎07-29-2010

All, thanks for the further updates on this.

From our view we are experiencing greater stability with .018 firmware on a number of 857's and 877's connected across the county.

We are to start evaluation of an Annex-M service using a Cisco 1841 with HWIC-1ADSL-M interface.

The initial observation is one of good (but variable) sync speed from 2300 to 2447 on the upstream. Less interesting is the 17Mbps downstream sync.

More relevant it would seem is the number of CRC errors on the upstream (virtually NIL) as compared to the disproportionately high number on the downstream. This is indicative of a connection that will prove to be unstable.

We have learned that the Dslam profile can be tuned siginficantly by a suitably supportive service provider.

One qu. is whether (and mileage may again vary between service provider) we can have the upstream characteristics unchanged, but perhaps have the downstream tuned such that CRC error count is reduced even at the expense of downstream bandwidth.

Thanks. Graham