The following article could not be properly formatted as an HTML page in this forum due to the amount of CLI output and spacing within the output. Therefore, most of the article can be found as an attachment.
This document contains a wealth of information on troubleshooting any kind of issue related to line cards, ports and Vlans, connectivity, dropped traffic, NPU and datalink counters, basic static routes, ARP, and subscriber monitoring. It could be useful for any kind of issue where the chassis is possibly not processing, transmitting, or receiving traffic as expected, even if you do not know whether the STxx is at fault or not.
Table of Contents
Overview.. 1
Basic Hardware Configuration and Troubleshooting. 2
Basic Port Configuration. 2
Viewing port state/status and configuration. 4
show card table all 4
PSC/PAC and Linecard relationships. 5
show port table all 6
show port info. 7
show card info. 9
show card hardware. 9
show card diag. 10
Further Linecard Configuration and Troubleshooting. 10
Mapping a port to a context and assigning IP addresses. 11
show ip interface. 12
Static route configuration. 12
show ip route. 13
Address Resolution (ARP) tables. 13
show ip arp. 13
Connectivity Test commands. 14
ping. 14
traceroute. 15
ppp echo-test 15
Other application commands. 15
System Counters and Data-rate/Throughput 16
Datalink and NPU counters. 16
show port datalink counters. 16
show port npu counters. 16
Example usage of datalink and npu counters. 17
=> RX Bytes BAD, HW error, Unknown VLAN tag, Port non-operational 17
=> RX Bytes BAD, RS SHORT CRC, RX NORM CRC, RX FALS CRS, RX SYM ERR.. 18
=> RX OVF. 19
Datarate/Throughput Analysis. 20
show port utilization table. 20
Tracing data for specific subscribers or protocols. 21
Quick Recover Methods. 22
card lc switch to <standby linecard #>. 22
card reboot <linecard #>. 23
card pac migrate from <PAC/PSC card #> to <PAC/PSC card #>. 23
Overview
This article attempts to discuss troubleshooting issues that could be related to linecard functionality. It includes troubleshooting linecard hardware errors, port configuration, packet flow issues, dropped or missing packets, routes and arp, throughput, etc.
Here is a partial list of the types of symptoms that one may observe with regards to this topic. They could be intermittent or continuous.
- failed called attempts
- port link bouncing
- various protocols experiencing issues (BGP, OSPF, etc.) due to what looks like connectivity issues
- calls being torn down unexpectedly
- authentication and/or accounting failures due to timeout and/or retry
- user not able to browse internet or reach certain addresses that are known to be working
- user complaints of sudden loss of connectivity
- failed ftp, telnet, or other service or application attempts
- unable to ping addresses that should be pingable from a particular context/interface and/or interface unable to be pinged from non-chassis address
- Foreign Agent (FA) and Home Agent (HA) connectivity issues
The list of items to check when troubleshooting connectivity issues is extensive, and you may already be familiar with a number of them. But it is better to have a complete list that will eventually lead to finding a root cause than an incomplete one that is missing the one step that would lead to an answer. It is difficult to give the most logical list of steps to take in every situation, and it will be up to your discretion to order the potential steps based on the information you have gathered or have been told thus far, your experience from troubleshooting similar issues in the past, your intuition, advice from others, availability to data from non-Starent tools (i.e. packet captures) and other network equipment, etc. Most importantly, at least you'll have a list that should lead you down the path to ultimately finding the root cause.
Basic Hardware Configuration and Troubleshooting
Commands mentioned:
show config
show card table all
show linecard table
show port table all
show port info
show card info
show card hardware
show card diags
Imported from Starent Networks Knowledgebase Article # 10360
