Odd lan speed issue.. desperate for help

Unanswered Question

Here is the strangest thing I have ever seen.

Working network, with Cat4006 at the core. Bringing in vmware, so bought 3750G-48-TS switch. Setup trunking from 3750 Gi1/0/48 to port 5/22 on 4006.

Ping works. Telnet works (with some commands pausing on the 3750). Everything else, does not work.

Unable to open 3750 web gui, just hangs.

We moved our servers to the 3750, and network applications were horribly slow, except if the apps were being tested from plugged into the same switch. Again all icmp traffic worked.

Moving the servers back to their old HP switch, everything works back to normal.

Any help would be appreciated.

I am attaching the trunking portions of each config below.

The 4006 is the root switch in the spanning-tree right now.

4006

-----------

set vlan 2 5/22-24

clear trunk 5/22 1,3,5-1005,1025-4094

set trunk 5/22 on dot1q 2,4

3750

------------

interface GigabitEthernet1/0/48

switchport trunk encapsulation dot1q

switchport trunk native vlan 2

switchport trunk allowed vlan 2,4

switchport mode trunk

Now these switches are relatively far apart, running Gig over normal Cat5 (not Cat5e), not sure if cabling could be a problem or not.

I have this problem too.
0 votes
  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Rolf Fischer Fri, 02/23/2007 - 23:48

One year ago we had a problem kind of simular to yours.

We found out that applications which use large IP-packets didn't work well but had no problems with ICMP,Telnet etc.

The reason was that those applications sometimes set the DF-bit (don't fragment) while using the full MTU and because of a new GRE-tunnel we had some new IP-overhead.

You can check this easily with some options in your ping, e.g. "ping -f -l 1460 ".

The exact length (here 1460) depends on your ping-application, Wireshark could help you finding out the length for 1500 bytes in IP.

I'm not sure if it helps but it dosn't cost you much efford.

If that's not the problem and your spanning-tree is stable I'd recommend to sniff and inspect TCP/IP behavior. If things go as crazy as you tell you shoud see something unusual.

Regarding the cabling you mentioned a 'show interface Gi...' and 'show controller Gi...' should show increasing errors in case of bad cabling.

I assume you've allready checked Duplex settings on the relevant interfaces - that's always the first thing I check when having a slow LAN.

After spending a long time testing, it looks like it might partially be the 4006 is unable to properly negotiate with the 3750. Here is my findings:

3750 4006 Works?

100/Full 100/Full No - Shuts down port

100/Auto 100/Auto No - 3750 negotiates half duplex, 4006 does full, both switches complain

100/Half Auto Yes

Auto 100/Half Yes

1000/Full 1000/Full No - Ports shut down

1000/Auto 1000/Auto No - Ports shut down

1000/Half 1000/Half No - Gig ports only support full duplex on 1000Mbps speed setting

Auto 1000/Full No - Ports shut down

1000/Full Auto No - Traffic is slow

While both devices were in auto, or the traffic is reported as slow, telnet and icmp work just fine.

With it hard coded to 100/half, it works properly. Now I am puzzled... looks like a problem with the 4006 gig ports I guess.

glen.grant Sat, 02/24/2007 - 02:14

The 3750 runs auto mdix . I would shut this feature off on your connecting port on the 3750, "no mdix auto" , this being said you must then use a crossover cable to connect the 2 sides as auto mdix is shutoff . This is the reason any hardcoded setting will not work on the 3750 as auto mdix only works in "auto" . After turning off auto mdix on the 3750 set both sides to "auto" for speed and duplex . The "speed and duplex" must be auto on both sides and it should negotiate 1000/full. Do not hardcode any speeds or duplex and it should negotiate gig correctly.

3750

interface x/x

speed auto

duplex auto

4006

set port x/x speed auto

set port x/x duplex auto

mahmoodmkl Sat, 02/24/2007 - 02:21

HI

As peryour configs why do u need to define the port 5/22 in vlan 2 as u r going to have this port as a trunk port.just define the port as the trunk port and clear the unwanted vlans from it.

i would recommend you to use cat6 cable between the switches as cat5 doesnt scale much for giga speeds.

Thanks

Mahmood

hoogen_82 Sat, 02/24/2007 - 05:36

Hi Mahmood,

In catos if you need to define your native vlan as anything other than vlan 1 this is the way it is done.

Cheers

Hoogen

Ok, I disabled the auto mdix, the connection stayed up because it is already a crossover connection.

Set the 3750 and 4006 both ends to auto negotiation, and it went to 1000/full.

However now I am back to the horrible traffic, and the 3750's web ui is so slow it is unusable, and the applications on the servers are horribly slow as well.

As soon as I drop down to 100/half on the 4006, and set the 3750 to auto, the link comes up, and apps work over the link again.

I am thinking it has to be cabling. I have looked at the controller stats, and there is some Deferred frames, late collisions, undersize frames, collision fragments, etc. Not a lot of them, but currently there is nothing plugged into the 3750 since rebooting the device to upgrade the IOS.

glen.grant Sat, 02/24/2007 - 16:24

If you are running it half duplex you are going to see fcs errors , collision fragments etc .. I agree it sounds like wiring is not right . To run gig you must have 4 good pairs of wires as gig speeds use all 4 pairs to run . Sounds like you have at least one bad pair . When its slow does it say 1000/full on both sides ???? Do you see errors when it comes in at gig ??? Because it it will only run at half duplex , a single pair of wires , this indicates to me your wiring is not right .

Danilo Dy Sat, 02/24/2007 - 07:50

Your 3750 port is 10/100/1000. Is your 4006 port 10/1000/1000 or 10/100 or 100 only?. While waiting for answer about your 4006, you should hardcode both switch port to full-duplex/100BaseT considering you also using Cat5 not Cat6 or Cat5e.

The errors you saw can be caused by speed/duplex mismatch, cabling type, cabling category, cabling poor crimping. Therefore the recommendation above is more appropriate to your problem.

Both ends are 10/100/1000 interfaces.

I am unable to hard code both ends to 100/half or 100/full. But if I hard code the 4006 to 100/half, and leave the 3750 in auto, the link comes up. If I hard code the 3750 to 100/half to match the 4006, link does not come up at all.

Isn't this a strange one?

I have the guys onsite trying to figure out the cabling, and possibly locate a fluke cable tester.

Danilo Dy Sat, 02/24/2007 - 08:46

Its a cabling problem.

- It should be crossed

- It should be Cat 6 or Cat 5e if going for 1000Base/Full. Else settle for 100Base/Full

- It should be properly crimped, all copper cables reach the stopper of the RJ45 head. Try tapping the RJ45 head with a hard object (plier or crimper) against the copper cables before crimping it.

Don't trust those pre-fabricated cables, they are not 100% perfect. This is the reason I always bring my own AMP RJ45 heads, AMP Crimper, Cable Tester, 2 x 5 meters Cat 5e, and cable extender whenever I go to the site of the network problem

scottmac Sat, 02/24/2007 - 10:47

If your cable are hand-made (versus purchased / commercial cables), then the issue is classic:

You have a split pair on pins 3&6 ... meaning that the conductors that make (what should be) the single pair connecting pins 3&6 are actually belonging to two different pairs.

THis is usually because the cable was made with pin order instead of EIA/TIA 568 (a or b) pair order.

EIA/TIA 568a looks like this, clip down, end away from you:

1 - White-Green

2 - Green

3 - White - Orange

4 - Blue

5 - White - Blue

6 - Orange

7 - Brown - WHite

8 - Brown

(EIA/TIA 568b reverse the positions of the orange and green pair)

You'll note that using 568 pin-outs, the conductors on pins 3&6 form one pair (Orange pair)

Ethernet / Fast Ethernet uses pins 1&2 and 3&6. Gig Ethernet uses all four pair (but 3&6 must still be a single pair)

If you construct the cable

1 - pair color 1a

2 - pair color 1b

3 - pair color 2a

4 - pair color 2b

5 - pair color 3a

6 - pair color 3b

7 - pair color 4a

8 - pair color 4b

You'll note that pins 3&6 connect to two different pair (a "split pair," , 2a and 3b)

BEcuase the conductors are no longer "twisted" (correctly) all of your crosstalk rejection properties go out the window ... the added crosstalk causes interference between the two functional pair (1&2, 3&6) and the cable becomes "broken," or at least extremely inefficient for full-duplex operation (that's why half works OK, but full duplex does not)

With GigEthernet using all four pair, with every pair working in both directions at the same time .. the crosstalk spec becomes extremely critical.

Use commercial cables. Under Cat6 spec, there is no such thing as a hand-made Cat6 jumper ... a human is far less likely to make a cable to Cat6 specification.

It is also less expensinve, the time it takes to properly create and certify a cable to spec costs *TIME* as well as materials .. generally an engineer's time is much more valuable thatn using it to create cabling.

As a final toss-in, jumpers should be stranded cabling (a pain in the butt to terminate), "in-wall" infrastructure should be solid. Stranded cabling has much more loss than solid conductor and should not be used for long runs (max spec is 5 meters at each end of the span, with the middle part being 90 meters of solid conductor).

Hope this is helpful,

Good Luck

Scott

gabriel.gearip Sat, 02/24/2007 - 11:44

Hi,

I had a similar problem some time ago, but the problem wasn't cabling.

You should check what is the maximum packet size that passes without failure between the switches (with ping -l). If packets exceeding a value all fail reaching the other end you _do not_ have a cabling issue.

If that is the case the first thing you should check for trunking problems - set different trunking negotiation modes or no negotiation at all.

HTH.

Gabi.

Actions

This Discussion