cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
2608
Views
4
Helpful
3
Replies

3750-X + C3KX-NM-10G = oddness

darenv
Level 1
Level 1

Folks

We have a brand new 2 x 3750-X stack running 15.0(1)SE3 with C3KX-NM-10Gs installed in each switch.  An HP C-Class server enclosure with Virtual Connect Flex-10 modules is connected to each of the 10G ports using twinax cables and they seem to behave OK apart from some random packet drop which appears to be within the enclosure and not without.  However, when we were investigating this, we found a little oddity ... even though we have jumbo frame support enabled on the 3750-Xs, we are getting a significant "Valid frames, too large" recieved count on the interface, and a "Too large" transmitted count.  I've looked around and can't find a definitive reason for this count to go up - it doesn't seem to be a jumbo frames thing, more an unexpected-additional-field-in-a-header type thing, like when you have a VLAN tag in a DTP frame.  Any thoughts as to why I might be getting these from a new server enclosure?

SW-STACK-CORE-3750X#sho int t2/1/1 controller              

TenGigabitEthernet2/1/1 is up, line protocol is up (connected)

  Hardware is Ten Gigabit Ethernet, address is a44c.116e.55b5 (bia a44c.116e.55b5)

  Description: *** Server Enclosure B2P3 ***

  MTU 9000 bytes, BW 10000000 Kbit/sec, DLY 10 usec,

     reliability 255/255, txload 1/255, rxload 1/255

  Encapsulation ARPA, loopback not set

  Keepalive not set

  Full-duplex, 10Gb/s, link type is auto, media type is SFP-10GBase-CX1

  input flow-control is off, output flow-control is unsupported

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:05, output 00:00:00, output hang never

  Last clearing of "show interface" counters 2d15h

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 3453000 bits/sec, 1142 packets/sec

  5 minute output rate 6076000 bits/sec, 1064 packets/sec

     52092348 packets input, 20546616470 bytes, 0 no buffer

     Received 15193 broadcasts (15193 multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 15193 multicast, 0 pause input

     0 input packets with dribble condition detected

     45009519 packets output, 23719388911 bytes, 0 underruns

     0 output errors, 0 collisions, 0 interface resets

     7596 unknown protocol drops

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 pause output

     0 output buffer failures, 0 output buffers swapped out

     Transmit TenGigabitEthernet2/1/1         Receive

  23722640051 Bytes                      20546716139 Bytes                   

     43629235 Unicast frames                52077194 Unicast frames          

       393269 Multicast frames                 15935 Multicast frames        

      1009161 Broadcast frames                     0 Broadcast frames        

            0 Too old frames             20336358788 Unicast bytes           

            0 Deferred frames                2053800 Multicast bytes         

            0 MTU exceeded frames                  0 Broadcast bytes         

            0 1 collision frames                   0 Alignment errors        

            0 2 collision frames                   0 FCS errors              

            0 3 collision frames                   0 Oversize frames         

            0 4 collision frames                   0 Undersize frames        

            0 5 collision frames                   0 Collision fragments     

            0 6 collision frames      

            0 7 collision frames                1306 Minimum size frames     

            0 8 collision frames            25046107 65 to 127 byte frames   

            0 9 collision frames            10356470 128 to 255 byte frames  

            0 10 collision frames            5009947 256 to 511 byte frames  

            0 11 collision frames            2082849 512 to 1023 byte frames 

            0 12 collision frames            5590720 1024 to 1518 byte frames

            0 13 collision frames                  0 Overrun frames          

            0 14 collision frames                  0 Pause frames            

            0 15 collision frames     

            0 Excessive collisions                 0 Symbol error frames     

            0 Late collisions                      0 Invalid frames, too large

            0 VLAN discard frames            4005730 Valid frames, too large 

            0 Excess defer frames                  0 Invalid frames, too small

        17812 64 byte frames                       0 Valid frames, too small 

     16287332 127 byte frames         

     14170891 255 byte frames                      0 Too old frames          

      1172987 511 byte frames                      0 Valid oversize frames   

       611101 1023 byte frames                     0 System FCS error frames 

      2147552 1518 byte frames                     0 RxPortFifoFull drop frame

     10623990 Too large frames        

            0 Good (1 coll) frames    

            0 Good (>1 coll) frames   

3 Replies 3

Leo Laohoo
Hall of Fame
Hall of Fame

There's a scarcity in documentation which would explain, line-by-line, as to the meaning of the output above.

I, for one, am no expert but I've seen "valid frames, too large" even with ordinary PC and I just tend to ignore it.   As far as I can see, the output to your "sh controller" is pretty much a two-thumbs-up for me.  No issues there.  No alignment, no FCS, no Symbols errors on the receive side.  On the transmit side, I don't see any collisions frames.  Again.  Looks good.

I, however, am a bit curious as to the amount of "unknown protocol drops" in the "sh interface" output.  I've never seen that before.

I am also curious about the SFP "media type" of "SFP-10GBase-CX1".  I've never seen an output like this before.

Thanks for that, leolaohoo - I thought much the same but with the other issues around the Virtual Connect side of things, I need to be sure

The "unknown protocol drops" may be LLDP, which I know is turned on down at the HP end of things, but not on the 3750-X.  Then again, knowing HP it could be any number of other odd things ...!

The media type is how the 3750-X represents the SFP-H10GB-CU5M twinax cables when connected to the C3KX-NM-10G - I've only used them before in Nexus 2Ks and 5Ks so not seen this before myself!

AnthonyKett
Level 1
Level 1

Hi there DarrenV.

I have the same 3750x x 2 Stack, same C7000 HP enclosure (with Flex 10/D, 10gbit card) and am seeing the exact same increment of "Valid Frames, too large". Same IOS version as well.

slight difference in the transceivers i'm using (we have the basic SR fibre 10gbit transceivers) .

I cannot see any drops on the interface statistics but the "show controller ethe ten 1/1/1" is showing the "valid frames, too large" increment all the time at quite a high rate

we've recently had some of our linux vm guests on the chassis start to get packetloss and although i've given the switch a clean bill of health, that statistic is rather annoying me. I'm sure the packetloss is at the HP/Vmware level but without doing a packet capture on the 10gbit interface not sure what else i can do to look at those (i.e. whether they are jumbo frames prancing through the network even though MTU is set at 1500 on the interfaces or they are other protocols like LLDP etc).

i know this thread is a little old now, but i thought i'd also join it and see if you happen to have had any problems from what you stated above.

thanks all.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Innovations in Cisco Full Stack Observability - A new webinar from Cisco