cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
11879
Views
5
Helpful
9
Replies

C3KX-SM-10G Capabilities?

bakaneko59
Level 1
Level 1

We have a small network which is completely serviced by a single StackWise stack of Catalyst 3750G and 3750E switches.  There are no external connections.  We recently purchased a 3750X switch with the C3KX-SM-10G Service Module for expansion, and for it's ability to allow Netflow monitoring of our network.  However, I apparently misinterpreted the capabilities of the service module from the information I used to order it.  I was hoping to be able to monitor all of my port-to-port on-switch traffic with Netflow, but I now understand that only the SFP ports on the service module itself can be monitored.  We have nothing connected to those ports as we do not have a distributed switch network (once again, only a single stack).

Is there any kind of workaround for this that would allow me to use Netflow to monitor my on-switch traffic?  I was thinking or maybe just connecting one of the SFP ports directly to my monitoring server, but there doesn't appear to be a 1000BaseT copper SFP module available for it, and even then the port-to-port traffic wouldn't really be travelling on the SFP.  Did I waste my money buying the module?

Thanks,

Bill

9 Replies 9

Marvin Rhoads
Hall of Fame
Hall of Fame

Netflow is usually supported on routers as it depends on having a visibility of the Layer 3 conversations.

Do you have multiple VLANs on your switch and is it acting a the router betwen the subnets serviced on those VLANs? If so, you should be able to collect and export Netflow statistics of those inter-VLAN flows.

You will not - either with the service module or any other Netflow collector - see intra-VLAN conversations as they do not go via any Layer 3 interface.

Yes, I have multiple VLANs - my servers are on one, iSCSI on another, and workstations on a third, and the switch is routing between them.  Port-to-port may have been an unfortunate term for me to use.  It is the traffic between ports on different VLANs that I wish to be able to monitor, even though the ports are on the same switch stack.

So we should be able to get these flows even without it going through the SFP ports?  When we try to configure the flow, all the examples I see include commands similar to this:

interface TenGigabitEthernet1/1/2

ip flow ingress

ip flow egress

and

interface TenGigabitEthernet1/1/2

ip flow monitor flowrecordname1 layer2-switched input

ip flow monitor flowrecordname2 output

always referring to the 10GB ports.  That seems to indicate that the input and output can only be read from the SFP 10GB ports.   In particular, I'm trying to follow the information shown on this site - http://www.plixer.com/blog/network-traffic-analysis/3kx-netflow-configuration-catalyst-3750-netflow-support/.  Is that just the way this guy wrote it?  Could I use any of the ports?

We're trying to use Solarwinds Netflow Monitor (eval mode at present - will purchase if we get it working right) to do the analysis from a dedicated (virtual) server.

Any further help is welcome.

Good question. I'm not sure about the answer and I don't have a 3750X with the service module available to test it out on. Since you do, go ahead and give it a try. Please let us know what you find out.

I was reading about the 'smartlog' feature when researching this. If you have access-lists separating the VLANs, you may be able to leverage that.

I have confirmed that you can do NetFlow over the gigabit ports, so you don't need to use a 10 gig port.  I tried using @cweinhold's config but it didn't work for some reason in our environment.   I made a step by step post that shows the config that worked for us.  Hope it helps!

Marvin,

There are two tricks.

The first is to force locally-switched traffic to the service module ports. Cisco has a white paper that describes two ways of doing this. Remote SPAN seems to be the best if you aren't planning on using the uplinks, though it does sacrifice netflow visibility into physical port and vlan information.

The second trick is to keep the ten gig port to go up, so that traffic gets forwarded. The simplest way to do this is to install a 1-meter twinax cable between the two uplink ports (p/n SFP-H10GB-CU1M, list price $150). [Updated 4-27-2012 because the Twinax cable does not work] To do this you will need a pair of fiber SFP's -- either GLC-SX-MM for 1G or SFP-10G-SR for 10GE -- and a short fiber run between them. You may be able to get away with a single SFP and a fiber loopback cable.

Now, before you go and create a 10 Gbps loop in your network, make sure that one port is configured to carry the RSPAN vlan, and the other port is set to drop every packet it receives. I.e., something like this:

    vlan 950

    remote-span

    vlan 951

    shutdown

    interface ten 1/1

    switchport trunk encapsulation dot1q

    switchport trunk allowed vlan 950

    switchport mode trunk

    switchport nonegotiate

    ip flow monitor myFlowMonitor layer2-switched output

    no cdp enable

    int ten 1/2

    no cdp enable

    switchport mode access

    switchport nonegotiate

    switcport access vlan 951

I've tested this and it works fine.

-Craig

but how incredibly dirty and inconvenient is this?

Can't grasp that there is no decent way to do this...

Update: I was able to make this work with a single 10G SFP+ (p/n SFP-10G-SR) and a multimode fiber loopback cable (about $25 from cdw.com).The configuration looks like this:

vlan 950
 remote-span

mac access-list extended blackhole-l2
 deny any any

ip access-list extended blackhole-l3
 deny ip any any

interface TenGigabitEthernet1/1
 switchport trunk encapsulation dot1q
 switchport trunk allowed vlan 950
 switchport mode trunk
 switchport nonegotiate
 ip flow monitor  layer2-switched output
 ip access-group blackhole-l3 in
 mac access-group blackhole-l2 in
 no cdp enable
 spanning-tree bpduguard enable

monitor session 1 source vlan [list of active vlans]
monitor session 1 destination remote vlan 950

With this config, one of the 10G ports is still available for server or uplink use.

-Craig

Craig,

Is there any kind of CPU hit doing this?  So I either need to rip/replace 3750x's or buy a $2,500 module per site to get Netflow visibility.  I also have a Layer 3 port (no switchport) can I add that to the SPAN as well?

I don't know about performance. The main switch handles the SPAN session and probably also does the UDP Netflow export. The SM is responsible for all Netflow processing, so it shouldn't impact the main switch at all.

You should be able to add anything to the SPAN session, including routed interfaces..

-Craig

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Innovations in Cisco Full Stack Observability - A new webinar from Cisco