IDSM2 in inline mode can be a complex configuration. Not in terms of configuration on IDSM2 itself, but in terms of configuring 6500/7600 so that traffic can flow through IDSM2. This document tries to explain the basic concepts behind the inline mode design.
IDSM2 does not do IP routing. It cannot route between Vlans.
IDSM2 can only bridge Vlans (Operates at layer 2).
IDSM2 in inline mode necessitates an additional artificial Vlan on the SAME subnet as the Vlan you wish to sense. A layer 3 switch interface needs to be configured within this additional artificial Vlan.
In a nutshell, we need to create 2 Vlans that share one same ip subnet and put SVI on only one of the Vlans.
One Vlan is one broadcast domain. This means is a broadcast frame sent will only be flooded to all ports in the Vlan and will not transmitted outside it.
We need to route a packet if we need to go from one broadcast domain to the other, i.e from one Vlan to the other.
If we bridge Vlans together, we merge broadcast domains. A broadcast frame sent by host is sent to all ports in both the Vlans that are bridged together.
L3 broadcast allows IP packets to reach all hosts on the local subnet. (But packets will not be forwarded out of other router interfaces to other subnets.)
With bridging, we can merge broadcast domains.
Data flow between IDSM2 and 6500 happens via ports on the back-plane.
6500 dataport 1 connects to IDSM gig0/7. 6500 dataport 2 connects to IDSM gig0/8
IDSM will bridge gig0/7 and gig0/8 together.
Vlan assignment to ports can be done only on Cat6500 side.
If Dataport1 is in vlan x and Dataport 2 is in Vlan y then, IDSM is in fact bridging Vlan x & y due to the architecture.
For IDSM2 to monitor vlan 100, we create another Vlan 300.
Both Vlan 100 & 300 belong to same Ip subnet 100.1.1.x
The point to be noted is L3 interface Vlan only exists on one Vlan and NOT both.
The default gateway for Vlan 100 should be ip address configured on interface vlan 300.
To understand why traffic flows through the IDSM by this design necessitates good understanding of ARP.
ARP for destination in same subnet: ARP broadcast request packet sent directed to the destination.
ARP for destination in different subnet: ARP broadcast request packet directed to the default gateway.
The configuration works for all traffic leaving and entering the Vlan but not for traffic local to the Vlan.
A. Traffic that leaves vlan 100 or enters vlan 100 shall get monitored by IDSM2.
- When Host A tries to reach a destination in different subnet or different Vlan, it Arps for the default gateway. The default gateway for Vlan 100 is the SVI on the newly created Vlan 300.
- The Arp broadcast request packet sent by Host A in Vlan 100 is received on Gig1/1 on the switch and is flooded in the Vlan 100. Dataport1 which also belongs to Vlan 100 receives it and then Gig 0/7 of IDSM receives it over the back-plane connection. IDSM bridges Gig0/7 and Gig0/8 and packet goes out of Gig0/8 into Vlan 300 and reaches SVI of Vlan 300 from where is routed to its destination.
- The switch maintains the source mac addresses per Vlan based in the table & learns mac address of Host A on port gig1/1 for Vlan 100 and Dataport 2 for Vlan 300 (Since same packet is bridged from Vlan 100 to Vlan 300 by passing from gig0/7 to gig0/8).
- From the point of view of switch Host A for Vlan 300 can be reached through dataport2.
- Hence any packet sent to Host A in Vlan 300 is sent to dataport2. This is how return traffic destined to Host A also goes through IDSM2.
B. Traffic local to Vlan 100 shall never reach the IDSM2 and so will not get monitored.
- When a host A in Vlan 100 sends a packet to host B in vlan 100, Host A arps directly for Host B. And Host B replies (unicast) to Host A. Switch learns the source mac of Host A on Gig 1/1. It also learns the source mac of Host B via arp reply on the port Host B is connected to. The packet never needs to go to the default gateway since both hosts are in the same Vlan.
- The traffic between Host A and B is switched directly by the switch, and this traffic never hits the IDSM2.
The only traffic that needs to go the default gateway will be passing through the IDSM2.
In other words only traffic that leaves the Vlan or is routed to it will get monitored by the IDSM since its bridging the vlans together.