Re: SG500X-24P, IP layer issue ?

ArnaudG · ‎11-03-2017

Dear Cisco,

I'm encountering an issue with a SG500X-24poe switch on a specific vlan.

Firmware 1.4.8.6

Boot 1.4.0.02

I'm logged with ssh directly into my switch and got a device(ip camera) configured in 172.20.230.101 connected onto a access port (vlan 2253)

My switch has an IP in that VLAN and can ping itself

When I try to ping the camera, it fails.

But the ARP shows the correct ip, port and vlan...

When I switch to another VLAN, the device is pinging normally.

Moreover I have a mirror setup with sames switch (same config) and device thats working fine.

I got no specific rules like ACL or else.

See the console output below

let me know if you require any other information

Any clue to help me solve this mystery ?

a92-sw-stk-s12-poe#clear arp-cache
a92-sw-stk-s12-poe#show arp
Total number of entries: 1

VLAN Interface IP address HW address status
--------------------- --------------- ------------------- ---------------
vlan 2251 te1/1/1 172.20.0.3 e0:d1:73:fb:e3:74 dynamic

a92-sw-stk-s12-poe#ping 172.20.230.101
Pinging 172.20.230.101 with 18 bytes of data:
PING: no reply from 172.20.230.101
PING: timeout
PING: no reply from 172.20.230.101
PING: timeout
PING: no reply from 172.20.230.101
PING: timeout
PING: no reply from 172.20.230.101
PING: timeout
----172.20.230.101 PING Statistics----
4 packets transmitted, 0 packets received, 100% packet loss
a92-sw-stk-s12-poe#show arp
Total number of entries: 2

VLAN Interface IP address HW address status
--------------------- --------------- ------------------- ---------------
vlan 2251 te1/1/1 172.20.0.3 e0:d1:73:fb:e3:74 dynamic
vlan 2253 gi1/1/19 172.20.230.101 00:1b:a2:00:a2:b5 dynamic

Hulk8647 · ‎11-03-2017

Do you have anything else on that same VLAN as the camera that could test ping? You're not doing anything funny like hsrp, MAC sticky? Can post config of the switch?
Can you post "show ip route" and traceroute to camera?

ArnaudG · ‎11-04-2017

Hello Zack,

thanks for your feed back here are my answers :

Do you have anything else on that same VLAN as the camera that could test ping?

yes we figured out this issue with a machine with a direct connection on the same VLAN. This machine was able to ping others cameras (from the "mirror" setup)

You're not doing anything funny like hsrp, MAC sticky?

No I don`t think so

Can post config of the switch?

config-file-header
a92-sw-stk-s12-poe
v1.4.8.6 / R800_NIK_1_4_202_008
CLI v1.0
set system queues-mode 4
file SSD indicator encrypted
@
ssd-control-start
ssd config
ssd file passphrase control unrestricted
no ssd file integrity control
ssd-control-end cb0a3fdb1f3b1af4Xe4430033719968X0
!
vlan database
default-vlan vlan 2252
exit
vlan database
vlan 1,xxxx,xxxx,xxxx-xxxx,2253,xxxx,xxxx
exit
voice vlan oui-table add 0001e3 Siemens_AG_phone________
voice vlan oui-table add 00036b Cisco_phone_____________
voice vlan oui-table add 00096e Avaya___________________
voice vlan oui-table add 000fe2 H3C_Aolynk______________
voice vlan oui-table add 0060b9 Philips_and_NEC_AG_phone
voice vlan oui-table add 00d01e Pingtel_phone___________
voice vlan oui-table add 00e075 Polycom/Veritel_phone___
voice vlan oui-table add 00e0bb 3Com_phone______________
bonjour interface range vlan 1
hostname a92-sw-stk-s12-poe
logging host 172.16.1.1
logging origin-id hostname
username cisco password encrypted blablah privilege 15
ip ssh server
snmp-server server
snmp-server community secret!!! ro xxx.xxx.xxx.xxx view Default
clock timezone " " +1
clock summer-time web recurring eu
clock source sntp
sntp unicast client enable
sntp unicast client poll
sntp server xxx.xxx.xxx.xxx poll
!
interface vlan xxxx
name dmx
!
interface vlan xxxx
name audio
!
interface vlan xxxx
name regie-video
!
interface vlan xxxx
name management
ip address xxx.xxx.xxx.xxx 255.255.255.0
!
interface vlan xxxx
name vlan
!vla
interface vlan 2253
name gige
ip address 172.20.230.199 255.255.255.0
!
interface vlan xxxx
name vlan
!
interface vlan xxxx
name vlan
!
interface vlan xxxx
name vlan
!
interface vlan xxxx
name vlan
!
interface vlan xxxx
name vlan
!
interface vlan xxxx
name vlan
!
interface vlan xxxx
name vlan
!
interface vlan xxxx
name vlan
!
interface gigabitethernet1/1/1
description port
switchport mode access
!
interface gigabitethernet1/1/2
description port
switchport mode access
!
interface gigabitethernet1/1/3
description port
switchport mode access
!
interface gigabitethernet1/1/4
description port
switchport mode access
!
interface gigabitethernet1/1/5
description port
switchport mode access
!
interface gigabitethernet1/1/6
description port
switchport mode access
!
interface gigabitethernet1/1/7
description port
switchport mode access
!
interface gigabitethernet1/1/8
description port
switchport mode access
!
interface gigabitethernet1/1/9
description port
switchport mode access
!
interface gigabitethernet1/1/10
description port
switchport mode access
!
interface gigabitethernet1/1/11
description uplink
switchport trunk allowed vlan add xxxx,2253-xxxx,xxxx
!
interface gigabitethernet1/1/12
description port
switchport mode access
!
interface gigabitethernet1/1/13
description port
switchport mode access
switchport access vlan xxxx
!
interface gigabitethernet1/1/14
description port
switchport mode access
switchport access vlan xxxx
!
interface gigabitethernet1/1/15
description port
switchport mode access
switchport access vlan xxxx
!
interface gigabitethernet1/1/16
description port
switchport mode access
switchport access vlan xxxx
!
interface gigabitethernet1/1/17
description port
switchport mode access
switchport access vlan xxxx
!
interface gigabitethernet1/1/18
description port
switchport mode access
!
interface gigabitethernet1/1/19
description camera-gige
switchport mode access
switchport access vlan 2253
!
interface gigabitethernet1/1/20
description camera-gige
switchport mode access
switchport access vlan 2253
!
interface gigabitethernet1/1/21
description camera-gige
switchport mode access
switchport access vlan 2253
!
interface gigabitethernet1/1/22
description camera-gige
switchport mode access
switchport access vlan 2253
!
interface gigabitethernet1/1/23
description uplink
switchport trunk allowed vlan add xxxx,2253-xxxx,xxxx
!
interface gigabitethernet1/1/24
description port
switchport mode access
switchport access vlan xxxx
!
interface tengigabitethernet1/1/1
description uplink
switchport trunk allowed vlan add xxxx,2253-xxxx,xxxx
!
interface tengigabitethernet1/1/2
description uplink
switchport trunk allowed vlan add xxxx,2253-xxxx,xxxx
!
exit
banner login ^C
a92-sw-stk-s12-poe
^C
banner exec ^C
a92-sw-stk-s12-poe
^C
macro auto disabled
ip default-gateway 172.20.0.3

Can you post "show ip route" and traceroute to camera?

a92-sw-stk-s12-poe#show ip route address 172.20.230.101
Maximum Parallel Paths: 1 (1 after reset)
IP Forwarding: enabled
Codes: > - best, C - connected, S - static,
       R - RIP


S   0.0.0.0/0 [1/1] via 172.20.0.3, 18:28:50, vlan 2251
C   172.20.230.0/24 is directly connected, vlan 2253

a92-sw-stk-s12-poe#traceroute ip 172.20.230.101
Tracing the route to 172.20.230.101 (172.20.230.101) from , 30 hops max, 18 byte packets
Type Esc to abort.
 1   *  *  *
 2   *  *  *
 3   *  *  *
 4   *  *  *
 5   *  *  *
 6   *  *  *
 7   *  *  *
 8   *  *  *
 9   *  *  *
10   *  *  *
11   *
Trace aborted.

ArnaudG · ‎11-04-2017

In addiction.

When I login into the "mirror"setup switch, I cant ping 172.20.230.199.

It`s like if vlan 2253 was dead on that switch :-|

Hulk8647 · ‎11-05-2017

Your camera is on 2253. Your default vlan says its 2252. If that intentional? Ping your camera using source ip.

Ex. "ping CAMERA IP source vlan 2053"

Let me know what happens

ArnaudG · ‎11-06-2017

Hello !

Yes it's as design.

2252 is my default vlan for all devices of my network
2251 for switches management
2253 for cameras...

Command you gave doesn't seems to work, needs to specify IP address instead.

a92-sw-stk-s12-poe(config-if-range)#do ping 172.20.230.101 source 172.20.230.199
Pinging 172.20.230.101 with 18 bytes of data:
PING: no reply from 172.20.230.101
PING: timeout
PING: no reply from 172.20.230.101
PING: timeout
PING: no reply from 172.20.230.101
PING: timeout
PING: no reply from 172.20.230.101
PING: timeout
----172.20.230.101 PING Statistics----
4 packets transmitted, 0 packets received, 100% packet loss

Here is my routing table

a92-sw-stk-s12-poe(config-if-range)#do show ip rout
Maximum Parallel Paths: 1 (1 after reset)
IP Forwarding: enabled
Codes: > - best, C - connected, S - static,
       R - RIP


S   0.0.0.0/0 [1/1] via 172.20.0.3, 63:01:01, vlan 2251
C   172.20.0.0/24 is directly connected, vlan 2251
C   172.20.230.0/24 is directly connected, vlan 2253

Hulk8647 · ‎11-06-2017

The only thing I can think of is that your camera might have a wrong config. Like, wrong gateway. Have you tried hard resetting network settings for your camera?

If you did try, is there any way you can find out exactly what the config of the camera is? After that, I would configure ACL capture to see if traffic is not being sent out or not returned.

ArnaudG · ‎11-06-2017

My camera are not supposed to be routed on the network.

The machines supposed to access it are simply plugged into an acces port 2253.

The ip 172.20.230.199 that I setted up on the switch in Vlan 2253 was just here for testing purpose, in order to be as close as possible from the cameras.

Morevover these cameras do not even offer the possibility to have a gateway.

I doesnt look like that my camera has bad configuration as I can access them as soon as I change their access port's untagged vlan.

A soon as I do the change, even if my IP settings are wrong (not in the right subnet), the camera software is able to discover them on the network (through some kind of broadcast magic packet)

Which is not true when I switch back to VLAN 2253...

What do you mean by ACL capture ? Is this an option on the switch ?

Hulk8647 · ‎11-06-2017

I still think it's the camera. Here is what I would check/do

- hard reset the camera, clear arp, clean mac-address on the switch, go to the router behind the switch and clear DHCP binding and arp there too. Plug the camera back in and verify connectivity from switch and router.

- Check router for the new DHCP binding, ping it from the router and then switch

ArnaudG · ‎11-09-2017

Hi Zack,

Sorry for late answer.
To be more precise, my setup consist in 2 exact same switches (I just made a diff of the config file, port tagging, everything is the same)
where 6 exactly-the-same PoE cameras are plugged (3 per switch)
This is not a DHCP related topic, all cameras are static addressed
This is not a routing topic, everybody is on the same VLAN.
I hard reset the camera multiple time (unplugging, replugging...) cleared ARP cache on the switch nothing shows up.

As soon as I change the 3 camera's VLANs on the switch from 2253 to 2252, they popup in the IDS camera software interface (the host computer has two physical network interface, on in each VLAN)

For me there is really something wrong with the switch but cannot find what... Or I am missing something big in the rest of my setup.

Tonight I will swap my switches uplink on my root switch to ensure that the problem is located downstream in the config of the switches or the cameras and not around my root switch/host
After this I may probably want to reset the switch from scratch... which is a big deal because the switch is 50m high hanging on a platform :-D

Thanks for being so helpful and patient :D

ktonev · ‎11-09-2017

Hi,

From which device are you trying to ping the camera?

Is it connected on the same switch and same VLAN?

ArnaudG · ‎11-09-2017

Hello ktonev

I've started to try from a computer located on another branch on the network. this computer has direct link on the vlan.

As I was unsuccessful I ssh into the switch where cams are plugged and added an IP on the VLAN to ping them more directly.

I can see and ping the cameras only when I configure cameras port access 2252 which is my network default vlan.

As soon as I switch to 2253, the cameras disappear from the IP world, but I can still see correct MAC and, IP, port and vlan in my ARP table

Hulk8647 · ‎11-09-2017

can you past these, don't erase any vlans or IP info unless it's an outside IP.

sh ip int brief | ex un

show vlan brief

sh int status

sh int switchport

show int gigabitethernet1/1/19

sh mac address-table interface gi1/1/19

show arp

ArnaudG · ‎11-13-2017

Hello Zack, thanks for the follow up

Here are the command output. Some commands were not available onto SG200, I adapted.

sh ip int brief | ex un

a92-sw-stk-s12-poe#show ip int


    IP Address        I/F    I/F Status  Type   Directed  Prec Redirect Status
                             admin/oper         Broadcast
------------------ --------- ---------- ------- --------- ---- -------- ------
172.20.0.101/24    vlan 2251 UP/UP      Static  disable   No   enable   Valid
172.20.230.123/24  vlan 2253 UP/UP      Static  disable   No   enable   Valid

show vlan brief

a92-sw-stk-s12-poe#show vlan
Created by: D-Default, S-Static, G-GVRP, R-Radius Assigned VLAN, V-Voice VLAN

Vlan       Name           Tagged Ports      UnTagged Ports      Created by
---- ----------------- ------------------ ------------------ ----------------
 1           1                                                      S
1380    vlan1380                                                    S
2220    vlan2220                                                    S
2250    vlan2250                                                    S
2251    vlan2251       gi1/1/11,gi1/1/23,      gi1/1/24             S
                       te1/1/1-2
2252    default                           gi1/1/1-12,               D
                                          gi1/1/18,
                                          gi1/1/22-23,
                                          gi1/1/25-48,
                                          te1/1/1-4,
                                          gi2/1/1-48,
                                          te2/1/1-4,
                                          gi3/1/1-48,
                                          te3/1/1-4,
                                          gi4/1/1-48,
                                          te4/1/1-4,
                                          gi5/1/1-48,
                                          te5/1/1-4,
                                          gi6/1/1-48,
                                          te6/1/1-4,
                                          gi7/1/1-48,
                                          te7/1/1-4,
                                          gi8/1/1-48,
                                          te8/1/1-4,Po1-32
2253       Gige            te1/1/1-2         gi1/1/19-21            S
2254     vlan2254      gi1/1/11,gi1/1/23,                           S
                       te1/1/1-2
2255     vlan2255                                                   S
2256    vlan2256        gi1/1/11,gi1/1/23,   gi1/1/13-17            S
                       te1/1/1-2
2257      avail1                                                    S
2258      avail2                                                    S
2259      avail3                                                    V
2400     vlan2400                                                   S
2510     vlan2510                                                   S

sh int status

a92-sw-stk-s12-poe#sho int stat
                                             Flow Link          Back   Mdix
Port     Type         Duplex  Speed Neg      ctrl State       Pressure Mode
-------- ------------ ------  ----- -------- ---- ----------- -------- -------
gi1/1/1  1G-Copper    Full    100   Enabled  Off  Up          Disabled On
gi1/1/2  1G-Copper    Full    100   Enabled  Off  Up          Disabled Off
gi1/1/3  1G-Copper    Full    100   Enabled  Off  Up          Disabled Off
gi1/1/4  1G-Copper    Full    100   Enabled  Off  Up          Disabled Off
gi1/1/5  1G-Copper    Half    10    Enabled  Off  Up          Disabled On
gi1/1/6  1G-Copper      --      --     --     --  Down           --     --
gi1/1/7  1G-Copper      --      --     --     --  Down           --     --
gi1/1/8  1G-Copper    Full    100   Enabled  Off  Up          Disabled On
gi1/1/9  1G-Copper      --      --     --     --  Down           --     --
gi1/1/10 1G-Copper      --      --     --     --  Down           --     --
gi1/1/11 1G-Copper      --      --     --     --  Down           --     --
gi1/1/12 1G-Copper      --      --     --     --  Down           --     --
gi1/1/13 1G-Copper    Full    1000  Enabled  Off  Up          Disabled On
gi1/1/14 1G-Copper    Full    1000  Enabled  Off  Up          Disabled On
gi1/1/15 1G-Copper    Full    1000  Enabled  Off  Up          Disabled On
gi1/1/16 1G-Copper    Full    1000  Enabled  Off  Up          Disabled On
gi1/1/17 1G-Copper      --      --     --     --  Down           --     --
gi1/1/18 1G-Copper      --      --     --     --  Down           --     --
gi1/1/19 1G-Copper    Full    1000  Enabled  Off  Up          Disabled On
gi1/1/20 1G-Copper    Full    1000  Enabled  Off  Up          Disabled On
gi1/1/21 1G-Copper    Full    1000  Enabled  Off  Up          Disabled Off
gi1/1/22 1G-Copper      --      --     --     --  Down           --     --
gi1/1/23 1G-Copper      --      --     --     --  Down           --     --
gi1/1/24 1G-Copper      --      --     --     --  Down           --     --
te1/1/1  10G-Fiber    Full    10000 Disabled Off  Up          Disabled Off
te1/1/2  10G-Fiber      --      --     --     --  Down           --     --

sh int switchport

a92-sw-stk-s12-poe#sho int switc GE1/1/19
Added by: D-Default, S-Static, G-GVRP, R-Radius Assigned VLAN, T-Guest VLAN, V-Voice VLAN
Port : gi1/1/19
Port Mode: Access
Gvrp Status: disabled
Ingress Filtering: true
Acceptable Frame Type: admitAll
Ingress UnTagged VLAN ( NATIVE ): 2253

Port is member in:

Vlan               Name               Egress rule     Added by
---- -------------------------------- ----------- ----------------
2253               Gige                Untagged          S


Forbidden VLANS:
Vlan               Name
---- --------------------------------


Classification rules:

Protocol based VLANs:
  Group ID   Vlan ID
------------ -------


Mac based VLANs:
  Group ID   Vlan ID
------------ -------

show int gigabitethernet1/1/19

no such command... here are a few different command interfaces

a92-sw-stk-s12-poe#show int access-lists ge1/1/19
Interface                  ACLs
---------          -----------------------
gi1/1/19         N/A
a92-sw-stk-s12-poe#show int counters ge1/1/19

      Port       InUcastPkts  InMcastPkts  InBcastPkts    InOctets
---------------- ------------ ------------ ------------ ------------
    gi1/1/19      4002009754       0          866373    600132335998
                                                        4

      Port       OutUcastPkts OutMcastPkts OutBcastPkts  OutOctets
---------------- ------------ ------------ ------------ ------------
    gi1/1/19       21533187     9461343      9248221     5017689453

Alignment Errors: 0
FCS Errors: 0
Single Collision Frames: 0
Multiple Collision Frames: 0
SQE Test Errors: 0
Deferred Transmissions: 0
Late Collisions: 0
Excessive Collisions: 0
Carrier Sense Errors: 0
Oversize Packets: 0
Internal MAC Rx Errors: 0
Symbol Errors: 0
Received Pause Frames: 0
Transmitted Pause Frames: 0
a92-sw-stk-s12-poe#show int status ge1/1/19
                                             Flow Link          Back   Mdix
Port     Type         Duplex  Speed Neg      ctrl State       Pressure Mode
-------- ------------ ------  ----- -------- ---- ----------- -------- -------
gi1/1/19 1G-Copper    Full    1000  Enabled  Off  Up          Disabled On

sh mac address-table interface gi1/1/19

a92-sw-stk-s12-poe#sho mac address-table inter ge1/1/19
Flags: I - Internal usage VLAN
Aging time is 300 sec

    Vlan          Mac Address         Port       Type
------------ --------------------- ---------- ----------
    2253       00:1b:a2:00:a2:b5    gi1/1/19   dynamic

show arp

a92-sw-stk-s12-poe#show arp

Total number of entries: 2


  VLAN    Interface     IP address        HW address          status
--------------------- --------------- ------------------- ---------------
vlan 2251  te1/1/1    172.20.0.3      e0:d1:73:fb:e3:74   dynamic
vlan 2253  gi1/1/19   172.20.230.101  00:1b:a2:00:a2:b5   dynamic

Hulk8647 · ‎11-13-2017

See what happens when you configure access-list capture below.

Configure the ACL

access-list 110 permit ip host 172.20.230.199 host 172.20.230.101
access-list 110 permit ip host 172.20.230.101 host 172.20.230.199
access-list 110 permit ip host 172.20.0.101 host 172.20.230.101
access-list 110 permit ip host 172.20.230.101 host 172.20.0.101

Assign the ACL to the camera interface

interface gi1/1/19
ip access-group 110 in

Enable debug for the ACL

#debug ip packet 110 detail

ping the camera from the switch couple times. Once done, you can check the ACL capture logs.

show access-list 110