×

Warning message

  • Cisco Support Forums is in Read Only mode while the site is being migrated.
  • Cisco Support Forums is in Read Only mode while the site is being migrated.

Strange switching behaviour

Unanswered Question
Jul 19th, 2012
User Badges:

  Hi guys,


This is my first post here so be kind....


  I have a switching topology question that has me troubled. 


Basically I have 6 CODECS on a LAN with UDP Unicast video from each box, I have multiple clients viewing the video so around 700 Mbps throughput.


I have a Cisco SG30-10 Small business switch with a dedicated port for each CODEC, all traffic leaves the switch through one interface to a Firewall.


If i have this topology then I get some pretty bad loss on the viewing clients, if I change the topology so that each CODEC loops through its neighbour so only one has a link to the switch then all is well.


Obviously this will not do as a single failure will bring down the network.  The switch has its default config and I am sure that the single link can handle the load.


Any and all thoughts welcome (within reason).


Many thanks in advance.


Dan.

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (1 ratings)
Loading.
Peter Paluch Fri, 07/20/2012 - 10:10
User Badges:
  • Cisco Employee,

Hello Dan,


If I understand you correctly then you are saying that if each CODEC has its own uplink to the switch then bad things happen. If you daisy-chain the codecs and connect this entire chain of 6 CODECs using a single uplink to the switch then everything's fine. Do I get this correctly?


This is quite a strange issue indeed. What I suggest - if you can afford making such disruptive changes to the network - is to make the following test: disconnect all CODECs from the switch, and start connecting them to the switch, one by one, each having its own uplink, and before adding a new CODEC, verify if the clients are experiencing any losses, breaks or outages. Stop adding new CODECs as soon as the connection of a new CODEC results in decreased performance/experience. Note the count of connected CODECs here. Then, try to replace this CODEC with a different CODEC and see if the problem goes away of whether the problem is generally related to the number of attached CODECs.


With each added CODEC, make sure that the switch port autonegotiates full duplex and appropriate speed. Half duplex is your enemy.


Let me know the results please!


Best regards,

Peter

daniel-oneill Fri, 07/20/2012 - 13:21
User Badges:

Thank you Peter,

Yes you understand my topology correctly.

I am in a fortunate position at the moment of being in a pre production test environment, this will however go into a live network environment soon so I need to get this sorted.

I believe from previous tests that it is in fact related to load, having said that, I like the idea of trying to isolate a bad codec.

I will try this Monday morning.

It's funny you should say that negotiation is my enemy, I have found that there seems to be little or no difference between when I change a switch port to 100M full or have it set at auto (reported as 1000M full).  Having said that, it may be masked by the other traffic throughput being so high.  I need to test this further.  The codec backplane is 1000 auto, or so the manufacturer has told me. 


The thing that really gets me is how can a daisy chained topology perform much better than a switched one?

Something strange is going on but finding it is a nightmare, using udp isn't helping either as capturing a tcp trace would be easier.  I have to use udp though.


I'm sure this has a simple solution, it's a pretty simple topology.


Dan.

Peter Paluch Fri, 07/20/2012 - 15:08
User Badges:
  • Cisco Employee,

Hello Dan,


It's funny you should say that negotiation is my enemy, I have found  that there seems to be little or no difference between when I change a  switch port to 100M full or have it set at auto (reported as 1000M  full).


In fact, I have said that half duplex is your enemy, not the autonegotiation. You have to be careful with static setting of duplex and speed. Many switches deactivate the autonegotiation completely if the port is manually configured with both duplex and speed setting, often causing duplex mismatches with attached devices that expect that the duplex can be negotiated. Therefore, my suggestion is to leave the speed and duplex unconfigured and just verify if the duplex and speed are sensible.


The thing that really gets me is how can a daisy chained topology perform much better than a switched one?


I am not sure myself. How did you actually come to the idea of daisy chaining the codecs, anyway?


Best regards,

Peter

daniel-oneill Mon, 07/23/2012 - 01:14
User Badges:

Thanks Peter,


You are right about the Auto Negotiation settings, I wasnt thinking clearly when I posted last.  I will leave all set to Auto.

The idea to daisy chain came about through testing load through the system.  All was well until we ramped up the throughput to a couple of hundred Mbps.  We need 1Gbps ( or as close as possible).  The codecs have dual network ports for redundancy and an uplink port for attaching a local NVR (Recording device).

I was clutching at straws really but the uplink port going to the next devices network port works flawlesly.

I have contacted the manufacturer of the codecs but their support team are unable to tell me why this topology works, they could only say that the uplink ports were not designed for attaching to the network.

Unfortunately I can not leave it in this state for obvious reasons so more testing is required.....


Thanks,


Dan.

Peter Paluch Mon, 07/23/2012 - 02:00
User Badges:
  • Cisco Employee,

Hi Dan,


I see. So, if the ports are left to autonegotiate the speed and duplex, what speed and duplex did they negotiate? Also, were you able of doing the experiments with successive attaching the codecs to the switch?


Please keep me informed. Thank you!


Best regards,

Peter

daniel-oneill Mon, 07/23/2012 - 02:39
User Badges:

Hi Peter,


When the switch settings are left at Auto (Factory settings) they report 1000M Full.  I have not been able to isolate a bad codec, thinking about it, if one was faulty then it would possibly show when daisy chaining which it has not.

I'm starting to think that maybe the switch backplane can't handle the processing required when I use 7 ports (six in one out)?

If all traffic comes in one port then out one port everything is ok.

I may try getting hold of a bigger switch for testing, worth a try I think.

What are your thoughts on the the posibility of the switch being under powered? Its a SG 300-10 Small business switch.


Many thanks.


Dan.

Peter Paluch Mon, 07/23/2012 - 05:04
User Badges:
  • Cisco Employee,

Hello Dan,


I have not been able to isolate a bad codec, thinking about it, if one  was faulty then it would possibly show when daisy chaining which it has  not.


This is true. Still, we may find out about the capacity limitation of this switch's fabric if we start adding the codecs one by one and observe when does the quality go down.


I'm starting to think that maybe the switch backplane can't handle the processing required when I use 7 ports (six in one out)?


We should not dismiss this possibility. Still, assuming that the total amount of ingress traffic from these codecs does not exceed the capacity of the egress port, I would be surprised to learn that this switch is not capable of sustained switching these flows. How large is the flow generated by a single codec?


The datasheet about the SG300-10 switch suggests that the total capacity of the switching fabric is 20Gbps:


http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps10898/data_sheet_c78-610061.html


Trying out a larger switch is also worth it.


Best regards,

Peter

daniel-oneill Mon, 07/23/2012 - 05:37
User Badges:

Hi Peter,


Each codec device has four cards, each card will support eight x 6Mbps streams max.

So each codec device (4x cards) will have a maximum output of 192Mbps (each card producing 48Mbps).

This would be just over 1Gbps, we actually set the cards to 4Mbps per stream, so 128Mbps per device = 768Mbps max total.

This is below what I would think the switch could handle per port.  We see this issue at far less throughput though.


Thanks,


Dan.

Alessio Andreoli Mon, 07/23/2012 - 05:43
User Badges:
  • Silver, 250 points or more

Hi Daniel,

did you make sure that these application are not "multicast" applications? maybe it is the firewall blocking the multicast addresses that is changing the application to use replicated unicast rather than a normal optimised multicast routing...



Alessio

daniel-oneill Mon, 07/23/2012 - 05:57
User Badges:

Hi Alessio,


We have to use Unicast for our system so no multicast is configured at all.  Basically our clients will request a "stream" and this is delivered via UDP Unicast video.



Thanks,


Dan.

daniel-oneill Tue, 07/24/2012 - 07:30
User Badges:

Solved.


I have been looking at the options within the CLI and stumbled upon a fix for my issue.


For all 6 ingress ports from the codecs I have set the Interface Flow Control to On.


This has now solved the problem completely.


Thanks to Peter and Alessio for your help.


Dan.

Peter Paluch Tue, 07/24/2012 - 08:41
User Badges:
  • Cisco Employee,

Daniel,


Actually, I would have never thought this would solve the problem. Are you telling me that you had the flow control deactivated, and after activating it, the problem was solved?


+5 points for you!


Best regards,

Peter

daniel-oneill Tue, 07/24/2012 - 09:02
User Badges:

Hi Peter,


Yes Flowcontrol is disabled by default on these SG300 switches, I beleive that actually it is on most devices as it causes some problems.  I was trying to get anything to work to be honest in the CLI, this was the last thing I tried.


It seems to make no difference if this is enabled on the egress interface to the firewall but for whatever reason it must be on for the codecs to work in a correctly configured topology.


I dont really understand how flowcontrol would help real time udp video in this case so I need to do some reading!


Thanks,


Dan.

Actions

This Discussion

Related Content