A check sum error in an Evertz closed caption generator (HD9084) caused a primary and two secondary encoders (D9036) to fail.
When the issue first began, we got a call into our TOC that there was no closed captions on a particluar service. In the investigation we found that the primary encoder was giving us a warning about missing AFD. We founfd that strange and thought nothing of it. The TOCthen witnessed a video hit and decided it was safer to get off the primary encoder. The manually switched to the S1 encoder and within one hour the S1 encoder failed to the S2 encoder which failed back to the Primary.
At this point, even though we are still on the air, we have no secondaries for the entire mux. This was on a Thursday and Cisco had a engineer here by Monday. Thats 4 long days with no back ups!
It took Cisco 3 days to actually fix the problem.
It is still unknown on why the bad CC took down these encoders.
Also, we suspect that the AFD errors were actually CC errors but the D9036 has no handle on displaying CC errors.
Has anyone seen this before?
We have since engaged the following SOP:
If there is to be a manual switch for issues with no clear reason, the tech is to place the S2 encoder off line first. This way should this happen again, at least the error will not cascade to the second stand by. We would then take which ever encoder is showing the problem off line so the PNC will not switch it again. Finally, we place the S2 standby back online to back up the rest of the mux.
I apologize if this post is a bit confusing. The entire incident is confusing.
This document is an early notification of a behaviour change that will be introduced in IOS XR release 6.5.
IOS XR configuration principles relevant for this article are:
On router platforms all interfaces must be by defaul...
With XR 4.2.0 the ASR9000 is releasing a new line of hardware models. This amongst others is the RSP440, the next generation RSP with faster switch fabric along with Typhoon based Linecards, the next generation network processor.
The Cisco EPN system incorporates a network architecture designed to consolidate multiples services on a single Multiprotocol Label Switching (MPLS) transport network. This network is designed primarily based on Application ...