cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
12259
Views
11
Helpful
15
Comments
xthuijs
Cisco Employee
Cisco Employee

Introduction

This document details out the capabilities of the asr9000 when it comes to frequency synchronisation. It talks about the concept of SYNC-E (synchronous Ethernet), which allows an ethernet device to leverage the synchronous (hierarchy) that SONET and SDH networks provide.

A clock can be derived from a circuit or provided externally to the timing interfaces on the RSP and distributed throughout the system.

 

Verification and troubleshooting is also included in this article.

 

 

Key Concepts

Sources are a piece of hardware which inputs frequency signals into the system, or transmit them out of the system.

The three kinds of sources on ASR9000 are

  • Line Interfaces (ex SyncE, SONET)
  • Clock Interfaces (ex BITS/DTI)
  • Internal Oscillator (ST3 oscillator for ASR9000)

 

Selection Points:

Point where choice is made between various frequency signals

Can take it’s input directly from a source or from the output of another selection point.

 

RSP Selector Diagram

 

Screen Shot 2013-04-01 at 10.22.22 PM.png

As seen from diagram above, T0-SEL-B output (selected reference input at selector T0-SEL-B) is used to drive the line interface output. Similarly T4-SEL-C is used to drive the clock-interface output on the BITS ports.

 

Default Configuration

  Global Configuration

  If omitted no frequency synchronization will occur.

 

Line Interface configuration, the default behavior is as follows:

  SyncE: no synchronization, no ESMC messages are sent/received.

  SONET: legacy behavior depending on the ‘clock source’ and ‘s1byte ignore’ configuration.

 

  This configuration will enable frequency configuration on the system:

 

RP/0/0/CPU0:ios#conf

RP/0/0/CPU0:ios(config)#frequency synchronization

RP/0/0/CPU0:ios(config-freqsync)#commit

 

Optional submode global configuration also exists for the following:

  •   to select the clock interface timing mode, either for independent mode or system mode.
  •   to select the QL option type, either for option 1 or 2.

 

Clock Interface configuration

By default the system’s selected frequency signal will be used for clock transmit(if setup for transmit), but does not enable the use of the interface as an input.

 

interface <intf>

    frequency synchronization

    selection input

    ssm disable

    priority <pri>

    quality transmit { lowest <ql-option> <ql> [ highest <ql> ] | highest <ql-option> <ql> | exact <ql-option> <ql> }

    quality receive { lowest <ql-option> <ql> [ highest <ql> ] | highest <ql-option> <ql> | exact <ql-option> <ql> }

    wait-to-restore <time>

 

  The above configuration appears under both interface ‘frequency synchronization’ submode and clock-interface ‘frequency synchronization’

submode.

 

SSM disable:

     Receive: following QL value is used.

         Option 1: DNU

         Option 2: STU

 

    Transmit:

         syncE: No ESMC packets are sent

         SONET/clock-int: DNU

 

Quality transmit:

     Exact QL: Send exactly this QL, unless DNU would otherwise be sent.

    Highest QL: Do not send a QL higher than this; if the selected source has a higher QL than this, then send this QL instead.

     Lowest QL: Do not send a QL lower than this; if the selected source has a lower QL than this, then DNU is sent instead (for clock-interface which does not support SSM output will be squelched)

      Highest and Lowest QL: Combination of the above.

 

Quality receive:

     Exact QL: Use exactly this QL regardless of what value was received, unless the received value was DNU.

     Highest QL: Do not use a QL higher than this; if the received value is higher than this, then use this QL instead.

     Lowest QL: Do not use a QL lower than this; if the received value is lower than this, then DNU is used instead.

     Highest and Lowest QL: Combination of the above.

     For clock-interface which does not support SSM only exact QL can be used.

 

Clock-interface Configuration

Each RSP contains a pair of redundant BITS/UTI ports labeled “SYNC 0” and “SYNC 1”.

In addition to the local frequency synchronization configuration which can be setup for each clockinterface, ‘port-parameters’ for each clockinterface

also need to be setup.

 

Clock-interface modes supported on ASR9000:

  •   Bits-Input: T1(1544Kb/s), E1(2048Kb/s), 2048KHz, 64K
  •   Bits-output: T1(1544Kb/s), E1(2048Kb/s), 2048KHz, 6312Mhz
  •   UTI input

 

Also port configuration should be identical on redundant RSP’s.

 

Clock-interface Interoperability

Table below explains the clock-interface modes supported on ASR9000 and the configuration allowed on the other port when one port is already configured.

 

Screen Shot 2013-04-01 at 10.37.44 PM.png

Interface State and Input QL

Once frequency sync has been configured, need to verify that frequency signals and QL information is received from all sources.

The line interface and clock interface state can be dumped via the following show commands:

  •   show frequency synchronization interface <intf>
  •   show frequency synchronization interface brief
  •   show frequency synchronization clock-interface loc <nodeid>
  •   show frequency synchronization clock-interface brief loc <nodeid>

 

Possible reasons for interface state being down/not selected

  1.      Configuration errors (ex: for T1 bits-input global QL option 2 should have been configured for SSM reception).
  2.      Alarm raised by hardware for possible line defects (ex LOS/AIS/RAI on the bits-input interface).
  3.      SSM is disabled via configuration.
  4.      Interface is not assigned for input selection.

 

 

Show command example

 

RP/0/RSP0/CPU0:ptlm-RO-12#show frequency synchronization clock-interface brief location 0/rsp0/CPU0

Fri Nov 20 03:13:28.398 UTC

Flags: > - Up D - Down S - Assigned for selection

d - SSM Disabled s - Output squelched L - Looped back

Node 0/RSP0/CPU0:

==============

Fl Clock Interface QLrcv QLuse Pri QLsnd Output driven by

===== =================== ====== ====== === ====== ========================

Dd Sync0 n/a Fail 1 n/a n/a

> Sync1 n/a n/a n/a STU GigabitEthernet0/4/0/0

>S Internal0 n/a ST3 255 n/a n/a

 

In the example above ‘Sync0’ clock-interface has been configured for BITS-input(T1 esf ami) mode and ‘Sync1’ is configured for BITS-output

(T1 esf ami) mode.   Sync0 interface state is observed as DOWN.   Sync1 interface state is observed as UP, with its output driven from lineinterface

input(GigabitEthernet0/4/0/0 here) and SSM Qlsend = STU.

 

RP/0/RSP0/CPU0:ptlm-RO-12# show frequency synchronization clock-interface location 0/RSP0/CPU0

Fri Nov 20 03:13:23.712 UTC

Node 0/RSP0/CPU0:

==============

Clock interface Sync0 (Down: ALARM)

Wait-to-restore time 0 minutes

SSM supported

Input:

Down

Effective QL: Failed, Priority: 1

Output is disabled

Next selection points: T0-SEL-B

Clock interface Sync1 (Up)

Wait-to-restore time 5 minutes

SSM supported and enabled

Input is disabled

Output:

Selected source: GigabitEthernet0/4/0/0

Selected source QL: Opt-II,2/STU

Effective QL: Opt-II,2/STU

Next selection points: None

Clock interface Internal0 (Up)

Assigned as input for selection

Input:

Default QL: Opt-II,2/ST3

Effective QL: Opt-II,2/ST3, Priority: 255

Next selection points: T0-SEL-B

 

As seen in the detailed clock-interface state dump, it can be observed again that the clock-interface Sync0 interface state is DOWN, also indicates that an alarm is raised against the same.   Now check the bi-state-alarm logs for exact failure reason. In our example loss of frame condition was detected by hardware on the Sync0 bits-input T1 signal due to which the interface state is DOWN.

 

RP/0/RSP0/CPU0:ptlm-RO-12# show logging events buffer bistate-alarms-set | i RSP0

#379 : :RP/0/RSP0/CPU0:Nov 20 02:57:09.662 2009:tempo[400]: %PLATFORMCLKCTRL-3-RX_ALARM_LOF : Loss of frame condition raised for Rx mode T1 on sync 0 clock-interface

 

Source Selection

Once the frequency signals are up for selection (no config errors, interface state UP and QL recd/assigned), it is important to understand how these signals flow in the system.   The signals from each nominated source are used as inputs to selection points.

The output of some selection points is used as input to other selection points, on the same or different nodes.

The output of some selection points is used to drive the system clock or clock-interface output.

 

In our example lets trace the route of syncE input received on GigabitEthernet0/4/0/0 interface, and examine how it is used to drive the clock-interface output. Also trace the route of Sync0 input on RSP and examine how it selected to drive the system clock (line interface output).

The show command dump “show frequency sync selection location 0/4/CPU0” (see upcoming slide) displays the selections made on the

linecard 0/4/CPU0.   The nominated source GigabitEthernet0/4/0/0 is fed to selection point EZ_RX_0_9 .

The selected reference from selection points EZ_RX_0_9 , EZ_RX_10_19 , EZ_RX_20_29 and EZ_RX_30_39 is fed to local selection point

ETH_RXMUX (see Node scoped field).   Selection point ETH_RXMUX in our example receives only one input and nominates that for selection, also forwards the signal to selection points T0-SEL-B and T4-SEL-A on the RSP (see Chassis scoped field)

 

The show command dump “show frequency sync selection location 0/RSP0/CPU0” displays the selections made on the RSP 0/RSP0/CPU0.   Selection point TO-SEL-B which is used to drive the entire system clock (or line interface output), receives three available input sources for selection.

 

  • Sync0 [0/RSP0/CPU0] from external clock-interface input  
  • GigabitEthernet0/4/0/0 from selection point ETH_RXMUX on node 0/4/CPU0
  • Local Internal oscillator input

 

Selection point TO-SEL-B selects Sync0 input to drive the line interface output based on selection algorithm (Sync0 has the highest QL recd(PRS) in our case).   The output of selection points T0-SEL-B and T4-SEL-A is also forwarded to selection point T4-SEL-C.

The selected reference from T4-SEL-C is used to drive the clock-interface (BITS) output.

 

Note: T4-SEL-C will choose the output from T4-SEL-A or T0-SEL-B purely based on the global timing-mode configured. In our case system is configured for independent timing-mode, hence T4-SEL-C selects the output of T4-SEL-A (i.e GigabitEthernet0/4/0/0) to drive the clock-interface output

 

Screen Shot 2013-04-01 at 10.49.07 PM.png

 

Screen Shot 2013-04-01 at 10.49.18 PM.png

 

 

Clock Interface Operation Modes

Default Mode:

 

The default mode is to auto-detect clock-interface loopbacks, and have the output of the clock-interface only driven by line interface

input (i.e T4-SEL-C always selects the output of T4-SEL-A).   Loopback is detected in the following scenarios:

  • SSM is not supported or disabled on either the input or the output clockinterface, and the clock-interface output is not squelched.
  • SSM is supported and enabled on both the clock-interface input and output,and the recd QL for the clock-interface input is the same as the send QL forthe clock-interface output.

 

Note: If the clock-interface doesn't support SSMs, or SSMs are disabled on the input clock-interface, then the QL received is assigned

to be the same as that for the source being used to drive the output clock-interfaces on the node.

Note: The input QL for the clock-interface can still be modified by the 'clock-interface quality receive' configuration.

 

In the example below the clock-interfaces on the RSP are configured in the following manner:

  •   Sync0: UTI input (does not support SSM), hence the QL received is assigned to the same value as the QL send on the Sync1 clockinterface (ST3E in our example)
  •   Sync1: T1 BITS-OUT (whose output is driven by line interface input GigabitEthernet0/4/0/0, which has an input QL = ST3E).

  As expected in default mode clock-interface loopback is detected as QLrecv (on Sync0 interface) = QLsend (on Sync1 interface).

 

RP/0/RSP0/CPU0:ptlm-RO-12# show frequency synchronization clock-int brief location 0/rsp0/CPU0

Fri Nov 20 22:09:45.908 UTC

Flags: > - Up D - Down S - Assigned for selection

d - SSM Disabled s - Output squelched L - Looped back

Node 0/RSP0/CPU0:

==============

Fl Clock Interface QLrcv QLuse Pri QLsnd Output driven by

=== ============ ==== ==== == ==== =====================

>SL Sync0 None ST3E 1 n/a n/a

> Sync1 n/a n/a n/a ST3E GigabitEthernet0/4/0/0

>S Internal0 n/a ST3 255 n/a n/a

 

Independent Mode

Similar to default mode, but has the clockinterface loopback detection disabled.

System Mode

Clock-interface loopback detection is disabled.   Also the clock-interface output is now driven by the same frequency source which drives the line

interface output. (i.e T4-SEL-C will always select the output from selection point T0-SEL-B).

 

Output QL

Determine which selection point is being used to drive the output for clock-interfaces or line interfaces on the node (For ASR9000 T0-SEL-B is used to drive line interface output and T4-SEL-C is used to drive clock-interface output).

Check which source has been selected at that selection point, and what the QL is for that source (this is the output QL which would be transmitted).

If the interface is down or SSM is disabled, then no QL can be sent, so the first step is to check for these (alternatively also check if QLsend value on the interface has been overridden by means of configuration).

Note: Timing loop avoidance - if the source is being used to drive its own output, then the output QL is set to DNU.

 

When is the interface output squelched ?

  •   For line interface, when SSM is disabled, output will be squelched.
  •   For clock-interface, output will be squelched (or send AIS) when no input source is available to drive the clock-interface output (i.e selection point T4-SEL-C table has no selected input).
  •   Note: For clock-interface BITS output, T1/E1 mode which support framing will send AIS, else output will be squelched (for 2M/6M mode).
  •   Note: Clock-interface output on the STBY RSP card will always be disabled (until card is in STBY state).

 

 

Show Commands

Show Interfaces:

  show frequency synchronization interfaces [ brief | summary [ location <node> ] | <interface> ]

 

Show Clock-interfaces:

  show frequency synchronization clock-interfaces [ brief ] [ location <node> ]

  show controllers timing Controller clock (cisco-support)

 

Show Selection:

  show frequency synchronization selection [ summary | location <node> ]

 

Show Config-errors:

  show frequency synchronization configuration-errors [ location <node> ]

 

Show Alarms:

  show logging events buffers bistate-alarms-set

 

Show Trace (cisco-support):

  show frequency synchronization trace { fast | slow } [ options ]

  show controllers timing Controller trace [options]

 

Show Tech (cisco-support):

  show tech-support frequency synchronization

 

Clear and Debug commands

Clear ESMC Stats:

  clear frequency synchronization esmc statistics interface { <interface> | all | summary location { <node> | all} }

 

Clear WTR:

  clear frequency synchronization wait-to-restore { interface { <interface> | all } | clockinterface { sync <port-num> location <node> | all } | all }

 

Debug Packets:

  debug frequency synchronization esmc packets [ sent | received ] [ interface <intf> ] [hexdump ]

 

Debug Selection:

  debug frequency synchronization selection [ location <node> ]

 

Debug Support (cisco-support):

  debug frequency synchronization support [ location <node> ]

 

Debug Internal (cisco-support):

  debug frequency synchronization internal module { all | <module> } [ location <node> ]

 

 

 

 

Xander Thuijs CCIE #6775

Principal Engineer ASR9000

 

With thanks to the great Freq Sync XR team!

Comments

Hi Xander!

I am new in synchronization and I have some questions to you. Our company have LTE network which takes synchronization from GPS. Our management wants to migrate to synch. through packet network. Our network consists of ASR9010, 901, MWR2941, ME3600. So, all equipment supports this feature. So,

1. As I understand, to synchronize LTE network I need to provide timing synchronization through 1588 (PTP protocol)

2. What can be the source of time synch.? As I understand it can be GPS or internal oscillator. 

3. Can we provide time synch. with source E1 2048Mb/s or it it only for frequency synch.?

4. Can we use GLONASS or GALILEO systems as backup for GPS?

 

Thanks!

xthuijs
Cisco Employee
Cisco Employee

hi there!

1588 is a very reliable and accurate methodology to distribute time and frequency. for asr9000 it requires a license and typhoon based hardware (because of the timestamping of the packets participating in the PTP protocol).

An alternative for time would be the use of NTP(v4) and an alternative for frequency distribution would be syncE.

All of them supported on asr9000 and XR.

The time source can be coming from the GPS taken into the RSP via the 1pps and 10Mhz ports together, or the RS422 port.

The internal oscillator can be used also, but generally in mobile networks you want to take the clock/time from an "authorative source" and distribute it wide out the network. Hence GPS is the most preferred option or taking the frequency from eg a BITS circuit (e1/t1 based line).

BITS cant be used for time. It can be used to take in or distribute out frequency. The signal is either E1 or T1 based.

I believe that GALILEO is a european based navigation system similar to gPS. So as long as the receiver provides the 10Mhz/1pps or RS422 based outputs then you are all set using that as a source.

Basically you take the clock/time from somewhere, sync the internal clocks to it and then distribute it out. The distribution downstream can be 1588 based or NTPv4/SyncE.

The intake of a clock can come from a circuit or BITS or the gPS ports on the RSP440.

cheers!

xander

Thijs Van Severen
Cisco Employee
Cisco Employee

Hi Xander

I have posted a question that is related to the above

(see https://supportforums.cisco.com/discussion/12406041/configure-gps-sync-asr9006)

 

Bottom line :

I have a ASR9006 that i want to sync to GPS and then use the ASR as stratum 1 NTP server for my lab

I have GPS 1pps and 10Mhz available at the ASR, but ToD will be a problem since the GPS receiver is located in a different part of the lab and i dont have a direct connection to it (but if this is an absolute requirement i might be able to work something out)

 

Is this at all possible ?  and if yes, what would be the best way to go about it ?

Thanks !

 

Grtz

Thijs

xthuijs
Cisco Employee
Cisco Employee

Hi Thijs,

all 3, 1pps, 10Mhz and rj45 need to be connected to the GPS receiver. Technically you can use the frequency only (from 10Mhz) but this is not a supported configuration and the DTI logic will wine about a missing TOD and raise an alarm for it.

I have an open request for this to allow frequency only from GPS: CSCun86423

Note that for the RJ45, only the RS422 is wired to the fpga (not the rs232) and you'll need to use pins 7/8 for that.

regards

xander

Thijs Van Severen
Cisco Employee
Cisco Employee

hi Xander

Thanks for the info, so if i manage to connect the ToD i can create a stratum 1 NTP server on the ASR ?

 

grtz

Thijs

xthuijs
Cisco Employee
Cisco Employee

Yup indeed, I just responded to your discussion thread also with some more detail there.

If you sync the date/time from the gps, you can distribute that clock as a stratum 1 ntp master.

cheers!

xander

mohamed_elkordy
Community Member

Hi Xander,

please advise if I can get the clock source from a channelized SONET interface and use it as an input source for SyncE instead of the clock interface. if yes, can you provide me with the SONET configuration

 

Thanks,

Mohamed

hengkycung
Level 1
Level 1

Hi Xander,

Nice to see you again.

I have a question, not sure if i'm in the right thread to ask this.

So, our customer just installed Pandorafms, and there is a feature called GIS (Geographic Information System). If the feature wanted to be run, on the remote should be installed pandora_agent. Is that possible to install that? Or is there any configuration that can be support/allow this feature, so the Pandorafms can grep data through ASR9k.

Thanks.

Please take a look : http://wiki.pandorafms.com/index.php?title=Pandora:Documentation_en:GIS#The_GIS_Maps

xthuijs
Cisco Employee
Cisco Employee

hi hengky,

if I understood the feature correctly, then there is a piece of sw that needs to be run on the device. now that part can't be done with XR since 3rd party apps cant be installed on the qnx/xr sw OS.

theoretically, you could do it on a Virtual machine of a VSM but I dont really see what the value of this func is.

other than that, the pandora is merely a monitoring tool that can run offbox with snmp doing some network discovery and (basic) performance monitoring via snmp and cli, and that should go fine without issue.

cheers

xander

hengkycung
Level 1
Level 1

Hi Xander,

Yes, right. It should be a piece of sw installed to the device.

I already told them before about that, in ios XR we cannot do installation of 3rd party sw.

If i understand what they need, they only need to know/show the location of their device under Pandora. They wanted to see their device from P router (HQ) to the remote site (other city) in a map.

Is that any feature on ASR9k can support this requirement? can we add configuration or something (MIB) like longitude/latitude to the router?

Thanks.

Rakloman
Level 1
Level 1

Hi Xander,

Are there any changes in SyncE implementation between Trident and Typhoon linecards, ie. RSP-4G & RSP-440?

Thanks.

xthuijs
Cisco Employee
Cisco Employee

hi karlo,

functionally they are operating the same way. although the hardware that uses it is obviously different. both RSP's have a DTI (timing interface) that is used to sync the clock internally and distribute it towards the linecards. the linecards can use that reference clock to clock out the data on the ports.

regards

xander

ituskes
Level 1
Level 1

Hi Xander,

I trying to configure ASR 9000 (RSP440SE and A9K-MOD80-SE) to be

1. Grand Master - source pps, 10Mhz and ToD (RS422)

2. Boundary Clock - source 1588

1. I get error configure the ToD:

clock-interface sync 1 location 0/RSP0/CPU0

port-parameters

  gps-input tod-format gprmc pps-input rs422

!!% Not supported

syncctrl[432]: %PLATFORM-FSYNC-3-CLOCK_PORT_CONFIG_NOT_SUPPORTED_IN_HARDWARE : The configuration for the synchronization port #1 in GPS mode is not supported in hardware.

1. I can not find information how to configure the 1588 port for source.

Does it need license to be able to see/configure?

running 5.3.3 

disk0:asr9k-os-mbi-5.3.3.CSCuz09772-1.0.0/0x100305/mbiasr9k-rsp3.vm

Thanks, Imre

ituskes
Level 1
Level 1


Use source pps, 10Mhz and ToD (RS422) - configure sync 2
sh running-config clock-interface sync 2 location 0/RSP0/CPU0
Thu Sep 8 09:18:26.717 CET
clock-interface sync 2 location 0/RSP0/CPU0
port-parameters
gps-input tod-format gprmc pps-input ttl
!
frequency synchronization
selection input
priority 1
wait-to-restore 0
ssm disable
time-of-day-priority 1
quality receive exact itu-t option 2 generation 2 PRS
!
!

pps-input ttl - Coax input

RS422 - 4800b 8N1

NMEA string format stricked:

$GPRMC,101025,A,4741.0650,N,01737.0714,E,,,08916,,,A*44

wenlzhao
Cisco Employee
Cisco Employee

Hi Xander,

 

As far I know the sync-E also have locked, available & unmonitored statue.

Can I know what exact meaning of these status?

I have treid to check this in standard document, cannot find any comment about this.

 

thanks,

 

/Wenlong.

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Quick Links