Hello Giuseppe,

Thank you for prompt revert. Below is the asked output , PLEASE let me know if anything else is needed from my end.

One question here router does the lable swap 2 times , isn't this strange or this is normal behavior with GRE tunnel

R3#show ver | i WS-C6504-E
cisco WS-C6504-E (R7000) processor (revision 2.0) with 516096K/8192K bytes of memory.

R3#show module
Mod Ports Card Type                              Model              Serial No.
--- ----- -------------------------------------- ------------------ -----------
  1    9  Supervisor Engine 32 8GE (Active)      WS-SUP32-GE-3B     SALXXXXXXXX
  3   48  48-port 10/100/1000 RJ45 EtherModule   WS-X6148A-GE-TX   SALXXXXXXXX

Mod MAC addresses                       Hw    Fw           Sw           Status
--- ---------------------------------- ------ ------------ ------------ -------
  1  0xxx.xxxx.xxxe to 0xxx.xxxx.xxx9   4.4   12.2(18r)SX5 12.2(33)SXI8 Ok
  3  6xxx.xxxx.xxx0 to 6xxx.xxxx.xxxf   4.1   8.4(1)       12.2(33)SXI8 Ok

Mod  Sub-Module                  Model              Serial       Hw     Status
---- --------------------------- ------------------ ----------- ------- -------
  1  Policy Feature Card 3       WS-F6K-PFC3B       SALXXXXXXXX 2.3    Ok
  1  Cat6k MSFC 2A daughterboard WS-F6K-MSFC2A     SALXXXXXXXX 3.1    Ok

Mod  Online Diag Status
---- -------------------
  1  Pass
  3  Pass

Traceroute from R3 to R1 - Success

R3#traceroute 192.168.254.204

Type escape sequence to abort.
Tracing the route to 192.168.254.204

  1 172.16.13.10 [MPLS: Label 453 Exp 0] 8 msec 12 msec 8 msec => R2
  2 172.16.13.198 8 msec *  8 msec                             => R1

Traceroute from R4 to R1 in same loop

R4#traceroute 192.168.254.204

Type escape sequence to abort.
Tracing the route to 192.168.254.204

  1 172.16.13.34 [MPLS: Label 346 Exp 0] 4 msec 4 msec 4 msec  => R4
  2 172.16.13.9 [MPLS: Label 453 Exp 0] 236 msec 232 msec 236 msec => R3
  3 172.16.13.33 [MPLS: Label 389 Exp 0] 236 msec 232 msec 236 msec => R3
  4 172.16.3.33 [MPLS: Label 317 Exp 0] 236 msec 232 msec 236 msec => R4
  5 172.16.0.5 [MPLS: Label 412 Exp 0] 236 msec 232 msec 236 msec
*
*
*

But Trace from R4(or R5 , R6 & R7) to R2 - Able to ping but not traceroute

R4#traceroute 192.168.254.27

Type escape sequence to abort.
Tracing the route to 192.168.254.27

  1 172.16.13.34 [MPLS: Label 542 Exp 0] 4 msec 4 msec 4 msec => R4
  2 172.16.13.9 0 msec 4 msec 0 msec       => R3
  3  *  *  *
  4  *  *  *
  5  *  *  *
  6  *

R4#ping 192.168.254.27

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.254.27, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/4/8 ms

Hello Yagnesh,

from the show module we see that there is only the sup32 supervisor and the 48 ports ethernet linecard, so the GRE tunnel is implemented on the supervisor for sure.

About your question:

>>One question here router does the lable swap 2 times , isn't this strange or this is normal behavior with GRE tunnel 

I don't know if it is correct when a GRE tunnel is involved, but I remember to have seen tracerouote over MPLS where a specific PE node appeared two times. (but it was usually the last one or the penultimate hop)

It can be helpful to review how traceroute works over an MPLS network.

I have found the following document that describes the process:

http://www.cisco.com/c/en/us/support/docs/multiprotocol-label-switching-mpls/mpls/26585-mpls-traceroute.html

According to the document, the originating router starts to send packets with IP TTL=1, but the next router in the path instead of answering back to the original sender forwards the ICMP unreachable message along the path to the final destination of the LSP. It is the final destination of the LSP that then sends back over a different LSP with destination the originator router the ICMP unreachable message(s).

And this should happen at each hop on the path of the LSP to the final destination.

I wonder if the problem is only limited to traceroute you show that the ping works to R2.

>>>R4#ping 10.214.254.27

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.214.254.27, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 4/4/8 ms

Hope to help

Giuseppe

Hello Giuseppe,

From ping command I just want to show you that we can ping R2 here as in that case we are removing label due to paneltimate hope , so its not going to routing loops like R1.

And no this not trace route issue , we can't ping the R1 .

This can be iOS bus?

What should be my next step ?

Thanks for your help so far 

Hello Yagnesh,

>> And no this not trace route issue , we can't ping the R1 .

You have a global connectivity issue for devices downstream the GRE tunnel that requires labeled frames to be sent over the tunnel.

I would open a service request with Cisco TAC to have them investigate the issue.

Of course, you cannot remove the GRE tunnel that allows you to go through a non MPLS IP only network.

If you are using service provider services like an MPLS L3 VPN to interconnect the two parts of your network you can ask the provider a quotation for Carrier supporting Carrier service. This is  to avoid the GRE tunnel.

Hope to help

Giuseppe

Hi Giuseppe ,Thanks for your revert .

We have GRE tunnel over internet & advertising router loopback over it.

Goal here is to connect R1 to route reflecter (R7) for connectivity to vpnv4 cloud.

Will ask Cisco guys & update the forum in case we get resolution.

We got below update from Cisco TAC

MPLS over GRE is not supported officially on PFC3B that equipped the SUP32.

This Feature is limited to SIP-400 with GE SPA as core facing interface and running SRB1 or greater using Sup720 and RSP720.