I would like to know if anyone knew of a way to get sub 1s convergence for MPBGP in case of a PE failure ?
Right no I can get a convergence time of around 4-5s.
The cust lan has 2 connection to the mpls network.
The dely come fro mthe fact that the VRF routing table is not updated with the backup route as soon as the route to the failed Pe is removed from the core igp.
Thanks for any input.
HI, [Pls Rate if HELPS]
With Primary & Secondary Link is Immediate. Say if your Primary running on BGP towards the Customer is DOWN, Secondary is STATIC, then immediately your Secondary Routes will be available in Routing Table. This will involve only the MAX of 2 / 3 drops (ie., Packet Lost).
Between your PE's you will have only one BGP Session running, so that the Convergence will actually depend on No.. of Routes on BGP Table, Connectivity between th PE's. As per my understanding & knowledge, it should be less than "30" secs.
Hope I am Informative.
Pls Rate if HELPS
Guru Prasad R
I will continue to research this issue, i am down to 0.6-1.1 s but will have see if I can get to no packet loss somehow.
using tracking of the bgp routes
track 1 ip route 0.0.0.0 0.0.0.0 reachability
ip vrf TEST
delay up 20 (delay is used to wait for bgp and ldp to converge before sending traffic on it)
standby timers msec 200 msec 800
bgp scan-time import 5
bgp scan-time 5
the connection setup is fairly simple,
two routers at customer with 1 hsrp using those timers, tracking the bgp route
each router is connected to the mpls core (mpls to the cpe)
you need to use RD/router/vrf so that the routes from your cpe are seen 2 time with different RD and can be swapped between them easily.
we are using RR but we nee to change someting as we need to make sure the bgp can import 2 route (command I don't remember. if you give me your e-amil addres I can send you some more detailled configs
can you please send me confis for test topology. I am working for PE redeudancy for our MPLS network. Here each CE is connected to Redundanat PE on each side. I am just wondering how can I acheive subsecond convergence for PE failure. Because for VPN traffic convergence my MP-BGP convergence comes into picture. And this convergence should run into seconds taking into account bgp scan times, bgp adevtisement intervals.....
Can you pls help with your configs for a test topology.
In your set-up who is tracking the default route and which feature is using it ?
Why do you mean by "mpls to the CPE" ? Did you configure CSC ?
Anyway to improve convergence time during PE failure in a MPLS-VPN backbone the key is your IGP.
You need to tune your IGP for sub-second convergence and be sure your PEs/RR support BGP Next-Hop-Tracking and BGP selective NHT if you have a default/summary route in your IGP.
If your IGP converge in less than one second, you can decrease the BGP NHT delay from 5s (default) to 1s.
As you already said to bypass the import scanner it's important to have RD per VRF so remote PEs will have both routes in their VRF BGP table.
Also if it's not the default yet, set the MRAI of your PE-CE eBGP and MP-IBGP session to 0s.
To achieve 0 packet lost, you should look at BGP local protect feature (7600 only):
In any case, you need also to be sure your CEs converge quickly as well.
Finally, it's not recommended to play with the scanner and import timer it could seriously impact the CPU.
I am concerned for my MP-iBGP convergence for my end to end VPN routes for PE failure scenario. How much is the min & max times for this convergence that I can acheive. Which method should I use???
My CE is connected to Redundant PE on both the ends & I am using BFD based convergence for my PE-CE routing protocol.
Forget the BGP local protect as it covers PE-CE link failure and not PE failure.
Sorry for the mistake.
So everything else in my previous post is still valid to improve convergence time in case of PE failure:
1- Tune your IGP for fast-convergence
2- Tune BGP: NHT feature, MRAI set to 0,..
3- Use one RD per vrf and per PE to avoid the import scanner on the remote PE
The end2end convergence time depends also on the time your CE needs to converge, the number of routes in the VRF,.. but if you implement all those steps, you should be able to converge in few seconds.
can you please explain point 3. I am not able to understand how will it improve convergence.
ie. Use one RD per vrf and per PE to avoid the import scanner on the remote PE
Using On RD per vrf/pe mean that when you export routes from redundant routers, they will be propagater as 2 saparate route in your mpls backbone -> bgp will be able to use the alternate route faster.
RD is prepended to the IPv4 update to create a unique VPNv4 update.
When two PE are connected to the same site, they will export the same IPv4 update.
1- If they use the same RD, they will create the same VPNv4 update and the RR will select only one as best and propagate it to the remote PE.
If the PE selected as best by the RR failed, you loose time as you need to wait for the RR to send the other update. Then on the remote PE you need to wait for the import scanner (every 15s) which will import the translated VPNv4 to IPv4 routes into the VRF.
If you use the one RD per PE per VRF, the RR sees two different VPNv4 updates even if it's the same IPv4 prefix. Remote PEs will install both IPv4 routes into the VRF BGP table and select the best one to install into the VRF RIB.
If the selected PE as best failed, remote PE will converge as soon as they received a withdraw from the RR of even quicker if BGP NHT is supported.
As you said using unique RD per VRF going to help convergance much better and also avoid import scan delay.
We have been discussing this for our network where we have PE1-CE1 (primary) and PE2-CE2(backup), we make that backup using AS path prepand.
So I belive if i use same RD , PE2 will select best path as PE1 due to BGP path selection algorithm (AS path) and if PE1-CE1 link fails due to same RD you also wait complete withdrawl process to happen and then PE2 remove that and selects local as best and then again advertise the same ( again MRIA comes in to picture).
So, My argument was if we use unique RD all this above avoided and remote PE install alternate backup path in FIB quickly once it get withdraw from PE1 due to PE-CE link fail. PE1 failure i guess BGP NH will help better.
Now argument to this was amount of memory utilization due to unique RD per VRF , i had impression that FIB ( in line card) still will have one path only and RIB on RP assuming 12K always have much higher memory so should not be major concern.
Is my understanding valid ? or you have anything to add. You inputs are highly apprcieated.
In your case, unique RD doesn't resolve your issue because PE2 will prefer its iBGP route from PE1 over its eBGP one received from CE2.
So as a result PE2 will not send its eBGP route to the RR so the remote PE will receive only one route even if you are using a unique RD.
To be useful, PE1 and PE2 must prefer their eBGP route, so the RR will receive both of them and forward both of them (thanks to unique RD which make the VPNv4 prefix unique)
On remote PE, it will increase the VRF memory as you store more routes but 12K scale well so except if you want the full routing in all your VRF's, you should be fine.
Regarding your question about BGP local convergence, it will be supported in XR via the BGP PIC edge feature. Please contact your Cisco account team for more details about the road-map.
In my case uf we use unique RD, shouldn't PE2 select local ( learned by EBGP) and routes learned by PE1 (via iBGP)as they are still two different VPNV4 prefix ( due to unique RD) ?
And regarding remote PE, it increase VRF memory which is RP but not memory (FIB)on line card -for example on 12K ? Is that correct ?
Thanks for clarification on BGP local convergance, will chasse Account team on the same.
RD is not involved in BGP best path selection. If you're doing as-prepend on CE2, PE2 will prefer it's iBGP routes.
Using unique RD is only useful for RR which will forward all the VPNv4 prefixes as they are unique.
RIB and FIB contain only best path so you're correct. It's the BGP table which will increase
Thanks for clarification. so In vpn if i want one side primary and other backup what is best way ? - I should not loose advantage of unique RD.
And as you said RIB and FIB contain best path and BGP table increase-that is on RP only right no impact on memory of line card ? - just to ensure.
What you should do is to define two standard BGP communities:
C1 for nominal routes and C2 for backup routes.
You configure PE1 and PE2 to add one of those communities to the route they are exporting. Then you configure the remote PE to set a different LP based on the community via an import-map.
Is there anytoher technique under BGP timer besides Hold timer and Keepalive that can improve convergence in MPLS L3 VPN network.
I know about BFD, some work is going on under BGP timer side to improve convergence
It's not just a question of timers and it's not recommended to configure aggressive BGP timers as it will increase CPU processing on the RR if you have many PEs. It could also lead to session flapping.
The mechanism you can use to improve MPLS-VPN convergence time depends of the type of failure:
1- A core link failed: Tune your IGP for fast failure detection and fast-convergence. MPLS-TE FRR is also a solution. BGP is not involved.
Use LDP-IGP sync feature to avoid blackholing MPLS-VPN traffic during Down-UP event.
2- PE failure: We don't use the global scanner (every 60s) anymore for BGP peer failure detection. We have a new mechanism which is event driven called BGP Next-Hop Tracking.
So now your detection time is based on the time for your IGP to converge + BGP NHT initial delay (5s by default)
3- PE-CE link failure or CE failure for a dual attached site.
Convergence is based on fast link failure detection and on BGP convergence in the backbone. To improve the BGP part:
- Use one RD per vrf and per PE so the remote PEs will receive and install in their local vrf BGP RIB both routes (you bypass this way the import scanner)
If your PE is a 7600, you can use the following BGP local convergence feature:
Fast-convergence is a big subject involving many techniques so you should not focus on one feature only.
Have you heard of VPN FRR. Huawei devices are already supporting this feature. They are using this feature for acheiving <200 ms convergence time during PE node failure secanrios.
Actually we are designing MPLS backbone for GSM voice traffic. We are implementing MPLS TE FRR for core link & P node failure to acheive convergence time around 200ms. But I am concerned for PE node failure scenarions. According to me convergence time should happen in seconds & not ms for PE node failure scenario. To address this Huawei came up with VPN FRR solution. Wanted to know whether VPN FRR is in cisco's roadmap. Or does cisco also support it in some IOS.
FRR is supported for a long time in IOS, this is a Traffic Engineering feature.
FRR is not a solution for PE crash it is more used for Link failure and intermediate P/PE router failure.
Endpoint PE failure is not a good case for Fast reroute I think.
more info here :
on a side note, frr can be also triggered by BFD.
Hope it helps
TE FRR is different than VPN FRR. VPN FRR is specifically designed for PE failures but it is not yer developed for Cisco.
Only Huawei supports VPN FRR i believe.
Does BGP local convergance feature avilable in 12K (IOS/IOS-XR)?
I see that it is like per CPE one lable than per prefix label for VPN.
I m not sure how this will be applicable to your scenario. but if the links are POS u can use the loss of signal (LOS) to detect the failure and it can trigger igp convergence more rapidly