Cant ping spoke to spoke IPSEC/GRE Deployment

Answered Question
Feb 21st, 2010

Greetings,  i have a hub and spoke IPSEC/GRE configuration connecting together 5 routers. I can ping from spoke to spoke when connected to each spoke router but if i add the inside interface as the source address i cant ping between spokes.


I think it may be a NAT or Crypto issue but any assistance would be much appreciated, yes i could create seperate tunnels between spokes or use DMVPN but the customer wants all traffic to go via the HUB site.


Ping another spoke router

#ping 192.168.20.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.20.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 120/123/124 ms

Traceroute to another spoke router


#traceroute 192.168.20.1

Type escape sequence to abort.
Tracing the route to 192.168.20.1

  1 10.255.255.13 56 msec 56 msec 52 msec
  2 10.255.255.2 116 msec *  116 msec

Ping another spoke router with source-interface statement

#ping 192.168.20.1 source vlan 1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.20.1, timeout is 2 seconds:
Packet sent with a source address of 192.168.50.1
.....
Success rate is 0 percent (0/5)


Show IP Route on spoke router with LAN IP 192.168.50.1

bhx-ro-877#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is 0.0.0.0 to network 0.0.0.0

D    192.168.30.0/24 [90/2882560] via 10.255.255.13, 01:35:17, Tunnel0
     81.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C       81.**.**.**/32 is directly connected, Dialer1
C       81.**.**.**/24 is directly connected, Dialer1
D    192.168.10.0/24 [90/1602560] via 10.255.255.13, 02:46:54, Tunnel0
D    192.168.40.0/24 [90/2882560] via 10.255.255.13, 02:46:54, Tunnel0
D    192.168.20.0/24 [90/2882560] via 10.255.255.13, 02:46:54, Tunnel0
     10.0.0.0/30 is subnetted, 4 subnets
D       10.255.255.8 [90/2880000] via 10.255.255.13, 02:46:54, Tunnel0
C       10.255.255.12 is directly connected, Tunnel0
D       10.255.255.0 [90/2880000] via 10.255.255.13, 02:46:54, Tunnel0
D       10.255.255.4 [90/2880000] via 10.255.255.13, 02:46:54, Tunnel0
C    192.168.50.0/24 is directly connected, Vlan1
S*   0.0.0.0/0 is directly connected, Dialer1

Crypto Map and Tunnel Interface on Spoke Router - 192.168.50.1

interface Tunnel0
bandwidth 8000
ip address 10.255.255.14 255.255.255.252
ip mtu 1420
ip nbar protocol-discovery
qos pre-classify
tunnel source Dialer1
tunnel destination 81.**.**.**
tunnel path-mtu-discovery
crypto map cryptomap

!

crypto map cryptomap 1 ipsec-isakmp
description ****** Link to HUB ******
set peer 81.**.**.**
set security-association lifetime seconds 86400
set transform-set 3DesL2L
set pfs group2
match address 101

!

access-list 100 remark ****** NAT ACL ******
access-list 100 deny   ip 192.168.50.0 0.0.0.255 192.168.0.0 0.0.255.255
access-list 100 deny   ip 192.168.50.0 0.0.0.255 172.31.255.0 0.0.0.255
access-list 100 permit ip 192.168.50.0 0.0.0.255 any
access-list 101 remark ****** Link to hqhnswth-ro-877 ******
access-list 101 permit ip 192.168.50.0 0.0.0.255 192.168.0.0 0.0.255.255

!

route-map nonat permit 1
match ip address 100

!

ip nat inside source route-map nonat interface Dialer1 overload

!

router eigrp 1
passive-interface Dialer1
passive-interface Vlan1
network 10.255.255.0 0.0.0.255
network 192.168.50.0
no auto-summary
eigrp stub connected summary

Crypto Map and Tunnel Interface on HUB Router - 192.168.10.1

interface Tunnel3
bandwidth 8000
ip address 10.255.255.13 255.255.255.252
ip mtu 1420
ip nbar protocol-discovery
qos pre-classify
tunnel source Dialer1
tunnel destination 81.**.**.**
tunnel path-mtu-discovery
crypto map hqmap

!

interface Tunnel0
bandwidth 8000
ip address 10.255.255.1 255.255.255.252
ip mtu 1420
tunnel source Dialer1
tunnel destination 217.**.**.**
tunnel path-mtu-discovery
crypto map hqmap

!

crypto map hqmap 1 ipsec-isakmp
description ****** Link to 192.168.20.0/24******
set peer 217.**.**.**
set security-association lifetime seconds 86400
set transform-set 3DesL2L
set pfs group2
match address 101
!
crypto map hqmap 4 ipsec-isakmp
description ****** Link to 192.168.50.0/24 ******
set peer 81.**.**.**
set security-association lifetime seconds 86400
set transform-set 3DesL2L
set pfs group2
match address 104

!

access-list 100 remark ****** NAT ACL ******
access-list 100 deny   ip 192.168.10.0 0.0.0.255 192.168.0.0 0.0.255.255
access-list 100 permit ip 192.168.10.0 0.0.0.255 any

!
access-list 101 remark ****** Link to 192.168.20.0/24 ******
access-list 101 permit ip 192.168.10.0 0.0.0.255 192.168.20.0 0.0.0.255

!
access-list 104 remark ****** Link to 192.168.50.0/24 ******
access-list 104 permit ip 192.168.10.0 0.0.0.255 192.168.50.0 0.0.0.255

!

ip nat inside source route-map nonat interface Dialer1 overload

!

router eigrp 1
passive-interface Dialer1
passive-interface Vlan1
network 10.255.255.0 0.0.0.255
network 192.168.0.0 0.0.255.255
no auto-summary

Show IP Route on HUB Router


#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is 0.0.0.0 to network 0.0.0.0

D    192.168.30.0/24 [90/1602560] via 10.255.255.6, 01:45:27, Tunnel1
     81.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C       81.**.**.**/24 is directly connected, Dialer1
C       81.
**.**.**//32 is directly connected, Dialer1
C    192.168.10.0/24 is directly connected, Vlan1
D    192.168.40.0/24 [90/1602560] via 10.255.255.10, 1d23h, Tunnel2
D    192.168.20.0/24 [90/1602560] via 10.255.255.2, 06:11:01, Tunnel0
     10.0.0.0/30 is subnetted, 4 subnets
C       10.255.255.8 is directly connected, Tunnel2
C       10.255.255.12 is directly connected, Tunnel3
C       10.255.255.0 is directly connected, Tunnel0
C       10.255.255.4 is directly connected, Tunnel1
D    192.168.50.0/24 [90/1602560] via 10.255.255.14, 02:57:08, Tunnel3
S*   0.0.0.0/0 is directly connected, Dialer1

Ping spoke from HUB Router

#ping 192.168.20.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.20.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 72/72/76 ms

Ping spoke from HUB Router using "source-interface" statement

#ping 192.168.20.1 source vlan 1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.20.1, timeout is 2 seconds:
Packet sent with a source address of 192.168.10.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 72/72/76 ms

I have this problem too.
0 votes
Correct Answer by Lei Tian about 6 years 9 months ago

Hi Mark,

/* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;}

Yes, you can apply ACL on tunnel interface to deny certain type of traffic on either direction.

/* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;}

After read your requirement, I think reflexive acl might be a good fit here. Say provider site is spoke 1 and the customer site is spoke 2; any traffic originate by spoke 1 and return traffic can pass, but traffic originate by spoke 2 would be blocked. Here is  the configuration guide if you are interested with reflexive acl.

http://www.cisco.com/en/US/docs/ios/sec_data_plane/configuration/guide/sec_cfg_ip_filter_ps6350_TSD_Products_Configuration_Guide_Chapter.html#wp1000897

HTH,

Lei Tian

Correct Answer by Lei Tian about 6 years 9 months ago

Hi Mark,

What you have seen is the correct behavior. In Phase 3, spoke will always point to hub as the next-hop in the routing table; what changed is the nhrp map. If you goto the spoke router and show ip nhrp brief, you will notice there will be one entry for remote spoke tunnel's (10.255.255.2 in your case), and another entry for remote spoke's LAN subnet (192.168.20.0 in your case). The later entry will be used when traffic send to 192.168.20.0 subnet.

And because of this feature, phase 3 allows you on hub router only advertise summary route to spokes.

HTH,

Lei Tian

Correct Answer by Lei Tian about 6 years 9 months ago

/* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;}

Hi Mark,

As I mentioned you still have "no ip next-hop-self eigrp 1" on your hub; you need remove that statement for phase 3 to work properly.


If i remove the statement " no ip next-hop-self eigrp 1" ie change it to " ip next-hop-self eigrp 1" the configuration becomes Phase2 and all spokes talk to one another via the HUB. Although when i removed the statement i dont think i allowed time for the tunnel interface to be reset so ill try that later.


/* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;}

Actually, it is the other way around. with "no ip next-hop-self eigrp 1" it will work as phase 2, and phase 3 do not need that statement on hub. After established  the dynamic tunnel, traffic from spoke to spoke will be sent via the ipsec tunnel and will not process by hub router. Be aware remove "no ip next-hop-self eigrp 1" on hub router will reset all eigrp neighbors.

These are little difference between phase 3 and phase 2. When you send traffic from spoke 1 to spoke 2,in phase 2 ipsec tunnel is initialed by spoke 1; in phase 3 ipsec tunnel is initialed by spoke 2. From user point of view, you might not even notice the difference. What matter is in phase 2, before spoke to spoke tunnel established, traffic will be process switched by hub router; in phase 3 all traffic are cef switched. That gives phase 3 a big plus when you have lot spokes. Phase 3 also build up spoke to spoke tunnel quicker than phase 2. You can verify that by doing ping from spoke 1 to spoke 2 with phase 2 and phase 3, and compare the number of pkt been encrypted from "show crypto ipsec sa".

In term of best practice, it doesn’t hurt to put spoke as eigrp stub network, but it wont help anything unless you have dual hub. For dual hub situation, you want put spoke as stub network so it won’t become transit site.

This is a good document for migrating DMVPN phase 2 to phase 3; you can find some best practices in the doc as well.

http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6586/ps6660/ps6808/prod_white_paper0900aecd8055c34e_ps6658_Products_White_Paper.html

HTH,

Lei Tian

Correct Answer by dbass about 6 years 9 months ago

It looks like your original problem is with your crypto ACL.  What you are trying to do is encrypt GRE traffic from the remote site to the hub and send the actual user data within the GRE tunnel.  So, this is what needs to happen: the router gets a packet from 192.168.50.x that is destined to 192.168.20.x.  It looks up the route and determines the next hop is the other end of the GRE tunnel, and should forward it out that interface.  When it forwards it out that interface it encapsulates the original packet with GRE and sends it to the physical interface to be transmitted on to the wire.  That is when the crypto engine examines the packet and determines that it should be encrypted, which it then encrypts and transmits the packet. Now, because your crypto ACL is wrong the router is conflicted in whether to encrypt the original packet in IPSEC or GRE (then IPSEC), and dropping it.  I would be willing to bet that if you do a "sh cry ips sa" on both ends that you will see incrementing send errors or encap/decap errors if there is even an SA established.  Hope I explained it in a manner that helps you understand it better.

The reason why doing a straight ping works, but an extended (with sourced from the internal network) doesn't also directly ties to the incorrect crypto ACL.  The regular ping will be sent with a source address of 10.255.255.14, which is not referenced in the crypto ACL so there is no conflict.

The correct crypto ACL would have a source address of the public address on the dialer interface (outside interface) and destination of the public address on the hub site, and the opposite for the hub site crypto ACL.  You can specify GRE as the protocol (ie: permit gre host x.x.x.x host x.x.x.x), or  all IP (ie: permit ip host x.x.x.x host x.x.x.x) depending on what you prefer.  Sometime's it helps to be able to ping the public address with it not being encrypted, which is why I prefer to just specifically do the GRE only traffic.

One thing to remember for future use as well...GRE/IPSEC and DMVPN is the same technology at heart, so if you cannot get the former working you will run in to the same problems with the later.  In fact the later is more complicated of a config and technology, so more likely it will be harder to get up without a thorough understanding of the technology.  This goes for any technology...ie, if you can't get EIGRP running between 2 routers switching to BGP is probably not going to get you very far.

Correct Answer by Lei Tian about 6 years 9 months ago

st1\:*{behavior:url(#ieooui) } /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;}

Hi Mark,

It looks about right. Just 2 things need to change

1, change the ipsec mode to transport instead of tunnel.

2, remove network 0.0.0.0 statement from eigrp process, you don’t want advertise your tunnel end point thought tunnel; otherwise you will have recursive lookup.

HTH,

Lei Tian

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 5 (8 ratings)
Loading.
Lei Tian Sun, 02/21/2010 - 09:51

Hi Mark,

Can you post "show ip route" output from the other spoke?

Thanks

Lei Tian

Mark Rigby Sun, 02/21/2010 - 09:56

Thank you for your post, please see the output from a second spoke, this is the one im pinging in the above examples with LAN IP 192.168.20.1.

Regards

#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is 0.0.0.0 to network 0.0.0.0

D    192.168.30.0/24 [90/2882560] via 10.255.255.1, 02:06:42, Tunnel0
     81.0.0.0/32 is subnetted, 1 subnets
C       81.**.**.** is directly connected, Dialer1
D    192.168.10.0/24 [90/1602560] via 10.255.255.1, 06:32:16, Tunnel0
D    192.168.40.0/24 [90/2882560] via 10.255.255.1, 06:32:16, Tunnel0
C    192.168.20.0/24 is directly connected, Vlan1
     10.0.0.0/30 is subnetted, 4 subnets
D       10.255.255.8 [90/2880000] via 10.255.255.1, 06:32:16, Tunnel0
D       10.255.255.12 [90/2880000] via 10.255.255.1, 06:32:16, Tunnel0
C       10.255.255.0 is directly connected, Tunnel0
D       10.255.255.4 [90/2880000] via 10.255.255.1, 06:32:16, Tunnel0
D    192.168.50.0/24 [90/2882560] via 10.255.255.1, 03:18:22, Tunnel0
C    217.**.**.**24 is directly connected, Dialer1
S*   0.0.0.0/0 is directly connected, Dialer1

Lei Tian Sun, 02/21/2010 - 11:49

Hi Mark,

The routing looks fine. I suspect your ICMP traffic is not been encrypted at all. If you issue "show crypto ipsec sa" on your spoke do you see number of packets been encrypted and decrypted?

Thanks

Lei Tian

Lei Tian Sun, 02/21/2010 - 11:51

Hi Mark,

Just noticed one thing, the interested ACL on your hub site is not same as the ACL on your spoke router.

Please change the ACL 101 to

access-list 101 per ip 192.168.0.0 0.0.255.255 192.168.20.0 0.0.0.255

ACL 104 to

access-list 104 per ip 192.168.0.0 0.0.255.255 192.168.50.0 0.0.0.255

HTH,

Lei Tian

Mark Rigby Mon, 02/22/2010 - 02:31

Thank you Lei, i shall try changing the statements on the HUB router later on today and let you know how we get on.


Regards


Lei Tian Mon, 02/22/2010 - 07:02

st1\:*{behavior:url(#ieooui) } /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;}

You welcome Mark.

As a note here, when using GRE+IPSEC, it is easier apply crypto-map on the physical interface and configure the ACL to permit GRE traffic between tunnel endpoints.

Another way to do it is as p.bevilacqua mentioned uses phase 1 DMVPN for hub and spoke only network.

HTH,

Lei Tian

Mark Rigby Mon, 02/22/2010 - 07:53

Thank for the replies guys, yes i think it's time to go down the DMVPN route rather than trying to get a non standard configuration working properly.


Can i ask what the difference is between DMVPN Hub and Spoke and DMVPN Full Mesh? Would i be correct in making the following assumptions.


  • DMVPN Hub and Spoke - All traffic goes via the hub site?
  • DMVPN Full Mesh - Spoke routers can initiate runnels with other spoke routers using the hub to determine the IP address of the second spoke router?


Regards

Mark Rigby Mon, 02/22/2010 - 08:54

Cheers this is my full mesh config so far for the HUB site and a single spoke so far. Just need to add a dynamic cryto map for VPN clients.


! HUB SITE !

crypto isakmp policy 1
hash sha
authentication pre-share
encryption 3des
group 2
lifetime 86400
!
crypto isakmp key hhDT8Fv1ZcM3T address 0.0.0.0 0.0.0.0
!
crypto ipsec transform-set 3DesSecure esp-3des esp-sha-hmac
mode tunnel
!
crypto ipsec profile dmvpn
set transform-set 3DesSecure
set security-association lifetime seconds 86400
set security-association lifetime kilobytes 4608000
!
interface Tunnel 0
description ****** DMVPN GRE Tunnel ******
ip address 10.255.255.1 255.255.255.0
bandwidth 1000
delay 1000
ip nhrp holdtime 360
ip nhrp network-id 100000
ip nhrp authentication 1zhGo5W3m26Cr
ip mtu 1400
ip tcp adjust-mss 1360
ip nhrp map multicast dynamic
tunnel source Dialer1
tunnel mode gre multipoint
tunnel key 100000
tunnel protection ipsec profile dmvpn
no ip split-horizon eigrp 1
no ip next-hop-self eigrp 1
!
router eigrp 1
network 10.255.255.1 0.0.0.0
network 0.0.0.0 255.255.255.255
no auto-summary
!
interface Dialer1
description ****** Outside Interface ******
ip nat outside
!
interface Vlan1
description ****** Inside Interface ******
ip nat inside
!
ip nat inside source list 100 interface Dialer1 overload
!
access-list 100 remark ****** NAT ACL ******
access-list 100 deny ip 192.168.0.0 0.0.255.255 192.168.0.0 0.0.255.255
access-list 100 permit ip 192.168.10.0 0.0.0.255

!

! SPOKE SITE !

crypto isakmp policy 10
hash sha
authentication pre-share
encryption 3des
group 2
lifetime 86400
!
crypto isakmp key hhDT8Fv1ZcM3T address 0.0.0.0 0.0.0.0
!
crypto ipsec transform-set 3DesSecure esp-3des esp-sha-hmac
mode tunnel
!
crypto ipsec profile dmvpn
set transform-set 3DesSecure
set security-association lifetime seconds 86400
set security-association lifetime kilobytes 4608000
!
interface Tunnel 0
description ****** DMVPN GRE Tunnel ******
ip address 10.255.255.2 255.255.255.0
bandwidth 1000
delay 1000
ip nhrp holdtime 360
ip nhrp network-id 100000
ip nhrp authentication 1zhGo5W3m26Cr
ip mtu 1400
ip tcp adjust-mss 1360
ip nhrp nhs 10.255.255.1
ip nhrp map multicast **HUB External IP Address **
ip nhrp map 10.255.255.1
**HUB External IP Address **
tunnel source Dialer1
tunnel mode gre multipoint
tunnel key 100000
tunnel protection ipsec profile dmvpn
!
router eigrp 1
network 10.255.255.2 0.0.0.0
network 0.0.0.0 255.255.255.255
no auto-summary
!
interface Dialer1
description ****** Outside Interface ******
ip nat outside
!
interface Vlan1
description ****** Inside Interface ******
ip nat inside
!
ip nat inside source list 100 interface Dialer1 overload
!
access-list 100 remark ****** NAT ACL ******
access-list 100 deny ip 192.168.0.0 0.0.255.255 192.168.0.0 0.0.255.255
access-list 100 permit ip 192.168.20.0 0.0.0.255

Correct Answer
Lei Tian Mon, 02/22/2010 - 09:18

st1\:*{behavior:url(#ieooui) } /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;}

Hi Mark,

It looks about right. Just 2 things need to change

1, change the ipsec mode to transport instead of tunnel.

2, remove network 0.0.0.0 statement from eigrp process, you don’t want advertise your tunnel end point thought tunnel; otherwise you will have recursive lookup.

HTH,

Lei Tian

Mark Rigby Mon, 02/22/2010 - 09:22

Thank you for the recommendations, would you suggest advertising the network 192.168.0.0/16 as all the sites fall under this range?


Regards

Lei Tian Mon, 02/22/2010 - 09:33

st1\:*{behavior:url(#ieooui) } /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;}

Hi Mark,

192.168.0.0/16 is fine. The bottom line is only advertising the protected LAN segment, 192.168.0.0 on your case and tunnel IP 10.255.255.0 on your case through tunnel.

HTH,


Lei Tian

Correct Answer
dbass Mon, 02/22/2010 - 13:30

It looks like your original problem is with your crypto ACL.  What you are trying to do is encrypt GRE traffic from the remote site to the hub and send the actual user data within the GRE tunnel.  So, this is what needs to happen: the router gets a packet from 192.168.50.x that is destined to 192.168.20.x.  It looks up the route and determines the next hop is the other end of the GRE tunnel, and should forward it out that interface.  When it forwards it out that interface it encapsulates the original packet with GRE and sends it to the physical interface to be transmitted on to the wire.  That is when the crypto engine examines the packet and determines that it should be encrypted, which it then encrypts and transmits the packet. Now, because your crypto ACL is wrong the router is conflicted in whether to encrypt the original packet in IPSEC or GRE (then IPSEC), and dropping it.  I would be willing to bet that if you do a "sh cry ips sa" on both ends that you will see incrementing send errors or encap/decap errors if there is even an SA established.  Hope I explained it in a manner that helps you understand it better.

The reason why doing a straight ping works, but an extended (with sourced from the internal network) doesn't also directly ties to the incorrect crypto ACL.  The regular ping will be sent with a source address of 10.255.255.14, which is not referenced in the crypto ACL so there is no conflict.

The correct crypto ACL would have a source address of the public address on the dialer interface (outside interface) and destination of the public address on the hub site, and the opposite for the hub site crypto ACL.  You can specify GRE as the protocol (ie: permit gre host x.x.x.x host x.x.x.x), or  all IP (ie: permit ip host x.x.x.x host x.x.x.x) depending on what you prefer.  Sometime's it helps to be able to ping the public address with it not being encrypted, which is why I prefer to just specifically do the GRE only traffic.

One thing to remember for future use as well...GRE/IPSEC and DMVPN is the same technology at heart, so if you cannot get the former working you will run in to the same problems with the later.  In fact the later is more complicated of a config and technology, so more likely it will be harder to get up without a thorough understanding of the technology.  This goes for any technology...ie, if you can't get EIGRP running between 2 routers switching to BGP is probably not going to get you very far.

Paolo Bevilacqua Mon, 02/22/2010 - 13:35

It looks like your original problem is with your crypto ACL. ...

Rated this post. Thank you for taking the time for explaining things.

The forum needs posts like your and not the too frequent noise by removed wannabes.

Lei Tian Mon, 02/22/2010 - 16:35

Hi p.bevilacqua,

I don't know what did I do to make you say so, but I got your point here. I will try to explain more if it is needed.

Thanks,

Lei Tian

Mark Rigby Mon, 02/22/2010 - 16:47

Wow that was easier than i expected, Phase 3 DMVPN up and running! Just added the following commands to the hub config and all is well, thank you for your help Lei.


Regards


ip nhrp shortcut
ip nhrp redirect


Spoke Router:

#sh ip nhrp summary  
IP NHRP cache 4 entries, 1312 bytes
    1 static  3 dynamic  0 incomplete

sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is 0.0.0.0 to network 0.0.0.0

D    192.168.30.0/24 [90/3074560] via 10.255.255.3, 00:06:30, Tunnel0
     81.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C       81.**.**.**/32 is directly connected, Dialer1
C       81.**.**.0/24 is directly connected, Dialer1
D    192.168.10.0/24 [90/2818560] via 10.255.255.1, 00:06:30, Tunnel0
D    192.168.40.0/24 [90/3074560] via 10.255.255.4, 00:06:30, Tunnel0
D    192.168.20.0/24 [90/3074560] via 10.255.255.2, 00:06:30, Tunnel0
     10.0.0.0/24 is subnetted, 1 subnets
C       10.255.255.0 is directly connected, Tunnel0
C    192.168.50.0/24 is directly connected, Vlan1
S*   0.0.0.0/0 is directly connected, Dialer1

Hub Router

#sh ip nhrp summary
IP NHRP cache 4 entries, 1312 bytes
    0 static  4 dynamic  0 incomplete

#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is 0.0.0.0 to network 0.0.0.0

D    192.168.30.0/24 [90/2818560] via 10.255.255.3, 00:07:42, Tunnel0
     81.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C       81.**.**.0/24 is directly connected, Dialer1
C       81.**.**.**/32 is directly connected, Dialer1
C    192.168.10.0/24 is directly connected, Vlan1
D    192.168.40.0/24 [90/2818560] via 10.255.255.4, 00:07:43, Tunnel0
D    192.168.20.0/24 [90/2818560] via 10.255.255.2, 00:07:43, Tunnel0
     10.0.0.0/24 is subnetted, 1 subnets
C       10.255.255.0 is directly connected, Tunnel0
D    192.168.50.0/24 [90/2818560] via 10.255.255.5, 00:07:41, Tunnel0
S*   0.0.0.0/0 is directly connected, Dialer1

Lei Tian Mon, 02/22/2010 - 19:12
/* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;}

Hi Mark,

I am glad that I can help you. One thing I noticed you still have "no ip next-hop-self eigrp 1" under your hub tunnel interface. The show ip route output on spokes points to remote site spoke router as next-hop. You might want to remove "no ip next-hop-self eigrp 1" on hub router.

With " no ip next-hop-self eigrp1" on hub router, spoke's NHRP resolution request will be processed like phase 2. Which is spoke 1 send nhrp request for spoke 2; hub find the NHRP mapping in its database and reply spoke 1; spoke 1 initial the tunnel with spoke 2. Before the spoke to spoke tunnel established, all traffic will through hub.

For phase 3 to work properly you need to remove "no ip next-hop-self eigrp1" on hub router. So the hub router instead of doing NHRP map lookup and reply NHRP request, it just redirect the NHRP request to the remote spoke.

HTH,

Lei Tian

Mark Rigby Tue, 02/23/2010 - 02:56

Thank you Lei, this is the tunnel configuration i have for the HUB at present, i have tested spoke to spoke connectivtiy with ICMP. With first set of requests go via the HUB, the second go direct to the other spoke.


If i remove the statement " no ip next-hop-self eigrp 1" ie change it to " ip next-hop-self eigrp 1" the configuration becomes Phase2 and all spokes talk to one another via the HUB. Although when i removed the statement i dont think i allowed time for the tunnel interface to be reset so ill try that later.


Also is it advisable to add the "EIGRP STUB" command to spoke routers?

Your advice is much appreciated.

Initial Ping

#ping 192.168.20.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.20.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 100/121/132 ms

Second Ping

#ping 192.168.20.1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.20.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 68/70/72 ms

These sites are DSL so the response time is somewhat higher.

HUB Tunnel Config

interface Tunnel0
description ****** DMVPN GRE Tunnel ******
bandwidth 1000
ip address 10.255.255.1 255.255.255.0
no ip redirects
ip mtu 1400
no ip next-hop-self eigrp 1
ip nhrp authentication RUcW52LQ
ip nhrp map multicast dynamic
ip nhrp network-id 100000
ip nhrp holdtime 360
ip nhrp redirect
ip tcp adjust-mss 1360
no ip split-horizon eigrp 1
delay 1000
qos pre-classify
tunnel source Dialer1
tunnel mode gre multipoint
tunnel key 100000
tunnel protection ipsec profile DMVPN

Spoke Tunnel Config


interface Tunnel0
description ****** DMVPN GRE Tunnel ******
bandwidth 1000
ip address 10.255.255.5 255.255.255.0
no ip redirects
ip mtu 1400
ip nbar protocol-discovery
ip nhrp authentication RUcW52LQ
ip nhrp map multicast **.**.**.**
ip nhrp map 10.255.255.1
**.**.**.**
ip nhrp network-id 100000
ip nhrp holdtime 360
ip nhrp nhs 10.255.255.1
ip nhrp shortcut
ip nhrp redirect
ip tcp adjust-mss 1360
delay 1000
qos pre-classify
tunnel source Dialer1
tunnel mode gre multipoint
tunnel key 100000
tunnel protection ipsec profile DMVPN

Correct Answer
Lei Tian Tue, 02/23/2010 - 07:05

/* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;}

Hi Mark,

As I mentioned you still have "no ip next-hop-self eigrp 1" on your hub; you need remove that statement for phase 3 to work properly.


If i remove the statement " no ip next-hop-self eigrp 1" ie change it to " ip next-hop-self eigrp 1" the configuration becomes Phase2 and all spokes talk to one another via the HUB. Although when i removed the statement i dont think i allowed time for the tunnel interface to be reset so ill try that later.


/* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;}

Actually, it is the other way around. with "no ip next-hop-self eigrp 1" it will work as phase 2, and phase 3 do not need that statement on hub. After established  the dynamic tunnel, traffic from spoke to spoke will be sent via the ipsec tunnel and will not process by hub router. Be aware remove "no ip next-hop-self eigrp 1" on hub router will reset all eigrp neighbors.

These are little difference between phase 3 and phase 2. When you send traffic from spoke 1 to spoke 2,in phase 2 ipsec tunnel is initialed by spoke 1; in phase 3 ipsec tunnel is initialed by spoke 2. From user point of view, you might not even notice the difference. What matter is in phase 2, before spoke to spoke tunnel established, traffic will be process switched by hub router; in phase 3 all traffic are cef switched. That gives phase 3 a big plus when you have lot spokes. Phase 3 also build up spoke to spoke tunnel quicker than phase 2. You can verify that by doing ping from spoke 1 to spoke 2 with phase 2 and phase 3, and compare the number of pkt been encrypted from "show crypto ipsec sa".

In term of best practice, it doesn’t hurt to put spoke as eigrp stub network, but it wont help anything unless you have dual hub. For dual hub situation, you want put spoke as stub network so it won’t become transit site.

This is a good document for migrating DMVPN phase 2 to phase 3; you can find some best practices in the doc as well.

http://www.cisco.com/en/US/prod/collateral/iosswrel/ps6537/ps6586/ps6660/ps6808/prod_white_paper0900aecd8055c34e_ps6658_Products_White_Paper.html

HTH,

Lei Tian

Mark Rigby Wed, 02/24/2010 - 03:09

Excellent information Lei, i think its all working as expected now. All strange thing is that when i ping spoke to spoke the dynamic tunnel is created but the routing table still points to the hub at the next hop.

HUB Config


interface Tunnel0
description ****** DMVPN GRE Tunnel ******
bandwidth 1000
ip address 10.255.255.1 255.255.255.0
no ip redirects
ip mtu 1400
ip nhrp authentication RUcW52LQ
ip nhrp map multicast dynamic
ip nhrp network-id 100000
ip nhrp holdtime 360
ip nhrp redirect
ip tcp adjust-mss 1360
no ip split-horizon eigrp 1
delay 1000
qos pre-classify
tunnel source Dialer1
tunnel mode gre multipoint
tunnel key 100000
tunnel protection ipsec profile DMVPN

!

router eigrp 1
passive-interface Dialer1
passive-interface Vlan1
network 10.255.255.1 0.0.0.0
network 192.168.0.0 0.0.255.255
no auto-summary

!

#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is 0.0.0.0 to network 0.0.0.0

D    192.168.30.0/24 [90/2818560] via 10.255.255.3, 00:09:48, Tunnel0
     81.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C       81.**.**.**/24 is directly connected, Dialer1
C       81.
**.**.**/32 is directly connected, Dialer1
C    192.168.10.0/24 is directly connected, Vlan1
D    192.168.40.0/24 [90/2818560] via 10.255.255.4, 00:09:49, Tunnel0
D    192.168.20.0/24 [90/2818560] via 10.255.255.2, 00:09:47, Tunnel0
     10.0.0.0/24 is subnetted, 1 subnets
C       10.255.255.0 is directly connected, Tunnel0
D    192.168.50.0/24 [90/2818560] via 10.255.255.5, 00:10:09, Tunnel0
S*   0.0.0.0/0 is directly connected, Dialer1

Spoke Config


interface Tunnel0
description ****** DMVPN GRE Tunnel ******
bandwidth 1000
ip address 10.255.255.5 255.255.255.0
no ip redirects
ip mtu 1400
ip nbar protocol-discovery
ip nhrp authentication RUcW52LQ
ip nhrp map multicast **.**.**.**
ip nhrp map 10.255.255.1
**.**.**.**
ip nhrp network-id 100000
ip nhrp holdtime 360
ip nhrp nhs 10.255.255.1
ip nhrp shortcut
ip nhrp redirect
ip tcp adjust-mss 1360
delay 1000
qos pre-classify
tunnel source Dialer1
tunnel mode gre multipoint
tunnel key 100000
tunnel protection ipsec profile DMVPN

!

router eigrp 1
passive-interface Dialer1
passive-interface Vlan1
network 10.255.255.5 0.0.0.0
network 192.168.50.0
no auto-summary

!

#ping 192.168.20.1 source vlan 1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.20.1, timeout is 2 seconds:
Packet sent with a source address of 192.168.50.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 124/125/128 ms

!

#ping 192.168.20.1 source vlan 1

Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.20.1, timeout is 2 seconds:
Packet sent with a source address of 192.168.50.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 96/98/100 ms

!

#traceroute 192.168.20.1 source vlan 1

Type escape sequence to abort.
Tracing the route to 192.168.20.1

  1 10.255.255.2 68 msec *  68 msec

#sh ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is 0.0.0.0 to network 0.0.0.0

D    192.168.30.0/24 [90/3074560] via 10.255.255.1, 00:12:31, Tunnel0
     81.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C       81.**.**.**/24 is directly connected, Dialer1
C       81.**.**.**/32 is directly connected, Dialer1
D    192.168.10.0/24 [90/2818560] via 10.255.255.1, 00:12:52, Tunnel0
D    192.168.40.0/24 [90/3074560] via 10.255.255.1, 00:12:31, Tunnel0
D    192.168.20.0/24 [90/3074560] via 10.255.255.1, 00:12:29, Tunnel0
     10.0.0.0/24 is subnetted, 1 subnets
C       10.255.255.0 is directly connected, Tunnel0
C    192.168.50.0/24 is directly connected, Vlan1
S*   0.0.0.0/0 is directly connected, Dialer1

Correct Answer
Lei Tian Wed, 02/24/2010 - 05:14

Hi Mark,

What you have seen is the correct behavior. In Phase 3, spoke will always point to hub as the next-hop in the routing table; what changed is the nhrp map. If you goto the spoke router and show ip nhrp brief, you will notice there will be one entry for remote spoke tunnel's (10.255.255.2 in your case), and another entry for remote spoke's LAN subnet (192.168.20.0 in your case). The later entry will be used when traffic send to 192.168.20.0 subnet.

And because of this feature, phase 3 allows you on hub router only advertise summary route to spokes.

HTH,

Lei Tian

Mark Rigby Wed, 02/24/2010 - 05:52

Yep all is fine there, really appreciate your time Lei, it's certainly a more efficient way of working..right thats one down, next project is is B2B DMVPN!


Regards

Mark Rigby Wed, 02/24/2010 - 10:02

Much appreciated, one quick query can you apply an inbound ACL on a tunnel interface to deny certain subnets/ports/protocols etc from specific spokes.


For example a third party IT services provider requires sporadic access to the customers DMVPN network in order to manage internal Client/Server assets.


The IT Services provider would like to establish temporary IPSEC tunnels with the customer for the purpose of securing management traffic between the customer internal LAN's and the IT Service provider, the IT Service provider wants to join to the customers DMVPN network as an additional spoke site but does not want the customer to be able access their own network.


Regards





Correct Answer
Lei Tian Wed, 02/24/2010 - 18:56

Hi Mark,

/* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;}

Yes, you can apply ACL on tunnel interface to deny certain type of traffic on either direction.

/* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;}

After read your requirement, I think reflexive acl might be a good fit here. Say provider site is spoke 1 and the customer site is spoke 2; any traffic originate by spoke 1 and return traffic can pass, but traffic originate by spoke 2 would be blocked. Here is  the configuration guide if you are interested with reflexive acl.

http://www.cisco.com/en/US/docs/ios/sec_data_plane/configuration/guide/sec_cfg_ip_filter_ps6350_TSD_Products_Configuration_Guide_Chapter.html#wp1000897

HTH,

Lei Tian

Paolo Bevilacqua Tue, 02/23/2010 - 04:55

I don't know what did I do to make you say so, but I got your point here. I will try to explain more if it is needed.

Hi,

In no way I was referring to your contributions, rather to people in other threads that think they can voice their uninformed opinion just because they have played a couple hours with 2500s.

Actions

This Discussion