Cisco Support Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Announcements

Welcome to Cisco Support Community. We would love to have your feedback.

For an introduction to the new site, click here. If you'd prefer to explore, try our test area to get started. And see here for current known issues.

New Member

High CPU with error "%ADJ-3-RESOLVE_REQ:" in Catalyst 4500-X VSS after making L3 function (static routing)

We have a VSS based on 2x WS-C4500X-16., The VSS is used as Layer 2 Switch for diffrents Vlan in our DC.
 
After making the VSS as a Layer 3 gateway for our production VLAN and added 2 routes for routing purposes, we encountered a network down time with high CPU in the VSS and  a huges log messages : 
 
.May 14 12:11:25.947: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.2.7.22 Vlan100
.May 14 12:11:34.516: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.2.7.22 Vlan100
.May 14 12:11:40.072: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.1.1.254 Vlan100
.May 14 12:11:49.682: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.2.1.253 Vlan100
.May 14 12:11:55.079: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.1.1.254 Vlan100
.May 14 12:12:00.926: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.2.7.40 Vlan100
.May 14 12:12:06.701: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.2.8.32 Vlan100
.May 14 12:12:12.624: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.2.7.40 Vlan100
.May 14 12:12:21.627: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.2.7.40 Vlan100
.May 14 12:12:32.261: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.2.8.32 Vlan100
.May 14 12:12:41.801: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.2.2.105 Vlan100
.May 14 12:12:49.633: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.2.1.253 Vlan100
.May 14 12:12:54.831: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.1.1.254 Vlan100
.May 14 12:12:59.960: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.1.1.254 Vlan100
.May 14 12:13:08.745: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.2.1.253 Vlan100
.May 14 12:13:16.138: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.2.1.253 Vlan100
.May 14 12:13:22.393: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.2.1.253 Vlan100
.May 14 12:13:31.415: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.2.1.141 Vlan100
.May 14 12:13:38.944: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.2.2.215 Vlan100
.May 14 12:13:45.972: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.2.1.253 Vlan100
 
 
Bellow are the show version of our VSS, 
 
Cisco IOS Software, IOS-XE Software, Catalyst 4500 L3 Switch Software (cat4500e-UNIVERSALK9-M), Version 03.04.00.SG RELEASE SOFTWARE (fc3)
Copyright (c) 1986-2012 by Cisco Systems, Inc.
Compiled Wed 05-Dec-12 04:38 by prod_rel_team
 
ROM: 15.0(1r)SG10
S_C4500X_01 uptime is 33 weeks, 1 day, 14 minutes
Uptime for this control processor is 33 weeks, 1 day, 16 minutes
System returned to ROM by power-on
System restarted at 11:59:10 UTC Tue Sep 24 2013
Running default software
Jawa Revision 2, Winter Revision 0x0.0x40
 
Last reload reason: power-on
 
 
License Information for 'WS-C4500X-16'
    License Level: ipbase   Type: Permanent
    Next reboot license Level: ipbase
 
cisco WS-C4500X-16 (MPC8572) processor (revision 9) with 4194304K/20480K bytes of memory.
Processor board ID JAE173303CF
MPC8572 CPU at 1.5GHz, Cisco Catalyst 4500X
Last reset from PowerUp
4 Virtual Ethernet interfaces
32 Ten Gigabit Ethernet interfaces
511K bytes of non-volatile configuration memory.
 
Configuration register is 0x2101
 
Can you help please, 
 
 
  • LAN Switching and Routing
2 REPLIES
Cisco Employee

these message are seen may be

these message are seen may be because for following reason,
 the adjacency table might  not have updated with the updated ARP entry when the CEF trying to  locate valid adjacency for destination prefix/ip address. This  might seems to  be the timing issue, which is temporary.

if this continues check for the hsrp/glbp  interfaces.

Regards,

Naveen

***rate if it is helpful***

 

New Member

Hi, thanks for your reply,

Hi,

 

thanks for your reply, but there is no hsrp configured, just an interface vlan. with 2 static routes and the problem was there for more than an hour before we decided to rollback.

Is there a BugId with this problem in Cisco DataBase.

here is a show ip route 

 

S_C4500X_01#      show ip route
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area 
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       + - replicated route, % - next hop override

Gateway of last resort is 10.2.1.253 to network 0.0.0.0

S*    0.0.0.0/0 [1/0] via 10.2.1.253
      10.0.0.0/8 is variably subnetted, 2 subnets, 2 masks
C        10.0.0.0/8 is directly connected, Vlan100
L        10.1.1.250/32 is directly connected, Vlan100
      172.31.0.0/16 is variably subnetted, 2 subnets, 2 masks
C        172.31.0.0/16 is directly connected, Vlan120
L        172.31.0.1/32 is directly connected, Vlan120
S     192.1.0.0/16 [1/0] via 10.1.1.254

 

 

and the show ip cef: 

 

 

_C4500X_01#        show ip cef 
.May 14 12:13:57.859: %ADJ-3-RESOLVE_REQ: Adj resolve request: Failed to resolve 10.2.1.158 Vlan100 f
Prefix               Next Hop             Interface
0.0.0.0/0            10.2.1.253           Vlan100
0.0.0.0/8            drop
0.0.0.0/32           receive              
10.0.0.0/8           attached             Vlan100
10.0.0.0/32          receive              Vlan100
10.1.1.6/32          attached             Vlan100
10.1.1.17/32         attached             Vlan100
10.1.1.40/32         attached             Vlan100
10.1.1.41/32         attached             Vlan100
10.1.1.50/32         attached             Vlan100
10.1.1.60/32         attached             Vlan100
10.1.1.99/32         attached             Vlan100
10.1.1.121/32        attached             Vlan100
10.1.1.122/32        attached             Vlan100
10.1.1.124/32        attached             Vlan100
10.1.1.125/32        attached             Vlan100
10.1.1.126/32        attached             Vlan100
10.1.1.225/32        attached             Vlan100
10.1.1.227/32        attached             Vlan100
10.1.1.250/32        receive              Vlan100
10.1.1.254/32        10.1.1.254           Vlan100
10.2.1.3/32          attached             Vlan100
10.2.1.4/32          attached             Vlan100
10.2.1.6/32          attached             Vlan100
10.2.1.8/32          attached             Vlan100
10.2.1.9/32          attached             Vlan100
10.2.1.18/32         attached             Vlan100
10.2.1.23/32         attached             Vlan100
10.2.1.24/32         attached             Vlan100
Prefix               Next Hop             Interface
10.2.1.26/32         attached             Vlan100
10.2.1.28/32         attached             Vlan100
10.2.1.29/32         attached             Vlan100
10.2.1.31/32         attached             Vlan100
10.2.1.103/32        attached             Vlan100
10.2.1.108/32        attached             Vlan100
10.2.1.109/32        attached             Vlan100
10.2.1.124/32        attached             Vlan100
10.2.1.129/32        attached             Vlan100
10.2.1.137/32        attached             Vlan100
10.2.1.139/32        attached             Vlan100
10.2.1.143/32        attached             Vlan100
10.2.1.144/32        attached             Vlan100
10.2.1.159/32        attached             Vlan100
10.2.1.167/32        attached             Vlan100
10.2.1.174/32        attached             Vlan100
10.2.1.175/32        attached             Vlan100
10.2.1.176/32        attached             Vlan100
10.2.1.181/32        attached             Vlan100

10.2.4.38/32         attached             Vlan100
10.2.4.39/32         attached             Vlan100
10.2.4.43/32         attached             Vlan100
10.2.4.47/32         attached             Vlan100
10.2.4.51/32         attached             Vlan100
10.2.4.63/32         attached             Vlan100
10.2.4.65/32         attached             Vlan100
10.2.4.69/32         attached             Vlan100
10.2.4.71/32         attached             Vlan100
10.2.4.73/32         attached             Vlan100
10.2.4.102/32        attached             Vlan100
10.2.4.106/32        attached             Vlan100
10.2.4.107/32        attached             Vlan100
10.2.4.113/32        attached             Vlan100
10.2.4.116/32        attached             Vlan100
10.2.4.119/32        attached             Vlan100
10.2.4.120/32        attached             Vlan100
10.2.4.122/32        attached             Vlan100
10.2.4.141/32        attached             Vlan100
10.2.4.148/32        attached             Vlan100
10.2.6.7/32          attached             Vlan100
Prefix               Next Hop             Interface
10.2.6.16/32         attached             Vlan100
10.2.6.31/32         attached             Vlan100
10.2.7.14/32         attached             Vlan100
10.2.7.22/32         attached             Vlan100
10.2.7.24/32         attached             Vlan100
10.2.7.34/32         attached             Vlan100
10.2.7.37/32         attached             Vlan100
10.2.7.41/32         attached             Vlan100
10.2.7.48/32         attached             Vlan100
10.2.8.18/32         attached             Vlan100
10.2.8.32/32         attached             Vlan100
10.2.8.59/32         attached             Vlan100
10.2.8.70/32         attached             Vlan100
10.2.8.85/32         attached             Vlan100
10.2.8.88/32         attached             Vlan100
10.2.8.104/32        attached             Vlan100
10.2.8.135/32        attached             Vlan100
10.2.99.10/32        attached             Vlan100
10.2.99.54/32        attached             Vlan100
10.255.255.255/32    receive              Vlan100
127.0.0.0/8          drop
172.31.0.0/16        attached             Vlan120
172.31.0.0/32        receive              Vlan120
172.31.0.1/32        receive              Vlan120
172.31.0.5/32        attached             Vlan120
172.31.0.29/32       attached             Vlan120
172.31.255.255/32    receive              Vlan120
192.1.0.0/16         10.1.1.254           Vlan100
224.0.0.0/4          drop
224.0.0.0/24         receive              
Prefix               Next Hop             Interface
240.0.0.0/4          drop

 

and show ip arp 

 

Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  10.1.1.1                0   aa00.0400.c286  ARPA   Vlan100
Internet  10.1.1.6                0   0050.5689.24b8  ARPA   Vlan100
Internet  10.1.1.10               0   0050.5694.7d20  ARPA   Vlan100
Internet  10.1.1.11               0   0050.5694.7d20  ARPA   Vlan100
Internet  10.1.1.12               0   0050.5694.6ae7  ARPA   Vlan100
Internet  10.1.1.13               0   0050.5694.6ae7  ARPA   Vlan100
Internet  10.1.1.14               0   0050.568a.6321  ARPA   Vlan100
Internet  10.1.1.16               0   0050.5694.0ab5  ARPA   Vlan100
Internet  10.1.1.17               0   0050.5694.493d  ARPA   Vlan100
Internet  10.1.1.40               0   0013.19b0.9c40  ARPA   Vlan100
Internet  10.1.1.41               0   1c17.d35a.c840  ARPA   Vlan100
Internet  10.1.1.50               0   0002.b9b4.a5c0  ARPA   Vlan100
Internet  10.1.1.60               0   000a.410f.e500  ARPA   Vlan100
Internet  10.1.1.71               -   0008.e3ff.fc28  ARPA   Vlan100
Internet  10.1.1.96               0   e02f.6d12.4df3  ARPA   Vlan100
Internet  10.1.1.98               0   0050.5696.6d86  ARPA   Vlan100
Internet  10.1.1.99               0   0050.5696.6d88  ARPA   Vlan100
Internet  10.1.1.121              0   e02f.6d12.4dea  ARPA   Vlan100
Internet  10.1.1.122              0   e02f.6d12.4e61  ARPA   Vlan100
Internet  10.1.1.123              0   e02f.6d5b.c10e  ARPA   Vlan100
Internet  10.1.1.124              0   e02f.6d17.c869  ARPA   Vlan100
Internet  10.1.1.125              0   e02f.6d5b.c217  ARPA   Vlan100
Internet  10.1.1.126              0   e02f.6d17.c8ec  ARPA   Vlan100
Internet  10.1.1.127              0   e02f.6d17.c876  ARPA   Vlan100
Internet  10.1.1.128              0   e02f.6d5b.bef3  ARPA   Vlan100
Internet  10.1.1.202              0   0000.85b7.9031  ARPA   Vlan100
Internet  10.1.1.222              0   000f.f84d.2ca9  ARPA   Vlan100
Internet  10.1.1.225              0   000f.f84d.3963  ARPA   Vlan100
Internet  10.1.1.227              0   00c0.ee26.9367  ARPA   Vlan100
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  10.1.1.250              -   0008.e3ff.fc28  ARPA   Vlan100
Internet  10.1.1.254              0   0000.0c07.ac07  ARPA   Vlan100
Internet  10.2.1.2                0   0011.4333.bcda  ARPA   Vlan100
Internet  10.2.1.3                0   0050.5689.5d38  ARPA   Vlan100
Internet  10.2.1.4                0   0050.5689.0404  ARPA   Vlan100
Internet  10.2.1.6                0   0050.5689.6d3b  ARPA   Vlan100
Internet  10.2.1.7                0   1cc1.def4.6940  ARPA   Vlan100
Internet  10.2.1.8                0   0050.5689.330e  ARPA   Vlan100
Internet  10.2.1.9                0   0012.793a.3ccc  ARPA   Vlan100
Internet  10.2.1.10               0   0012.7990.e5d3  ARPA   Vlan100
Internet  10.2.1.13               0   0050.568a.6dcf  ARPA   Vlan100
Internet  10.2.1.15               0   0050.568a.60ff  ARPA   Vlan100
Internet  10.2.1.18               0   0050.5689.091b  ARPA   Vlan100
Internet  10.2.1.20               0   0050.5689.451c  ARPA   Vlan100
Internet  10.2.1.21               0   0050.568a.0cf4  ARPA   Vlan100
Internet  10.2.1.22               0   0050.5689.6c59  ARPA   Vlan100
Internet  10.2.1.23               0   0050.5696.6d9e  ARPA   Vlan100
Internet  10.2.1.24               0   0050.5689.76c4  ARPA   Vlan100
Internet  10.2.1.26               0   0050.5689.2f4e  ARPA   Vlan100
Internet  10.2.1.27               0   0050.5689.0632  ARPA   Vlan100
Internet  10.2.1.28               0   0050.5689.1ce9  ARPA   Vlan100
Internet  10.2.1.29               0   0050.5689.6aaa  ARPA   Vlan100
Internet  10.2.1.31               0   0050.5689.0d1a  ARPA   Vlan100
Internet  10.2.1.37               0   0050.5696.6d81  ARPA   Vlan100
Internet  10.2.1.103              0   d4be.d9be.8eef  ARPA   Vlan100
Internet  10.2.1.106              0   14fe.b5e1.c595  ARPA   Vlan100
Internet  10.2.1.107              0   0023.ae7d.a966  ARPA   Vlan100
Internet  10.2.1.108              0   d4be.d9c8.6770  ARPA   Vlan100
Internet  10.2.1.109              0   14fe.b5e9.c5b5  ARPA   Vlan100
Internet  10.2.1.110              0   14fe.b5ea.5f9d  ARPA   Vlan100
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  10.2.1.111              0   001e.c959.d4f0  ARPA   Vlan100
Internet  10.2.1.114              0   b8ac.6f48.4538  ARPA   Vlan100
Internet  10.2.1.115              0   14fe.b5e1.ed89  ARPA   Vlan100
Internet  10.2.1.116              0   7845.c409.1959  ARPA   Vlan100

 

Thanks

Lotfi

665
Views
0
Helpful
2
Replies