cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
4236
Views
25
Helpful
8
Replies

Issues with SNMP

roysm
Level 1
Level 1

Hi

I am trying to get snmp monitoring setup so that I can monitor the controllers, leafs and spines. Getting it working for spines and leafs seems to be straightforward but not so for the controllers. I need this to work using OOB management and I have created an oob contract with the udp/161 filter, as suggested in various documents I have read, but I then lose access to the APICs.

What I'm looking for is a definitive step-by-step guide to setting up snmp for both controllers and leaf/spines. Is there such a thing? If not, can anyone share how they have setup snmp for their APICs and leafs/spines?

Many thanks

Roy

8 Replies 8

Tomas de Leon
Cisco Employee
Cisco Employee

Roy,

Check out my document here:

Technote: SNMP in the ACI Fabric

https://supportforums.cisco.com/document/12961146/technote-snmp-aci-fabric

T.

Tomas

Thanks for the links. I had already looked at these and although they are comprehensive and I have followed them, I still have issues.

Does it make any difference if snmp is configured to go through the oob management or is inband management better? I have set my oob contract with filters for udp/161 and udp/162 but still nothing. The leaf/spines are all fine but the controllers just will not talk to my management server.

Thanks

Roy

Management preference for you and your environment is a decision best made by your networking team. That said, what ever management strategy that you choose does matter.

In regards to INB vs. OOB, remember INB & OOB is for “management” of ACI nodes. That is APICs, Leaf nodes, Spine Nodes, etc…
External Data collectors like Syslog, SNMP, and Callhome features communicate to the ACI nodes via INB, OOB, or Both.

If both in-band and out-of-band management policies for ACI nodes (APIC\Leaf\Spine) are configured correctly & successfully deployed; the in-band management interface is the preferred management interface. Even if you configure a policy to use the out-of-band management EPG, the ACI node will still use the in-band management interface.

The exceptions to the in-band management connectivity preference are:
* The Destination IP address for the policy resides in the out-of-band management network.
* The ACI Node receives a request packet destined to the node's IP address of the out-of-band management interface. The ACI Node will reply to the request using the out-of-band management interface.

Originally, there was no way to change this management connectivity preference for ACI nodes. So in the latest versions ACI firmware, a feature was added to toggle between in-band and out-of-band connectivity on the APIC server for management access to devices such as authentication servers or SNMP servers external to the ACI fabric. A requirement for this feature for the toggling between in-band and out-of-band connectivity to have effect, Both in-band or out-of-band management must already be configured.

Note: As noted above, this toggling between in-band and out-of-band connectivity feature will only change the management connectivity preference on the APIC controllers not the leaf or spine nodes. The leaf & spine nodes will continue to use the in-band management connectivity preference.

In troubleshooting your issue, here are some things to try:

  • if you are using INBAND management with OOB, disable INB management by simply unconfiguring INB static node management addresses for all of the nodes.  Then make sure the management EPG for your SNMP configuration is set to OOB.
  • For OOB & INB, make sure that you have Static node management addresses configured for ALL nodes including the APICs.  Most ACI Administrators forget to configure static node management addresses for APICs since they configured them during the setup script.
  • For contracts for the OOB, use the "default" common filter to allow all for the testing of the feature.  Also, use 0.0.0.0 for the subnet on the external management instance for testing feature.
  • Make sure that your SNMP Collector's IP is configured in the SNMP Client groups in the SNMP fabric policies.
  • Are you using SNMP v2 or v3?  If v3 make sure all of the v3 parameters are configured.
  • What does the following commands show on the APIC(s):
    • netstat -aon | grep ":161"
    • netstat -lp | grep "snmp"
    • show snmp clientgroups   ( you can xxxx out the IP addresses)
    • moquery -c vzEntry | egrep -A 2 "161"
    • moquery -c vzEntry | egrep -A 2 "162"
    • moquery -c mgmtSubnet
    • moquery -c mgmtRsOoBCons
    • moquery -c vzOOBBrCP 
    • moquery -c snmpRsEpg | egrep "snmp.RsEpg|dn|tDn"
    • ps aux | grep snmp

Also what version are you running?

Thanks

T.

Tomas

Just so you know we are running v.2.1.1h.

I think I have it sorted now. It seems the oob EPG and external Management Network Instance does not like a custom contract. Once I replaced my own contract with the common/default I can now connect to the APICs form my snmp server.

thanks for the help

Roy

Roy,

I am glad that it is working.  For this question and for future questions, please mark your questions answered so that others can see the results for similar issues.

Thanks

T. 

russhe
Cisco Employee
Cisco Employee

Hey Roy,

I know you already solved the issue, just wanted to clarify the contract issue.

You can 100% create an oob contract and use it in the mgmt tenant to permit traffic. It sounds like when you were doing this though, you may have only added the filter for SNMP and not allowed any other traffic as you said you lost connectivity to your APICs. If you want to use a contract in the mgmt tenant, I would:

  1. Create the contract, naming it something other than default to avoid overlap with the common tenant
  2. Add an allow_all filter and the SNMP filter.

Hope that helps clear up the issue

Hello, I am running into the same issue as the user above.

I have SNMP configured and polling successfully for my APIC's but I am unable to do so for my leaf/spine switches.

Per the above, I have 0.0.0.0/0 set as the subnet for the External Management Network Instance Profile and my OOB contract is permitting any.

My nodes are configured with static IP addresses in the OOB range.

My external polling servers are configured in the client groups.

 

In response to the additional troubleshooting requests from the APIC:

  • netstat -aon | grep ":161"
    • udp 0 0 0.0.0.0:161 0.0.0.0:* off (0.00/0/0)
      udp6 0 0 :::161 :::* off (0.00/0/0)
  • netstat -lp | grep "snmp"
    • (No info could be read for "-p": geteuid()=15374 but you should be root.)
      udp 0 0 0.0.0.0:snmp 0.0.0.0:* -
      udp 0 0 0.0.0.0:snmptrap 0.0.0.0:* -
      udp6 0 0 [::]:snmp [::]:* -
      udp6 0 0 [::]:snmptrap [::]:* -
  • show snmp clientgroups   ( you can xxxx out the IP addresses)
    • SNMP Policy Name Description Client Entries Associated Management EPG
      -------------------- -------------------- -------------------- -------------------- --------------------
      default ERP-Solarwinds x.x.x.x default (Out-Of-Band)
      default ERP-Solarwinds-P2 x.x.x.x default (Out-Of-Band)
      ERP-SNMP ERP-SNMP-CG x.x.x.x,x.x.x.x default (Out-Of-Band)
  • moquery -c vzEntry | egrep -A 2 "161"
    • No output
  • moquery -c vzEntry | egrep -A 2 "162"
    • No output
  • moquery -c mgmtSubnet
    • Total Objects shown: 1

      # mgmt.Subnet
      ip : 0.0.0.0/0
      annotation :
      childAction :
      descr :
      dn : uni/tn-mgmt/extmgmt-default/instp-OOB-MGMT/subnet-[0.0.0.0/0]
      extMngdBy :
      lcOwn : local
      modTs : 2023-01-26T07:43:48.730-05:00
      name :
      nameAlias :
      rn : subnet-[0.0.0.0/0]
      status :
      uid : 15374

  • moquery -c mgmtRsOoBCons
    • Total Objects shown: 1

      # mgmt.RsOoBCons
      tnVzOOBBrCPName : OOB-default
      annotation :
      childAction :
      deplInfo :
      dn : uni/tn-mgmt/extmgmt-default/instp-OOB-MGMT/rsooBCons-OOB-default
      extMngdBy :
      forceResolve : yes
      lcOwn : local
      modTs : 2015-09-09T07:13:51.360-05:00
      monPolDn : uni/tn-common/monepg-default
      prio : unspecified
      rType : mo
      rn : rsooBCons-OOB-default
      state : formed
      stateQual : none
      status :
      tCl : vzOOBBrCP
      tContextDn :
      tDn : uni/tn-mgmt/oobbrc-OOB-default
      tRn : oobbrc-OOB-default
      tType : name
      triggerSt : triggerable
      uid : 15374

  • moquery -c vzOOBBrCP 
    • Total Objects shown: 1

      # mgmt.RsOoBCons
      tnVzOOBBrCPName : OOB-default
      annotation :
      childAction :
      deplInfo :
      dn : uni/tn-mgmt/extmgmt-default/instp-OOB-MGMT/rsooBCons-OOB-default
      extMngdBy :
      forceResolve : yes
      lcOwn : local
      modTs : 2015-09-09T07:13:51.360-05:00
      monPolDn : uni/tn-common/monepg-default
      prio : unspecified
      rType : mo
      rn : rsooBCons-OOB-default
      state : formed
      stateQual : none
      status :
      tCl : vzOOBBrCP
      tContextDn :
      tDn : uni/tn-mgmt/oobbrc-OOB-default
      tRn : oobbrc-OOB-default
      tType : name
      triggerSt : triggerable
      uid : 15374

      ERPTRAPIC1# moquery -c vzOOBBrCP
      Total Objects shown: 3

      # vz.OOBBrCP
      name : default
      annotation :
      childAction :
      configIssues :
      descr :
      dn : uni/tn-mgmt/oobbrc-default
      extMngdBy :
      lcOwn : local
      modTs : 2020-01-13T04:37:04.170-05:00
      monPolDn : uni/tn-common/monepg-default
      nameAlias :
      ownerKey :
      ownerTag :
      prio : unspecified
      reevaluateAll : no
      rn : oobbrc-default
      scope : global
      status :
      targetDscp : unspecified
      uid : 16005

      # vz.OOBBrCP
      name : OOB-default
      annotation :
      childAction :
      configIssues :
      descr :
      dn : uni/tn-mgmt/oobbrc-OOB-default
      extMngdBy :
      lcOwn : local
      modTs : 2020-01-13T04:31:09.015-05:00
      monPolDn : uni/tn-common/monepg-default
      nameAlias :
      ownerKey :
      ownerTag :
      prio : unspecified
      reevaluateAll : no
      rn : oobbrc-OOB-default
      scope : context
      status :
      targetDscp : unspecified
      uid : 15374

      # vz.OOBBrCP
      name : default
      annotation :
      childAction :
      configIssues :
      descr :
      dn : uni/tn-common/oobbrc-default
      extMngdBy :
      lcOwn : local
      modTs : 2015-08-25T04:23:57.118-05:00
      monPolDn : uni/tn-common/monepg-default
      nameAlias :
      ownerKey :
      ownerTag :
      prio : unspecified
      reevaluateAll : no
      rn : oobbrc-default
      scope : context
      status :
      targetDscp : unspecified
      uid : 0

  • moquery -c snmpRsEpg | egrep "snmp.RsEpg|dn|tDn"
    • # snmp.RsEpg
      dn : uni/fabric/snmppol-default/clgrp-ERP-Solarwinds-P2/rsepg
      tDn : uni/tn-mgmt/mgmtp-default/oob-default
      # snmp.RsEpg
      dn : uni/fabric/snmppol-ERP-SNMP/clgrp-ERP-SNMP-CG/rsepg
      tDn : uni/tn-mgmt/mgmtp-default/oob-default
      # snmp.RsEpg
      dn : uni/fabric/snmppol-default/clgrp-ERP-Solarwinds/rsepg
      tDn : uni/tn-mgmt/mgmtp-default/oob-default
  • ps aux | grep snmp
    • ifc 4662 1.6 0.3 772016 213432 ? Ssl 2020 24971:11 /mgmt//bin/snmpd.bin -f -p /tmp/snmpd2.pid -a -A -Lf /var/run/mgmt/log/snmpd.log -c /data//snmp/snmpd.conf
      ifc 11655 0.0 0.0 91848 9924 ? Ss 2020 0:00 /mgmt//bin/snmptrapd.bin -f -p /tmp/snmptrapd2.pid udp:162 udp6:162 -u ifc -a -c /data//snmp/snmptrapd.conf -C -d -f
      admin 18843 0.0 0.0 9060 920 pts/0 S+ 19:46 0:00 grep snmp

 

Again, SNMP works to the APICs but not to the leaf/spine switches. Additionally I am able to ping and on a packet capture from teh ASA firewall I can see the SNMP response coming back from the leaf/spine (although failing). Sorry to be so late to the party on this one, hopefully someone can provide assistance.

Tomas de Leon
Cisco Employee
Cisco Employee

also,

Ask the ACI Experts: SNMP in the ACI Fabric

https://supportforums.cisco.com/blog/13100731/ask-aci-experts-snmp-aci-fabric

Getting Started

Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community:

Save 25% on Day-2 Operations Add-On License