Config archive of Cisco 4404/4402 WLC devices

Unanswered Question

Hi,


I am currently having an issue backing up WLC devices in CiscoWorks LMS 3.2. i can see that from an inventory perspective they are discovered correctly. I have tested the credentials on the CLI and that works fine, but when doing a archive collection from the ciscoworks application it fails, stating that it is a timeout issue.


I have noticed that when trying to connect via the CLI using SSH example below, it does not just prompt for a password, it prompts for the username again, i am not sure if this is the issue


ssh -l test 10.1.2.3


Any assistance will be much appreciated


Kind Regards

James

  • 1
  • 2
  • 3
  • 4
  • 5
Overall Rating: 0 (0 ratings)
Loading.
Joe Clarke Tue, 04/13/2010 - 12:31
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

What errors do you get when trying to archive the configs?  No, the SSH prompting is not an issue.  However, I am tracking an issue with SSHv2

between LMS and the WLC.  The problem manifests itself as timeouts causing thread lock-ups in the ConfigMgmtServer daemon.

Hi Joe,


That is exactly the issue, I have run the below commands to can see this timeout issue


/opt/CSCOpx/lib/jre/bin/java -classpath /opt/CSCOpx/MDC/tomcat/shared/lib/cmdsvc.jar com.cisco.nm.lib.cmdsvc.Detect IP_Address_not_shown
java.net.SocketTimeoutException: Read timed out
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(SocketInputStream.java:129)
        at java.net.SocketInputStream.read(SocketInputStream.java:90)
        at com.cisco.nm.lib.cmdsvc.Detect.getOutput(Detect.java:244)
        at com.cisco.nm.lib.cmdsvc.Detect.getSshVersionStr(Detect.java:182)
        at com.cisco.nm.lib.cmdsvc.Detect.probe(Detect.java:88)
        at com.cisco.nm.lib.cmdsvc.Detect.probe(Detect.java:52)
        at com.cisco.nm.lib.cmdsvc.Detect.probe(Detect.java:33)
        at com.cisco.nm.lib.cmdsvc.Detect.main(Detect.java:263)


Let me know what you think.


Kind Regards

James

Hi


Thats the thing, I can SSH to the WLC from the LMS server and successfully login using TACACS which is the same credentials configured for this device in common services.


So I am not sure if this is a bug.


I have run some extra debug commands that performs an actual SSH CLI test and I get the same issue Socket read timeout.


Kind Regards

James

Joe Clarke Tue, 04/13/2010 - 23:26
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

The error seen when running the CLI tool may not be the same error that is seen when running config archive.  First, I need to see the output from the telnet to port 22 from the LMS server to the WLC.  Next, I need to know the exact error you see when performing a Sync Archive to this device.  Next, I need to see the dcmaservice.log after performing the Sync Archive job.


It would also be helpful to see a sniffer trace of all tcp/22 traffic between the LMS server and the device when performing the Sync Archive job.  This will help analyze the timing.


--


Please support CSC Helps Haiti


https://supportforums.cisco.com/docs/DOC-8895

https://supportforums.cisco.com

Hi


telnet output on port 22:


Trying *.*.*.*
Connected to *.*.*.*
Escape character is '^]'.
SSH-2.0-OpenSSH_4.0

Protocol mismatch.
Connection to *.*.*.* closed by foreign host.

Failed info in job output:


CM0151 PRIMARY RUNNING Config fetch failed for hostname Cause: fig from device: java.net.SocketTimeoutException: Read timed out Action: Check if protocol is supported by device and required device package is installed. Check device credentials. Increase timeout value, if required.



dcmaservice.log  output:


[ Tue Apr 13  10:46:48 BST 2010 ],INFO ,[Thread-5],com.cisco.nm.rmeng.dcma.configmanager.CfgThreadManager,triggerConfigFetch,52,#### Start of Sweep Tue Apr 13 10:46:48 BST 201
0 ####
[ Tue Apr 13  10:46:48 BST 2010 ],WARN ,[Thread-6033],com.cisco.nm.rmeng.util.DCRWrapperAPIs,getResultFromQuery,3315,SQLException occurred as connection closed. Re-connecting
to DB...
[ Tue Apr 13  10:46:48 BST 2010 ],INFO ,[Thread-6033],com.cisco.nm.rmeng.util.rmedaa.RMEDeviceContext,getCmdSvc,769,Inside RMEDeviceContext's getCmdSvc ....
[ Tue Apr 13  10:46:48 BST 2010 ],INFO ,[Thread-6033],com.cisco.nm.rmeng.util.rmedaa.RMEDeviceContext,getCmdSvc,774,Protocol and Platforms passed = SSH , RME_WLC
[ Tue Apr 13  10:46:48 BST 2010 ],INFO ,[Thread-6033],com.cisco.nm.rmeng.util.rmedaa.RMEDeviceContext,getCmdSvc,786,Iam inside ssh ....
[ Tue Apr 13  10:46:48 BST 2010 ],INFO ,[Thread-6033],com.cisco.nm.rmeng.util.rmedaa.RMEDeviceContext,getCmdSvc,787,Initial time_out : 0
[ Tue Apr 13  10:46:48 BST 2010 ],INFO ,[Thread-6033],com.cisco.nm.rmeng.util.rmedaa.RMEDeviceContext,getCmdSvc,792,Computed time_out : 60
[ Tue Apr 13  10:46:48 BST 2010 ],INFO ,[Thread-6033],com.cisco.nm.rmeng.util.rmedaa.RMEDeviceContext,getCmdSvc,800,After computing time_out : 60
[ Tue Apr 13  10:46:48 BST 2010 ],INFO ,[Thread-6033],com.cisco.nm.rmeng.util.rmedaa.RMEDeviceContext,getSshCmdSvc,819,inside getSshCmdSvc with timeout : 60000
[ Tue Apr 13  10:46:48 BST 2010 ],INFO ,[Thread-6033],com.cisco.nm.rmeng.util.rmedaa.RMEDeviceContext,getSshProtocols,872,Inside getsshprotocols with time out : 60000
[ Tue Apr 13  10:46:54 BST 2010 ],INFO ,[Thread-6033],com.cisco.nm.rmeng.util.rmedaa.RMEDeviceContext,getSshCmdSvc,826,SSH2 is running
[ Tue Apr 13  10:47:00 BST 2010 ],ERROR,[Thread-6033],com.cisco.nm.xms.xdi.transport.cmdsvc.LogAdapter,error,19,Warning: Received USERAUTH_SUCCESS in response to 'none' authen
tication request
[ Tue Apr 13  10:47:23 BST 2010 ],INFO ,[Thread-6033],com.cisco.nm.xms.xdi.pkgs.SharedDcmaWLC.transport.WLCCliOperator,getCmd2Fetch,342,SNMP initialization completed
[ Tue Apr 13  10:47:23 BST 2010 ],INFO ,[Thread-6033],com.cisco.nm.xms.xdi.pkgs.SharedDcmaWLC.transport.WLCCliOperator,getCmd2Fetch,349,Current Running image version:NoSuchIns
tance
[ Tue Apr 13  10:47:23 BST 2010 ],INFO ,[Thread-6033],com.cisco.nm.xms.xdi.pkgs.SharedDcmaWLC.transport.WLCCliOperator,getCmd2Fetch,351,Using the latest command as image veris
on is greater than 5.2
[ Tue Apr 13  10:48:25 BST 2010 ],ERROR,[Thread-6033],com.cisco.nm.xms.xdi.transport.cmdsvc.LogAdapter,error,19,IOException received during block() of Channel[UInt32[ 0 ]:UInt
32[ 0 ]]
[ Tue Apr 13  10:48:26 BST 2010 ],ERROR,[Thread-6033],com.cisco.nm.rmeng.dcma.configmanager.ConfigManager,updateArchiveForDevice,678,PRIMARY RUNNING Config fetch Failed for
hostname
[ Tue Apr 13  10:48:26 BST 2010 ],INFO ,[Thread-1],com.cisco.nm.rmeng.dcma.configmanager.CfgThreadManager,run,99,#### End of Sweep Tue Apr 13 10:48:26 BST 2010 ####


SNOOP output:


LMS-SERVER -> WLC-DEVICE TCP D=22 S=56325 Syn Seq=1825561549 Len=0 Win=49640 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56325 Ack=669720941 Seq=1825561550 Len=0 Win=50040
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56325 Ack=669720961 Seq=1825561550 Len=0 Win=50040
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56325 Fin Ack=669720961 Seq=1825561550 Len=0 Win=50040
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Syn Seq=2323886065 Len=0 Win=32850 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56325 Ack=669720962 Seq=1825561551 Len=0 Win=50040
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673581283 Seq=2323886066 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673581303 Seq=2323886066 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673581303 Seq=2323886066 Len=15 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673581303 Seq=2323886081 Len=248 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673581943 Seq=2323886329 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673581943 Seq=2323886329 Len=144 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673582407 Seq=2323886473 Len=16 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673582407 Seq=2323886489 Len=52 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673582459 Seq=2323886541 Len=68 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673582495 Seq=2323886609 Len=60 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673582547 Seq=2323886669 Len=76 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673582583 Seq=2323886745 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673582707 Seq=2323886789 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673582775 Seq=2323886789 Len=52 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673582879 Seq=2323886841 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673582879 Seq=2323886841 Len=52 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673582931 Seq=2323886893 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673582931 Seq=2323886937 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583039 Seq=2323886981 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583083 Seq=2323887025 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583127 Seq=2323887069 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583171 Seq=2323887113 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583215 Seq=2323887157 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583259 Seq=2323887201 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583303 Seq=2323887245 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583347 Seq=2323887289 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673583407 Seq=2323887333 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583407 Seq=2323887333 Len=60 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583467 Seq=2323887393 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673583535 Seq=2323887437 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583535 Seq=2323887437 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673583595 Seq=2323887481 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583595 Seq=2323887481 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583639 Seq=2323887525 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673583707 Seq=2323887569 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583707 Seq=2323887569 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583775 Seq=2323887613 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583843 Seq=2323887657 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583887 Seq=2323887701 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583931 Seq=2323887745 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673583975 Seq=2323887789 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584019 Seq=2323887833 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584063 Seq=2323887877 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584107 Seq=2323887921 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584151 Seq=2323887965 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673584219 Seq=2323888009 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584219 Seq=2323888009 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673584287 Seq=2323888053 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584287 Seq=2323888053 Len=52 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584339 Seq=2323888105 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673584415 Seq=2323888149 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584415 Seq=2323888149 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673584483 Seq=2323888193 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584483 Seq=2323888193 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584527 Seq=2323888237 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673584587 Seq=2323888281 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584587 Seq=2323888281 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584647 Seq=2323888325 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584707 Seq=2323888369 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584751 Seq=2323888413 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584795 Seq=2323888457 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584839 Seq=2323888501 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584883 Seq=2323888545 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584927 Seq=2323888589 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673584971 Seq=2323888633 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673585015 Seq=2323888677 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673585075 Seq=2323888721 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE UDP D=161 S=53008 LEN=77
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673585075 Seq=2323888721 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673585135 Seq=2323888765 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673585135 Seq=2323888765 Len=68 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673585203 Seq=2323888833 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673585735 Seq=2323888877 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673585735 Seq=2323888877 Len=44 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673585735 Seq=2323888921 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673587523 Seq=2323888921 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673587523 Seq=2323888921 Len=36 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Push Ack=673587523 Seq=2323888957 Len=128 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Ack=673587524 Seq=2323889085 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56332 Fin Ack=673587524 Seq=2323889085 Len=0 Win=33072 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56446 Syn Seq=3874994962 Len=0 Win=49640 Options=
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56446 Ack=754769594 Seq=3874994963 Len=0 Win=50040
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56446 Fin Ack=754769594 Seq=3874994963 Len=0 Win=50040
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56446 Rst Seq=3874994964 Len=0 Win=50040
LMS-SERVER -> WLC-DEVICE TCP D=22 S=56446 Rst Seq=3874994964 Len=0 Win=0


Kind Regards

James

Joe Clarke Wed, 04/14/2010 - 09:07
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

I need to see the full log and binary sniffer file.  Since it doesn't look like you can share this information on the forum, I suggest you open a TAC Service Request so the additional analysis can be done.  If it turns out you do require my patch, that can be made available to you via the Service Request.

Joe Clarke Thu, 04/15/2010 - 10:32
User Badges:
  • Cisco Employee,
  • Hall of Fame,

    Founding Member

You should open a service request so the data can be archived.  You will need an SR open in order to get the patch anyway.

Actions

This Discussion