01-04-2010 01:15 PM
After a Device Package update, I see the following procs left running:
root 8268 1 24 15:39:15 ? 1:11 /opt/CSCOpx/objects/dmgt/dmgtd.sol
root 10304 2006 0 15:51:01 ? 0:00 /opt/CSCOpx/bin/in.tftpd -s /tftpboot
casuser 23883 1 0 Dec 23 ? 0:10 /opt/vgi/csco/CSCOpx/objects/wfengine/program/PolicyScheduler
casuser 23878 1 0 Dec 23 ? 0:05 /opt/vgi/csco/CSCOpx/objects/wfengine/program/DataStore
casuser 23880 1 0 Dec 23 ? 0:12 /opt/vgi/csco/CSCOpx/objects/wfengine/program/GGLogger
casuser 23881 1 0 Dec 23 ? 0:05 /opt/vgi/csco/CSCOpx/objects/wfengine/program/GGProxy
casuser 23889 1 0 Dec 23 ? 0:04 /opt/vgi/csco/CSCOpx/objects/wfengine/program/GGHelper
casuser 23879 1 0 Dec 23 ? 0:06 /opt/vgi/csco/CSCOpx/objects/wfengine/program/GGScript
casuser 23882 1 0 Dec 23 ? 1:25 /opt/vgi/csco/CSCOpx/objects/wfengine/program/OpsBREngine
casuser 11683 1 24 15:59:39 ? 1:32 /opt/vgi/csco/CSCOpx/lib/jre/bin/java -classpath /opt/vgi/csco/CSCOpx/lib/class
I let it stand for 3-5 mins, but didn't see any more procs coming back up, so I manually restarted dmgtd. My question here is: Does it really take that "long" to bring everything but up, or was I not patient enough?
After dmgtd stop, I still see some CSCOpx procs running:
root 10304 2006 0 15:51:01 ? 0:00 /opt/CSCOpx/bin/in.tftpd -s /tftpboot
casuser 23883 1 0 Dec 23 ? 0:10 /opt/vgi/csco/CSCOpx/objects/wfengine/program/PolicyScheduler
casuser 23878 1 0 Dec 23 ? 0:05 /opt/vgi/csco/CSCOpx/objects/wfengine/program/DataStore
casuser 23880 1 0 Dec 23 ? 0:12 /opt/vgi/csco/CSCOpx/objects/wfengine/program/GGLogger
casuser 23881 1 0 Dec 23 ? 0:05 /opt/vgi/csco/CSCOpx/objects/wfengine/program/GGProxy
casuser 23889 1 0 Dec 23 ? 0:04 /opt/vgi/csco/CSCOpx/objects/wfengine/program/GGHelper
casuser 23879 1 0 Dec 23 ? 0:06 /opt/vgi/csco/CSCOpx/objects/wfengine/program/GGScript
casuser 23882 1 0 Dec 23 ? 1:25 /opt/vgi/csco/CSCOpx/objects/wfengine/program/OpsBREngine
casuser 11683 1 25 15:59:39 ? 4:11 /opt/vgi/csco/CSCOpx/lib/jre/bin/java -classpath /opt/vgi/csco/CSCOpx/lib/class
Is that "normal"? I thought in earlier versions there's none left "standing" after stopping dmgtd.
01-04-2010 01:23 PM
Doing what you did is bad. It will take a long time for all the processes to come back after a package update. The reason for that, is first dmgtd will shutdown each process (dmgtd itself will stay running). Then, it will install the packages, then proceed to restart each process. The whole thing can take 30 minutes. By restarting processes after five minutes, you may have corrupted your package maps and/or repositories.
01-04-2010 01:25 PM
Doh! Is there any way to "verify" whether I got away with it unscathed or not?
01-04-2010 01:29 PM
I'm running Device Pkg update again. I get prompted for a single new pkg being available.
1. | Cat3560 | DevicePackage | CiscoView | 5.0 | 6.0 | Cat3560.cv50.v6-0.readme | NA | NA |
Here'st the Event Log of the previous Device Pkg update:
Number of Packages Selected for Install : 57
For Product(s) : CiscoWorks Common Services,LMS Portal,CiscoWorks Assistant,CiscoView,Resource Manager Essentials,Campus Manager,Internetwork Performance Monitor
Install Invoked by user : admin
The Package(s) Selected for Install :
DeviceData, Mdf, Cat2350, Cat2960, Cat2975, Cat3560, miniRMON, PoE, CVCrossLaunch, Cat6000IOS, CVGenericPackage, CBS31x0, MWR, Rtr10000, Rtr3900, Rtr2900, Rtr1900, Rtr1800, NGMARShare, Rtr3800, Rtr7000, Rtr800, Routershare, SwitchAddlets, Cat2960, Cat3560, Cat3750, Cat4000IOS, CatGL3, IAD2400, LibCommon, LibDcma, LibInventory, LibNetconfig, LibSwim, Rtr3200, RtrUBR, SharedDcmaIOS, SharedDcmaSS, SharedInventoryCatIOS, SharedInventoryMDS, SharedInventoryRouter, SharedNetconfigIOS, SharedNetconfigSS, SharedNetshowIOS, SharedNetshowSS, SharedSwim2900XL, Rtr3900, Rtr1900, Rtr2900, Rtr800, SharedSwimIOS, CE500, ASR, Nexus, SharedSwimMDS9000, CMDeviceUpdates
Successfully installed packages (2):
DeviceData(1.6) ,Mdf(1.36)
Installation Completed for : CiscoWorks Common Services
No package(s) to install for : LMS Portal
No package(s) to install for : CiscoWorks Assistant
WARNING : Cat3750(8.0):Version 8.0 already installed
WARNING : CE500(5.0):Version 5.0 already installed
WARNING : IAD2400(7.0):Version 7.0 already installed
WARNING : Cat4000IOS(16.0):Version 16.0 already installed
Successfully installed packages (21):
SwitchAddlets(1.26) ,Rtr3800(9.0) ,Rtr10000(10.2) ,NGMARShare(1.11) ,CVCrossLaunch(2.2) ,Rtr7000(22.0) ,Cat2960(6.0) ,Rtr1900(1.0) ,MWR(1.0) ,miniRMON(3.3) ,PoE(1.3) ,CVGenericPackage(1.3) ,Rtr2900(2.0) ,Cat2975(1.0) ,Cat2350(1.0) ,Rtr3900(1.1) ,Rtr800(14.0) ,CBS31x0(3.0) ,Routershare(1.2) ,Cat6000IOS(30.2) ,Rtr1800(8.0)
Installation Completed for : CiscoView
Successfully installed packages (32):
SharedInventoryRouter(2.5.2) ,SharedSwim2900XL(2.2.2) ,Cat3750(6.2) ,LibCommon(2.4.2) ,RtrUBR(1.3) ,SharedInventoryMDS(1.5.1) ,CE500(6.1) ,SharedInventoryCatIOS(2.3.2) ,SharedSwimMDS9000(1.6.2) ,SharedNetshowIOS(2.2.3) ,Cat3560(7.3) ,LibNetconfig(2.2.2) ,Cat2960(6.3) ,SharedDcmaIOS(2.4.3) ,Rtr1900(2.2) ,SharedNetconfigSS(1.2.1) ,ASR(2.1) ,SharedDcmaSS(2.2.1) ,IAD2400(1.3) ,Cat4000IOS(7.2) ,LibInventory(3.1.4) ,LibSwim(2.5.3) ,Rtr2900(1.1) ,SharedNetconfigIOS(2.3.2) ,CatGL3(5.2) ,Rtr3900(1.0) ,Rtr800(6.3) ,SharedNetshowSS(1.1.1) ,SharedSwimIOS(2.5.3) ,LibDcma(2.3.2) ,Rtr3200(5.1) ,Nexus(2.3)
Installation Completed for : Resource Manager Essentials
Successfully installed packages (1):
CMDeviceUpdates(6.0)
Installation Completed for : Campus Manager
No package(s) to install for : Internetwork Performance Monitor
Does it mean no harm done to the Device Pkg map?
01-04-2010 01:34 PM
This means the package map is okay. Corruption there usually brings up 500 server errors. As for validating the RME package repository, you should have 330 packages on an up-to-date 4.3.1 server.
01-04-2010 01:46 PM
That's great news: I do have 330 RME pkgs, and 746 RME device types to match.
So I let it come back autonomously this time. Should I be concerned about some of the downed or failed processes according to the current proc status?
1. | 1395 | Administrator has shut down this server | 0 | 0 | 0 | N/A | Not applicable | |
2. | ANIDbEngine | Program started - No mgt msgs received | 18767 | 0 | 0 | 01/04/10 16:37:53 | Not applicable | |
3. | ANIServer | Running with busy flag set | 19377 | 0 | 0 | 01/04/10 16:40:49 | Not applicable | |
4. | CampusOGSServer | Administrator has shut down this server | 0 | 0 | 0 | 01/04/10 16:11:30 | 01/04/10 16:32:15 | |
5. | ChangeAudit | Administrator has shut down this server | 0 | -113 | 0 | 01/04/10 16:11:59 | 01/04/10 16:31:56 | |
6. | CmfDbEngine | Program started - No mgt msgs received | 19381 | 0 | 0 | 01/04/10 16:40:54 | Not applicable | |
7. | CmfDbMonitor | Failed to run | 0 | 0 | 6 | 01/04/10 16:11:14 | 01/04/10 16:40:58 | |
8. | CMFOGSServer | Program started - No mgt msgs received | 19460 | 0 | 0 | 01/04/10 16:41:10 | Not applicable | |
9. | ConfigMgmtServer | Administrator has shut down this server | 0 | -113 | 0 | 01/04/10 16:11:39 | 01/04/10 16:32:05 | |
10. | CSDiscovery | Administrator has shut down this server | 0 | 0 | 0 | N/A | Not applicable | |
11. | CSRegistryServer | Running normally | 18809 | 0 | 0 | 01/04/10 16:38:02 | Not applicable | |
12. | CTMJrmServer | Running normally | 19572 | 0 | 0 | 01/04/10 16:41:31 | Not applicable | |
13. | CWAPublisher | Program started - No mgt msgs received | 19652 | 0 | 0 | 01/04/10 16:41:50 | Not applicable | |
14. | DCRDevicePoll | Administrator has shut down this server | 0 | 0 | 0 | N/A | Not applicable | |
15. | DCRServer | Running normally | 19400 | 0 | 0 | 01/04/10 16:40:58 | Not applicable | |
16. | diskWatcher | Running normally | 19630 | 0 | 0 | 01/04/10 16:41:41 | Not applicable | |
17. | EDS | Running normally | 19293 | 0 | 0 | 01/04/10 16:40:22 | Not applicable | |
18. | EDS-GCF | Administrator has shut down this server | 0 | 0 | 0 | 01/04/10 16:11:39 | 01/04/10 16:33:46 | |
19. | ESS | Program started - No mgt msgs received | 18673 | 0 | 0 | 01/04/10 16:37:35 | Not applicable | |
20. | EssentialsDM | Running normally | 19468 | 0 | 0 | 01/04/10 16:41:15 | Not applicable | |
21. | EssMonitor | Running normally | 18677 | 0 | 0 | 01/04/10 16:37:39 | Not applicable | |
22. | EventFramework | Program started - No mgt msgs received | 18708 | 0 | 0 | 01/04/10 16:37:45 | Not applicable | |
23. | FDRewinder | Administrator has shut down this server | 0 | 0 | 0 | N/A | Not applicable | |
24. | ICServer | Administrator has shut down this server | 0 | 0 | 0 | 01/04/10 16:11:39 | 01/04/10 16:32:07 | |
25. | IpmDbEngine | Program started - No mgt msgs received | 18727 | 0 | 0 | 01/04/10 16:37:49 | Not applicable | |
26. | IPMOGSServer | Administrator has shut down this server | 0 | -113 | 0 | 01/04/10 16:12:03 | 01/04/10 16:31:52 | |
27. | IPMProcess | Program started - No mgt msgs received | 19595 | 0 | 0 | 01/04/10 16:41:37 | Not applicable | |
28. | jrm | Running normally | 19538 | 0 | 0 | 01/04/10 16:41:24 | Not applicable | |
29. | LicenseServer | Program started - No mgt msgs received | 19242 | 0 | 0 | 01/04/10 16:40:13 | Not applicable | |
30. | MACUHIC | Running normally | 18780 | 0 | 0 | 01/04/10 16:37:57 | Not applicable | |
31. | NameServer | Program started - No mgt msgs received | 19246 | 0 | 0 | 01/04/10 16:40:14 | Not applicable | |
32. | NameServiceMonitor | Running normally | 19287 | 0 | 0 | 01/04/10 16:40:18 | Not applicable | |
33. | OpsxmlDbEngine | Program started - No mgt msgs received | 18796 | 0 | 0 | 01/04/10 16:37:58 | Not applicable | |
34. | OpsXMLRuntime | Program started - No mgt msgs received | 19641 | 0 | 0 | 01/04/10 16:41:46 | Not applicable | |
35. | PMCOGSServer | Administrator has shut down this server | 0 | 0 | 0 | 01/04/10 16:11:39 | 01/04/10 16:32:09 | |
36. | ProcSysBus | Program started - No mgt msgs received | 19634 | 0 | 0 | 01/04/10 16:41:41 | Not applicable | |
37. | RMEDbEngine | Program started - No mgt msgs received | 18713 | 0 | 0 | 01/04/10 16:37:45 | Not applicable | |
38. | RMEOGSServer | Program started - No mgt msgs received | 19478 | 0 | 0 | 01/04/10 16:41:19 | Not applicable | |
39. | SyslogAnalyzer | Administrator has shut down this server | 0 | -113 | 0 | 01/04/10 16:11:59 | 01/04/10 16:31:55 | |
40. | SyslogCollector | Program started - No mgt msgs received | 18710 | 0 | 0 | 01/04/10 16:37:45 | Not applicable | |
41. | TomcatMonitor | Running normally | 18864 | 0 | 0 | 01/04/10 16:38:07 | Not applicable | |
42. | UTLITE | Running normally | 18784 | 0 | 0 | 01/04/10 16:37:58 | Not applicable | |
43. | UTMajorAcquisition | Administrator has shut down this server | 0 | 0 | 0 | N/A | Not applicable | |
44. | UTManager | Administrator has shut down this server | 0 | 2 | 0 | 01/04/10 16:11:39 | 01/04/10 16:32:04 | |
45. | VNMServer | Program started - No mgt msgs received | 18792 | 0 | 0 | 01/04/10 16:37:58 | Not applicable | |
46. | WlseUHIC | Running normally | 18788 | 0 | 0 | 01/04/10 16:37:58 | Not applicable |
01-04-2010 01:49 PM
Did you run the installation again? If so, and if PSU is still performing the installation and daemon restart, then no. Else, you do have some problems. There appears to be some issues with the cmf database as well as other processes. If these processes don't come back up automatically in about 30 minutes, then you definitely have a problem.
01-05-2010 06:29 AM
I simply let Device Pkg Update install the CiscoView Cat3560 pkg pictured earlier. It's then left unattended. The reason I developed the inclination to manually intervene at all was because I had a similar issue with LMS procs (*OGS* ones) not coming back properly in LMS 3.1 on my production box (alluded to in this thread: https://supportforums.cisco.com/message/2006396#2006396), while LMS 3.2 is currently only on my test box.
Here's the proc status around 5pm yesterday:
1. | 1395 | Administrator has shut down this server | 0 | 0 | 0 | N/A | Not applicable | |
2. | ANIDbEngine | Program started - No mgt msgs received | 18767 | 0 | 0 | 01/04/10 16:37:53 | Not applicable | |
3. | ANIServer | Running with busy flag set | 19377 | 0 | 0 | 01/04/10 16:40:49 | Not applicable | |
4. | CampusOGSServer | Administrator has shut down this server | 0 | 0 | 0 | 01/04/10 16:11:30 | 01/04/10 16:32:15 | |
5. | ChangeAudit | Administrator has shut down this server | 0 | -113 | 0 | 01/04/10 16:11:59 | 01/04/10 16:31:56 | |
6. | CmfDbEngine | Program started - No mgt msgs received | 19381 | 0 | 0 | 01/04/10 16:40:54 | Not applicable | |
7. | CmfDbMonitor | Failed to run | 0 | 0 | 6 | 01/04/10 16:11:14 | 01/04/10 16:40:58 | |
8. | CMFOGSServer | Program started - No mgt msgs received | 19460 | 0 | 0 | 01/04/10 16:41:10 | Not applicable | |
9. | ConfigMgmtServer | Administrator has shut down this server | 0 | -113 | 0 | 01/04/10 16:11:39 | 01/04/10 16:32:05 | |
10. | CSDiscovery | Administrator has shut down this server | 0 | 0 | 0 | N/A | Not applicable | |
11. | CSRegistryServer | Running normally | 18809 | 0 | 0 | 01/04/10 16:38:02 | Not applicable | |
12. | CTMJrmServer | Running normally | 19572 | 0 | 0 | 01/04/10 16:41:31 | Not applicable | |
13. | CWAPublisher | Administrator has shut down this server | 0 | 0 | 0 | 01/04/10 16:41:50 | 01/04/10 16:44:38 | |
14. | DCRDevicePoll | Administrator has shut down this server | 0 | 0 | 0 | N/A | Not applicable | |
15. | DCRServer | Running normally | 19400 | 0 | 0 | 01/04/10 16:40:58 | Not applicable | |
16. | diskWatcher | Running normally | 19630 | 0 | 0 | 01/04/10 16:41:41 | Not applicable | |
17. | EDS | Running normally | 19293 | 0 | 0 | 01/04/10 16:40:22 | Not applicable | |
18. | EDS-GCF | Administrator has shut down this server | 0 | 0 | 0 | 01/04/10 16:11:39 | 01/04/10 16:33:46 | |
19. | ESS | Program started - No mgt msgs received | 18673 | 0 | 0 | 01/04/10 16:37:35 | Not applicable | |
20. | EssentialsDM | Running normally | 19468 | 0 | 0 | 01/04/10 16:41:15 | Not applicable | |
21. | EssMonitor | Running normally | 18677 | 0 | 0 | 01/04/10 16:37:39 | Not applicable | |
22. | EventFramework | Program started - No mgt msgs received | 18708 | 0 | 0 | 01/04/10 16:37:45 | Not applicable | |
23. | FDRewinder | Administrator has shut down this server | 0 | 0 | 0 | N/A | Not applicable | |
24. | ICServer | Administrator has shut down this server | 0 | 0 | 0 | 01/04/10 16:11:39 | 01/04/10 16:32:07 | |
25. | IpmDbEngine | Program started - No mgt msgs received | 18727 | 0 | 0 | 01/04/10 16:37:49 | Not applicable | |
26. | IPMOGSServer | Administrator has shut down this server | 0 | -113 | 0 | 01/04/10 16:12:03 | 01/04/10 16:31:52 | |
27. | IPMProcess | Program started - No mgt msgs received | 19595 | 0 | 0 | 01/04/10 16:41:37 | Not applicable | |
28. | jrm | Running normally | 19538 | 0 | 0 | 01/04/10 16:41:24 | Not applicable | |
29. | LicenseServer | Program started - No mgt msgs received | 19242 | 0 | 0 | 01/04/10 16:40:13 | Not applicable | |
30. | MACUHIC | Running normally | 18780 | 0 | 0 | 01/04/10 16:37:57 | Not applicable | |
31. | NameServer | Program started - No mgt msgs received | 19246 | 0 | 0 | 01/04/10 16:40:14 | Not applicable | |
32. | NameServiceMonitor | Running normally | 19287 | 0 | 0 | 01/04/10 16:40:18 | Not applicable | |
33. | OpsxmlDbEngine | Program started - No mgt msgs received | 18796 | 0 | 0 | 01/04/10 16:37:58 | Not applicable | |
34. | OpsXMLRuntime | Program started - No mgt msgs received | 19641 | 0 | 0 | 01/04/10 16:41:46 | Not applicable | |
35. | PMCOGSServer | Administrator has shut down this server | 0 | 0 | 0 | 01/04/10 16:11:39 | 01/04/10 16:32:09 | |
36. | ProcSysBus | Program started - No mgt msgs received | 19634 | 0 | 0 | 01/04/10 16:41:41 | Not applicable | |
37. | RMEDbEngine | Program started - No mgt msgs received | 18713 | 0 | 0 | 01/04/10 16:37:45 | Not applicable | |
38. | RMEOGSServer | Program started - No mgt msgs received | 19478 | 0 | 0 | 01/04/10 16:41:19 | Not applicable | |
39. | SyslogAnalyzer | Administrator has shut down this server | 0 | -113 | 0 | 01/04/10 16:11:59 | 01/04/10 16:31:55 | |
40. | SyslogCollector | Program started - No mgt msgs received | 18710 | 0 | 0 | 01/04/10 16:37:45 | Not applicable | |
41. | TomcatMonitor | Running normally | 18864 | 0 | 0 | 01/04/10 16:38:07 | Not applicable | |
42. | UTLITE | Running normally | 18784 | 0 | 0 | 01/04/10 16:37:58 | Not applicable | |
43. | UTMajorAcquisition | Administrator has shut down this server | 0 | 0 | 0 | N/A | Not applicable | |
44. | UTManager | Administrator has shut down this server | 0 | 2 | 0 | 01/04/10 16:11:39 | 01/04/10 16:32:04 | |
45. | VNMServer | Program started - No mgt msgs received | 18792 | 0 | 0 | 01/04/10 16:37:58 | Not applicable | |
46. | WlseUHIC | Running normally | 18788 | 0 | 0 | 01/04/10 16:37:58 | Not applicable |
Here's the status overnight this morning:
1. | 1177 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
2. | 1195 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
3. | 1293 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
4. | 1390 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
5. | 1391 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
6. | 1393 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
7. | 1394 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
8. | 1395 | Administrator has shut down this server | 0 | 0 | 0 | N/A | Not applicable | |
9. | 1398 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
10. | 1399 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
11. | 1400 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
12. | 1401 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
13. | 1402 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
14. | 1403 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
15. | 1404 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
16. | 1405 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
17. | 1406 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
18. | 1407 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
19. | 1408 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
20. | 1409 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
21. | 1410 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
22. | 1562 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
23. | 1993 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
24. | 1996 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
25. | 2034 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
26. | ANIDbEngine | Program started - No mgt msgs received | 18767 | 0 | 0 | 01/04/10 16:37:53 | Not applicable | |
27. | ANIServer | Running with busy flag set | 19377 | 0 | 0 | 01/04/10 16:40:49 | Not applicable | |
28. | CampusOGSServer | Administrator has shut down this server | 0 | 0 | 0 | 01/04/10 16:11:30 | 01/04/10 16:32:15 | |
29. | ChangeAudit | Administrator has shut down this server | 0 | -113 | 0 | 01/04/10 16:11:59 | 01/04/10 16:31:56 | |
30. | CmfDbEngine | Program started - No mgt msgs received | 19381 | 0 | 0 | 01/04/10 16:40:54 | Not applicable | |
31. | CmfDbMonitor | Failed to run | 0 | 0 | 6 | 01/04/10 16:11:14 | 01/04/10 16:40:58 | |
32. | CMFOGSServer | Program started - No mgt msgs received | 19460 | 0 | 0 | 01/04/10 16:41:10 | Not applicable | |
33. | ConfigMgmtServer | Administrator has shut down this server | 0 | -113 | 0 | 01/04/10 16:11:39 | 01/04/10 16:32:05 | |
34. | CSDiscovery | Administrator has shut down this server | 0 | 0 | 0 | N/A | Not applicable | |
35. | CSRegistryServer | Running normally | 18809 | 0 | 0 | 01/04/10 16:38:02 | Not applicable | |
36. | CTMJrmServer | Running normally | 19572 | 0 | 0 | 01/04/10 16:41:31 | Not applicable | |
37. | CWAPublisher | Administrator has shut down this server | 0 | 0 | 0 | 01/04/10 16:41:50 | 01/04/10 16:44:38 | |
38. | DCRDevicePoll | Administrator has shut down this server | 0 | 0 | 0 | N/A | Not applicable | |
39. | DCRServer | Running normally | 19400 | 0 | 0 | 01/04/10 16:40:58 | Not applicable | |
40. | diskWatcher | Running normally | 19630 | 0 | 0 | 01/04/10 16:41:41 | Not applicable | |
41. | EDS | Running normally | 19293 | 0 | 0 | 01/04/10 16:40:22 | Not applicable | |
42. | EDS-GCF | Administrator has shut down this server | 0 | 0 | 0 | 01/04/10 16:11:39 | 01/04/10 16:33:46 | |
43. | ESS | Program started - No mgt msgs received | 18673 | 0 | 0 | 01/04/10 16:37:35 | Not applicable | |
44. | EssentialsDM | Running normally | 19468 | 0 | 0 | 01/04/10 16:41:15 | Not applicable | |
45. | EssMonitor | Running normally | 18677 | 0 | 0 | 01/04/10 16:37:39 | Not applicable | |
46. | EventFramework | Program started - No mgt msgs received | 18708 | 0 | 0 | 01/04/10 16:37:45 | Not applicable | |
47. | FDRewinder | Administrator has shut down this server | 0 | 0 | 0 | N/A | Not applicable | |
48. | ICServer | Administrator has shut down this server | 0 | 0 | 0 | 01/04/10 16:11:39 | 01/04/10 16:32:07 | |
49. | IpmDbEngine | Program started - No mgt msgs received | 18727 | 0 | 0 | 01/04/10 16:37:49 | Not applicable | |
50. | IPMOGSServer | Administrator has shut down this server | 0 | -113 | 0 | 01/04/10 16:12:03 | 01/04/10 16:31:52 | |
51. | IPMProcess | Program started - No mgt msgs received | 19595 | 0 | 0 | 01/04/10 16:41:37 | Not applicable | |
52. | jrm | Running normally | 19538 | 0 | 0 | 01/04/10 16:41:24 | Not applicable | |
53. | LicenseServer | Program started - No mgt msgs received | 19242 | 0 | 0 | 01/04/10 16:40:13 | Not applicable | |
54. | MACUHIC | Running normally | 18780 | 0 | 0 | 01/04/10 16:37:57 | Not applicable | |
55. | NameServer | Program started - No mgt msgs received | 19246 | 0 | 0 | 01/04/10 16:40:14 | Not applicable | |
56. | NameServiceMonitor | Running normally | 19287 | 0 | 0 | 01/04/10 16:40:18 | Not applicable | |
57. | OpsxmlDbEngine | Program started - No mgt msgs received | 18796 | 0 | 0 | 01/04/10 16:37:58 | Not applicable | |
58. | OpsXMLRuntime | Program started - No mgt msgs received | 19641 | 0 | 0 | 01/04/10 16:41:46 | Not applicable | |
59. | PMCOGSServer | Administrator has shut down this server | 0 | 0 | 0 | 01/04/10 16:11:39 | 01/04/10 16:32:09 | |
60. | ProcSysBus | Program started - No mgt msgs received | 19634 | 0 | 0 | 01/04/10 16:41:41 | Not applicable | |
61. | RMEDbEngine | Program started - No mgt msgs received | 18713 | 0 | 0 | 01/04/10 16:37:45 | Not applicable | |
62. | RMEOGSServer | Program started - No mgt msgs received | 19478 | 0 | 0 | 01/04/10 16:41:19 | Not applicable | |
63. | SyslogAnalyzer | Administrator has shut down this server | 0 | -113 | 0 | 01/04/10 16:11:59 | 01/04/10 16:31:55 | |
64. | SyslogCollector | Program started - No mgt msgs received | 18710 | 0 | 0 | 01/04/10 16:37:45 | Not applicable | |
65. | TomcatMonitor | Running normally | 18864 | 0 | 0 | 01/04/10 16:38:07 | Not applicable | |
66. | UTLITE | Running normally | 18784 | 0 | 0 | 01/04/10 16:37:58 | Not applicable | |
67. | UTMajorAcquisition | Transient terminated | 0 | 0 | 0 | 01/05/10 00:00:07 | 01/05/10 00:34:51 | |
68. | UTManager | Administrator has shut down this server | 0 | 2 | 0 | 01/04/10 16:11:39 | 01/04/10 16:32:04 | |
69. | VNMServer | Program started - No mgt msgs received | 18792 | 0 | 0 | 01/04/10 16:37:58 | Not applicable | |
70. | WlseUHIC | Running normally | 18788 | 0 | 0 | 01/04/10 16:37:58 | Not applicable |
I also want to note that after my manual dmgtd restart, and prior to the installation of the single CiscoView Cat3560 pkg mentioned above, the only processes that didn't come back were:
CWAPublisher Administrator has shut down this server 0 0 0 01/04/10 1#:##:## 01/04/10 1#:##:##
DCRDevicePoll Administrator has shut down this server 0 0 0 N/A Not applicable
01-05-2010 09:01 AM
You definitely have a process problem. Has the /opt/CSCOpx/Psu.Lock disappeared? If so, go ahead and restart dmgtd manually, and see if the processes come back up.
01-05-2010 09:11 AM
After dmgtd stop, I still have these procs up. Should I go ahead with starting dmgtd, or is there something awry here?
casuser 23883 1 0 Dec 23 ? 0:11 /opt/vgi/csco/CSCOpx/objects/wfengine/program/PolicyScheduler
casuser 23878 1 0 Dec 23 ? 0:06 /opt/vgi/csco/CSCOpx/objects/wfengine/program/DataStore
casuser 23880 1 0 Dec 23 ? 0:13 /opt/vgi/csco/CSCOpx/objects/wfengine/program/GGLogger
casuser 23881 1 0 Dec 23 ? 0:05 /opt/vgi/csco/CSCOpx/objects/wfengine/program/GGProxy
casuser 23889 1 0 Dec 23 ? 0:04 /opt/vgi/csco/CSCOpx/objects/wfengine/program/GGHelper
casuser 23879 1 0 Dec 23 ? 0:07 /opt/vgi/csco/CSCOpx/objects/wfengine/program/GGScript
casuser 23882 1 0 Dec 23 ? 1:31 /opt/vgi/csco/CSCOpx/objects/wfengine/program/OpsBREngine
01-05-2010 09:13 AM
No, these need to be stopped manually before dmgtd can be properly restarted. Send them a SIGTERM, and if that doesn't work, move to a SIGKILL.
01-05-2010 09:34 AM
Those procs were killed with SIGTERM just fine. Here's the current proc status. Everything seems to have come up fine. Overall, this seems to mirror what I've seen with LMS 3.1 Device Pkg Update regarding its inability to restart everything properly though.
1. | 1177 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
2. | 1195 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
3. | 1293 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
4. | 1390 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
5. | 1391 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
6. | 1392 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
7. | 1393 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
8. | 1394 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
9. | 1395 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
10. | 1398 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
11. | 1399 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
12. | 1400 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
13. | 1401 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
14. | 1402 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
15. | 1403 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
16. | 1404 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
17. | 1405 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
18. | 1406 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
19. | 1407 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
20. | 1408 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
21. | 1409 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
22. | 1410 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
23. | 1562 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
24. | 1993 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
25. | 1996 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
26. | 2034 | Never started | 0 | 0 | 0 | N/A | Not applicable | |
27. | ANIDbEngine | Program started - No mgt msgs received | 19 | 0 | 0 | 01/05/10 12:23:35 | Not applicable | |
28. | ANIServer | Running with busy flag set | 585 | 0 | 0 | 01/05/10 12:26:33 | Not applicable | |
29. | CampusOGSServer | Program started - No mgt msgs received | 657 | 0 | 0 | 01/05/10 12:26:57 | Not applicable | |
30. | ChangeAudit | Program started - No mgt msgs received | 798 | 0 | 0 | 01/05/10 12:27:33 | Not applicable | |
31. | CmfDbEngine | Program started - No mgt msgs received | 586 | 0 | 0 | 01/05/10 12:26:37 | Not applicable | |
32. | CmfDbMonitor | Running normally | 622 | 0 | 0 | 01/05/10 12:26:41 | Not applicable | |
33. | CMFOGSServer | Program started - No mgt msgs received | 655 | 0 | 0 | 01/05/10 12:26:53 | Not applicable | |
34. | ConfigMgmtServer | Program started - No mgt msgs received | 704 | 0 | 0 | 01/05/10 12:27:06 | Not applicable | |
35. | CSDiscovery | Never started | 0 | 0 | 0 | N/A | Not applicable | |
36. | CSRegistryServer | Running normally | 71 | 0 | 0 | 01/05/10 12:23:43 | Not applicable | |
37. | CTMJrmServer | Running normally | 784 | 0 | 0 | 01/05/10 12:27:24 | Not applicable | |
38. | CWAPublisher | Program started - No mgt msgs received | 856 | 0 | 0 | 01/05/10 12:27:45 | Not applicable | |
39. | DCRDevicePoll | Never started | 0 | 0 | 0 | N/A | Not applicable | |
40. | DCRServer | Running normally | 623 | 0 | 0 | 01/05/10 12:26:42 | Not applicable | |
41. | diskWatcher | Running normally | 835 | 0 | 0 | 01/05/10 12:27:37 | Not applicable | |
42. | EDS | Running normally | 507 | 0 | 0 | 01/05/10 12:26:06 | Not applicable | |
43. | EDS-GCF | Running normally | 706 | 0 | 0 | 01/05/10 12:27:07 | Not applicable | |
44. | ESS | Program started - No mgt msgs received | 29954 | 0 | 0 | 01/05/10 12:23:17 | Not applicable | |
45. | EssentialsDM | Running normally | 658 | 0 | 0 | 01/05/10 12:26:57 | Not applicable | |
46. | EssMonitor | Running normally | 29955 | 0 | 0 | 01/05/10 12:23:21 | Not applicable | |
47. | EventFramework | Program started - No mgt msgs received | 29991 | 0 | 0 | 01/05/10 12:23:27 | Not applicable | |
48. | FDRewinder | Never started | 0 | 0 | 0 | N/A | Not applicable | |
49. | ICServer | Running normally | 703 | 0 | 0 | 01/05/10 12:27:06 | Not applicable | |
50. | IpmDbEngine | Program started - No mgt msgs received | 8 | 0 | 0 | 01/05/10 12:23:31 | Not applicable | |
51. | IPMOGSServer | Program started - No mgt msgs received | 834 | 0 | 0 | 01/05/10 12:27:37 | Not applicable | |
52. | IPMProcess | Program started - No mgt msgs received | 800 | 0 | 0 | 01/05/10 12:27:33 | Not applicable | |
53. | jrm | Running normally | 709 | 0 | 0 | 01/05/10 12:27:07 | Not applicable | |
54. | LicenseServer | Program started - No mgt msgs received | 465 | 0 | 0 | 01/05/10 12:25:58 | Not applicable | |
55. | MACUHIC | Running normally | 54 | 0 | 0 | 01/05/10 12:23:39 | Not applicable | |
56. | NameServer | Program started - No mgt msgs received | 466 | 0 | 0 | 01/05/10 12:25:58 | Not applicable | |
57. | NameServiceMonitor | Running normally | 503 | 0 | 0 | 01/05/10 12:26:02 | Not applicable | |
58. | OpsxmlDbEngine | Program started - No mgt msgs received | 58 | 0 | 0 | 01/05/10 12:23:39 | Not applicable | |
59. | OpsXMLRuntime | Program started - No mgt msgs received | 851 | 0 | 0 | 01/05/10 12:27:41 | Not applicable | |
60. | PMCOGSServer | Program started - No mgt msgs received | 702 | 0 | 0 | 01/05/10 12:27:06 | Not applicable | |
61. | ProcSysBus | Program started - No mgt msgs received | 836 | 0 | 0 | 01/05/10 12:27:37 | Not applicable | |
62. | RMEDbEngine | Program started - No mgt msgs received | 29993 | 0 | 0 | 01/05/10 12:23:27 | Not applicable | |
63. | RMEOGSServer | Program started - No mgt msgs received | 665 | 0 | 0 | 01/05/10 12:27:01 | Not applicable | |
64. | SyslogAnalyzer | Program started - No mgt msgs received | 799 | 0 | 0 | 01/05/10 12:27:33 | Not applicable | |
65. | SyslogCollector | Program started - No mgt msgs received | 29992 | 0 | 0 | 01/05/10 12:23:27 | Not applicable | |
66. | TomcatMonitor | Running normally | 118 | 0 | 0 | 01/05/10 12:23:47 | Not applicable | |
67. | UTLITE | Running normally | 55 | 0 | 0 | 01/05/10 12:23:39 | Not applicable | |
68. | UTMajorAcquisition | Never started | 0 | 0 | 0 | N/A | Not applicable | |
69. | UTManager | Running normally | 705 | 0 | 0 | 01/05/10 12:27:06 | Not applicable | |
70. | VNMServer | Program started - No mgt msgs received | 57 | 0 | 0 | 01/05/10 12:23:39 | Not applicable | |
71. | WlseUHIC | Running normally | 56 | 0 | 0 | 01/05/10 12:23:39 | Not applicable |
01-05-2010 10:47 AM
I have never seen such a problem on Solaris with LMS 3.1 or 3.2. Device package upgrades always restore a working system. If you can reliably reproduce, it may be helpful to open a TAC case on your next package update so one can look at the processes as they come back up.
Find answers to your questions by entering keywords or phrases in the Search bar above. New here? Use these resources to familiarize yourself with the community: