After reboot vmware-statsmonitor service doesn’t start automatically. In some situation, this service even doesn’t start manually and fail with following error.
root@buildvcenter [ ~ ]# service-control --start vmware-statsmonitor Operation not cancellable. Please wait for it to finish... Performing start operation on service statsmonitor... channel 3: open failed: administratively prohibited: open failed Error executing start on service statsmonitor. Details { "detail": [ { "args": [ "statsmonitor" ], "id": "install.ciscommon.service.failstart", "localized": "An error occurred while starting service 'statsmonitor'", "translatable": "An error occurred while starting service '%(0)s'" } ], "problemId": null, "resolution": null, "componentKey": null }
Following line can be seen in Vmon log (/var/log/vmware/vmon/vmon-syslog.log)
2019-08-08T09:26:42.809220-07:00 warning vmon Service api healthcheck command returned unknown exit code 1 2019-08-08T09:26:42.809574-07:00 notice vmon Re-check service health since it is still initializing. 2019-08-08T09:26:45.810708-07:00 notice vmon Constructed command: /usr/bin/python /usr/lib/vmware-vmon/vmonApiHealthCmd.py -n vmware-statsmonitor -f /var/vmware/applmgmt/statsmonitor_health.xml 2019-08-08T09:26:51.166333-07:00 warning vmon Service api-health command's stderr: Error getting service health. Error: Failed to read health xml file: /var/vmware/applmgmt/statsmonitor_health.xml. Error: [Errno 2] No such file or directory: '/var/vmware/applmgmt/statsmonitor_health.xml' 2019-08-08T09:26:51.166701-07:00 warning vmon 2019-08-08T09:26:51.194457-07:00 warning vmon Service api healthcheck command returned unknown exit code 1 2019-08-08T09:26:51.194832-07:00 notice vmon Re-check service health since it is still initializing. 2019-08-08T09:26:54.195953-07:00 notice vmon Constructed command: /usr/bin/python /usr/lib/vmware-vmon/vmonApiHealthCmd.py -n vmware-statsmonitor -f /var/vmware/applmgmt/statsmonitor_health.xml 2019-08-08T09:26:58.391456-07:00 notice vmon Service start operation timed out. 2019-08-08T09:26:58.394656-07:00 notice vmon Cancelling execution of pid 38052 2019-08-08T09:26:58.395009-07:00 warning vmon Found empty StopSignal parameter in config file. Defaulting to SIGTERM
This issue happens because of startup delay and can be fix by following steps:-
- Take snapshot of vCenter to be in safer side.
- Take SSH to VCSA using root login.
- Modify statsmonitor service config for vMon to set higher start up timeout:
sed -i '/StartTimeout/d' /etc/vmware/vmware-vmon/svcCfgfiles/statsmonitor.json sed -i '/ApiHealthFile/a "StartTimeout": 600,' /etc/vmware/vmware-vmon/svcCfgfiles/statsmonitor.json kill -HUP $(cat /var/run/vmon.pid)
- Stop and start stats-monitor service explicitly.
/usr/lib/vmware-vmon/vmon-cli -k statsmonitor /usr/lib/vmware-vmon/vmon-cli -i statsmonitor
- Then restart vCenter to see if that service start automatically