ESXi host unresponsive due to missing entries in endpoints.conf

Following symptoms can be seen.

  • Accessing ESXi host using web (https://hostnameORIP/ui) shows Error 404 not found.
  • Inside vCenter (flash/html), ESXi host status shows Not responding.
  • SSH to host works however Hostd service status show not running .
[root@TH-P-VDI1:~] /etc/init.d/hostd status
hostd is not running.

When we manually start this service, it stop after couple of minutes.

[root@esxi-host1:~] /etc/init.d/hostd start

At the end of hostd.log (/var/log) following lines can be seen.

2019-05-15T20:37:49.154Z error hostd[89FFB80] [Originator@6876 sub=Httpnfcsvc] Proxy endpoint list does not contain /ha-nfc entry
2019-05-15T20:37:49.154Z panic hostd[89FFB80] [Originator@6876 sub=Default] error: N5Vmomi5Fault11SystemError9ExceptionE(vmodl.fault.SystemError)
[context]zKq7AUoCAgAAACLjjQAOaG9zdGQAAJ+9LWxpYnZtYWNvcmUuc28AAPJpEgA/1g0B700qaG9zdGQAAf7LLAGNYDAB6lkwABVPDwGv+ZUBl9KVAcSQlQFJ3ykCZ4sBbGliYy5zby42AAEd9yk=[/context]
2019-05-15T20:37:49.157Z panic hostd[89FFB80] [Originator@6876 sub=Default] backtrace:

[backtrace begin] product: VMware ESX, version: 6.5.0, build: build-9298722, tag: hostd, cpu: x86, os: esx, buildType: release
backtrace[00] libvmacore.so[0x002DBD9F]: Vmacore::System::Stacktrace::CaptureWork(unsigned int)
backtrace[01] libvmacore.so[0x001269F2]: Vmacore::System::SystemFactoryImpl::CreateQuickBacktrace(Vmacore::Ref&)
backtrace[02] libvmacore.so[0x000DD63F]: Vmacore::Throwable::Throwable(std::string const&)
backtrace[03] hostd[0x002A4DEF]
backtrace[04] hostd[0x002CCBFE]
backtrace[05] hostd[0x0030608D]
backtrace[06] hostd[0x003059EA]
backtrace[07] libvmacore.so[0x000F4F15]
backtrace[08] hostd[0x0095F9AF]
backtrace[09] hostd[0x0095D297]
backtrace[10] hostd[0x009590C4]
backtrace[11] hostd[0x0029DF49]
backtrace[12] libc.so.6[0x00018B67]
backtrace[13] hostd[0x0029F71D]
[backtrace end]

 

Line ‘Proxy endpoint list does not contain /ha-nfc entry’ indicate the issue with RHTTPPRoxy service. This issue comes because either endpoints.conf is missing OR it doesn’t contain appropriate endpoints information. Use following steps to fix the issue.

Take backup of endpoints.conf file.

[root@esxi-1:~] cp /etc/vmware/rhttpproxy/endpoints.conf /etc/vmware/rhttpproxy/Backup-endpoints.conf

 

Modify endpoints.conf file using text editor. Get the lines from any working ESXi of same version from production OR test environment then paste. Typically endpoints.conf should contain following lines.

/    local    8309    redirect    allow
/cgi-bin    local    8303    redirect    allow
/client/clients.xml    local    8309    allow    allow
/fdm    local    9089    reject    allow
/folder    local    8309    redirect    allow
/guestFile    local    8309    redirect    allow
/ha-nfc    local    12001    allow    allow
/host    local    8309    redirect    allow
/mob    namedpipe    /var/run/vmware/proxy-mob    redirect    allow
/mobfdm    local    9090    redirect    allow
/nfc    local    12000    allow    allow
/screen    local    8309    redirect    allow
/sdk    local    8307    redirect    allow
/sdkTunnel    namedpipetunnel    /var/run/vmware/proxy-sdk-tunnel    allow    reject
/ticket    tickettunnel    /var/run/vmware/ticket/%1    redirect    allow
/tmp    local    8309    redirect    allow
/vpxa    local    8089    reject    allow
/vsan    local    9096    redirect    allow
/vsanperf    localtunnel    9095    allow    reject
/vvold    local    8090    allow    allow
/wsman    local    8889    redirect    allow

 

Once done then start hostd services.

[root@esxi-host1:~] /etc/init.d/hostd start

Now host should start responding to vCenter.

 

Advertisements