Strange problem: VMware Inventory Service cannot start due to Microsoft Update (July 2016)

In one datacenter of my company, there was a strange problem last month of VMware vSphere.

Here is our structure:
SqlServer instance for vSphere is installed dedicated on cluster server. SqlServer version is 2012 Enterprise and Windows is 2012R2 Standard
vCenter is installed on another server.

After we applied July patches from Microsoft and restart the vCenter server, nearly all services from VMware cannot start. In event viewer, one event said:

The VMware Inventory Service service terminated with the following service-specific error:
Incorrect function.

As the request from VMware support, I uninstalled all updates related and reboot but it won’t help. I thought there may be some error related to the database connection. I ran the connection test from ODBC setup window, test finished succeeded. So I collected and submitted the log file generated from vCenter server command line. You know it’s really huge.

2 weeks later, I got an email from the high level engineers of VMware. By digging the log, they found that the vCenter cannot connect to SqlServer and the tcp service port is specified. At that moment, thanks to documentation I wrote :), I found that tcp service port is different than it should be. When I checked the SqlServer, I found the reason really weird: the TCP port of this instances is modified, automatically and silently. Due to I really used vSphere client just before upgrading Windows, I’m quite sure this issue is related to at least one of the patch which applied on the database cluster, launched by Microsoft in July 2016. And after a search, there is another instance of SqlServer which has the port changed. Due to the port changed and our firewall policy is set based tcp port, the client, vCenter in this issue, cannot connect to this SqlServer instance. After the port setting changed back and instance restarted, vCenter is back to normal.

Don’t ask me why ODBC test passed without any problem. If you know the answer, I’m listening as well.

Work through: Upgrade vSphere 5.5 to 6

This is a record for upgrading vSphere 5.5 to 6. In this scenario, database for vCenter is a powered by SqlServer on a dedicated server.

 

Phase 1: Prepare

  • Before upgrading, something need to be ready.
  • Password of vCenter domain account.
  • Password of vCenter SSO.
  • ISO files for vCenter and ESXi. These files should be tested by checking MD5 or SHA hash codes.
  • License Keys. vCenter and ESXi keys can be gotten from VMware site by upgrading the currents.
  • vCenter server and databases should be backed up.
  • vSphere Update Manager Client (not server side) and vSphere Client should be uninstalled.

 

Phase 2: Upgrade vCenter

Mount the ISO of vCenter and perform the upgrading. Passwords of SSO and account will be required by this process.

The current instance of vCenter will be detected by setup program and all settings will be preserved. Current settings will be exported and will be imported automatically but not be removed after.

This step will need more than 1 hour.

At the end of this process, the post upgrade steps will be prompted including entering new license key and deleting the exported setting folder. You have to wait several minutes for web client service initializing. This is a good time for upgrading Update Manager Server and PowerCLI.

 

Phase 3: Install vSphere Client

Install vSphere Client and Update Manager Extension. If database error is occurred in extension installing, you need to check the account for running the VMware vSphere Update Manager Service. It should be a domain account but will be replaced to system account by setup process incorrectly.

 

Phase 4: Upgrade VM tools

You have to upgrade all VM tools one by one.

Get ready for embracing vSphere 6

Although there are many official documents about vSphere 6 launched recently, the release date of this product is still not uncovered.

One sure thing is there will be a new product, vFabric Postgres database, to replace the SQL server express database. The external database is still supported as well.

I will post a workthough for upgrading after it launched.

Enable Remote Management for Hyper-V Server

After a clean installation process, firewall of Hyper-V Server is set to deny any remote connection by default.

If you need to management other than using console, you may want to enable inbound rules for Remote Management and Remote Desktop by running these commands in the console.

To enable the Remote Management:

cscript C:\windows\system32\scregedit.wsf /im 1

To enable the Remote Desktop:

cscript C:\windows\system32\scregedit.wsf /ar 0

And, you need to enable the remote desktop from the sconfig (the blue console window) also.

AVMA: An easier way to activate Windows Server 2012 R2 in Hyper-V

Microsoft provides a new way to activate VMs hosted in Hyper-V, named Automatic Virtual Machine Activation (AVMA). It makes the activation step of VMs much easier.

Requirement:

  • Hypervisor: Windows Server 2012 R2 Datacenter with Hyper-V role. Other versions are not supported. Dedicated Hyper-V Server is not supported.
  • VM: Windows Server 2012 R2 Datacenter, Standard and Essentials.

Steps:

  1. Prepare and activate your Hypervisor;
  2. Install supported OS as VM with the key listed below, or change the key of an installed VM with command “slmgr /ipk <key>”.

Keys:

  • Windows Server 2012 R2 Datacenter: Y4TGP-NPTV9-HTC2H-7MGQ3-DV4TW
  • Windows Server 2012 R2 Standard: DBGBW-NPF86-BJVTX-K3WKJ-MTB6V
  • Windows Server 2012 R2 Essentials: K2XGM-NMBT3-2R6Q8-WF2FK-P36R2

These keys are supported to being used in any unattend.exe setup file also.

 

Source: http://technet.microsoft.com/en-us/library/dn303421.aspx