In one datacenter of my company, there was a strange problem last month of VMware vSphere.
Here is our structure:
SqlServer instance for vSphere is installed dedicated on cluster server. SqlServer version is 2012 Enterprise and Windows is 2012R2 Standard
vCenter is installed on another server.
After we applied July patches from Microsoft and restart the vCenter server, nearly all services from VMware cannot start. In event viewer, one event said:
The VMware Inventory Service service terminated with the following service-specific error:
As the request from VMware support, I uninstalled all updates related and reboot but it won’t help. I thought there may be some error related to the database connection. I ran the connection test from ODBC setup window, test finished succeeded. So I collected and submitted the log file generated from vCenter server command line. You know it’s really huge.
2 weeks later, I got an email from the high level engineers of VMware. By digging the log, they found that the vCenter cannot connect to SqlServer and the tcp service port is specified. At that moment, thanks to documentation I wrote :), I found that tcp service port is different than it should be. When I checked the SqlServer, I found the reason really weird: the TCP port of this instances is modified, automatically and silently. Due to I really used vSphere client just before upgrading Windows, I’m quite sure this issue is related to at least one of the patch which applied on the database cluster, launched by Microsoft in July 2016. And after a search, there is another instance of SqlServer which has the port changed. Due to the port changed and our firewall policy is set based tcp port, the client, vCenter in this issue, cannot connect to this SqlServer instance. After the port setting changed back and instance restarted, vCenter is back to normal.
Don’t ask me why ODBC test passed without any problem. If you know the answer, I’m listening as well.
This is a record for upgrading vSphere 5.5 to 6. In this scenario, database for vCenter is a powered by SqlServer on a dedicated server.
Phase 1: Prepare
- Before upgrading, something need to be ready.
- Password of vCenter domain account.
- Password of vCenter SSO.
- ISO files for vCenter and ESXi. These files should be tested by checking MD5 or SHA hash codes.
- License Keys. vCenter and ESXi keys can be gotten from VMware site by upgrading the currents.
- vCenter server and databases should be backed up.
- vSphere Update Manager Client (not server side) and vSphere Client should be uninstalled.
Phase 2: Upgrade vCenter
Mount the ISO of vCenter and perform the upgrading. Passwords of SSO and account will be required by this process.
The current instance of vCenter will be detected by setup program and all settings will be preserved. Current settings will be exported and will be imported automatically but not be removed after.
This step will need more than 1 hour.
At the end of this process, the post upgrade steps will be prompted including entering new license key and deleting the exported setting folder. You have to wait several minutes for web client service initializing. This is a good time for upgrading Update Manager Server and PowerCLI.
Phase 3: Install vSphere Client
Install vSphere Client and Update Manager Extension. If database error is occurred in extension installing, you need to check the account for running the VMware vSphere Update Manager Service. It should be a domain account but will be replaced to system account by setup process incorrectly.
Phase 4: Upgrade VM tools
You have to upgrade all VM tools one by one.
Although there are many official documents about vSphere 6 launched recently, the release date of this product is still not uncovered.
One sure thing is there will be a new product, vFabric Postgres database, to replace the SQL server express database. The external database is still supported as well.
I will post a workthough for upgrading after it launched.
I’ve just change this site from VM based Windows Azure into a Web site based mode to mitigate the jobs for maintenance.
It still slow and not stable as predictable.
Azure service is really not good, especially for PHP+MySQL.
After a clean installation process, firewall of Hyper-V Server is set to deny any remote connection by default.
If you need to management other than using console, you may want to enable inbound rules for Remote Management and Remote Desktop by running these commands in the console.
To enable the Remote Management:
cscript C:\windows\system32\scregedit.wsf /im 1
To enable the Remote Desktop:
cscript C:\windows\system32\scregedit.wsf /ar 0
And, you need to enable the remote desktop from the sconfig (the blue console window) also.
Microsoft provides a new way to activate VMs hosted in Hyper-V, named Automatic Virtual Machine Activation (AVMA). It makes the activation step of VMs much easier.
- Hypervisor: Windows Server 2012 R2 Datacenter with Hyper-V role. Other versions are not supported. Dedicated Hyper-V Server is not supported.
- VM: Windows Server 2012 R2 Datacenter, Standard and Essentials.
- Prepare and activate your Hypervisor;
- Install supported OS as VM with the key listed below, or change the key of an installed VM with command “slmgr /ipk <key>”.
- Windows Server 2012 R2 Datacenter: Y4TGP-NPTV9-HTC2H-7MGQ3-DV4TW
- Windows Server 2012 R2 Standard: DBGBW-NPF86-BJVTX-K3WKJ-MTB6V
- Windows Server 2012 R2 Essentials: K2XGM-NMBT3-2R6Q8-WF2FK-P36R2
These keys are supported to being used in any unattend.exe setup file also.
This document is about upgrading vSphere 5.1 to 5.1 Update 1.
Phase 1: Preparation
You need these listed below:
- vCenter Installation Disc (iso): VMware-VIMSetup-all-5.1.0-1065152.iso;
- ESXi update package (zip, optional, will be used in Update Manager): update-from-esxi5.1-5.1_update01.zip;
- SSO admin password;
- vCenter service account password;
- An account within local administrators group of vCenter server; and,
- If you have Update Manager Client Plug-in installed on the same server of vCenter, it should be uninstalled before upgrading.
Phase 2: Upgrading vCenter and additions
- Insert vCenter Installation disc or mount iso.
- You cannot perform an upgrading installation by VMware vCenter Simple Install. It is required to upgrade all components manually as the same sequence as items listed in VMware vCenter Installer: vCenter Single Sign On, VMware vCenter Inventory Service, VMware vCenter Server, VMware vSphere Client (optional), VMware vSphere Web Client (optional) and VMware vSphere Update Manager.
- While upgrading the SSO (vCenter Single Sign On), you will be notified to enter the SSO admin password. A restart is required after performing SSO upgrading.
- Nothing more information is required while upgrading Inventory Service.
- Password of vCenter service account is required while upgrading vCenter Server. It is possible but highly not recommended to change service address or port while upgrading.
- All other 3 upgrading tasks are easy to perform. All you need is press Next button after starting the processes.
- (Optional) Check the server status by logging on from vSphere Client.
- (Optional) Install the Update Manager Client Plug-in now.
Phase 3: Upgrading ESXi
- Log on vSphere Client with Update Manager Client Plug-in.
- (Optional) Uploading ESXi upgrading package
- In the Update Manager Tab of the top level of vCenter Hosts and Clusters (top item from the left panel), select Admin View.
- In the Patch Repository tab, click Import Patches and upload update-from-esxi5.1-5.1_update01.zip. It will save downloading time of ESXi upgrading process.
- Go back to Compliance View.
- Scan your ESXi server.
- (Optional) Stage. You may want to stage one server first to test your patch repository before staging all servers.
- Remediate. You may need to set the servers in maintenance mode but not necessary. ESXi servers will be rebooted automatically after remediation.
Phase 4: VMtool upgrading
Upgrade VMtool on each virtual machine.
Phase 5: vSphere Client upgrading
Upgrade vSphere Client on all installed computer.
Phase 6: Troubleshooting
If there are some templates in your vSphere, you may need to convert them to virtual machine to upgrading VMtool. If this function is grayed out, you can re-enabled it by removing the related template from inventory and adding it back.
After deployment of Hyper-V Server, I trapped in trouble of linking Hyper-V Server to SCVMM.
Finally, I find a way to shutdown firewall of Hyper-V Server, which helps me resolved the problem.
- Log on to Hyper-V Server;
- Type this into cmd window:
netsh advfirewall set allprofiles state off
I have some hyper-v servers with broadcom NICs. Due to lack of UI (device manager and property window of NIC), it’s not easy to enable Jumbo Frame (MTU9000) setting.
After a google dig and some experiment, I find a way to achieve that.
- Install BACS (Broadcom Advanced Control Suite). You may want to use this tool for creating team also. Remember the 4-digit number within brackets before the NIC you want to change MTU.
- Open regedit, expand HKEY_LOCAL_MACHINE\CurrentControlSet\Control\Class\4D36E972-E325-11CE-BFC1-08002BE10318}\(4-digit-number-of-NIC) on the left panel, you will get a value named “*JumboMTU” on the right. Change it to 9014 or any other value you need.
- Type this command “netsh int ipv4 sh int” to list all network. Remember the Name of the connection you want to change MTU.
- Type this command to change it to 9000 or any other value. The value should be the value you’ve set in step 2, minus 14:
netsh interface ipv4 set subinterface “Connection Name” mtu=9000 store=persistent
- Reboot to apply.
We changed our domain controllers of our company these days. Of course, we did that by SOP:
- use dcpromo to promote new servers as domain controllers;
- migrate PDC and other critical roles to the new domain controllers;
- use dcpromo to depromote the old domain controllers.
After these steps, all Windows servers works great.
But we can use neither vSphere Client nor vSphere Web Client to login vCenter with a domain account any more.
By asking VMware support, we realize we have to adjust Single Sign On for this change.
- Open vSphere Web Client, using admin@system-Domain to login;
- Select Sign-On and Discovery – Configuration from the left panel;
- You will the a record related to your domain on the right;
- Select the record and make a screen capture of all information;
(You cannot edit it directly, cos it won’t save successfully.)
- Delete that record and create a new one, with the same setting and new LDAP address;
(For Authentication Type, the existed one may set to Reuse Session, but you have to change it Password and leave an account with password below. I don’t know the reason.)
- Save it.