IIS Windows SSO with Multiple Hostnames

We had an issue were an old server (cobalt1.contoso.com) using IIS SSO was depcreated and a new server (cobalt2.contoso.com) took its place. The DNS record for the older server was updated with the new server's IP address, but while SSO worked when accessing cobalt2.contoso.com, it would prompt for credentials when accessing server1.contoso.com. Both hostnames were properly added to the intranet zone under the security tab in internet options. To resolve the issue: - Delete "cobalt1.contoso.com" computer object in Active Directory. - From elevated CLI on cobalt2 run “netdom computername cobalt2 /ADD:cobalt1.constos.com”. - From elevated CLI on cobalt2 run “ipconfig /registerdns”. This adds "HOST/cobalt1.contoso.com" to the "servicePrincipleName" attribute and "cobalt1.contoso.com" to the "msDS-AdditionalDnsHostName" attribute for the cobalt2 computer object. The hostname used for SSO needs needs to be in those attributes in addition to being in the intranet zone under the security tab in internet options.

8bit skull displayed on RDP login

When logging into a system via RDP what looks like a skull may be momentarily displayed:

After some testing, I believe it's caused by a slow transition from the user icon from the lock/login screen to the actual desktop, and during the transition that user icon can become an 8bit image like you saw. During testing there were a couple of time when the user icon would start to change into the "skull" but never completed because it went to the desktop too quickly.

Veeam MoRef (VM ID) matching

A common way to upgrade a VMware hosts is installing a fresh copy of ESXi and selecting the option to leave the existing datastore intact, and then added the VMs from the datastore to its inventory. Since the datastore hasn’t changed, the CBT data is still valid. However, when adding to inventory, the MoRef (VM ID) of the VM is changed, so Veeam sees it as a brand new VM. Therefore, you’d need to create new backup and replication jobs for the VMs, which when first run can take a very long time.

However, the Veeam SQL database can be edited to change the old MoRef to the new MoRef. Please note this only works if the datastore has not changed, if it has, then the CBT data is invalid, and you want new backups/replication to avoid data corruption.

NOTE: To directly edit query results in MS SQL, select “Edit Top 200 Rows” for the table you want, then in the tool bar click the “SQL” button which opens the query pane. You can then put your query in that pane, run it to update the results below, and directly edit them.





1. Before upgrading, connect to the host via PowerCLI and run the command below to output a list of VMs and their IDs. Save this data.

Get-VM | Get-View | Select-Object Name,MoRef




2. Upgrade the host, add the VMs to inventory, then re-run the command to find the new MoRefs.

3. Open the Veeam console, go to “Inventory” and properties on the upgraded server. Go to the “credentials” page and click “Apply”; if you need to update the credential information do it before clicking “Apply”. On the certificate prompt click “Continue”, and then finish. Once it’s done updating the host information, close and re-open the console.

4. Open SQL Management Studio on the Veeam backup server, look at the [Hosts] table, and record the ID of the target host that you upgraded.

SELECT * FROM [VeeamBackup].[dbo].[Hosts]




5. If you have replication jobs whose replicas are on the upgraded host, edit the [Replicas] table and update the old replica MoRef in the [target_vm_ref] column with the new replica MoRef. If you have multiple replicas on multiple hosts of a VM, reference the [target_location] column to locate the correct VM. If you’re only updating backup jobs you can skip this step.

SELECT * FROM [VeeamBackup].[dbo].[Replicas]




6. Edit the [BObjects] table. Display all rows from the [host_id] of the host you upgraded and update the MoRefs in the [object_id] column. If you have an old MoRef for a VM which doesn’t exist anymore and would duplicate one of the new MoRefs, increase its MoRef out of the range of IDs used, e.g. IDs “100003’ and “100004” below.

SELECT * FROM [VeeamBackup].[dbo].[BObjects] WHERE [host_id] = ‘<Upgraded Host ID>'




7. Run your backup and replication jobs. They should run normally, just like before the upgrade. Please be sure to commit any SQL changes before running the jobs, and it’s a good idea to simply restart the entire backup server if possible.

NOTE: If you already upgraded your host without recording the MoRefs first, you can still find what the old MoRefs were. If you refer back to step 6, you can resolve the unique ID for the VM in the [id] column to the VM name by finding that unique ID in the [BObject_id] column of the [BObjectsSensitiveInfo] table.

SELECT * FROM [VeeamBackup].[dbo].[BObjectsSensitiveInfo] WHERE [BObject_id] = '<VM unique ID>'

Cannot activate MS SQL 2017 Evaulation after expiration

You can install MS SQL as an evaluation version for 180 days, after which the SQL service will refuse to start, with an application log event saying the evaluation period has expired.

If the service is still running, but you're outside the 180 days, the edition upgrade freezes, because during the upgrade process it checks the SQL service and will stop it because its past 180 days, at which point the edition upgrade cannot continue because the service isn't running.

Microsoft provides a CLI command which allows you to activate it, but it too will either freeze or fails to actually activate SQL.

To fix this, if it's a domain attached SQL server, log in as a domain admin, then disconnect the network, then manually set the date back so it's back within the 180 day evaluation window. Then start the SQL service and do the activation through the GUI and it should work. You can check by going to properties for the SQL instance the SSMS. Once it's properly activated, you can set the clock back and reconnect the network.

Private SSL certificates

If your network uses a non-public domain name then you can't simply buy a public trust SSL certificate if you want to install certificates on your domain controllers to use LDAPS. Usually the recommendation is to install a Microsoft Certificate Authority server on the network and roll your own certificates. However, that's additional servers to manage and it can open a large security hole if not done perfectly right.

Instead, we found several companies which will sell you a private certificate. The cost is roughly the same as a public SSL certificate, and it saves you the hassle of standing up PKI on your network. Since it's not a public trust certificate, you'll also receive a root and intermediate certificate with the certificate you buy. We use Entrust Datacard, and have been very happy with them.

On the domain controller you add the root and intermediate certificates to the computer account and the computer certificate into the private store for the ADDS service account. Any computer which will communicate with the server over a secure protocol will also need the root and intermediate certificates installed. Once the certificates are in place, the DC should immediately start accepting LDAPS connections

Hot-spare warning on Dell servers

When assigning a global hot spare in a system with multiple arrays, the warning "The global hot-spare drive Disk 0 in Backplane 1 of Integrated RAID Controller 1 does not cover all the virtual drives (VDs) available on the controller" or "Insufficient space on the global hot spare Disk 0 in Backplane 1 of Integrated RAID Controller 1" might appear if one of the arrays either uses drives larger than the hot spare, or if the hot spare is an HDD and one of the arrays is SSD.

Find IP address on Cisco switch

I found two ways to locate an the port of an IP address on a Cisco switch.

If the IP address is assigned by DHCP and you have DCHP Snooping turned on (which you should), then you can use "sh ip dhcp snooping binding" to show all DHCP IP leases that went through that switch. 

If DHCP Snooping is not turned on and DHCP isn't used to assign the IP address, first ping the IP you're looking for on the switch. Then, run "sh ip arp" and get the mac address of the IP. Finally, use "sh mac address-table | include <mac address>" to show the interface the IP is on.