If you have been reading up about the features of Hyper-V 2.0, you probably read about Clustered Shared Volumes or CSVs. Basically, CSVs are the main ingredient to the (not so) secret sauce that makes Live Migration work.

A clustered shared volume can be created from any existing volume in a failover cluster. If you have an existing volume formatted with NTFS, you can mark it as a CSV and it will show up in the Failover Cluster Manager as a CSV. Note that existing data on the NTFS volume is not touched in any way.

On the compatibility side you should know the following:

1. CSVs in Windows Server 2008 R2 only support Hyper-V
2. There are no special hardware requirements and you can use the same technologies as a standard cluster disk: iSCSI, FC, SAS
3. There is no limit on directory structure
4. No agents or other software needs to be installed
5. Not another file system – it is standard NTFS

Note that this is not a clustered file system like VMware VMFS of Sanbolic’s Melio FS. There is no distributed lock manager or anything like that. What CSVs deliver is a distributed access file system.

CSVs provide a single consistent file name space. This means that every node in the cluster has the same path to the volumes. You will see the CSVs under the %windir%\ClusterStorage directory. For example:

• c:\ClusterStorage\Volume1\<root>
• c:\ClusterStorage\Volume2\<root>

The only thing you need to do after adding the CSVs is to put VHDs on the clustered shared volume (using the paths shown above) and create the virtual machine using Hyper-V Manager making sure that you put the configuration files on the CSV as well. You can then use the Failover Cluster Manager console to make the virtual machine highly available just like how you do it now. Basically, the only thing that changes here is the creation of the CSV and making sure that you place your virtual machine on the CSV. All the other steps are exactly the same as how it currently works.

With the current implementation of Hyper-V, you need to put each virtual machine in its own LUN if you want the virtual machines to failover to different hosts or if you want to quick migrate the virtual machines independently from the others. CSVs in Windows Server 2008 R2 remove this limitation. This also results in less wasted storage space and basically a better storage management story.

From an architectural perspective you should know that only one node in the failover cluster still owns the CSV LUN. That node is called the coordinator and it basically manages how the disk access should work. There is one such coordinator for each shared volume. You do not have to worry about selecting a coordinator as that is taken care of automatically. When the coordinator fails, there is no impact on virtual machines that are not running on the coordinator itself.

Microsoft did not provide any clear information about how the access to disk is actually performed. Basically the answer to our questions was that the coordinator decides what the best path to the storage is: direct access or over the network. Yes, you read that right: access to the LUN from one node might be redirected over the network via SMB to another node that writes to the LUN. Again, I have no information about when that exactly happens.

We also learned that the implementation of CSVs is done with a mini file system driver called CSVfilter.sys. Being a new filter driver, there might be some impact on other components such as backup software. Microsoft is working with backup and storage vendors and its own DPM team to clear up these issues. What I can say is that there is a new VSS API call called PrepareVolumeForSnapshotSet and that this needs to be used by VSS Requesters. The node that initiates the VSS backup also needs to become the coordinator node (happens automatically) because it needs to manage exactly what happens with the CSV volumes. The CSV volumes stay online during the backup but there will be no direct I/O until the backup ends. I guess that means that access to the LUN from other nodes will go over the network to the coordinator and from there to the LUN.

What is my take on this? Well, it seems like an overly complex solution to make something like Live Migration and HA work for individual virtual machines on a single bigger LUN. I guess that results from the fact that Microsoft decided to work with the existing NTFS file system instead of creating a dedicated file system optimized for virtual machine storage. On the other hand it does deliver extra features such as SAN I/O redirection over the network when the storage path to the SAN array completely fails.

More information about CSVs can be found in the R2 reviewers guide.

This week, I had to change the VirtualCenter certificate in our test environment. Changing the certificate is not that straightforward but there is quite some information available on the net. I used the information in this post: http://lraikhman.blogsite.org/?p=41.

Basically, you use OpenSSL to generate a public/private key pair and a certificate request. You then pass the certificate request to a certificate authority. I used a Microsoft Enterprise CA running on Windows Server 2008. When you obtain the certificate from the CA, you create a pfx file with OpenSSL that contains the private key and the certificate. You can then copy the files to your VirtualCenter server.

Note that when you generate the pfx file with OpenSSL, you will be asked for a password. To use the pfx file, for example to extract the certificate or the private key, the password is needed.

Now if you follow the steps you will be unable to use your existing sysprep customizations in VirtualCenter because the administrator password that is encrypted in the customization file cannot be decrypted. You will see something like:

The reason for this error is quite simple: VirtualCenter cannot get to the data in the pfx file because the password you used to protect the pfx is not correct. It seems that the password you need to use for the pfx file is testpassword. I got that information from here.

To summarize: to change the VC certificate use the steps in http://lraikhman.blogsite.org/?p=41 but when you are asked for the password during pfx creation use testpassword.

A hot topic with any virtualization platform is virtual machine backup. Besides the traditional backup with a backup agent in the guest you can also backup the complete virtual machine as it sits on the file system on the virtualization host (in this case Windows Server 2008). Naturally, you will not want to shutdown your virtual machines just to take a backup so Hyper-V supports hot backups of running virtual machines with the Hyper-V VSS Writer.

Let's first see if the Hyper-V VSS Writer is available on the host. Issue the following command:

vssadmin list writers 

In some instances, the Hyper-V VSS writer is not listed even though Hyper-V is installed on the host. This could be due to broken symbolic links in the following directory: C:\ProgramData\Microsoft\Windows\Hyper-V\Virtual Machines. Remove the broken link files and issue the vssadmin list writers command again.

We will now perform a manual backup with the help of the Windows Server 2008 diskshadow.exe command. Open a command prompt and issue the diskshadow command. You will get a prompt inside diskshadow. Issue the following commands:

set context persistent

add volume <driveletter> alias <alias_you_choose>
(for example add volume s: alias svolume)

set verbose on

create

When you enter the create command, the VSS operation will create a new shadow copy. In the output you should see something like:

At the end of the output, you should see:

You can now enter the following command in diskshadow:

expose %alias% x:
(for example expose %svolume% x:)

The last command mounts the shadow copy on drive x: so that you have access to the backup. If you look at the contents of drive x:, it will be the contents of the volume you specified at the time the backup was taken. You can now copy the files from drive x: to some other location.

Note that the Hyper-V VSS writer will take care of the consistency of the virtual machines both outside and inside the vhd file. As long as you are using the integration components in the virtual machine (as you should), the virtual machine will stay running during the operation. If the volume contains virtual machines that do not have integration components, these virtual machines will be suspended to disk before the shadow copy is made and resumed afterwards. In that case, the shadow copy will contain the .bin and .vsv file with the state of the virtual machine at the time of the backup.

Inside the virtual machine that has the integration components you will see events in event viewer that indicate VSS activity:

A couple of notes on this process:

• You can use add volume multiple times for different volumes. Naturally, for each volume a shadow copy will be created.
• If you turn off the backup support of the integration components (can be done from Hyper-V manager or SCVMM 2008), the writer will suspend/resume the virtual machine.
• You can integrate the Hyper-V VSS Writer with Windows Server 2008 Backup. See http://technet.microsoft.com/en-us/magazine/cc895627.aspx for more information. By default, Windows Server Backup is not configured to use the Hyper-V VSS Writer.
• Backup software vendors are working to include support for the Hyper-V VSS Writer. BackupExec for instance should have support in version 12.5.

A following post will discuss how to restore a virtual machine from the backup location.

My colleague, Tim Jacobs, has published a good article covering virtual machine backups and VSS integration. In my opinion, VSS clearly is the way to go because the VMware sync driver or other proprietary solutions just don't cut it. Microsoft already included VSS support in Virtual Server 2005 R2 SP1 and also has it in Hyper-V.

Check out the article on his blog.

In the previous post, I talked about creating highly available virtual machines on Hyper-V and the fact that SCVMM can only create virtual machines on unused storage. The standard tools such as Hyper-V Manager and Failover Cluster Manager do support multiple virtual machines on the same volume (with a hotfix). So what happens to SCVMM in that case? Have a look at the screenshot below:

The virtual machines are in an unsupported configuration and cannot be managed from SCVMM (no starting, stopping etc...). This will probably be fixed in the final release but if you are testing the product now you know what to expect.

When you need to create a highly available virtual machine running on Microsoft's Hyper-V you need to do so on a Failover Cluster. In this post I will show you how to create a highly available virtual machine on a two-node Windows Server 2008 cluster. The cluster is using simple desktop machines with storage on an MSA2012i SAN (iSCSI).

There are two ways to create the virtual machine:

1. Use the Hyper-V Manager in conjunction with Failover Cluster Management.
2. Use System Center Virtual Machine Manager 2008 (SCVMM).

The first method is a bit clumsy because it involves two different consoles. That method, however, has more flexibility as you will see later. Let's take a look at the first method and see how you could create a virtual machine in the cluster on a shared volume.

Creating the virtual machine without SCVMM
Without SCVMM you need Hyper-V Manager and Failover Cluster Manager. To start, you need to have a volume available to the cluster. In my case, I had a volume with drive letter S: as available storage to the cluster as shown in the screenshot below.

Even a volume that is not part of a service or application (a resource group in old terminology) is owned by a particular host. In the screenshot you see that hyperv2 owns the disk.

The next step is to use Hyper-V Manager to create a virtual machine. It is important that you store both the virtual machine configuration and hard disk(s) on shared storage. To create the virtual machine, open Hyper-V Manager on the host that owns the disk and create a new virtual machine as shown below. This is the same as on a standalone host as this tool is not aware of the cluster.

A wizard will start to create the virtual machine. Just follow the wizard and make sure you use shared storage. We can now proceed to the next step and that is adding the machine to the cluster.

Now open Failover Cluster Management, right click Services and Applications and click Configure a Service or Application. A wizard will start. Follow the wizard until the screen below and select Virtual Machine (only available when Hyper-V is installed):

When you click Next, the wizard will enumerate the virtual machines on shared storage so that you can select them:

After the virtual machine has been selected, continue the wizard. The end result will be a new service called Virtual Machine with two resources (well, actually three): the virtual machine configuration, the virtual machine and the storage volume. The screenshot below illustrates what you see in the Failover Cluster Management console after running the wizard a few times to add other virtual machines on the same volume.

You can now start the virtual machine directly from Failover Cluster Management and install a guest operating system. If you need to attach an iso first, you will need to use Hyper-V Manager. The Failover Cluster Management tool provides an easy way to start Hyper-V Manager. Just right click the service (here called by default Virtual Machine).

From the moment the virtual machine is part of the cluster you will need to perform actions such as stopping, shutting down and saving state from Failover Cluster Management. If you use Hyper-V Manager to stop the virtual machine for example, the cluster will restart the virtual machine again (with the default resource settings).

Creating the virtual machine with SCVMM
SCVMM, being the 'single pane of glass' for virtualization management provides a way to create new virtual machines on both clusters and standalone Hyper-V hosts. Before you can create a virtual machine on a cluster you need to add the cluster to SCVMM. From the Actions pane, select Add Host as shown below:

In the Add Hosts wizard, type the cluster name in Host server name and press Enter. The cluster should be added to the list as shown below:

Now continue the wizard to deploy the SCVMM agent on each node in the cluster (=automatic). The cluster will be shown in SCVMM as shown below:

Adding a virtual machine is easy but the current beta of SCVMM has some limitations:

• SCVMM only supports one virtual machine per cluster volume. This effectively means I need one LUN per virtual machine, something I need anyway if I want to 'Quick Migrate' individual virtual machines.
• SCVMM only supports volumes with drive letters (no mount points).

Before you create the virtual machine, make sure there is available storage in the cluster that is not used by any other virtual machine. To create the virtual machine, just select New Virtual Machine from the Actions pane at the right to launch the wizard. The wizard is straightforward but you should not forget to configure the virtual machine as highly available. You do so from the Configure Hardware dialog in the wizard. Scroll to the bottom where there is an Availability section. When the wizard is finished, SCVMM will have created a new service in the cluster called SCVMM vmname Resources. You can now start the virtual machine from SCVMM and start installing a guest operating system.

Conclusion
Creating a highly available virtual machine is easy but as you have seen, there are several ways to do it. There are also some discrepancies between what you can do with and without SCVMM. The final version of SCVMM will have to solve this so that you can truly use it as your primary management tool for Hyper-V.

REMARK: I my setup I installed the following hotfix: http://support.microsoft.com/?id=951308. This hotfix provides more functionality and virtual machine control for Hyper-V in a failover cluster.

Application link is a feature VMware added to the Thinstall product they acquired a while ago. As the feature name implies, it allows you to link ThinApp'ed applications. You could for example link a browser to a Flash plugin or a Java plugin. In this post I will show you the basics of application link. I will link Opera 9.5 to a Flash plugin. The steps to get this to work are as follows:

1. Create separate projects for Opera 9.5 and the Flash plugin.
2. Configure the package.ini of Opera 9.5 with the application link.
3. Deliver the packages to end-users taking into account the placement of the plugins.

This post does not discuss the first step. Check the two previous blog posts to see how to do that. When you create the project for the Flash plugin, download a standalone installer for Flash and use it during Setup Capture. I ended up with the following projects:

The second step, configuring the application link, is easy. In the package.ini of Opera 9.51 you will find the following line:

Just remove the ; in front of the highlighted line and build the project using build.bat. With this default configuration (plugins\*.exe) you need to put a plugins folder in the folder that contains Opera.exe:

The Flash project resulted in a file called NPSWF32_FlashUtil.exe and that file is put in the plugins folder. Because application link is configured with *.exe, the plugin will be picked up at runtime. It does not matter where the exe's are located. They can be put on the local workstation, a USB drive or a network share.

In the previous post I showed you how to install ThinApp and run a Setup Capture. Unless you are completely new to the technology, those things are rather straightforward. Now we will take a look at some other aspects of the package building process.

Setup Capture generates a project that you can customize before the build process. The default project location is in C:\Program Files\VMware\VMware ThinApp\Captures\. As an example, I chose to virtualize Opera 9.5 and that resulted in this project folder: C:\Program Files\VMware\VMware ThinApp\Captures\Opera 9.51.

When you open the project folder you will get something like in the screenshot below.

One of the most important files is the package.ini file because it contains parameters that drive the build process. The VMware site has full details about the settings in package.ini here.

Some interesting parameters can be found in the general purpose parameters section:

• SandboxName: the sandbox is the location where the user's application settings are stored. By default that is %appdata%\Thinstall\SANDBOXNAME. %appdata% is a directory in the user's profile that can be redirected to a network location using Group Policies. Note that VMware did not change Thinstall to ThinApp here.
• PermittedGroups: to configure who is allowed to run the application
• RemoveSandboxonExit: to clear the sandbox when the user quits the application, useful when you always want the user to start from the default settings in the package.
• SandboxNetworkDrives: to allow or disallow access to network drives from the package. Access is allowed by default.
• SandboxRemovableDisk: similar to the network drives setting but for removable disks.

Near the end of package.ini you will find the entry points to the application. In my case there is a section called [Opera.exe]. In some cases you will find other entry points that you don't need. In my case I had entry points called [cmd.exe], [regedit.exe] and [iexplore.exe]. Although you can disable these entry points during Setup Capture I usually remove them from package.ini afterwards as well.

Now that you know about package.ini, let's build the project. In the project folder there is a batch file called build.bat. Just run that file to start the build. It's that simple! You will get a new folder called bin. In my case there are two files:

You can just take the opera.exe application and run it on another computer. One of the strengths of ThinApp is the fact that it is agentless. It is agentless because the virtualization engine is actually included in each executable that is generated.

When you run a ThinApp'ed application you get the following in the right bottom corner of the screen:

The application then starts as if it was installed locally. In virtual Opera, I added my blog to the list of bookmarks. Since that is a user preference, it is saved in the sandbox at C:\Documents and Settings\USERNAME\Application Data\Thinstall\Opera 9.51. You should note that by default, if you install add-ons like Flash, they are also installed to the sandbox. When you delete the sandbox directory and start the application again you start with default settings in the package. Talk about a simple way to reset the user environment on a per application basis.

The features described above are essentially the same as in Thinstall 3.0. VMware added some new features like Application Sync and Application Link but that is for another post.

I just downloaded ThinApp 4.0 from the VMware website. The screens below show the installation screens and setup capture screens to have some idea about what the product looks like.

The installer is very small (around 7MB). I created a Windows XP virtual machine in VMware Workstation, installed ThinApp 4.0 and created a snapshot. The snapshot allows me to revert to it after creating a package in order to start with a clean machine whenever I create a package. Note that it is recommended to start the ThinApp setup capture process from a network share instead of installing it on your capture workstation.

First, let's start the installation.

You have to enter a serial number and license display name. You can get a trial key at www.vmware.com for 60 days. The trial includes VMware Workstation 6 as well if you don't have it yet.

After installation you get the following shortcuts in the Start Menu:

With ThinApp Setup Capture you capture the installation of a piece of software. When launched, you get the following screens:

Of course, you need to run setup capture on a clean computer. The next dialog tells you that but also allows you to set advanced settings.

The advanced settings:

You normally don't need to change anything in the Advanced settings. Continuing with the setup wizard, the prescan starts. Scanning is very fast.

When the prescan finishes you get the dialog below. Now it is time to start an installation. I installed Opera 9 (not shown).

After installation, you click Next in the dialog above. ThinApp can now start the postscan in order to determine the differences.

Now you can select the entry points. Multiple entry points are possible, for example when you virtualize Office with Excel, Word and PowerPoint.

You can configure the application so that only specific users can run it. AD groups are used for this purpose. The sandbox location is a location where ThinApp can store the user's settings. If the user sets favorites in Opera for example, they are stored in the sandbox.

The next dialog asks for the isolation mode. The explanations in the dialog speak for themselves.

You can accept the default location for the project. The MSI generation checkbox is not checked by default. It generates a standard MSI that can be deployed to user's desktops with tools such as Altiris or SCCM 2007.

The project is then created.

In the final screen you can browse the project and make changes or build the project. I will not do that and click Finish to end the capture.

Note that when you just click Finish, your virtualized application is not created. You need to actually build the project for that. A next post will show you how the build process works without using the Build Now button in the dialog above.

From Arlindo's Blog I picked up that there is a hotfix to get SCVMM (System Center Virtual Machine Manager) 2008 to work with Hyper-V RC1. Now I can finally update my Hyper-V test system.

You can get the hotfix from http://connect.microsoft.com.

Douglas Brown has written a reviewers guide for Virtual Desktop Manager 2.1. I am not sure if this guide is really needed as the VMware documentation is clear and the product is simple to install and use. However, if you are looking for one document that contains an overview from start to finish it is a very good read.

Get it here.

As a side remark it seems that Provision Networks VAS (Virtual Access Suite) still has the upper hand when you look at the features and the price of the product. They just need to put some extra work in their documentation, support site and troubleshooting FAQs because those leave a lot to be desired.

This week somebody asked me if it was possible to sync a Windows Mobile device using a VMware VDM virtual desktop. The following screenshots show how this is done. I am running VDM 2.1 and the virtual desktop is Windows XP SP2 with ActiveSync 4.5. The client is Windows Vista SP1 with the VMware VDM Client and my Windows Mobile SmartPhone is connected with a USB cable.

From the VDM Client, the user logs on to the desktop by double clicking it:

The remote desktop session is started but it is wrapped in the VMware VDM Client's user interface. The window's title bar contains a menu:

If the Windows Mobile device is connected on the client, it will show up in the Devices menu:

The user just needs to click the device to "connect" it to the virtual desktop. As soon as the device is redirected it can be used. In my case, I need to provide a password to the device first:

The device is now connected and ActiveSync in the Windows XP virtual desktop can work with it:

The mobile device also shows up in My Computer as expected.

A couple of things you should know here. USB device redirection like this requires the VDM agent in the virtual desktop and VMware VDM Client on the client.

Hyper-V RC1 is available for download. More info can be found here. A couple of things to note:

• Saved-state files and online snapshots are different so discard these before upgrading.
• The current SCVMM 2008 Beta does not support RC1.
• There is support for Windows 2000 SP4 guests.
• Integration Components disk (iso) contain the binaries needed for all Windows guest operating systems including Windows Server 2008.
• Static IPv4 migration when you create a virtual network bound to an adapter with an existing fixed IPv4 address.

While we are on the subject of Hyper-V, check out the Microsoft Virtualization Team Blog. It contains a couple of interesting posts including some more WMI examples.

I stumbled across two configuration issues with the beta of SCVMM 2008 (available on http://connect.microsoft.com). Here they are:

1. SCVMM does not like : in a path

I use local storage on my ESX server and ESX automatically gives that storage a label in the form of hostname:storage#. When I tried to deploy a virtual machine to ESX with SCVMM, it failed saying that the volume name was not recognized. Removing the : in the label solved the issue.

2. Issue with USB flash devices in Hyper-V host

When I added my Hyper-V host to SCVMM everything seemed to work fine except the Refresh-Host job. It turns out that in the beta, that job fails when you have USB flash devices in the system. Because I use an HP Desktop machine with built-in card readers, I had to disable them all in device manager. After disabling those devices, Refresh-Host completed without errors, making the Hyper-V host available for deployments.

I downloaded and installed SCVMM 2008 today and installed it to check out the VMware integration. I created a new virtual machine with Windows Server 2008 x64 with 1,5GB of RAM and started the installation from an attached ISO. When I did that the installation failed during WAIK installation. I copied the WAIK files (from \Prerequisites\WAIK\1033) to the server and installed WAIK from there. I than reran the setup procedure and SCVMM installed fine.

The first thing you do after installation is to add some hosts. This beta of SCVMM 2008 supports the following virtualization hosts:

• Virtual Server 2005
• Hyper-V (on Windows Server 2008)
• VMware VI3 (with VirtualCenter)

To manage ESX you need to add a VirtualCenter server. SCVMM cannot connect to ESX servers directly. The result (click to enlarge):

In the screenshot above at the right, you'll see an action called Add VMware VirtualCenter. That action launches a wizard that asks you for the name of your VirtualCenter box and your credentials. Upon completion, the wizard adds folders to SCVMM for each VMware datacenter object and then adds the ESX boxes.

If you look at the screenshot further, the Summary tab shows some information about the ESX host like CPU, memory, storage and the virtual machines on the host. The Storage and Networking tab shows the following info:

When you click on the Virtual Machines button and select the ESX host you get a list of virtual machines:

From that list you can do what's expected: stop, start, suspend, modify the virtual machine settings, VMotion, etc... You can also connect to the console of virtual machines. The first time you do this you need to install an ActiveX control. The console looks like this:

That's all I have time for today. Next I will check out how SCVMM works with ISOs and virtual machine templates and how that ties in with VirtualCenter. I will report those findings later.

Scott Lowe has written an interesting post about enabling jumbo frames for VMware's software iSCSI initiator. VMware does not support this (yet) but it appears you can already enable it. For full details, check his blogpost.

A jumbo frame's size is typically set at 9000 versus the standard size of around 1500. Setting the frame size (or mtu) to 9000 actually lowers the amount of packet processing (assembly/disassembly) by a factor of six. The overhead for ethernet packets also decreased because you send less frames. In general, you should see a performance increase of around 20 to 30%. Note that you have to enable jumbo frames on all levels of your infrastructure: the ethernet switch, the server and the storage box (e.g. NetApp, EqualLogic, ...).

In my test environment I use a couple of base disks to easily create new virtual machines. I have a base disk for Windows XP SP3, Windows Server 2008 (32-bit and 64-bit), Windows Server 2008 Core and Windows Server 2003. Each time I create a new virtual machine, I create a differencing disk based on one of the base disks and tell the virtual machine to use that base disk. This functionality is the same as in Virtual PC and Virtual Server.

I wanted to automate the creation of differencing disks using a PowerShell script and it turns out to be very easy. The following two lines are all you need:

$img_svc=Get-WmiObject -Namespace root\virtualization -class msvm_imagemanagementservice

$img_svc.CreateDifferencingVirtualHardDisk("path for new disk", "path to parent disk")

You can find this information here but at this time the information there is pretty basic. You'll probably learn more by listing the WMI classes in PowerShell and playing with them a bit. To list the classes just use the following command:

Get-WmiObject -Namespace root\virtualization -list

I tried to find out how to create a virtual machine using WMI but that seems to be rather difficult. At first glance there is no simple method you can use with some parameters like the name, amount of memory, disk and so forth. Oh well, when SCVMM will have support for Hyper-V there will be native PowerShell commands to do just that so it's no big deal.