Connecting Windows Azure Virtual Machines with PowerShell

In my post on automating virtual machines I showed the basics for getting around and managing aspects of Windows Azure VMs. In this post I want to cover a few of the more complex scenarios such as connectivity between VMs, deploying into a virtual network and finally deploying a virtual machine automatically domain joined into an Active Directory Domain.

Connecting Virtual Machines

So far we have seen details on provisioning a single virtual machine. What happens if your application requires more than one VM? How to connect them? There are two options for connecting virtual machines. The first is to add multiple virtual machines to the same cloud service and the second is to provision each VM into the same virtual network. When machines are added to the same cloud service and are not within a virtual network they receive the benefits of being on the same network and receive built in name resolution with each other through Windows Azure provided DNS.


So how do you add two virtual machines to the same cloud service? When you create the first virtual machine using New-AzureVM or New-AzureQuickVM you are required to specify the -Location or -AffinityGroup parameter. When you specify either parameter it tells the cmdlets that you wish to create the cloud service at that time because the data center location can only be set on initial creation. To tell the cmdlets to create the VM in an existing cloud service you just omit the -Location/-AffinityGroup parameter.

Create a VM and a New Cloud Service (specify -Location/-AffinityGroup)

New-AzureVMConfig -ImageName $img -Name $vmn -InstanceSize Small | 
	Add-AzureProvisioningConfig -Windows -Password $PWD |
	New-AzureVM -ServiceName $svc -Location $loc

Create a VM and Adds to an Existing Cloud Service by (omit -Location/-AffinityGroup)

New-AzureVMConfig -ImageName $img -Name $vmn -InstanceSize Small | 
	Add-AzureProvisioningConfig -Windows -Password $PWD |
	New-AzureVM -ServiceName $svc 

Connecting Virtual Machines with Windows Azure Virtual Networks

The second way of provisioning connected virtual machines is by using a Windows Azure Virtual Network. With a Windows Azure Virtual Network the network can span cloud services. This enables scenarios such as virtual machines (or web and worker roles) in different cloud services to be fully connected in the cloud.

How do you provision VMs into a VNET with PowerShell?

Just like the -Location parameter a VNET can only be specified when creating the first VM in a cloud service (note that the subnet for each VM can be set per VM on provisioning). Additionally, VNETs require that the cloud service be deployed into the same affinity group as the VNET was created in. The New-AzureVM cmdlet requires -AffinityGroup instead of -Location when deploying to a VNET.

Joining a Virtual Network at Provision Time

New-AzureVMConfig -ImageName $img -Name $vmn -InstanceSize Small | 
	Add-AzureProvisioningConfig -Windows -Password $PWD |
       Set-AzureSubnet 'subnet' |
	New-AzureVM -ServiceName $svc -AffinityGroup 'myag' -VNetName 'VNET' 

Specifying DNS

One of the significant differences between deploying a virtual machine outside of a VNET and one within is inside of a VNET there is no Windows Azure Provided DNS for VM to VM name resolution. To provide for this you are allowed to specify DNS servers inside of the Virtual Network configuration. When deploying with PowerShell you also have the ability to specify DNS settings when you create the first VM. This is a very flexible approach because it allows you the ability to specify DNS at deployment time without the need to modify the underlying virtual network configuration.

Specifying DNS Server on Provisioning

In this example I am creating a DNS object that references a DNS server (10.1.1.4) and I specify it with New-AzureVM. All VMs created in this cloud service will inherit this DNS setting on boot.

$dns = New-AzureDns -Name 'onprem-dns' -IPAddress '10.1.1.4'
New-AzureVMConfig -ImageName $img -Name $vmn -InstanceSize Small | 
    Add-AzureProvisioningConfig -Windows -Password $PWD |
    Set-AzureSubnet 'subnet' |
    New-AzureVM -ServiceName $svc -AffinityGroup 'myag' -VNetName 'VNET' -DnsSettings $dns

Deploying a Virtual Machine into an Active Directory Domain

With Windows Azure Virtual Machines it is entirely possible to have a full Active Directory environment in the cloud. AD can either be hosted on-premises with connectivity provided by a site-to-site VPN tunnel using Windows Azure Virtual Networks OR you can host an AD domain directly in the cloud.

Once AD connectivity is in place you can use the PowerShell cmdlets to automatically join a Windows Virtual Machine directly to an Active Directory domain at provision time. For AD domain join to work you must specify the DNS server IP address for your Active Directory domain.
In this example New-AzureDNS is used to specify the DNS for the VM to point to an AD DNS Server in the cloud (10.2.0.4) which itself has been configured to point to an on-premise AD server (192.168.1.6) in a previous deployment. Setting DNS at this level is also useful because any future VMs added to this cloud service will inherit the DNS setting.

$subnet = 'APPSubnet'
$ou = 'OU=AzureVMs,DC=fabrikam,DC=com'
$dom = 'fabrikam'
$domjoin = 'fabrikam.com'
$domuser = 'administrator'

$domVM = New-AzureVMConfig -Name 'advm1' -InstanceSize Small -ImageName $image |      
	Add-AzureProvisioningConfig -WindowsDomain -JoinDomain $domjoin -Domain $dom -DomainPassword $pass -Password $pass -DomainUserName $domuser -MachineObjectOU $ou | 	
	Set-AzureSubnet -SubnetNames $subnet

$dns = New-AzureDns -Name 'clouddc-ad' -IPAddress '10.2.0.4' 

New-AzureVM -ServiceName 'app-cloudservice' -AffinityGroup 'ADAG' -VNetName 'HybridVNET' -DnsSettings $dns -VMs $domVM

If you would like to try some of this out on your own I highly suggest the Windows Azure Training Kit as a starting point. There are many hands on labs including deploying Active Directory and connecting multiple virtual machines.

Automating Windows Azure Virtual Machines with PowerShell

With the release of the June 2012 Windows Azure PowerShell cmdlets I’m excited to tell you that I have officially passed the buck!

In this release we have added some exciting functionality to the Windows Azure PowerShell cmdlets and have transitioned ownership and support of the cmdlets to the Windows Azure Engineering teams. The benefit to you as the cmdlet user is the code will be officially supported and much better documented with more frequent release cycles.

To make all of this happen did take a lot of coordination with the different teams so there is a difference in functionality regarding what was previously on CodePlex. For example, the Windows Azure Scaffolding cmdlets for Node.JS have been integrated along with a lot of new functionality for building scaffolding around PHP projects. However, for the first release we were not quite ready to bring forward all of the existing functionality in the Windows Azure cmdlets so with this release a few areas will be delayed: Diagnostics, Traffic Manager and the SQL Azure cmdlets will all come in later waves.

In this blog post I’m going to cover the some of the new functionality around automating Windows Azure Virtual Machines.

Getting Started

Download the Windows Azure PowerShell Cmdlets

Launch the cmdlets by clicking the Windows Azure PowerShell link on the start menu.
If you can’t find this link OR you want to use the cmdlets in your own editor you have to load the modules a little different than normal:

Manual Import of the PowerShell Modules

Import-Module 'C:\Program Files (x86)\Microsoft SDKs\Windows Azure\PowerShell\Azure\Azure.psd1'

(Note: We do plan to add a module in the Users module location soon – we did not forget)

Configuring Your Subscription

The easiest way to get started is to download and import your publish settings file. You can do this by using your web browser and browsing directly to the site: https://windows.azure.com/download/publishprofile.aspx OR by using a handy cmdlet that was previously part of the Node.JS cmdlets Get-AzurePublishSettingsFile which will launch a browser to this URL automatically.
Once you have downloaded the publish settings file it’s a simple matter of importing it.

Import-AzurePublishSettingsFile 'c:\temp\mysub.publishsettings'

To see the resulting configuration run the following command:

Get-AzureSubscription 

If you prefer to manually configure your subscription you can use Set-AzureSubscription to configure your subscription ID and management certificate.

$subid = '[YOUR-SUBSCRIPTION-ID]'
$cert = Get-Item Cert:\CurrentUser\My\YOURCERTTHUMBPRINT
Set-AzureSubscription -SubscriptionName 'testsub1' -SubscriptionId $subid -Certificate $cert 

Note the subscription settings that are imported or set manually are persisted in the following location:
C:\Users\user\AppData\Roaming\Windows Azure Powershell

This means you do not have to run Set-AzureSubscription for each script since it is already there!

The cmdlets do support multiple subscriptions and to allow you to choose which subscription to work on we provide the Select-AzureSubscription cmdlet to switch the active subscription.

Configure Storage
Finally, one new addition is the –CurrentStorageAccount parameter of Set-AzureSubscription. This parameter tells the cmdlets what storage account they should use if they need to access storage. When dealing with virtual machines this comes up quite a bit.

To set it you can run the following:

Get-AzureStorageAccount   # Enumerates your storage accounts 

If you do NOT have a storage account returned above you should create one first.
Run the following to determine the data center to create your storage account in. Ensure you pick a data center that shows support for PersistentVMRole.

Get-AzureLocation  

Create your storage account:

New-AzureStorageAccount -StorageAccountName 'myuniquelynamedstorage' -Location 'East US'

Finally, set the -CurrentStorageAccount property of your subscription. The -CurrentStorageAccount parameter specifies the storage account to use for VHD storage during the later exercises

Set-AzureSubscription -SubscriptionName 'testsub1'  -CurrentStorageAccount 'mystorage'

For the remainder of this blog post I’m going to walk you through some exercises from our hands on lab to get you familiar with some of the great things you can automate with Windows Azure Virtual Machines and PowerShell.

Before continuing you should start an elevated PowerShell session and run the following command:
Set-ExecutionPolicy RemoteSigned

Provisioning a Virtual Machine using PowerShell CmdLets

Create a new variable and assign it the location you used to create your storage account or whatever location your selected storage account is currently in.
The virtual machines have to be created in the same location/region as your storage account.

$dclocation = '[YOUR-LOCATION]'

Determine the name of the cloud service that will act as the container for your virtual machines:

Check to see if the name is available (Test returns True if the name already exists)

Test-AzureName -Service '[YOUR-CLOUD-SERVICE-NAME]'

Create a variable to use the unique name of your cloud service.

$cloudSvcName = '[YOUR-CLOUD-SERVICE-NAME]'

Determine which platform image your want to use as the basis of your virtual machine by running:

Get-AzureVMImage | select ImageName

Create a variable to hold virtual machine image name:

$image = '[YOUR-SELECTED-IMAGE-NAME]'

Now let’s create a virtual machine!

If you chose a Windows image you would use the following parameter set:

Quick Create of a Windows VM with PowerShell

$adminPassword = '[YOUR-PASSWORD]'
$vmname = 'mytestvm1'
New-AzureQuickVM -Windows -ServiceName $cloudSvcName -Name $vmname -ImageName $image -Password $adminPassword 

For Linux:
Quick Create of a Linux VM using PowerShell

$linuxuser = '[CHOOSE-USERNAME]'
$adminPassword = '[YOUR-PASSWORD]'
$vmname = 'mytestvm1'

New-AzureQuickVM -Linux -ServiceName $cloudSvcName -Name $vmname -ImageName $image -LinuxUser $linuxuser 

Once the virtual machines have been created you can check their status by running:

Viewing Virtual Machine Settings

Get-AzureVM -ServiceName $cloudSvcName 

This cmdlet when -Name is omitted returns all of the virtual machines within the cloud service. You can of course specify a single VM with the -Name parameter.

Restart, Shutdown and Start of a Windows Azure Virtual Machine

# Restart
Restart-AzureVM -ServiceName $cloudSvcName -Name $vmname

# Shutdown 
Stop-AzureVM -ServiceName $cloudSvcName -Name $vmname

# Start
Start-AzureVM -ServiceName $cloudSvcName -Name $vmname

This exercise showed how you can quickly provision a new virtual machine. What about if you wanted to provision a VM with some settings already configured?

Advanced Provisioning Options

So the following script may put you in PowerShell overload but let me explain it a bit.

The New-AzureVMConfig cmdlet allows you to create a virtual machine configuration object that you can then modify with other cmdlets. You can add data disks, configure endpoints and even modify the disk cache behavior for the OS Disk or the data disks at this stage. You can also add endpoints that are needed for the virtual machine such as port 80 and port 443 (we automatically add endpoints for RDP and SSH when you create a VM off of an image – there is a bug where we do not when creating off of a disk).

So each call to New-AzureVMConfig returns back a object you can then modify with Add-AzureDataDisk, Set-AzureOSDisk, Add-AzureEndpoint etc.. You then take the VM object(s) and pass them to the New-AzureVM cmdlet to actually create the VM(s).

Creating Windows Virtual Machines with PowerShell

   $vmname2 = 'mytestvm2'
   $vmname3 = 'mytestvm3'

   $vm2 = New-AzureVMConfig -Name $vmname2 -InstanceSize ExtraSmall -ImageName $image |
             Add-AzureProvisioningConfig -Windows -Password $adminPassword |
             Add-AzureDataDisk -CreateNew -DiskSizeInGB 50 -DiskLabel 'datadisk1' -LUN 0 |
             Add-AzureEndpoint -Protocol tcp -LocalPort 80 -PublicPort 80 -Name 'web' `
                 -LBSetName 'lbweb' -ProbePort 80 -ProbeProtocol http -ProbePath '/' 

   $vm3 = New-AzureVMConfig -Name $vmname3 -InstanceSize ExtraSmall -ImageName $image |
          Add-AzureProvisioningConfig -Windows -Password $adminPassword  |
           Add-AzureDataDisk -CreateNew -DiskSizeInGB 50 -DiskLabel 'datadisk2' -LUN 0  |
           Add-AzureEndpoint -Protocol tcp -LocalPort 80 -PublicPort 80 -Name 'web' `
                 -LBSetName 'lbweb' -ProbePort 80 -ProbeProtocol http -ProbePath '/' 

   New-AzureVM -ServiceName $cloudSvcName -VMs $vm2,$vm3

Creating Linux Virtual Machines with PowerShell

   $vmname2 = 'mytestvm2'
   $vmname3 = 'mytestvm3'

   $vm2 = New-AzureVMConfig -Name $vmname2 -InstanceSize ExtraSmall -ImageName $image |
          Add-AzureProvisioningConfig -Linux -LinuxUser $linuxUser -Password $adminPassword |
          Add-AzureDataDisk -CreateNew -DiskSizeInGB 50 -DiskLabel 'datadisk1' -LUN 0 |
           Add-AzureEndpoint -Protocol tcp -LocalPort 80 -PublicPort 80 -Name 'web' `
                 -LBSetName 'lbweb' -ProbePort 80 -ProbeProtocol http -ProbePath '/' 

   $vm3 = New-AzureVMConfig -Name $vmname3 -InstanceSize ExtraSmall -ImageName $image |
            Add-AzureProvisioningConfig -Linux -LinuxUser $linuxUser -Password $adminPassword |
            Add-AzureDataDisk -CreateNew -DiskSizeInGB 50 -DiskLabel 'datadisk2' -LUN 0 |
            Add-AzureEndpoint -Protocol tcp -LocalPort 80 -PublicPort 80 -Name 'web' `
                -LBSetName 'lbweb' -ProbePort 80 -ProbeProtocol http -ProbePath '/' 

   New-AzureVM -ServiceName $cloudSvcName -VMs $vm2,$vm3

In this example is I have added the -LBSetName, ProbePort, ProbeProtocol and ProbePath parameters. These are optional for creating a single endpoint but in this case they configure the load balancer for these two virtual machines.

Configuring Other Protocols for the Load Balancer

In addition to configuring HTTP requests, the PowerShell cmdlets can also configure endpoints for UDP and TCP connections as well.
For TCP sockets applications you can specify -ProbeProtocol as tcp instead of http and remove the -ProbePath parameter. The load balancer will attempt to connect to the TCP port specified in -ProbePort and if successful will keep the VM in the load balancer.
For load balancing UDP connections you will need to specify an alternate probe endpoint that can tell the load balancer whether the UDP service is available. This could be a simple http page that returns a 200 assuming the service is up.

Updating Existing Virtual Machines

Modifying an existing virtual machine requires retrieving the current settings by calling Get-AzureVM, modifying them and then calling the Update-AzureVM cmdlet to save the changes.
You can hot add and remove data disks and networking endpoints. Changing disk cache settings requires a reboot as does changing the virtual machine’s instance size.
The following example uses the Get-AzureVM cmdlet to retrieve the VM object and send it to the PowerShell Pipeline.

Add-AzureDataDisk with the CreateNew parameter allows you to dynamically add storage to the virtual machine. In this case we are calling it twice to attach to unformatted blank VHDs to the server each 50 gigs of storage each. The -LUN parameter tells the order of the device being attached and optionally uses the -MediaLocation to specify the location in Storage to keep the newly created VHDs.

Add-AzureDataDisk also supports the Import parameter to attach a disk in the disk library and -ImportFrom to attach a disk that already exists in storage.

The example also adds a new endpoint for TCP port 1433 internally that is listening externally on port 2000 using the Add-AzureEndpoint command.

$vmname = 'mytestvm1'

Get-AzureVM -Name $vmname -ServiceName $cloudSvcName |
    Add-AzureDataDisk -CreateNew -DiskSizeInGB 50 -DiskLabel 'datadisk1' -LUN 0 |
    Add-AzureDataDisk -CreateNew -DiskSizeInGB 50 -DiskLabel 'translogs1' -LUN 1 |
    Add-AzureEndpoint -Protocol tcp -LocalPort 1433 -PublicPort 2000 -Name 'sql' |
    Update-AzureVM 

Once the virtual machine has been updated you can RDP into the VM to configure the disks using Computer Manager -> Disk Management.

Get-AzureRemoteDesktopFile -ServiceName $cloudSvcName -Name $vmname -Launch 

Get-AzureRemoteDesktopFile also allows you to save the RDP files for your VMs to the local file system.

Saving RDP files for all VMs in a Cloud Service

Get-AzureVM -ServiceName fabrikam-cloudapps | foreach { 
	$rdpfile = 'D:\User-Data\Scratch\' + $_.Name + '.rdp'
	Get-AzureRemoteDesktopFile -ServiceName $cloudSvcName -Name $_.Name -LocalPath $rdpfile
}

Modifying Disk Cache Settings

There is a disk cache built into Windows Azure VMs. Essentially, Windows Azure can use the local physical disk attached to the VM for caching.

The OS disk can support ReadOnly and ReadWrite cache settings with the default being ReadOnly
Data Disks can support None, ReadOnly and ReadWrite with the default being None (The safest choice for data files)

Modifying the Data Disks Host Cache Settings

Get-AzureVM -ServiceName $cloudSvcName -Name $vmName  |
       Set-AzureDataDisk -HostCaching ReadWrite -LUN 0 |
       Set-AzureDataDisk -HostCaching ReadWrite -LUN 1 |
       Update-AzureVM

Get-AzureVM -ServiceName $cloudSvcName -Name $vmname | Get-AzureDataDisk

Modifying the OS Disk Host Cache Settings (Requires a Reboot)

    Get-AzureVM -ServiceName $cloudSvcName -Name $vmName |   # Retrieve existing VM 
        Set-AzureOSDisk -HostCaching ReadOnly | # Perform an operation that modifies the returned VM and pipe the results
        Update-AzureVM  # Update the modified VM 

Modifying the VM Size

You are allowed to change the instance size of your VM through the portal and through PowerShell.

Get-AzureVM -ServiceName $cloudSvcName -Name $vmName |
    Set-AzureVMSize -InstanceSize Medium |
    Update-AzureVM

TIP: I have separated these updates into separate examples but you should know that they can be combined into one batch update.

Modifying the OS Disk Cache Settings

    Get-AzureVM -ServiceName $cloudSvcName -Name $vmName |   # Retrieve existing VM 
        Set-AzureOSDisk -HostCaching ReadOnly | # Perform an operation that modifies the returned VM and pipe the results
        Update-AzureVM  # Update the modified VM 

P.S. I’ll document some of the changes to the existing deployment cmdlets soon. One thing you will immediately notice though is it is no longer necessary to call Get-OperationStatus after every cmdlet. The cmdlets now handle the waiting automatically.

With this post I hope I get you started learning how to create and manage your virtual machines with the new Windows Azure PowerShell cmdlets. Look for more articles in the very near future that detail some of the more advanced scenarios you can accomplish.

Thanks!
Michael Washam
Senior Technical Evangelist – Windows Azure

Updating SQL Azure Firewall Rules with Windows Azure PowerShell Cmdlets 2.0

You’ve deployed a few Sql Azure servers and through no fault of your own the requirement comes up to update all of the firewall rules for each of the Sql Azure Servers.

No Problem!

Adding new SQL Azure Firewall Rules

Get-SqlAzureServer -Certificate $cert -SubscriptionId $subscriptionid | foreach {
  $_ | New-SqlAzureFirewallRule -RuleName "NewRule1" -StartIpAddress "0.0.0.0" -EndIpAddress "1.1.2.2"
  $_ | New-SqlAzureFirewallRule -RuleName "NewRule2" -StartIpAddress "100.1.0.0" -EndIpAddress "100.15.0.0"
}

Removing Rules is Just as Easy

Get-SqlAzureServer -Certificate $cert -SubscriptionId $subscriptionid | foreach {
  $_ | Remove-SqlAzureFirewallRule -RuleName "OldRule1"
  $_ | Remove-SqlAzureFirewallRule -RuleName "OldRule2"
}

This of course requires the Windows Azure PowerShell Cmdlets 2.0

Automate It! Deploying a Windows Azure Service Package from PowerShell.

With the new version of Windows Azure PowerShell Cmdlets 2.0 we have made it possible to script the deployment of an entire hosted service directly from PowerShell!

As a proof of concept we have put together a sample script that reads settings from a configuration file and deploys a service package to your Windows Azure Account. What is new about this is if the hosted service or storage account do not exist yet we will create them using the settings you specify!

To get started ensure you have downloaded and installed the Windows Azure PowerShell Cmdlets 2.0.

Next download the sample script from here: https://mlwpublic.blob.core.windows.net/downloads/e2e-ps-deployment.zip

Once the file is unzipped you will need to modify the configuration.xml file with your own settings:

Image

Once configured copy your .cspkg file and .cscfg file to the \deploymentPackage folder beneath the script.

Finally, run the deploy.cmd file which will launch your PowerShell script and deploy the service!

Image

Operations demonstrated in this sample include:

  • New-HostedService / Get-HostedService
  • New-StorageAccount / Get-StorageAccount
  • New-Deployment / Get-Deployment
  • Add-Certificate
  • Move-Deployment (VIP Swap)

400 Bad Request when using Windows Azure PowerShell CmdLets

Experimenting with the PowerShell Cmdlets today I ran into a not-so-clear issue when using new-deployment and set-deployment.
Although, [-StorageServiceName ] is optional I recieved a 400 Bad Request when using the CmdLets without setting it.

The name is a little misleading as well. StorageServiceName really just means your StorageAccountName.

Added that and everything worked. Just a tip in case you are stuck.