Windows Azure IaaS Webcast Series Part One – Getting Started with Virtual Machines

I’ve started a series of webcasts focused on Windows Azure Infrastructure as a Service. This series is targeted at new users to IaaS and starts from the basics with creating virtual machines. I plan on building many of these webcasts to cover the getting started scenarios to more advanced topics.

If you have any scenario requests shoot me an email at: mwasham@microsoft.com or Tweet me at MWashamMS!

iaas-part-1

Bootstrapping a Virtual Machine with Windows Azure

In my Advanced IaaS Talk at Build I showed a demo where you can configure a PowerShell script that can download a zip file with multiple actions (unzip or execute) that gives you similar functionality to a Windows Azure startup task for web and worker roles.

This is a very simple example but it does show some of the capabilities you can do.

The bootstrap.ps1 file below is the main script that is responsible for downloading the bootstrap.zip file from your storage account. You should create a directory called c:\BootStrap and place the script inside and reference it from the startup task on your virtual machine.

In this demonstration I am pulling from a public storage container. If you want to contain anything secure in your bootstrap.zip you should modify the url below to use a shared access signature..

bootstrap.ps1

cls

$rootpath = 'C:\BootStrap\'
$manifest = 'C:\BootStrap\Working\manifest.xml'
$workingdir = 'C:\BootStrap\Working\'
$downloaddir = 'C:\BootStrap\Working\config1.zip'
$packagesource = 'http://YOURSTORAGEACCOUNT.blob.core.windows.net/bootstrap/config1.zip'

function GetPayload()
{
    $retries = 5
	while($retries -gt 0)
	{
		CheckFolder $workingdir
	    try {
		    $wc = New-Object System.Net.WebClient
		    $wc.DownloadFile($packagesource, $downloaddir)
			break
	     } 
	     catch [System.Net.WebException] {
		    # $_ is set to the ErrorRecord of the exception
	        if ($_.Exception.InnerException) {
	     	   $_.Exception.InnerException.Message | Out-File c:\BootStrap\error.txt -Append
	        } else {
	           $_.Exception.Message | Out-File c:\BootStrap\error.txt -Append
	        }
			Start-Sleep -Seconds 15
			$retries = $retries - 1
	     }
	 }
     UnzipFileTo $downloaddir $workingdir
}

function BootStrapVM()
{
  if((Test-Path HKLM:\Software\VMBootStrap) -eq $true)
  {
     Write-Host "Already Ran"
	 return
  }

  [xml] $manifest = Get-Content $manifest
  $counter = 0
  $manifest.StartupManifest.Items.Item | foreach { 
	$action = $_.action 
	$path = $_."#text"
	$target = $_.target 
	$sourcefullpath = $workingdir + $path
	
	switch($action)
	{
	   "execute" {
		  Write-Host "Executing command: " $sourcefullpath
		  ExecuteCommand $sourcefullpath
	   }
	   "unzip" {
		  $sourcefullpath = $workingdir + $path
		  Write-Host "Unzipping " $sourcefullpath " to " $target 
	      UnzipFileTo $sourcefullpath $target
	   }
	   
	}
	 
  }
  New-Item -Path HKLM:\Software -Name VMBootStrap –Force | Out-Null
  Set-Item -Path HKLM:\Software\VMBootStrap -Value "ran" | Out-Null
}

function ExecuteCommand($commandpath)
{
	& $commandpath
}

function UnzipFileTo($sourcepath, $destinationpath)
{
	CheckFolder $destinationpath
	$shell_app = new-object -com shell.application
	$zip_file = $shell_app.namespace($sourcepath)
	$destination = $shell_app.namespace($destinationpath)
	$destination.Copyhere($zip_file.items(), 16)
}

function CheckFolder($path)
{
	if((Test-Path $path) -eq $false)
	{
   		New-Item -ItemType directory -Path $path -Force | Out-Null
	}
}
GetPayload
BootStrapVM

Here are two sample configs I put together:

config1.zip Enables Remote PowerShell

http://mwweststorage.blob.core.windows.net/bootstrapblog/config1.zip

config2.zip Installs IIS, WebPI, MySQL and WordPress

http://mwweststorage.blob.core.windows.net/bootstrapblog/config2.zip

The config1.zip file contains a manifest.xml file and all of the entities you need to upload.

Example of the manifest.xml section of the config2.zip

 <StartupManifest>
   <Items>
     <Item action="execute">InstallRoles\ConfigureRoles.ps1</Item>
     <Item action="unzip" target="c:\webpi\">WebPI\WebpiCmd.zip</Item>
     <Item action="execute">InstallPackages\EnableWebPI.cmd</Item>
   </Items>
 </StartupManifest>

All that is needed to configure the script to run when your VM boots is to configure a startup task using group policy.

Run: mmc
File -> Add/Remove Snapin
Select -> Group Policy Object Editor (Local Computer)
Expand -> Computer Configuration -> Windows Settings -> Scripts Startup/Shutdown

Once the script is configured you will need to change the execution policy on the virtual machine before you run sysprep and capture on your image otherwise the script will not execute on the next boot.

 Set-ExecutionPolicy RemoteSigned

Finally, if you do enable to the remote PowerShell task here is the command line you can use to connect.

Enter-PSSession -ComputerName {hostname}.cloudapp.net -Authentication Basic -Credential Administrator -UseSSL -SessionOption (New-PSSessionOption -SkipCACheck -SkipCNCheck)

Advanced Windows Azure IaaS – Demo Code

First of all thanks to everyone that watched my session at Build. I’ve received quite a bit of positive feedback so far (one wasn’t so positive.. you know who you are ;-)).

.

One of the common requests I’ve received is to get access to the source code for the demo virtual machine manager app I showed.

I’ve posted the full source for this demo in a repo on GitHub.

Disclaimer: This demo was designed specifically to show off how to how to use the service management API for basic operations. The code has limited (to no) exception handling, best practices or anything else that would work in a production application 🙂

So now that you know the quality level of the code level of the demo I do want to explain a bit of how it works.

There are two projects: SMLibrary which is a very basic wrapper around the Windows Azure Service Management API and MyVMPortal is the other. MyVMPortal is an ASP.NET WebForms app that uses the API to create a VM.

To run it locally you will need to have a management certificate installed on your local machine that is also installed in your Windows Azure subscription.

Open the web.config page and add your subscription ID, cert thumbprint and storage account name(s) (separate them by comma if you want multiple storage accounts). The app by default is hard coded to deploy to West US. So unless you change that you will need to create your storage accounts in West US as well.

Long term it might be nice to further expand this demo to show other functionality for automating virtual machines. If you are interested in contributing please let me know.

Thanks!
Michael Washam

Windows Azure IaaS Overload

I took a short breather after TechEd North America and TechEd Europe back-to-back but I did want to put up a post to summarize the sessions around Windows Azure Virtual Machines and Virtual Networks from TechEd 2012. This is a big and extremely important launch for Windows Azure so we have quite a bit of coverage on the subject 🙂

If you were looking for a crash course on Windows Azure IaaS here it is!


Meet the New Windows Azure – Scott Guthrie

Windows Azure Virtual Machines and Virtual Networks – Mark Russinovich

Windows Azure IaaS and How it Works – Corey Sanders

Extending Enterprise Networks to Windows Azure using Windows Azure Virtual Networks – Ganesh Srinivasan

Deep Dive on Windows Azure Virtual Machines – Vijay Rajagopalan

Running Linux on Windows Azure Virtual Machines – Tad Brockway

Migrating Applications to Windows Azure Virtual Machines – Michael Washam

Deploying SharePoint Farms on Windows Azure Virtual Machines – Paul Stubbs

Migrating SQL Server database applications to Windows Azure Virtual Machines – Guy Bowerman, Madhan Arumugam

Running Active Directory on Windows Azure Virtual Machine – Dean Wells

How to Move and Enhance Existing Apps for Windows Azure – Tom Fuller, Greg Varveris, Purush Vankireddy

Windows Azure Diagnostics and PowerShell – Performance Counters

This is part 2 of a series.

With the introduction of Windows Azure PowerShell Cmdlets 2.0 we have added a lot of functionality around managing Windows Azure Diagnostics. This is a 2nd article in a series that covers various aspects of diagnostics and PowerShell. Click here to see the previous article on using tracing and the Windows Azure Log.

Just as in the other articles you will need to add the PowerShell Snapin (or module):

 
  Add-PsSnapin WAPPSCmdlets 

Next I have a handy helper function called GetDiagRoles that I use for all of my diagnostic examples:

Initialization and Helper Function

 
  $storageAccountName = "YourStorageAccountName" 
  $storageAccountKey = "YourStorageAccountKey" 
  $deploymentSlot = "Production" 
  $serviceName = "YourHostedService" 
  $subscriptionId = "YourSubscriptionId" 
  # Thumbprint for your cert goes below 
  $mgmtCert = Get-Item cert:\CurrentUser\My\D7BECD4D63EBAF86023BB4F1A5FBF5C2C924902A 

  function GetDiagRoles { 
    Get-HostedService -ServiceName $serviceName -SubscriptionId $subscriptionId -Certificate $mgmtCert | ` 
        Get-Deployment -Slot $deploymentslot | ` 
        Get-DiagnosticAwareRoles -StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey 
   } 

I have another helper function I have written that basically creates a new PerformanceCounterConfiguration object. You will need an array of these to pass to the Set-PerformanceCounter cmdlet. The function takes the performance counter specifier (more on this shortly) and the sample rate in seconds for each counter.

Helper Function that Initializes a Counter Object

 
function CPC($specifier, $SampleRate) { 
   $perfCounter = New-Object Microsoft.WindowsAzure.Diagnostics.PerformanceCounterConfiguration 
   $perfCounter.CounterSpecifier = $specifier 
   $perfCounter.SampleRate = [TimeSpan]::FromSeconds($SampleRate) 
   return $perfCounter 
} 

From there it is up to you to pass the performance counters you are interested in. I’ve written another function that wraps up a the functionality of creating a PowerShell array, populating it with the counters I am interested in and sets them for the diagnostic aware role I pass in from GetDiagRoles. Note: I’m also comparing the current role name to the name of one of my web roles so I’m only adding web related counters on instances that I care about. I retrieved the counter names by typing in: typeperf.exe /qx while logged into my Windows Azure roles via RDP. This way I can actually get counters specific to the machines in the service (and the processes such as RulesEngineService.exe (my sample NT service). Finally, it makes a call to the Set-PerformanceCounter cmdlet with the array of performance counters, and configures diagnostics to transfer these to Windows Azure Storage every 15 minutes.

Helper Function that Configures Counters for a Role

 
 function SetPerfmonCounters { 
    $sampleRate = 15 
    $counters = @() 
    $counters += CPC "\Processor(_Total)\% Processor Time" $sampleRate 
    $counters += CPC "\Memory\Available MBytes" $sampleRate 
    $counters += CPC "\PhysicalDisk(*)\Current Disk Queue Length" $sampleRate 
    $counters += CPC "\System\Context Switches/sec" $sampleRate 
    # Process specific counters (retrieved via typeperf.exe /qx) 
    $counters += CPC "\Process(RulesEngineService)\% Processor Time" $sampleRate 
    $counters += CPC "\Process(RulesEngineService)\Private Bytes" $sampleRate 
    # $RoleName is populated from a call to GetDiagRoles in the next snippet. 
    if($RoleName -eq "SomeWebRole") { 
        $counters += CPC "\W3SVC_W3WP(*)\Requests / Sec" $sampleRate 
        $counters += CPC "\ASP.NET\Requests Queued" $sampleRate 
        $counters += CPC "\ASP.NET\Error Events Raised" $sampleRate 
        $counters += CPC "\ASP.NET\Request Wait Time" $sampleRate 
        # Process specific counters (retrieved via typeperf.exe /qx  
        $counters += CPC "\Process(W3WP)\% Processor Time" $sampleRate 
        $counters += CPC "\Process(W3WP)\Private Bytes" $sampleRate 
     } 
     $input | Set-PerformanceCounter -PerformanceCounters $counters -TransferPeriod 15 
} 

Now linking all of this together to configure performance logging is pretty simple. In the snippet below I call GetDiagRoles which returns a collection of all of the diagnostic aware roles in my service. Which I then pipe individually to the SetPerfmonCounters function previously created.

Set Counters for each Role

 
  GetDiagRoles | foreach { 
       $_ | SetPerfmonCounters 
  } 

Now I am successfully logging perfmon data and transferring it to storage. What’s next? Well analysis and clean up of course! To analyze the data we have added another cmdlet called Get-PerfmonLogs that downloads the data and optionally converts it to .blg for analysis. The snippet below creates a perfmon log foreach role in your service using the BLG format. Note the Get-PerfmonLog cmdlet also supports -From and -To (or -FromUTC or -ToUTC) so you can selectively download performance counter data.

Download Perfmon Logs for Each Role

 
   GetDiagRoles | foreach { 
     $Filename = "c:\DiagData\" + $_.RoleName + "perf.blg" 
     $_ | Get-PerfmonLog -LocalPath $Filename -Format BLG 
   } 

Once the data is downloaded you can of course analyze it using Perfmon or utilize the Clear-PerfmonLogs to clean up the previously downloaded perfmon counters from Windows Azure Storage. Note all of the Clear-* diagnostic data cmdlets support a -From and -To parameter to allow you to manage what data to delete.

Analysis using Perfmon

Delete Perfmon Data from Storage for Each Role

 
     GetDiagRoles | foreach { 
     $_ | Clear-PerfmonLog -From "9/19/2011 6:00:00 AM" -To "9/19/2011 9:00:00 AM" 
} 

Finally, one last example. If you ever have the need to completely clear out and reset your perfmon counters there is a -Clear switch that gives you this functionality.

Reset Perfmon Logs for Each Role

 
  GetDiagRoles | foreach { 
       $_ | Set-PerfmonLog -Clear 
  } 

Starting a Service from an Azure Worker Role

Just a quick followup post from my previous entry regarding Azure firewall rules and endpoints.

Starting a Service Sample:

class ServiceMonitor
{
    public static bool CheckAndStartService(String ServiceName, int StartTimeOutSeconds)
    {
        ServiceController mySC = new ServiceController(ServiceName);
        if (IsServiceStatusOK(mySC.Status) == false)
        {
            System.Diagnostics.Trace.WriteLine("Starting Service.....");
            try
            {
                mySC.Start();
            }
            catch(Exception exc)
            {
                  System.Diagnostics.Trace.WriteLine("Exception Occurred:" + exc.Message);
            }
            mySC.WaitForStatus(ServiceControllerStatus.Running, new TimeSpan(0, 0, StartTimeOutSeconds));

            if (IsServiceStatusOK(mySC.Status))
                return true;
            else
                return false;
        }
        return true;
    }

    static private bool IsServiceStatusOK(ServiceControllerStatus Status)
    {
        if (Status != ServiceControllerStatus.Running && Status != ServiceControllerStatus.StartPending && Status != ServiceControllerStatus.ContinuePending)
        {
            return false;
        }
        return true;
    }
}

To use this helper class from your worker role:

 

public override void Run()
{
String ServiceName = "SimpleService";
int ServiceFailCount = 0;
const int MAXFAILS = 5;

while (true)
{
// if the service failed more than 5 times return
if (ServiceFailCount >= MAXFAILS)
{
Trace.TraceError(String.Format(String.Format("{0} has failed to start {1} times", ServiceName, ServiceFailCount)));
return;
}
try
{
// Check if the service is running
if (ServiceMonitor.CheckAndStartService(ServiceName, 10) == true)
{
Trace.TraceInformation("Service is Running");
}
else
{
// if not increment the fail count
ServiceFailCount++;
Trace.TraceError("Service is no longer Running");
}
}
catch (Exception e)
{
Trace.TraceError("Exception occurred: " + e.Message);
}
Thread.Sleep(10000);
}
}

Note: Using the ServiceController class from your worker role requires elevation. So you will need to add the following to your worker role configuration:


This should allow connectivity to the service via whatever internal endpoint you have configured without additional firewall configuration.