Showing posts with label devtestlab. Show all posts
Showing posts with label devtestlab. Show all posts

How to be efficient with our Azure Devtest Lab deployments

(Ce billet est en aussi disponible en fran├žais.)

The Devtest labs is a fantastic tool to quickly build environments for development & test purposes and for a classroom. It offers great tools to restrict the users without removing all their freedom. It will speed up the boarding, with its claimable VMs that are already created and are waiting for the user. Formulas will help ensure you that you always get the latest version of your artifact installed on those VMs. And finally, the auto-shutdown will keep your money where it should stay...in your pocket.


In this post, I will show you how to deploy an Azure Devtest Lab with an Azure Resource Manager (ARM) template, and create the claimable VMs based on your formulas in one shot.

Step 1 - The ARM template


First, we need an ARM template. You can start from scratch of course, but it may be a lot of work if you are just getting started. You can also pick one from GiHub and customize it.

What I recommended, is to create a simple Azure Devtest Lab directly from the Azure portal. Once your lab is created, go in the Automation script option of the resourcegroup and copy/paste the ARM template in your favorite text editor.
armTemplate
Now you must clean it. If you don't already know it, use the 5 Simple Steps to Get a Clean ARM Template method, it an excellent way to get started.
Once the template is clean we need to add a few things that didn't follow during the export. Usually, in an ARM template, you get one list named resources. However, a Devtest Lab also contains a list named resources but it's probably missing.
{
    "parameters": {},
    "variables": {},
    "resources": [],
}
See In the following example, I added the labs resources list just after the lab's location. This list must contain a virtualnetworks. It's also a good idea to add a schedules and a notificationChannels. Those two will be used to shut down automatically all the VMs and to send a notification to the user just before.

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        ...
    },
    "variables": {
        ...
    },
    "resources": [
        {
            "type": "Microsoft.DevTestLab/labs",
            "name": "[variables('LabName')]",
            "apiVersion": "2016-05-15",
            "location": "[resourceGroup().location]",
            "resources": [
                {
                    "apiVersion": "2017-04-26-preview",
                    "name": "[variables('virtualNetworksName')]",
                    "type": "virtualnetworks",
                    "dependsOn": [
                        "[resourceId('microsoft.devtestlab/labs', variables('LabName'))]"
                    ]
                },
                {
                    "apiVersion": "2017-04-26-preview",
                    "name": "LabVmsShutdown",
                    "type": "schedules",
                    "dependsOn": [
                        "[resourceId('Microsoft.DevTestLab/labs', variables('LabName'))]"
                    ],
                    "properties": {
                        "status": "Enabled",
                        "timeZoneId": "Eastern Standard Time",
                        "dailyRecurrence": {
                            "time": "[variables('ShutdowTime')]"
                        },
                        "taskType": "LabVmsShutdownTask",
                        "notificationSettings": {
                            "status": "Enabled",
                            "timeInMinutes": 30
                        }
                    }
                },
                {
                    "apiVersion": "2017-04-26-preview",
                    "name": "AutoShutdown",
                    "type": "notificationChannels",
                    "properties": {
                        "description": "This option will send notifications to the specified webhook URL before auto-shutdown of virtual machines occurs.",
                        "events": [
                            {
                                "eventName": "Autoshutdown"
                            }
                        ],
                        "emailRecipient": "[variables('emailRecipient')]"
                    },
                    "dependsOn": [
                        "[resourceId('Microsoft.DevTestLab/labs', variables('LabName'))]"
                    ]
                }
            ],
            "dependsOn": []
        }
        ...

Step 2 - The Formulas


Now that the Devtest lab is well defined, it's time to add our formulas. If you had created some already from the portal, don't look for them in the template. At the moment, export won't script the formulas.

A quick way to get the JSON of your formulas is to create them from the portal and then use Azure Resources Explorer to get the code.
resourceExplorer
In a web browser, navigate to https://resources.azure.com, to open your Resource Explorer. Select the subscription, resource group, and lab that you are working on. In the node Formulas (4) you should see your formulas, click one and let's bring that JSON into our ARM template. Copy-paste it at the Resource level (the prime one, not the one inside the Lab).

Step 2.5 - The Azure KeyVault


You shouldn't put any password inside your ARM template, however, having them pre-define inside the formulas is pretty convenient. One solution is to use an Azure KeyVault.

Let's assume the KeyVault already exists, I will explain how to create it later. In your parameter file, add a parameter named adminPassword and let's reference the KeyVault. We also need to specify the secret we want to use. In this case, we will put the password in a secret named vmPassword.
    "adminPassword": {
        "reference": {
            "keyVault": {
                "id": "/subscriptions/{xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx}/resourceGroups/cloud5mins/providers/Microsoft.KeyVault/vaults/Cloud5minsVault"
            },
            "secretName": "vmPassword"
        }
    }
Now to get the password in the ARM template just use a regular parameter, and voila!

Step 3 - The ARM Claimable VMs


Now we have a Lab and the formulas, the only thing missing is the claimable VM based on the formulas. It's impossible to create in one ARM template both formulas and VMs. The alternative is to use a script that will create our VMs just after the deployment.
az group deployment create --name test-1 --resource-group cloud5mins --template-file DevTest.json --parameters DevTest.parameters.json --verbose

az lab vm create --lab-name C5M-DevTestLab -g  cloud5mins --name FrankDevBox --formula SimpleDevBox  
As you can see in the second Azure CLI command, we are creating a virtual machine named FrankDevBox based on the formula SimpleDevBox. Note that we don't need to specify any credential because everything was pre-defined in the formula. Pretty neat!

Here a part of a script that will create if it doesn't exist a KeyVault and populate it. Then it will deploy our ARM template and finally, create our claimable VM. You can find all the code on my GitHub project: Azure-Devtest-Lab-efficient-deployment-sample.

[...]

# Checking for a KeyVault
searchKeyVault=$(az keyvault list -g $resourceGroupName --query "[?name=='$keyvaultName'].name" -o tsv )
lenResult=${#searchKeyVault}

if [ ! $lenResult -gt 0 ] ;then
    echo "---> Creating keyvault: " $keyvaultName
    az keyvault create --name $keyvaultName --resource-group $resourceGroupName --location $resourceGroupLocation --enabled-for-template-deployment true
else
    echo "---> The Keyvaul $keyvaultName already exists"
fi


echo "---> Populating KeyVault..."
az keyvault secret set --vault-name $keyvaultName --name 'vmPassword' --value 'cr@zySheep42!'


# Deploy the DevTest Lab

echo "---> Deploying..."
az group deployment create --name $deploymentName --resource-group $resourceGroupName --template-file $templateFilePath --parameters $parameterFilePath --verbose

# Create the VMs using the formula created in the deployment

labName=$(az resource list -g cloud5mins --resource-type "Microsoft.DevTestLab/labs" --query [*].[name] --output tsv)
formulaName=$(az lab formula list -g $resourceGroupName  --lab-name $labName --query [*].[name] --output tsv)

echo "---> Creating VM(s)..."
az lab vm create --lab-name $labName -g  $resourceGroupName --name FrankSDevBox --formula $formulaName 
echo "---> done <--- code="">

In a video, please!


I also have a video of this post if you prefer.



Conclusion


Would it be for developing, testing, or training, as soon as you are creating environments in Azure, the DevTest Labs are definitely a must. It's a very powerful tool that not enough people know. Give it a try and let me know what do you do with the Azure DevTest Lab?


References:

  • Azure-Devtest-Lab-efficient-deployment-sample: https://github.com/FBoucher/Azure-Devtest-Lab-efficient-deployment-sample
  • An Overview of Azure DevTest Labs: https://www.youtube.com/watch?v=caO7AzOUxhQ
  • Best practices Using Azure Resource Manager (ARM) Templates: https://www.youtube.com/watch?v=myYTGsONrn0&t=7s
  • 5 Simple Steps to Get a Clean ARM Template: http://www.frankysnotes.com/2018/05/5-simple-steps-to-get-clean-arm-template.html



~

How to save huge money by shutting down your VM automatically

Updated on 2018-03-14

Virtual machines (VM) are used in most solutions nowadays as a [ProcessName] server, temporary machine to run tests or make demos, and sometimes even as a development machine. One of the great benefits of the cloud is that you only pay for what you use. So unlike the old server, that you keep paying for, you won pay virtual machine's CPU for when you turned off! In this post, I explain how to do it with your existing machines and also what to do with all the future one that you will be creating.

(Ce billet en aussi disponible en fran├žais.)

Already have a VM up and running, here what to do


From the Azure portal (portal.azure.com), select the Virtual Machine (VM) that you which to edit. Then look at the option panel, on the left, for Auto-Shutdown in the Operations section. You should have something that looks like this:

auto-shutdown

At any time you can enable and disable that functionality, it won’t affect the running VM.

Now, to activate it click on the Enabled. Then Select the time you would like to see the VM shutdown. Be sure to select the good time zone, by default it’s UTC. You can adjust the at for UTC of change the time zone, both options are valid.

Now you could decide to enable the notification. That could be useful if you may want to postpone the shutdown for one or two hours, or integrate the shutdown to another process like backup, cleaning…

To activate the notification option just click on the enabled, and enter the email address. If you want to attach the shutdown to a Logic App or an Azure Functions use the webhook. Here an example of notification email, see the Postpone options link.

emailsample

What if you have many VMs running


Let's say you have already twenty (or more) VMs running, you could have executed a PowerShell script like:


$myMVsName = @("franDev1", "frankBuildserver", "demo_sales2018")

For ($i=0; $i -lt $myMVsName.Length; $i++) {     
    Set-AzureRmDtlAutoShutdownPolicy $myMVsName[$i]
    [...]
}

Update - 2018-03-14
Well, today this is only possible for VM part of a DevTest Labs. Not for "regular" VM. However, I'm sure that day will come pretty quick.Does that mean that you need to go in all your VMs and set it manually? No. You can use an Azure Automation that will stop a list of VM on a regular schedule. A big advantage of this solution is that you can be more creative since it offers a lot more flexibility. You could identify the VM to shutdown base on some TAGS, you could have a different schedule base on the week vs weekend. You could even have a task to start VMs in the morning... More to come on that topic in a future post... If you want to read about how to get started to Azure Automation click here.

Multiple VMs that already exist, no problem

Obviously, if you have multiple virtual machines that already exist it is not very efficient to change their configuration one by one via the portal. Here is a small script to change the configuration of a large amount of VM in one shot.


    '# Login-AzureRmAccount

    $Subscription = Get-AzureRmSubscription -SubscriptionName 'YOUR_SUBSCRIPTION_NAME'
    Select-AzureRmSubscription -Subscription $Subscription.Id

    $selectedVMs = Get-Azurermvm -ResourceGroupName cloud5mins
    foreach($vm in $selectedVMs) 
    { 
        $ResourceGroup = $vm.ResourceGroupName
        $vmName = $vm.Name
        $ScheduledShutdownResourceId = "/subscriptions/$Subscription/resourceGroups/$ResourceGroup/providers/microsoft.devtestlab/schedules/shutdown-computevm-$vmName"
    
        $Properties = @{}
        $Properties.Add('status', 'Enabled')
        $Properties.Add('targetResourceId', $vm.Id)
        $Properties.Add('taskType', 'ComputeVmShutdownTask')
        $Properties.Add('dailyRecurrence', @{'time'= 2100})
        $Properties.Add('timeZoneId', 'Eastern Standard Time')
        $Properties.Add('notificationSettings', @{status='Disabled'; timeInMinutes=60})

        New-AzureRmResource -Location $vm.Location -ResourceId $ScheduledShutdownResourceId -Properties $Properties -Force
    }


The variable $selectedVMs contains all the VMS that we wish to edit. In this sample, I only get VMs contained in the RessourceGroup cloud5mins, but there are no limits to what you can do. You could select all VMs with a specific OS, tags, location, name, etc.

The variable $ScheduledShutdownResourceId will be the identity for the configuration for the auto-shutdown we wish to inject. Note that the provider is microsoft.devtestlab.

Next, we create a collection of properties in $Properties. status the one that active or deactivate the auto-shutdonw. targetResourceId is the resourceID of the VM we target.

The only things left is to specify the time and timezone.

If you prefer, I also have a video version that explains all the steps.

How to shutdown automatically all your existing VMs



End Update

Let's create a VM with the auto-shutdown pre-configured with ARM


Of course, a much more efficient way to set the auto-shutdown is at the creation time by adding a new resource of type Microsoft.DevTestLab/schedules to your template. This option was previously only accessible for DevTestLab, but recently was made available to any VMs.
Here an example of the variables that could be added to your template.

"variables": {

    "ShutdowTime": "21:00",
    "TimeZone": "UTC",
    "emailRecipient": "frank@frankysnotes.com",
    "notificationLocale": "en",
    "timeInMinutes": 30
}

And here an example of Microsoft.DevTestLab/schedules resource. One of these should be added for every VM you wish to auto-shutdown. Because your script is for one server, however, only one instance is required.

{
    "name": "[concat('autoshutdown-', variables('vmName'))]",
    "type": "Microsoft.DevTestLab/schedules",
    "apiVersion": "2017-04-26-preview",
    "location": "[resourceGroup().location]",
    "properties": {
        "status": "Enabled",
        "taskType": "ComputeVmShutdownTask",
        "dailyRecurrence": {
            "time": "[variables('ShutdowTime')]"
        },
        "timeZoneId": "[variables('TimeZone')]",
        "targetResourceId": "[resourceId('Microsoft.Compute/virtualMachines', variables('vmName'))]",
        "notificationSettings": {
            "status": "Enabled",
            "emailRecipient": "[variables('emailRecipient')]",
            "notificationLocale": "[variables('notificationLocale')]",
            "timeInMinutes": "[variables('timeInMinutes')]"
        }
    },
    "dependsOn": [
        "[concat('Microsoft.Compute/virtualMachines/', variables('vmName'))]"
    ]
}