How to Create Your Custom Artifacts for DevTest Labs

Most of the time when we use an Azure Devtest Lab it to Test our own application. This means that will need to install them on the virtual machines, every time. To do that, we need to create a custom artifact and add it to our formulas or to our claimable VMs. Lucky for us, creating a custom artifact is much easier than you may think. In fact, this post I will show you how easy it can be.

Goal

I want to create an artifact available from a private repository (Git from dev.azure.com in this case) that will set the timezone inside the VM.

Getting started

First, let's use a section in the Azure portal that is very useful; the Get Started section. In the portal navigate to your DevTest Lab (1), and select the Getting Started option from the left menu bar (2). In this new bar scroll down to the Lear more area and select Sample artifacts and scripts (3).

GetStarted
That will open the DevTestLab artifacts, scripts and samples project from Azure on Github. Open the folder Artifact, to see the list of all the usual artifacts you find in the public repo that is available by default in the portal.

Notice how all artifacts are in their own folder. When you create a new artifact, you can always come here and pick something similar to what you are trying to do. This way, you won't start from scratch. Let's open windows-vsts-download-and-run-script. An artifact is defined in the file Artifactfile.json. This file is mandatory and cannot be renamed. You can put scripts, images, or anything else you need inside this folder.

Open the Artifactfile.json file and have a look.

ArtifactfileSample
As you can see it's a simple JSON file. In the section (A) you will define the title, description, publisher, OS and the Icon. Note that the Icon must be accessible publicly, it could be on github, a blob storage or on a website. Section (B) is to define all the parameters you may need to install your artifact on the VM. Finally In (C) it's the command to execute.

Create the Artifact

Here is the JSON for our windows-Set-TimeZone artifact. A made it very static by not passing any parameter, but in a reel situation, a timezone parameter would be better.
Artifactfile.json
{
    "$schema": "https://raw.githubusercontent.com/Azure/azure-devtestlab/master/schemas/2016-11-28/dtlArtifacts.json",
    "title": "Set TimeZone to Eastern Standard Time",
    "description": "Execute tzutil command on the VM set set the Time Zone",
    "publisher": "FBoucher",
    "tags": [
        "PowerShell"
    ],
    "iconUri": "https://raw.githubusercontent.com/Azure/azure-devtestlab/master/Artifacts/windows-run-powershell/powershell.png",
    "targetOsType": "Windows",
    "parameters": { },
    "runCommand": {
        "commandToExecute": "tzutil.exe /s \"Eastern Standard Time\""
    }
}

Create an artifact repository

For this post, I'm using Git from Azure Devops (dev.azure.com) previously named VSTS, but any private repository should works. If it's not already done create a project and go to the Repos section. Create a root folder named Artifacts or something else if you prefer. Then add a new folder for your artifact. To follow the best practices you should start with the name of your artifact by the name of the targeted OS; in my case windows-Set-TimeZone. Now add the file Artifactfile.json defined previously.

Note the url of the repository, it should be easy to get it by click on the Clone button that is on the top right of the screen.

clone

Add repository to DevTest Lab

Now we need to add this repository to our Devtest Labs. From the portal.azure.com, open the blade of your lab. From the left panel, click on Repository, then click the Add button.

CreateRepo
It's time to use the information noted previously. This is about the Repository, not the artifact.

Use the Artifact

The only thing left is to use our artifact. You can find it while creating a VM or a formula. When you have parameters define in your Artifactfile.json, the parameters will be listed in a form completly a the left.
artifactList
And if you try it, you will see that the time match the desired timezone. Here my PC is set to display with a format of 24H put it's the same... yep I'm in Eastern Standard Time.

voila

Add it to an ARM template

Doing it with the nice interface is good when you are learning. However, we all know that no DevOps will do that manually every time. So let's add our Repository to our ARM template. If you need more detail on the deployment method, I explain it in a previous post How to be efficient with our Azure Devtest Lab deployments.

When you don't know the type or the structure of a resource, you can always go in the Resource Explorer (resources.azure.com) there will be able to find your resource and see how it's defined.

ResourceExplorer
So for this post our artifactsources will look like this:
{
    "properties": {
        "displayName": "Cloud5mins",
        "uri": "https://fboucher.visualstudio.com/DefaultCollection/Cloud5minsArtifacts/_git/Cloud5minsArtifacts",
        "sourceType": "VsoGit",
        "folderPath": "/Artifacts",
        "armTemplateFolderPath": "",
        "branchRef": "master",
        "securityToken": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
        "status": "Enabled"
    }, 
    "name": "Cloud5minsRepo",
    "type": "Microsoft.DevTestLab/labs/artifactsources"
}
An artifactsources goes in the Resources list inside the Devtest Labs.

ARM
In an ARM template you have the main node Resources (A), then you will have the Lab node (B). Inside this node, you should see second resources list (C), where the Virtual Network is defined. The artifactsources should go there.

Then when you declare your formula, you just need to reference this repository, exactly like the public one.

In a video, please!

I also have a video of this post if you prefer.



Reference:

Reading Notes #344

CI-CD

Suggestion of the week


Cloud


Programming


Books

Five_cover
The Five Dysfunctions of a Team: A Leadership Fable (Patrick Lencioni) - I really enjoyed this book. The fact the first the material was passed as a story adds a lot of perspective and to our comprehension. In the last chapter the author return to the theories and gives more details. I completely devour that book; I'm looking forward to reading more.


Miscellaneous


~Enjoy


What happens when you mix Asp.Net Core, different versions of Docker and Azure for the first time

For a project I have, I wanted to validate if containers were easier to use compare to regular code with services in Azure. I needed to refresh myself with Docker, so I decide to do what I thought would be a simple test: Create an Asp.Net Core web site in a container and access it on my machine.

This post is about my journey to finally achieve this goal, as you may guess it didn't work on the first attempt.

The Goal


One reason why I was looking at containers, it's because it's supposed to be working everywhere right? Well yes but sometimes with a little of effort. The goal here is to be able to run the same container on my main PC, my surface, a Linux VM and of course in Azure.

The context


I have a different setup on my main machine and on my surface. On my PC, I'm using VirtualBox for my VMs so I'm not running Docker for windows, but Docker Toolbox. This flavor (older version) of Docker will create a VM in VitualBox instead of Hyper-V. I couldn't use Docker for Windows like on my Surface, because the two virtualization softwares don't run side by side.

I also wanted to use only tools available on each of this platform, so I decided not to use Visual Studio IDE (the big one). Moreover, I wanted to understand what was happening so I didn't want too much magic involve. Visual Studio is a fantastic tool and I love it. :)

Installing Docker


I needed to install Docker on my Surface. I downloaded Docker Community Edition (CE), and because Hyper-V was already installed everything ran smoothly. On Windows, you need to share the "C" drive from the Docker setting. However, I was getting a strange "bug" when trying to share mine. It was asking my to login with AzureAD and was ignoring my request by letting the share drive uncheckeddockerazureadcredentials.

Thanks to my new friend Tom Chantler, I did search for too long. See the thing is I'm using an AzureAD account to login, and something is not working right at the moment. As explained in Tom's post: Sharing your C drive with Docker for Windows when using Azure Active Directory, to walkaround this situation, I only had to create a new user account with the exact name as my AzureAD account, but without the AzureAD prefix (ex: AzureAD\FBoucher became FBoucher). Once that was done I could share the drive without any issue.

Let's get started with the documentation


The HelloWord container worked like a charm, so I was ready to create my Asp.Net Core website. My reflex was to go on docs.docker.com and follow the instruction from Create a Dockerfile for an ASP.NET Core application. I was probably doing something wrong, because it didn't work. So I decided to start from scratch and do every step manually... I always learn more that way.

Let's start by the beginning


Before moving everything in a container, we need a web application. This can be easily done from the terminal/ command prompt, with the commands:

dotnet new mvc -o dotnetcoredockerappservicedemo

cd dotnetcoredockerappservicedemo

dotnet restore

dotnet publish -c release -o app/ .

Here we create a new folder with a website using the mcv template. I then go in that new folder and restore the Nuget package. To test the we site locally simply use dotnet run. And finally, we build and publish the application into the subfolder app.


Moving to Docker


Now that we have our app it's time to containerize it. We need to add some Docker instruction in a dockerfile. Add a new file name dockerfile (no extension) to the root folder and copy/paste these commandes:

# dockerfile

FROM microsoft/dotnet:2.1-aspnetcore-runtime 
WORKDIR /app COPY /app /app 
ENTRYPOINT [ "dotnet" , "dotnetcoredockerappservicedemo.dll"]

To start Docker with Docker Tool just start the Docker Quickstart Terminal
This instruction will specify how to build our container. First, it will download the image microsoft/aspnetcore or microsoft/dotnet:2.1-aspnetcore-runtime. We specify the work directory, then copy the app folder to app folder inside the container. Finally, we specify the entry point of our application telling it to start with dotnet.
Like Git and it's gitIgnore file docker has the same thing with .dockerignore (no extension). Add that file into your folder to ignore the bin and obj folder.


# .dockerignore
bin\ obj\

Now that the instructions about how to build our container are completed, we can build our container. Execute the following command:

docker build -t dotnetcoredockerappservicedemo .

This will build dotnetcoredockerappservicedemo from the current folder.

Running Docker container locally


Everything is in place, the only thing missing is to run it. If you want to run it locally just go with this command:

docker run -p 8181:80 dotnetcoredockerappservicedemo

On my machine, the port 80 is always used. So I remap the port 80 to 8181, feel free to change it at your convenience. The website will be available at localhost:8181

If you are running Docker Tool (older version of Docker), you need to get the IP of your VM. To get it do

docker-machine ip

Running in the cloud


To run our container into Azure you will need to publish it to the cloud first. It could be on DockerHub or in a private registry on Azure. I decided to go with Azure. First, we need to create a registry, then publish our container.

az group create --name dotnetcoredockerappservicedemo --location eastus

az acr create --resource-group dotnetcoredockerappservicedemo --name frankContainerDemo01 --sku Basic --admin-enabled true

az acr credential show -n frankContainerDemo01

The last command az acr credential show will provides information to tag our container with our repository name and also gives us the credential to be able to push. Of course, you could go to the portal.azure.com and get the information from the Registry's Access Keys blade.

docker tag dotnetcoredockerappservicedemo frankcontainerdemo01.azurecr.io/dotnetcoredockerappservicedemo:v1

Let's connect our docker to our registry, and then push (upload) our container to Azure.


# The https:// is important...

docker login https://frankcontainerdemo01.azurecr.io -u frankContainerDemo01 -p <Password_Retreived>

docker push frankcontainerdemo01.azurecr.io/dotnetcoredockerappservicedemo:v1


Great the container is in Azure. Now let's create a quick webApp to see it. We could also use the Azure Container Instance (ACI) that would be only one command, but because the demo is a website, it won't make sense to use ACI for that.

To get an Application service, we need a Service plan, and then we will create an "empty" webapp. To do that we will specify the runtime without providing any code/binary/container. I wasn't able to create a webapp from a private Azure registry in one command, so this is why I'm doing it in two.

az appservice plan create --name demoplan --resource-group dotnetcoredockerappservicedemo --sku S1 --is-linux

az webapp create -g dotnetcoredockerappservicedemo -p demoplan -n frankdockerdemo --runtime "DOTNETCORE|2.1"

On Windows, I got the following error message: '2.1' is not recognized as an internal or external command, operable program or batch file. The PowerShell command line escape "--%" solves the problem: az --% webapp create -g dotnetcoredockerappservicedemo -p demoplan -n frankdockerdemo --runtime "DOTNETCORE|2.1"

If you check the website right now you should have page saying that the site is up but empty. Let's update the container settings with our registry and container settings.

az webapp config container set -n frankdockerdemo -g dotnetcoredockerappservicedemo --docker-custom-image-name frankcontainerdemo01.azurecr.io/dotnetcoredockerappservicedemo:v1 --docker-registry-server-url https://frankcontainerdemo01.azurecr.io --docker-registry-server-user frankContainerDemo01 --docker-registry-server-password <Password_Retreived> 

It's works of course!
final2


Conclusion


It's only four steps: create the .Net Core application, package it into a Docker container, publish our container into our Azure Registry, and create an application service base on that container. However, because all this tech are cross-platform, sometimes you get some little tiny differences between the platform, and those could become time-consuming. It was a great little project that turned out to be a lot more than expected, but I learn so much!

I'm very happy with the result... expect more of Docker in the future!


In a video, please!


I also have a video of this post if you prefer.




References