PowerBI and Microsoft Azure Consumption

Recently, I needed to check and compare Azure consumption for a client. What a repetitive task: download the csv files from the Azure billing portal, open it in Excel to clean/merge/customize it… Would it be great if it could be easily done? Well, it is! Power BI is the perfect tool to do that (and a lot more).  In this post, I will explain how I created my Power Query and solved different problem I encountered in my journey.

The Goal

I want PowerBI to read (dynamically) all csv files in a folder and updates all my charts and graph, so I can share them easily why my clients.

The Tools

To create Power Queries, you can use the new Power BI Desktop available online for free or Excel. With Excel 2016, the Power query editor tools is included, for the previous version you need to install the Microsoft Power Query for Excel add-in. In both cases, many tutorials explain how to get started with these tools (see the references at the end of this post).

The Problem

Creating our query should be pretty straight forward, since we can create a Power Query by selecting a folder as a source.

The problem is that our file contains three types of records: Provisioning Status, Statement, and Daily Usage. These “tables” are very different and don’t have the same number of columns. This is why when we try to merge them; we got some Error.

The Solution

The way to solve this problem is to create a function that will parse one file to extract one recordset, and call that function for all the file in the folder.

Note:

The simplest way to get started is to work with one file, then convert it to a function. The way to that is to replace the path of the csv file by a variable that will be passed as a parameter: (filePath) =>.

To keep the code as simple as possible, I kept the function and the looping query separated, but they can be merged in only query.

Extract “Daily Usage”

Here are the queries to extract the Daily Usage (third recordSet) from the csv file and some code description.

 // -- fcnCleanOneCSV_v2 ----------------------------------------

(filePath) =>
let
   fnRawFileContents = (fullpath as text) as table =>
let
   Value = Table.FromList(Lines.FromBinary(File.Contents(fullpath)),Splitter.SplitByNothing())
in Value,

   Source = fnRawFileContents(filePath),
   #"Daily Usage Row" = Table.SelectRows(Source, each Text.Contains([Column1], "Daily Usage")),
   #"DailyPosition" = Table.PositionOf(Source, #"Daily Usage Row" {0}),
   #"TopRemoved" = Table.Skip(Source, (DailyPosition + 1)),
   #"Result" = Table.PromoteHeaders(TopRemoved)
in 
   Result

The first part is to load the content of the file as a one column table. Then DailyPosition is used to store the position where Daily Usage data starts. This value is used in Table.Skip(Source, (DailyPosition + 1)) to keep only the rows after, since Daily usage is the last recordSet it works perfectly.

 //== Process Folder CSV_v2 for Daily Usage==============================

let
   Source = Folder.Files("C:\Azure_Consumption_demo\CSV_v2\"),
   MergedColumns = Table.CombineColumns(Source,{"Folder Path", "Name"},Combiner.CombineTextByDelimiter("", QuoteStyle.None),"Merged"),
   RemovedOtherColumns = Table.SelectColumns(MergedColumns,{"Merged"}),
   #"Results" = Table.AddColumn(RemovedOtherColumns , "GetCsvs", each fcnCleanOneCSV_v2([Merged])),
   #"Removed Columns" = Table.RemoveColumns(Results,{"Merged"}),
   #"Expanded GetCsvs" = Table.ExpandTableColumn(#"Removed Columns", "GetCsvs", {"Usage Date,Meter Category,Meter Id,Meter Sub-category,Meter Name,Meter Region,Unit,Consumed Quantity,Resource Location,Consumed Service,Resource Group,Instance Id,Tags,Additional Info,Service Info 1,Service Info 2"}, {"Usage Date,Meter Category,Meter Id,Meter Sub-category,Meter Name,Meter Region,Unit,Consumed Quantity,Resource Location,Consumed Service,Resource Group,Instance Id,Tags,Additional Info,Service Info 1,Service Info 2"}),


   #"Demoted Headers" = Table.DemoteHeaders(#"Expanded GetCsvs"),
   #"Split Column by Delimiter" = Table.SplitColumn(#"Demoted Headers","Column1",Splitter.SplitTextByDelimiter(","),{"Column1.1", "Column1.2", "Column1.3", "Column1.4", "Column1.5", "Column1.6", "Column1.7", "Column1.8", "Column1.9", "Column1.10", "Column1.11", "Column1.12", "Column1.13", "Column1.14", "Column1.15", "Column1.16"}),
   #"Changed Type" = Table.TransformColumnTypes(#"Split Column by Delimiter",{{"Column1.1", type text}, {"Column1.2", type text}, {"Column1.3", type text}, {"Column1.4", type text}, {"Column1.5", type text}, {"Column1.6", type text}, {"Column1.7", type text}, {"Column1.8", type text}, {"Column1.9", type text}, {"Column1.10", type text}, {"Column1.11", type text}, {"Column1.12", type text}, {"Column1.13", type text}, {"Column1.14", type text}, {"Column1.15", type text}, {"Column1.16", type text}}),
   #"Promoted Headers" = Table.PromoteHeaders(#"Changed Type"),
   #"Changed Type1" = Table.TransformColumnTypes(#"Promoted Headers",{{"Usage Date", type date}, {"Meter Region", type text}}),
   #"Replaced Value" = Table.ReplaceValue(#"Changed Type1","""","",Replacer.ReplaceText,{"Meter Category", "Meter Id", "Meter Sub-category", "Meter Name", "Meter Region", "Unit", "Resource Location", "Consumed Service", "Instance Id", "Tags", "Additional Info", "Service Info 1", "Service Info 2"}),
   #"Changed Type2" = Table.TransformColumnTypes(#"Replaced Value",{{"Consumed Quantity", type number}})
in
  #"Changed Type2"

From row 1 to 6, we get all the file in the folder then combine columns to get a full path for each file. We then pass that to our function previously defined. With the command Table.SplitColumn, on line 11, we re-built the result as a table with multiple columns.
The rest of the query is to clean-up the result by changing the column’s type or removing undesired character.

Extract “Statement”

To get the Statement recordSet, it’s the same thing except that we will Table.Range, since the rows that we are looking for are between Provisioning Status and Daily Usage.

//== fcnGetStatement ========================================== 

(filePath) =>
let
   fnRawFileContents = (fullpath as text) as table =>
let
   Value = Table.FromList(Lines.FromBinary(File.Contents(fullpath)),Splitter.SplitByNothing())
in Value,

    Source = fnRawFileContents(filePath),
    #"Daily Usage Row" = Table.SelectRows(Source, each Text.Contains([Column1], "Daily Usage")),
    #"DailyPosition" = Table.PositionOf(Source, #"Daily Usage Row" {0}),
    #"Statement Row" = Table.SelectRows(Source, each Text.Contains([Column1], "Statement")),
    #"StatementPosition" = Table.PositionOf(Source, #"Statement Row" {0}),
    #"SelectedRows" = Table.Range(Source,(StatementPosition+1),(DailyPosition - StatementPosition )-2),
    #"Result" = Table.PromoteHeaders(SelectedRows)
in
    Result

And once again we loop through every file and do some clean-up.

//== Query Statements ========================================

let
    Source = Folder.Files("C:\Azure_Consumption_demo\CSV_v2\"),
    MergedColumns = Table.CombineColumns(Source,{"Folder Path", "Name"},Combiner.CombineTextByDelimiter("", QuoteStyle.None),"Merged"),
    RemovedOtherColumns = Table.SelectColumns(MergedColumns,{"Merged"}),
    #"Results" = Table.AddColumn(RemovedOtherColumns , "GetCsvs", each fcnGetStatement([Merged])),
    #"Removed Columns" = Table.RemoveColumns(Results,{"Merged"}),
    #"Expanded GetCsvs" = Table.ExpandTableColumn(#"Removed Columns", "GetCsvs", {"Billing Period,Meter Category,Meter Sub-category,Meter Name,Meter Region,SKU,Unit,Consumed Quantity,Included Quantity,Within Commitment,Overage Quantity,Currency,Overage,Commitment Rate,Rate,Value"}, {"Billing Period,Meter Category,Meter Sub-category,Meter Name,Meter Region,SKU,Unit,Consumed Quantity,Included Quantity,Within Commitment,Overage Quantity,Currency,Overage,Commitment Rate,Rate,Value"}),


    #"Demoted Headers" = Table.DemoteHeaders(#"Expanded GetCsvs"),
    #"Split Column by Delimiter" = Table.SplitColumn(#"Demoted Headers","Column1",Splitter.SplitTextByDelimiter(",", QuoteStyle.Csv),{"Column1.1", "Column1.2", "Column1.3", "Column1.4", "Column1.5", "Column1.6", "Column1.7", "Column1.8", "Column1.9", "Column1.10", "Column1.11", "Column1.12", "Column1.13", "Column1.14", "Column1.15", "Column1.16"}),
    #"Promoted Headers" = Table.PromoteHeaders(#"Split Column by Delimiter"),
    #"Replaced Value" = Table.ReplaceValue(#"Promoted Headers","""","",Replacer.ReplaceText,{"Meter Category", "Meter Sub-category", "Meter Name", "Meter Region", "SKU", "Unit"})
in
    #"Replaced Value"

Once all that is done… Now the fun can begin!

References

• Microsoft Power Query for Excel
• Power BI Desktop
• Getting started with Power BI Desktop
• Introduction to Microsoft Power Query for Excel
• Power Query (informally known as “M”) Formula reference

What you shouldn't have missed in the last few weeks

In September, Microsoft did so many announcements it’s difficult to keep tracks of all. This short post he just to list the most important and to give you the opportunity to watch them again or the first time.

AzureCon 2015

The AzureCon was a virtual event on September 29 that was focusing on Microsoft Azure. Many View Party around the global was watching this even rich in announcements. You can watch here the AzureCon keynotes online.
But AzureCon was not only keynotes, it was more than 50 technical sessions covering every Azure’s feature. Get the the full list on Channel9 here.

Windows 10 Devices

At the beginning of October the Windows 10 Devices was a really amazing event. Microsoft was showing us all is new devices, and they were a lot! Don’t trust me, go see by yourself on this blog post by Terry Myerson.

 

What’s new

Get Started and deploy your first cloud solution in under 5 minutes. Find tons of short videos online that teach you how to quicky enjoy the power of Azure.



~Frank

What is an AzureCon View Party?

First what is AzureCon?

In less than a week Microsoft is doing a great event called AzureCon. This event is a virtual conference that will focus on Microsoft Azure. It is a virtual event because it's happening online. Even more, it will be available to watch it live for free! The lineup as been published and four great speaker will share with us the latest news about Azure.

What is a View Party?

A View Party is the chance to watch live the same content of all other, but in a group. It's an opportunity to ask your question while it's happening and gets answers from the MVPs or other viewers.

Where are those View Party?

By the time I'm writing this post, I don't know all of them, but please sharing is good, so if you know a view party is happening in your area share the info using the comment session. You could also send me an e-mail, and I will update this post. I will be at Ottawa, looking forward to meeting you there!

• Montreal
  MsDevMtl Community
  2000 McGill College, 5e étage, Montréal, QC, Montréal, QC
  Meetup
  
• Ottawa
  Ottawa IT Community
  100 Queen Street, Suite 500 , Ottawa, ON
  Meetup

Get more from Azure application Insights

Application Insights is an incredible tool that will brings to you a gigantic quantity of information about your application. In a previous post: Tons statistics and metrics for Microsoft Azure websites (not only in Asp.Net), I was explaining how to get started with App Insights. In this post, I want to explain how you can create your own customs events or metrics.

The setup

I will use an Asp.Net MVC named SimpleWebApp that will consume an Azure Api App named FrankApiApp. For the demo purposes I will have both projects in the same solution, but let's pretend that the ApiApp is not us, and that we don't control what inside.

To create an Azure Api App you will need Visual Studio 2015 or Visual Studio 2013 with minimum Azure SDK 2.6 installed. When creating the app, select Asp.Net Web Application, then Azure Api App (Preview).



For this post, we simply need one function. Here the code of the Get method that will wait and return us a string:

    // GET api/values
    public IEnumerable<string> Get()
    {
        var rnd = new Random();
        var waitFor = rnd.Next(500, 3000);

        System.Threading.Thread.Sleep(waitFor);
        var msg = String.Format("You have wait {0}. Sorry about that.", waitFor);

        return new string[] { waitFor.ToString(), msg };
    }

The application where telemetries will be added is a simple Asp.Net Web Application with MVC, created with the default template in Visual Studio. To add App Insights, just right-click on the project and select Add Application Insights Telemetry.



Now we need to add a reference of the Azure API App. That could be easily done. Right-click, once again, on the project and Select Add, then Azure API App Client. A new window will appear, and we only need to select our just deployed API App.

The Base

Of course, just by adding the code spinet to the head of _Layout.cshtml you will have a lot of information about your server, your application but also the client experience and browser. This is all very useful, and it worth the time to navigate/ dig in the Azure portal to understand how your application is going/ doing.

Custom Metric

An interesting information could be to measure the time for each call to FrankApiApp. Let's started by adding a custom metric. To do that we will add a function in the HomeController and a new tab in that menu of the _Layout.cshtml.

public class HomeController : Controller
{
    private TelemetryClient appInsights = new TelemetryClient();

    public ActionResult ApiCall()
    {
        var stopwatch = System.Diagnostics.Stopwatch.StartNew();
        var client = new FrankApiApp();
        var result = client.Values.Get();

        stopwatch.Stop();

        ViewBag.Message = result[1];

        appInsights.TrackMetric("TimeDoingNothing", stopwatch.Elapsed.TotalSeconds);

        return View("Contact");
    }
}   

First, we create an instance of TelemetryClient. Then before calling the FrankApiApp's Get method we start a stopwatch. And we just need to use the TrackMetric method to push the metric.

After deploying and navigating a little in the application, let see what's we have available for us. Open portal.azure.com, and navigate to your application Insights. Click on the Metrics Explorer, a new blade now open with three empty charts. Click on the timeline, then in the Metric texbox type "TimeDoingNothing" (new metric's name). Now, click on the grid (third chart), add the metric name and select Group By Continent. Interesting right?!

Even more

Application Insight gives the opportunity to track many things: PageViews, Events, Exception, Requests, Trace and of course Metrics. Here some alternative of our ApiCall function using the TrackEvent and TrackRequest.

// Using TrackEvent
public ActionResult ApiCall()
{
    var stopwatch = System.Diagnostics.Stopwatch.StartNew();

    var client = new FrankApiApp();
    var result = client.Values.Get();
    stopwatch.Stop();

    ViewBag.Message = result[1];

    var metrics = new Dictionary<string, double> {{"processingTime", stopwatch.Elapsed.TotalMilliseconds}};
    var properties = new Dictionary<string, string>  {{ "MessageSuggestedElapsed", result[0] } };

    // Send the event:
    appInsights.TrackEvent("APICallProcessed", properties, metrics);
    appInsights.TrackMetric("TimeDoingNothing", stopwatch.Elapsed.TotalSeconds);

    return View("Contact");
}



// Using TrackRequest
public ActionResult ApiCallWithTelemetry()
{
    appInsights.Context.Operation.Id = Guid.NewGuid().ToString();

    var stopwatch = System.Diagnostics.Stopwatch.StartNew();
    var client = new FrankApiApp();
    var result = client.Values.Get();

    stopwatch.Stop();

    ViewBag.Message = result[1];

    // Send the event:
    appInsights.TrackRequest("APICallRequest", DateTime.UtcNow, stopwatch.Elapsed, "200", true);
    appInsights.TrackMetric("TimeDoingNothing", stopwatch.Elapsed.TotalSeconds);

    return View("Contact");
}

Wrap it up

I couldn't show everything in one post, it's up to you to create new metrics, events and query them in Azure Application Insights. Share your discovery or questions in the comment section bellow.

References

• Application Insights API for custom events and metrics
• Visual Studio Application Insights

Azure SDK 2.7 my savior

Recently, I got a very unpleasant problem: I needed to remove all my Microsoft Azure accounts from Visual Studio. An error was popping every time Visual Studio was trying to connect to Azure preventing me to anything else. It could happen to everyone, so I decide to share that story so other could save time and energy trying to fix that.

The Context

Long time ago I created my Azure account using my Hotmail account. I modified the administrator to my Outlook.com when it came-up (approx. two years ago). When I installed Azure SKD 2.6, the problem started.

The Error

In Visual Studio 2013 Update 4 or VS 2015 RTM, every time VS was trying to connect to Azure, I was receiving this error message.


Every time Visual Studio 2013 Update 4 or VS 2015 try to connect to Azure...

The Fix

If you only need Visual Studio, remove the "damaged" Azure account. If you need Visual Studio to work with Azure you have two options:

1. Azure SDK 2.5 will work, if that is an acceptable workaround. You would need to make sure Azure SDK 2.6 was never installed on the machine, otherwise it will still have the offending dll.
2. Install Azure SDK 2.7 and/or Visual Studio 2015. In my case, it fixed everything!

~Frank

First VSCode Tasks in less than 5 minutes

I'm working on solution to automating the generation of my weekly post: Reading Notes. While this work is finished, I still need to do many steps manually. I introduced in a previous post VSCode, that already supports markdown. In fact, from any markdown file, you can press Ctrl+Shift+V to preview the oupput. However, today, it's not possible to do anything with this preview. This post explains how we can use VSCode task to fix this problem.

Goal

Create a task that will quickly transform my current open markdown file in HTML.

Step 1 - Get a Markdown parser

First, we need Markdown parser. I'm sure many different are available, but in this post, I will use markdown-js, a Node.js parser that is available on github. To install it, just execute the following command:

npm install -g markdown

Step 2 - Create a Task

Secondly, we need to create our task. Open Visual Studio Code, and type Ctrl+Shift+P to open the Command Palette and then start typing Configure Task. Press Enter or select “Configure Task Runner”. This will create a new file tasks.jon, with many different samples of tasks. Before you replace the content of the page by the following code, take a look at the comment block at the top of the page. You will see different context variables that are available.

// Simple Task to transform MarkDown to Html
{
    "version": "0.1.0",
    "command": "md2html",
    "isShellCommand": true,
    "args": ["${file}"]
}

This defines our task using the json format. The command name in this case is md2html and we pass the name of the current open file as parameter using the variable ${file}. Finally, the "isShellCommand" is set to true so the task will be triggered by pressing Ctrl+Shift+B.

Step 3 - Try it

To try it, open a markdown page, hit Ctrl+Shift+B, and voilà! You should see in the output the HTML code generated.

References

• Node.js
• Markdown-js
• Visual Studio Code - Docs

~Frank

Microsoft MVP Virtual Conference, one more day to go

Yesterday, was the first day of the #MVPvConf 2015. I already write about this event presented by MVP for everyone.

I really enjoy switching from one path to the other following the best fits of my interest. I even got some real though decisions to take because two sessions were at the same time! But wait; there is more!


Today it's Day 2, and many great presentations will by available.

Developer track with topics like: Roslyn Windows 10 ASP.NET Azure Cross-Platform...

IT Pro track with topics like: DevOps System Center Hyper-V Migration Office 365...

Consumer track with topics like: Pivot Table Data Windows 10 Cortana OneNote Security...

LATAM track with topics like: Power BI Exchange Enterprise Mobility Suite (EMS) Office 365...

Brazil track with topics like: Azure Active Directory Hybrid Cloud SQL Server...

Build your own agenda, and come join-us!

Meet my new best friend: Visual Studio Code

On the last week of April, Microsoft was having a huge event call //Build. During three (3) days many great conferences were presented. You can watch them on-demand on Channel 9. The keynote of day one is a must! It was during this keynote that Microsoft reveals many amazing news. One of them: Visual Studio Code. In this post, I indent to share my impression of this tool after one week of usage.

The Beast

You may be thinking: VSCode, it must be a light version Visual Studio... To that, I will reply it's not.

VSCode doesn't go without remembering other modern text editor like Sublime text, or brackets. The interface is far different from Visual Studio though. At this day, three themes are available but you can customize e v e r y t h i n g.

VSCode is free, and it runs on Windows, Linux and Mac, it includes great tooling for web technologies such as HTML, Asp.net, Nodejs, CSS, LESS, SASS and JSon. It has syntax highlighting and a true IntelliSense. It also included: package managers, repositories, Git experience, debug tools, tasks, and so more!

I could continue over and over, but all the features are well explained on the official website.

My experience

It took me less than a minute to install on my "old" surface one. The interface is fast, and everything was looking good... Until I try to do something.

First, I was a bit confused. How should I open or create a project? Where is the menu? How do I use the debugger? After few minutes reading the excellent documentation on the official website, everything became clear. Visual Studio Code is sharing his name with his big brother, but he is really different. VS Code is using a folder approach, and a lot of shortcuts (so you keep your hands on the keyboard). And you quickly learn to use the Command Palette (Ctrl + Shift + P) to do more specific work.

My initial test was why Ghost. I cloned the repository directly from Github to my computer. Then from VSCode, open the folder. Without changing / editing anything, VSCode knew my project was in JavaScript, and the coloring and IntelliSense were working. And(F5), the debugger and I felt already more at home.

A little message informed me that I needed to specify the starting point in the configuration file and voilà! I was debugging a Node.js project, that was easy.

Verdict

After a really short adaptation, Visual Studio Code reveals to be a reel gem. I strongly suggest that you spend few minutes reading the documentation before, to enjoy all his capabilities! The further I use it the more I become completely addicted. Visual Studio Code is an indispensable tool for everyone doing web development or looking for a powerful code editor.

 

 

Reference:

• Visual studio code: Official website where you can find documentation and download VSCode for your favourite platform.

The first ever Microsoft MVP Virtual Conference

Did you eared about that great free event that Microsoft and the MVPs are putting on in May? On the 14th and 15th (yes two days!), join Microsoft MVPs from the Americas’ region as they share their knowledge and real-world expertise during a free event, the MVP Virtual Conference.

by MVPs, for everyone

 

Gigantic event

The MVP Virtual Conference will showcase 95 sessions of content for IT Pros, Developers and Consumer experts designed to help you navigate life in a mobile-first, cloud-first world.  Microsoft’s Corporate Vice President of Developer Platform, Steve Guggenheimer, will be on hand to deliver the opening Key Note Address.
 

Still not sure if you will found what you are looking for?

The conference will have 5 tracks:

• IT Pro English
• Dev English
• Consumer English
• Portuguese mixed sessions
• Spanish mixed sessions

There is something for everyone! Learn from the best and brightest MVPs in the tech world today and develop some great skills!

Join Me!

Be sure to register quickly to hold your spot and tell your friends & colleagues.


 
 
 


The conference will be widely covered on social media, you can join the conversation by following @MVPAward and using the hashtag #MVPvConf.

From Ottawa to Montreal in April 2015

This April two event kept my attention, let me present them.

The MVP Cloud RoadShow - April 11

Where: Ottawa
Register: Meetup

Bored that every one is telling you to move to the cloud, but don't explain how to get there or are how it's working in the cloud?

The MVP Cloud RoadShow, is the perfect opportunity to meet Microsoft Azure MVPs. These specialists will talk about: Intune, Hybrid Identity, Rights Management, SQL Business Intelligence and Building a Lab in Azure.

Here all the information you need on the Meetup page.

Global Azure Bootcamp (Montreal edition) - April 25

Where: Montreal
Register: Meetup

This year is marks the third edition of this great event. In more than 190 locations around the globe, people will develop Cloud Computing applications for Azure, this is definitely a great learning opportunity. Many goods hands-on-labs:

• Infrastructure as a Service in Microsoft Azure (Virtual Machines)
• Getting started with Azure Websites and ASP.NET
• Building a web application with ASP.NET MVC using DocumentDB
• Get Started with the Azure WebJobs SDK
• Get started with Mobile Services
• How to Use Azure Redis Cache
• etc.

For all the detail about the Montreal edition be sure to go on the Meetup page. For more information about the global event go on http://global.azurebootcamp.net/

~Frank B.

Microsoft Azure MVP 2015

With April came the MVP nominations, and I'm very happy to receive the 2015 Microsoft Azure MVP award, and to join the great MVP's family.

I would like to thanks everyone who helps me to share my passion, push me to always move forward, stimulate me to get outside my comfort zone to meet other passionate around the globe. Be sure that this is only the beginning!

The Microsoft Most Valuable Professionals, or MVPs is an award that Microsoft gives to exceptional, independent community leaders who share their passion, technical expertise, and real-world knowledge of Microsoft products with others. It is part of Microsoft’s commitment to supporting and enriching technical communities.

To know more about the MVP program, or to know how you could become an MVP, visit Microsoft MVP website.


See you soon.

~ Frank B

Tons of statistics and metrics for Microsoft Azure websites (not only Asp.Net)

Your website is finally online. That great, good job. But now, you have a lot of questions: How much visits do I have? Which part of the site is mostly visited? Does the site performs well? In this post, I will show you some tools that exist in Microsoft Azure that will help you to get some answers.

Request and Error

When you create a website in Azure, you automatically got some monitoring. Go on portal.azure.com (the preview portal), and select your website.



This will be useful to see how many requests and errors you got. You could create some alerts, by clicking the "+" sign, and you would be notified by e-mail if the number of request is greater than x, over the last hour.

Analytics

The first time you will click on this section all the instruction will be given to you.

To collect end-user usage analytics about your application, insert the following script into each page you want to track. Place this code immediately before the closing </headtag, and before any other scripts. Your first data will appear automatically in just a few seconds.

To get analytics for the whole web site, in an Asp.Net MVC site, a good place will be: \Views\Shared\_Layout.cshtml.

In a Ghost blog, if you are using the default theme, the file will be /content/themes/casper/default.hbs. Otherwise, just replace "casper" by your theme name in the path.

Once your website is re-deployed, re-open the website blade and click again on the Analytics graph. And you will be able to see a lot of information: Session per browser, information on page's views, slowest pages, details on sessions, etc.





When you click on a graph, the Metrics Explorer blade will be visible. On the top of this blade, you will have many different options to customize your results. You will be able to add a chart, change the time range, add some filter and even set some alerts.





Moreover, if you click on those Charts, tables or even on a row, you will have more details and options to fine-tune the result.

Application Insights

You thought it was enough? Well, Microsoft Azure still has one more tool for you, that will cover in this post: Application Insights. With this one, you will be able to see the health of your application by adding some tests, custom events, logs, errors, etc.
Adding Application Insights to your Asp.Net website, can easily be done via Visual Studio, like I explained in a previous post.

Since Azure is compatible with many different languages, chances are that you are using one of those. Let say you are running a node.js Ghost blog, how could you add Application Insights? By using website extensions. To add an extension you could use the Kudu interface. This interface is easy to access. In a browser, type the URL of the website but inject "scm" between the name of the application and the azurewebsites.net. Something like http://mybookmanager.scm.azurewebsites.net/





Once you are in the Kudu interface, click on the tab Site Extensions, then section Gallery. Add the Application Insights Extension.
It's also possible to add a website extension using the website blade.





This will gives you a lot of information already, but to add more customs metric in .Net by sure to add the Application Insights SDK to your porject. In node.js use the applicationinsights package from npm,

~Frank Boucher

References

• Microsoft Azure
• Ghost
• Application Insights Extension
• Write custom telemetry with Application Insights API
• Applicationinsights package (npm)
• Kudu

Visual Studio Online is definitely more than text editor online!

5 powerful features that make Visual Studio online an indispensable tool

If you are a .Net developer then you know Visual Studio. Few months ago, Microsoft released a new version of this tool call Visual Studio 2013 Community edition. This version offers all the features and is completely free. Many of us already have downloaded and installed this version. Another version is also available, but doesn't have the popularity of his brothers... yet. Let's talk about Visual Studio Online.
 

Get Started

Did you know that Visual Studio Online is available for free? Go to http://www.visualstudio.com/ and create an free account that will included:

5 FREE Basic user licenses

Unlimited team projects and private code repos

FREE work item tracking for all users

FREE 60 minutes/month of build

and much more.... 

1- It's a Source Control

It may be called Visual Studio Online, but it has all the features from Team Foundation Server (TFS).
Let's create a new project. From your home page, under the tab Overview (selected by default) click on New. This will pop the form to create a new team project. One interesting thing to note is that both TFS and Git are supported as Version control system.
For the post I will create my project MyBookManager using Git.

2- It's an Agile board

Now that we have a project, it's time to describe it. Visual Studio Online has an entire section just for collaborative work. From the home screen of your VSO click on _Users_ tab. Create your users, roles and manage their permissions.
It's now time to time to describe all the features of the project. Click on _Overview_ tab to get back to Home. Select your project, then click on _Work_ tab. From here you will be able to create all the features you need, and split them in tasks and sub tasks; perfect to fill-up a backlog item list and plan your next sprint.
Once a sprint is started, it's also from here that you will be able to see and interact with the Board.

From this board, cards can be move by dragging them in the column that reflected the current status of the work. Open the on-premise Visual Studio. From the bar Team Explorer click on the little plug, and select: “Select Team Project…”. If your VSO Server is not present, just add it, then select the your project (ex: MyBookManager).
For this post I will create an empty Asp.Net MVC project. In the Models folder I add a new class Book with very basic code.

public class Book
{
     public int BookID { get; set; }
     public string Title { get; set; }
     public bool Read { get; set; }
}

Then by right-clicking on the folder Controllers select the Add new Scafolded Item option, and use that to create our CRUD controller and views for the Book model we just created. Let’s check-in the code, and go back to VSO. You should see the code.

To give more visibility to the work’s status, you can create very nice chart and pinned them on the Home page. To add a new chart, go in the Work tab select Queries.  You can use the existing query or simply create a new one. To add a chart to the Home page right-click on the query and select Pin to Home page.

3- It's a build / Deployment server

If it's one think that seem always complex for a developer is the automation build and deployment server setup. With VSO no need for all that complexity and resources, only follow a short wizard and your build machine will be setup in Azure ready to deploy as soon as you want.  You can setup an automatic build and deployment from the Azure Portal and Visual Studio. In this post, I will use Visual Studio.

From the Team Explorer bar in Visual Studio, click on Builds then New Build Definition, to get the wizard.



First step, just put a name who makes sense.



Step two, we need to choose what will trigger the build. I select continuous integration, so the build should start at every check-in.



When I create my project I select Git as a source control, so for the next step I will need to expand the Build process template zone, by clicking Show details.  Then from the list select GitContinuousDeploymentTemplate.12.xaml.

The build definition is done, save and let it there for now.

4- It's a Code Editor

If it's one feature that you could expect from a tool like VSO is to be able to edit code online.  The online version is obviously less powerful then the on-premise one, but still you will it very convenient and easy to use.

Let's change the button in the book list to be more modern. First, select Code tab.  Now let's find the Index.html file under the View folder. Once the file is selected, click on Edit button. It's time to add some bootstrap classes to transform the look of our button. As you can see even we are in a web  browser we have IntelliSense. 



Hit Save and it will commit our change. Since we create a continuous build, this one should kick-in and build and deploy our changes. We can see the status of our build by going in the Build tab in Visual Studio Online. It’s also possible to see it in the Home page by pinning it.
When the build will be done, our change will be visible in the website. Of course, this build is very simple, unit test and more complexes processes could also be implemented.

5- It's a Test server

Another great feature is the possibility to create a test plan for the quality assurance (QA) tester. You could create it via VSO as a test plan by going in the last tab Test. And create a checklist,  so they can follow it to test the application feature by feature.  You could also record the action with Microsoft Test Manager and then do a load test on the website by running simultaneously multiple tests on different instances in Azure.

Verdict

Visual Studio Online is a great tool and win to  be now more. What are the features you enjoy the most in VSO?


~Frank B

References

http://www.visualstudio.com/

Stop losing time installing your software

Automation is the key to many hard situations. Everyone knows that, yet a lot of tasks are done manually. In this post, I will share with you some easy steps to install all your applications.

Let's get started

For those how follow this blog I talked many times about Chocolatey. For all the others, Chocolatey is a Machine Package Manager, somewhat like apt-get (on Linux), but built with Windows in mind.
Assuming that you don't have Chocolatey installed, let's start by that. Open a command prompt (cmd.exe) as Administrator, and execute this command:

@powershell -NoProfile -ExecutionPolicy unrestricted -Command "iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))" && SET PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin

Now anytime you need an application you can simply do a choco install command. This will download the latest version of the package online the last package, and install it on your PC.
Here are some examples with popular applications:

Application	Command
Notepad++	choco install notepadplusplus
Atom	choco install atom
7-Zip	choco install 7zip
Skype	choco install skype

You can find the complete list of all applications on the Chocolatey web site. Many other commands are also available to search, list and update some packages:

Command	Description
choco install atom -Version 0.140.0	Install old version (0.140.0) of Atom
choco list nunit	Show all package that contain nUnit
choco update	updates chocolatey to the latest version
choco update notepadplusplus	updates NotePad++ to the latest version

Let's go a step further

Will it be nice if we could chain them? This is possible with Boxstarter, a repeatable, reboot resilient windows environment installations using Chocolatey packages. To install Fiddler, Atom, and Visual Studio 2013, simply type this command in Internet Explorer (IE) and Boxstarter will starts his magic.

http://boxstarter.org/package/fiddler4,atom,visualstudiocommunity2013

---

Note: This will work only in Internet Explorer (IE), on Chrome or Firefox you will need a "Click-Once" extension.

---

Turn it up to 11

Now that we know we can chain them, what is stooping us to create a script that will install ALL our favorite applications? Well, nothing!

Boxstarter is a very easy and powerful way to automate the installation of software. Here an example to show some of the great features supported by Boxstarter. In the following script I will: configure Windows, install the latest Windows updates, install Visual Studio Community 2013 with an extension and install many other applications.

# Windows Configuration
Update-ExecutionPolicy Unrestricted
Set-ExplorerOptions -showHidenFilesFoldersDrives -showProtectedOSFiles -showFileExtensions
Enable-RemoteDesktop
Disable-InternetExplorerESC
Disable-UAC #Win8
Set-TaskbarSmall

if (Test-PendingReboot) { Invoke-Reboot }

# Update Windows and reboot if necessary
Install-WindowsUpdate -AcceptEula
if (Test-PendingReboot) { Invoke-Reboot }

# Install Visual Studio Community 2013
choco install choco install visualstudiocommunity2013

# VS extensions
Install-ChocolateyVsixPackage PowerShellTools http://visualstudiogallery.msdn.microsoft.com/c9eb3ba8-0c59-4944-9a62-6eee37294597/file/112013/6/PowerShellTools.vsix


# Install WebPI (Platform Installer) 
choco install webpi

# Install
#  . SSDT
#  . Microsoft Azure SDK - 2.4.1
#  . Microsoft Azure SDK for .NET (VS 2012) - 2.4
C:\Program Files\Microsoft\Web Platform Installer>WebpiCmd.exe /Applications: SSDT, WindowsAzureSDK_2_4_1, VWDOrVs2012AzurePack.2.4


if (Test-PendingReboot) { Invoke-Reboot }

# Install Favourite Tools
choco install sourcetree
choco install resharper
choco install Atom
choco install LinqPad
choco install fiddler4

You can now execute this script executing the command in IE:

http://boxstarter.org/package/url?C:/dev/Demoscript.txt 

In this case, the file was on a local folder, put it could have been in a share folder on another server, or somewhere online like in gist.github.com.

Wrapping up

I hope it will help you to automate repetitive task and save time. For more information about Chocolatey or Boxstarter, go visit their respective web site. Any comments, suggestions and/or questions are welcome.


~Frank B

Toby, did you see what I just did!

Today, I was running around with my laptop trying to find someone to show him what I did. My problem was that since I'm working from home, I found no one except my dog... Toby, did you see what I just did! He was looking at me and didn't really care about that I was doing some C# in Atom, a regular text editor. So, here I am now, sharing my discovery with you.

The "What"

While reading some article on the Internet, I fall on a video talking about OmniSharp.

A family of Open Source projects, each with one goal - To enable great .NET development in YOUR editor of choice.
SO I decide to give it a try in one of my favorite text editors this time called Atom.

Less than two minutes after, I was running across my house....

The "How"

What I like about Atom that it is so easy to install and to customize. The easiest way to install it is via Chocolatey.

Chocolatey NuGet is a Machine Package Manager, somewhat like apt-get, but built with Windows in mind.

Assuming that you don't have Chocolatey installed, let's start by that. Open a command prompt (cmd.exe) as Administrator, and execute this command:

@powershell -NoProfile -ExecutionPolicy unrestricted -Command "iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))" && SET PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin

In a new command prompt again with the administrator permission, let's Install the text editor Atom:

cinst Atom

and Git:

cinst Git

Now to install the OmniSharp in Atom you have two options. You could do it through the Settings or using a console. For this post, I will use the second option. Let's open a new command prompt, always as Administrator.
The reason why I use a new prompt every time is to be sure that environment variable gets refreshed.

Execute these command:

apm install language-csharp
apm install autocomplete-plus-async
apm install omnisharp-atom

Now open Atom and let's put some code:

using System;

namespace ConsoleAppDemo
{
    class Program
    {
        static void Main(string[] args)
        {
          var myBook = new Book{Title="Get Started with OmniSharp"};
          Console.WriteLine(String.Format("Here is my review of: {0}", myBook.Title));
        }
    }

    public class Book
    {
      private string _review;

      public string Title{get;set;}

      public string Review{
        get{
          if(String.IsNullOrEmpty(_review))
          {
            _review = "This book is nice";
          }
          return _review;
        }
        set{
          _review = value;
        }
      }
    }
}

Nothing special until you start the OmniSharp server with Ctrl-Alt-o.

Boom!

As you can see now the intelisense, completion, code navigation and so more! If you click on the little green flame on the bottom left you see details about notification and error message.

The end

OmniSharp is a frontend compiler, not a complete compiler so it doesn't generate CLI code. But It's already a lot. Today, you can use OmniSharp with Sublime Text 3, Atom, Emacs, Brackets, Vim and with command line only. So whether on your old laptop or your new PC, whether you run Linux, Mac or Windows let's do some C#!


~Frank Boucher

Four Simple Tips to Improve your Asp.Net MVC Project

When it's time to do the re-factoring of a solution, it's always a good idea to clean the code before doing any re-factoring. In this post, I will share with you simple but very efficient ways to improve your solution.

1- Forget the magic string

By default in Asp.Net MVC magic strings are used every where.

return View("Index");

or

@Html.ActionLink("Delete Dinner", "Delete", "Dinners", new { id = Model.DinnerID }, null)

Nothing bad here, but nothing will tell you that to did a typo, or that the method name as changed. This is where T4MVC will become a great tool do add to all your project.

To add it a simple Nuget command is enough: Install-Package T4MVC. By doing this a T4 file (T4MVC.tt) will be added to your project that will generates a number of files. These generated files will simplify your life and gives you the opportunity to code using strongly type.

Here are few transformations:

// Before ----------------
  return View("Index");

// After with T4MVC
  return View(Views.Index);

An action link in a view.

// Before ----------------
  @Html.ActionLink("Delete Product", "Delete", "Products", new { id = Model.ProductID }, null)

// After with T4MVC
  @Html.ActionLink("Delete Product", MVC.Products.Delete(Model.ProductID))

An Ajax call.

// Before ----------------
<%= Ajax.ActionLink( "RSVP for this event",
                 "Register", "RSVP",
                 new { id=Model.DinnerID }, 
                 new AjaxOptions { UpdateTargetId="rsvpmsg", OnSuccess="AnimateRSVPMessage" }) %>

// After with T4MVC
<%= Ajax.ActionLink( "RSVP for this event",
                 MVC.RSVP.Register(Model.DinnerID),
                 new AjaxOptions { UpdateTargetId="rsvpmsg", OnSuccess="AnimateRSVPMessage" }) %>

A redirection.

// Before ----------------
return RedirectToAction("Details", new { id = product.ProductID });

// After with T4MVC
return RedirectToAction(MVC.Products.Details(product.ProductID));

When writing the code, it gives you intellisense where you normally would not have any. At compile time, it validates all the code so no typo or other misspelling errors are present.

2- Clean your views

You know all those "@using" on the top of each views that we copy over and over... It's time to remove them. The way to do it is by moving them to the web.config file in the "Views" folder.



So you can move the namespaces used globally

@using Microsoft.Security.Application
@using System.Globalization;

by including them to this section:

<system.web.webPages.razor>
  <pages pageBaseType="System.Web.Mvc.WebViewPage">
    <namespaces>
      <add namespace="System.Web.Mvc" />
      <add namespace="System.Web.Mvc.Ajax" />

      <add namespace="Microsoft.Security.Application" />
      <add namespace="System.Globalization" />

    </namespaces>
  </pages>
</system.web.webPages.razor>

3- Don't lose time debugging

To many people are losing time debugging their application or web site. Start using Glimpse right away! This will provide information in real time across all layers of your application from the UI to the server and database side. Perfect to know everything that happen on a click of a button: javascript validation, controller code and even the query in the database.







Install it in ten seconds with the Nuget command manager and pick the version you need.

PM> Install-Package Glimpse

Glimpse is secure and is configured to be accessible only from localhost by default. But don't trust me and try it by yourself, or go check this one minute Glimpse Heads Up Display youtube video.

4- Start monitoring your website health and usage

Move your website on Microsoft Azure and use the Application Insights. This will gives you the opportunity to monitor that availability, performance and usage of your live application.




To add it you got many possibilities, one of them from Visual Studio 2013, just right-click on the project and select Add Application Insights Telemetry, and voilà!

Now you just need to run or deploy the website and after few minutes or so you will have plenty of information, graphs waiting for you in the Azure Portal.







You will find a lot of information about Application Insights on the Microsoft Azure

Wrapping up

I hope it will help you, thanks for reading. Any comments, suggestions and/or questions are welcome.

~ Frank Boucher

References

• T4MVC Documentation: https://t4mvc.codeplex.com/documentation
• Glimpse Documentation: http://getglimpse.com
• Application Insights: http://azure.microsoft.com/en-us/documentation/articles/app-insights-start-monitoring-app-health-usage/