Showing posts with label storage. Show all posts
Showing posts with label storage. Show all posts

Reading Notes #403



Every week, I publish my reading notes. Those are the articles, blog posts, podcast episodes, and books that catch my interest and that I found interesting. It's a mix of the actuality and what I was looking for.

I passed the 4k subscribers mark on YouTube

The suggestion of the week


Cloud


Programming

  • ASP.NET Core updates in .NET Core 3.1 (Sourabh Shirhatti) - I'm very excited about this 3.1 version.I'm not sure why maybe it's because it is the long-time support (LTS). Nevertheless, I will update all my projects.

Miscellaneous

  • 7 Dangers of Micromanagement ( Jack Wallen) - Nice post to help you give the best version of yourself. (Note it's now 6, but it was 7 when I read it)

Books


Build Your Dream Stream

Ashley

Great book with many very interesting ideas.Not the same things that we ear elsewhere, or if it is, it's in very different words.








~

Reading Notes #395


Cloud


Programming


  • New workflow editor for GitHub Actions (Chris Patterson) - Have you tested the new GitHub action? If yes you will be pleased with this new editor...Ending the research of that missing space somewhere.

Miscellaneous

  • What You Need for Effective Remote Work (William Gant) - This is a full chapter of an up coming book about remote workers... If you are new to this adventure and even more if you are fulltime remote, this read is a must.

Books




Author: Gretchen Rubin 

I really enjoyed this book. I found very interesting the categorization of all those habits and comportment grouping. I like also the habits association to help to break some or creating new ones. It's obvious, but I didn't think about it before.



Reading Notes #392


The suggestion of the week


Cloud

  • Andrew Connell's Blog (Andrew Connell) - This nice post is the second of a series of three. It explains how to do every step but also why the author decided to do that.

Programming


Miscellaneous


~

Reading Notes #390


Suggestion of the week

Cloud

Programming

Podcasts

  • Economics of Kubernetes, with Owen Rog (Craig and Adam) - Really interesting episode. Of course all the news about Kubernetes were interesting, but even more the economics of cloud computing with the guess of the week, Owen Rog.

Miscellaneous

Reading Notes #387

Cloud


Programming


Podcasts


Books



Dare to Lead: Brave Work. Tough Conversations. Whole Hearts. (Brené Brown) - A nice book. pack with a lot of information. A lot's of stories to emphasize her points, I always like that. It was maybe a little too much cartesian for me... many steps. Or maybe I was not in a good mindset. Good book however.








~

Reading Notes #385


Suggestion of the week

Programming


~


The Dog-Not-a-Dog Workshop


I recently presented, a workshop at the TOHack to get started with Azure. The goal was to try different Azure services, and see how we could augment an existing website using serverless function and artificial intelligence.
(Aussi disponible en français)


During this workshop, a website is deployed automatically from GitHub. Then by adding an Azure Function and using the Vision API of Azure Cognitive Services, the final solution is able to detect when uploaded pictures are or not dogs and keep our image folder "clean". We call that application: The automatic Not a Dog application.

The step by step instruction with the code can be found on GitHub - Not-a-Dog-Workshop. The workshop can be done in about 45-60 minutes.

I also did a video that is available on my YouTube channel:



You have questions, you are blocked, it will be a pleasure to help you.


Reading Notes #384

Programming

  • Install WSL 2 on Windows 10 (Thomas Maurer) - Awesome tutorial. If like me you didn't want to wait until the next Windows release or take the time to compile and debug a deployment....this tutorial is for us!

Databases


Miscellaneous

~

Reading Notes #374

Cloud


Programming


Podcast

  • Hevesh5 - Making a YouTube Career from Viral Domino Art (#46) (That Creative Life) - Great show. An amazing story.
  • Azure Functions using Node with Simona Cotin (.NET Rocks!) - Great show. I just switch my website following that Jam stack pattern. I was planning to use Azure Functions to add a few little twists.... I'm happy to see that I not alone thinking like that!
  • 0230 - Alain Vezina - Le métier du DevOps (Visual Studio Talk Show) - Super épisode, très intéressant d'entendre parler du rôle de DevOps de quelqu'un qui le vie au quotidien. Merci de la suggestion, je crois, bien que je suis du pour relire The Pheonix Project.
  • Goal Setting Tips & Tracking KPIs (Video Pursuit Podcast) - Really interesting episode. Everybody is talking about matrix and KPI... But it's not frequent to hear about the "how". I really like how the goals are explained, achievable, but not easy... And how we should react when we don't reach them.

Miscellaneous


~ Good week!

How create a static website on Azure Storage

I have been waiting for this feature for so long! I know; it's not a major feature, but it fills an important gap in the Azure offer. We can now create static websites in the Azure Blob Storage (as I'm writing this post the service is still in preview). In this post, I will explain why I think it's a really good news, show how to create and publish on a static website.

Why It's an Awesome News


The cloud is the perfect place when you need to build something huge very quickly. It's also an excellent solution when you have a lot of variance in the number of resources it required. Because Azure is a service, it will provide you as many resources as you would like in few minutes. And when you are done with the resources you stop paying for them; and it's really great like that!
However, if the only thing you need was to host a little something like a blog or a little website for an event or some temporary publicity Azure was not the best place for it. I mean yes of course, you could build a service and host many little websites on it (Scott Hanselman as excellent posts about that like this one), but it felt always a bit overkill for most of the users. Some people kept an "old style" host provider just for that. I mean it's fine, it works... But with Azure storage, it will be really reliable, and at a lower cost! Let's see how we can create one.

Create a Static Website


To have the static website feature you need to create an Azure Blob Storage account the same way you created them before, however, it needs to be of kind General Purpose V2 (GPV2). Today if you install the Azure CLI Storage-extension Preview, you can use it to create one, or simply go on the portal.azure.com. Let's use the portal since it's more visual.

createStorage
Once the storage is created, open it. On the left menu of the storage blade, click on the Static website (preview) option. That will open the configuration page for our static website. First, click the Enabled button then enter the initial/ index document name (ex:index.html). Finally, click the Save button on the top of the blade.

ConfigureStatic
The shell for our website is now created. A new Azure Blob Storage container named $web h been created. The Primary and secondary endpoint should now be displayed (ex: https://frankdemo.z13.web.core.windows.net/). If you test this URL, you will see and message saying that the content doesn't exist... and it's normal.

emptywebsite

Create some content


This is the part where it all depends on your needs. You may already have some HTML pages ready, or you may want to code them all yourself, or the website may previously exist. For this post, I will create a brand-new blog using a static website generator named Wyam (if you would like to see how to do it with Jekyll, another generator, I used it in the video)
To create a new template with Wyam you use the following command in a command prompt. That will create a new website in the subfolder output.
wyam --recipe Blog --theme CleanBlog

Publish to Azure


It's now time to upload our content to the Azure blob Storage. The easiest is probably directly from the portal. To upload a file, click on the $web container, then the Upload button. From the new form, select the file and upload it.

portalUpload
The main problem with this method is the that it only works one file at the time... And a website usually has many of those...
A more efficient way would be to use Azure Explorer or some script. Azure Explorer doesn't support yet the Azure Storage Static Website, but it will be soon. So that leads us to scripts or command lines.

AzCopy


I really like AZCopy as it's very efficient and easy to use. Unfortunately, as I'm writing this post, AzCopy doesn't support the Azure Storage Static Website. I try to upload all content from the output folder (and sub folders)) with a command like this, but it fails.
azcopy --source ./output --destination https://frankdemo.blob.core.windows.net/$web --dest-key fec1acb473aa47cba3aa77fa6ca0c5fdfec1acb473aa47cba3aa77fa6ca0c5fd== --recursive

Azure CLI


An Azure CLI extension preview is also available. Like I mentioned previously, the extension gives you the possibility to create a static website or update the configuration, to upload files you have two options the batch would be more efficient of course, but the file by file option also works. Thanks to Carl-Hugo (@CarlHugoM) for your help with those commands.


az storage blob upload-batch -s "./output" -d $"web" --account-key fec1acb473aa47cba3aa77fa6ca0c5fdfec1acb473aa47cba3aa77fa6ca0c5fd== --account-name frankdemo

az storage blob upload -f "./output/index.html" -c $"web" -n index.html ---account-key fec1acb473aa47cba3aa77fa6ca0c5fdfec1acb473aa47cba3aa77fa6ca0c5fd== --account-name frankdemo

Visual Studio Code Azure Storage Extension

I finally tried the Visual Studio Code Stogare Extension. After installing it, you need to add a User Setting Ctrl + ,. Then add "azureStorage.preview.staticWebsites" : true to your configuration. Now you just need to click on the extension, then select Azure blob storage from your subscription, and right click to be able to upload a folder.

vscodeupload
Depending on how many files, and their sizes it will take a moment. VSCode will notify you when it's done. You will then be able to get back online and refresh your website to see the result.

website

Conclusion


I'm very happy to see that feature because it fills a need that was not really cover yet by the Microsoft offer. Right now, it's an early preview so even if the service is very stable, not all the tools support it but that only temporary. Right not you can set your custom domain name, however, HTTPS is not supported.
So what do we do with it? Should we wait or jump right on? Well as the best practices imply when a feature is in preview don't put your core business on it yet. If you are just looking to build a personal website, a little promo than... enjoy!

In video, please!


I also have a video of this post if you prefer.




References




Reading Notes #334

canadaflag

Suggestion of the week

  • HTTPS Is Easy! (Troy Hunt) - A wonderful series of 4 videos that explains how to get secure with https. A must!

Cloud


Programming


Miscellaneous


Reading Notes #333

flag-28555_640Cloud


Programming


Data


How to copy files between Azure subscription from Windows, Linux, OS X, or the cloud

(en français: ici)

Copy, Download or Upload from-to any combination of Windows, Linux, OS X, or the cloud


Data is and will always be our primary concern. Whether shaped as text files, images, VM VHDs or any other ways, at some point in time, our data will need to be moved. I already wrote about it previously, and the content of this post is still valuable today, but I wanted to share new options and convert all ground (meaning Linux, Windows and OS X).

Scenarios


Here few scenarios why you would want to move data.
  • Your Microsoft Azure trial is ending, and you wish to keep all the data.
  • You are creating a new web application, and all those images need to be moved to the Azure subscription.
  • You have a Virtual Machine that you would like to move to the cloud or to a different subscription.
  • ...

AZCopy


AzCopy is a fantastic command-line tool for copying data to and from Microsoft Azure Blob, File, and Table storage. At the moment, to write this post AzCopy is only available for Windows users. Another solution will be introduced later in this post for Mac and Linux users. Before AzCopy was only available on Windows. However, recently a second version built with .NET Core Framework is available. The commands are very similar but not exactly the same.

AzCopy on Windows


In his simplest expression, an AzCopy command looks like this:
AzCopy /Source:<source> /Dest:<destination> [Options]
If you earlier have installed an Azure SDK on your machine, you already have it. By default, AzCopy is installed to %ProgramFiles(x86)%\Microsoft SDKs\Azure\AzCopy (64-bit Windows) or %ProgramFiles%\Microsoft SDKs\Azure\AzCopy (32-bit Windows).

If you need only AzCopy for a server, you can download the latest version of AzCopy.
Let's see some frequent usage. First let's say you need do move all those images from your server to an Azure blob storage.
AzCopy /Source:C:\MyWebApp\images /Dest:https://frankysnotes.blob.core.windows.net/blog /DestKey:4YvvYDTg3UUpky8Rj5bDG4KO/R1FdtssxVnunsEd/4rAS04V2LkO0F8mXbddAv39WtCo5LW6JyvfhA== /S
CopyAllImages

Then to copy those images to another subscription very easy.
AzCopy /Source:https://frankysnotes.blob.core.windows.net/blog /Dest:https://frankshare.blob.core.windows.net/imagesbackup /SourceKey:4YvvYDTg3UUpky8Rj5bDG4KO/R1FdtssxVnunsEd/4rAS04V2LkO0F8mXbddAv39WtCo5LW6JyvfhA== /DestKey:EwXpZ2uZ3zrjEbpBGDfsefWkj3G2QY5fJcb6kMqV2A0+2TsGno+mk9vEXc5Uw1XiouvAiTS7Kr5OGzA== /S

AzCopy Parameters


These examples were simple, but AzCopy is a very powerful tool. I invite you to type one of the following commands to discover more about using AzCopy:
  • For detailed command-line help for AzCopy: AzCopy /?
  • For command-line examples: AzCopy /?:Samples

AzCopy on Linux


Before you could install AzCopy you will need to install .Net Core. This is done very simply with few commands.
curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg
sudo mv microsoft.gpg /etc/apt/trusted.gpg.d/microsoft.gpg
sudo sh -c 'echo "deb [arch=amd64] https://packages.microsoft.com/repos/microsoft-ubuntu-xenial-prod xenial main" > /etc/apt/sources.list.d/dotnetdev.list'
sudo apt-get update
sudo apt-get install dotnet-sdk-2.0.2
Then to install it, you just need to get it with a wget command, unzip it, and execute the install script.
wget -O azcopy.tar.gz https://aka.ms/downloadazcopyprlinux 
tar -xf azcopy.tar.gz 
sudo ./install.sh
In his simplest expression, the .Net Core version of AzCopy command looks like this:
azcopy --source <source> --destination <destination> [Options]
It is very similar to the original version, but parameters are using -- and - instead of the / and where a : was required, it's now a simple space.

Uploading to Azure


Here an example, to copy a single file GlobalDevopsBootcamp.jpg to an Azure Blob Storage. We pass the full local path to the file into --source, the destination is the full URI, and finally the destination blob storage key. Of course, you could also use SAS token if you prefer.
azcopy \
--source /home/frank/demo/GlobalDevopsBootcamp.jpg \
--destination https://frankysnotes.blob.core.windows.net/blog/GlobalDevopsBootcamp.jpg \
--dest-key 4YvvYDTg3UUpky8Rj5bDG4KO/R1FdtssxVnunsEd/4rAS04V2LkO0F8mXbddAv39WtCo5LW6JyvfhA== 

Copying Between Azure Subscriptions


To copy the image to a second Azure subscription, we use the command the source is now an Azure Storage URI, and we pass the source and the destination keys:
azcopy \
--source https://frankysnotes.blob.core.windows.net/blog/GlobalDevopsBootcamp.jpg \
--destination https://frankshare.blob.core.windows.net/imagesbackup/GlobalDevopsBootcamp.jpg \
--source-key 4YvvYDTg3UUpky8Rj5bDG4KO/R1FdtssxVnunsEd/4rAS04V2LkO0F8mXbddAv39WtCo5LW6JyvfhA== \
--dest-key EwXpZ2uZ3zrjEbpBGDfsefWkj3G2QY5fJcb6kMqV2A0+2TsGno+mk9vEXc5Uw1XiouvAiTS7Kr5OGzA== 

Azure CLI


Azure CLI is a set of cross-platform commands for the Azure Platform. It gives tools to manipulate all Azure components, but this post will focus on azure storage features.

There are two versions of the Azure Command-Line Interface (CLI) currently available:

  • Azure CLI 2.0: written in Python, conpatible only with the Resource Manager deployment model.
  • Azure CLI 1.0: written in Node.js, compatible with both the classic and Resource Manager deployment models.

Azure CLI 1.0 is deprecated and should only be used for support with the Azure Service Management (ASM) model with "classic" resources.

Installing Azure CLI


Let's start by installing Azure CLI. Of course, you can download an installer but since everything is evolving very fast with not getting it from Node Package Manager (npm). The install will be the same, you just need to specify the version if you absolutely need Azure CLI 1.0.

sudo npm install azure-cli -g

AzureCli_installed

To keep the previous scenario, let's try to copy all images to a blob storage. Unfortunately, Azure CLI doesn't offer the same flexibility as AzCopy,and you must upload the file one by one. However, to upload all images from a folder, we can easily put the command in a loop.

for f in Documents/images/*.jpg
do
   azure storage blob upload -a frankysnotes -k YoMjXMDe+694FGgOaN0oaRdOF6s1ktMgkB6pBx2vnAr8AOXm3HTF7tT0NQWvGrWnWj5m4X1U0HIPUIAA==  $f blogimages
done

azurecli_allimages

In the previous command -a was the account name, and -k was the Access key. This two information can easily be found in the Azure portal. From the portal (https://portal.azure.com), select the storage account. In the right band click-on Access keys.

StorageAccessKeys

To copy a file (ex: a VM disk aka VHD) from one storage to another one in a different subscription or region, it's really easy. This time we will use the command azure storage blob copy start and the -a and -k are related to our destination.

azure storage blob copy start 'https://frankysnotes.blob.core.windows.net/blogimages/20151011_151451.MOV' imagesbackup -k EwXpZ2uZ3zrjEbpBGDfsefWkj3GnuFdPCt2QY5fJcb6kMqV2A0+2TsGno+mk9vEXc5Uw1XiouvAiTS7Kr5OGzA== -a frankshare

The nice thing about this command is that it's asynchronous. To see the status of your copy just execute the command azure storage blob copy show

azure storage blob copy show -a frankshare -k YoMjXMDe+694FGgOaN0oPaRdOF6s1ktMgkB6pBx2vnAr8AOXm3HTF7tT0NQVxsqhWvGrWnWj5m4X1U0HIPUIAA== imagesbackup 20151011_151451.MOV

CopyStatus1


CopyStatus2

Azure CLI 2.0 (Windows, Linux, OS X, Docker, Cloud Shell)


The Azure CLI 2.0 is Azure's new command-line optimized for managing and administering Azure resources that work against the Azure Resource Manager. Like the previous version, it will work perfectly on Windows, Linux, OS X, Docker but also from the Cloud Shell!


Cloud Shell is available right from the Azure Portal, without any plugging.

Uploading to Azure


The command if the same as the previous version except that now the command is named az. Here an example to upload a single file into an Azure Blob Storage.

az storage blob upload --file /home/frank/demo/CloudIsStrong.jpg \
--account-name frankysnotes \
--container-name blogimages --name CloudIsStrong.jpg \
--account-key 4YvvYDTg3UUpky8Rj5bDG4KO/R1FdtssxVnunsEd/4rAS04V2LkO0F8mXbddAv39WtCo5LW6JyvfhA==

Copying Between Subscriptions


Let's now copy the file to another Azure subscription. A think to be aware is that --account-name and --account-key are for the destination, even if it's not specified.
az storage blob copy start \
--source-account-name frankysnotes  --source-account-key 4YvvYDTg3UUpky8Rj5bDG4KO/R1FdtssxVnunsEd/4rAS04V2LkO0F8mXbddAv39WtCo5LW6JyvfhA== \
--source-container blogimages --source-blob CloudIsStrong.jpg   \
--account-name frankshare  --account-key EwXpZ2uZ3zrjEbpBGDfsefWkj3G2QY5fJcb6kMqV2A0+2TsGno+mk9vEXc5Uw1XiouvAiTS7Kr5OGzA== \
--destination-container imagesbackup  \
--destination-blob CloudIsStrong.jpg 

In Video Please!


If you prefer, I also have a video version of that post.



One More Thing


Sometimes, we don't need to script things, and a graphic interface is much better. For this kind of situation, the must is the Azure Storage Explorer. It does a lot! Upload, download, and manage blobs, files, queues, tables, and Cosmos DB entities. And it works on Windows, macOS, and Linux!


It's just the beginning


This post was just an introduction to two very powerful tools. I strongly suggest to go read in the official documentation to learn more. Use the comment to share all your questions and suggestion.

References:

Reading Notes #318 - MVP Summit 2018

CDNMVP2018

Last week, it was the 25 edition of the MVP Summit. An event, where Microsoft invites all his MVP to get to Seattle and spend some time with the products teams and learn on the latest news and best practices.

This year was particularly inspiriting by the Microsoft roadmap, of course,  but even more by all the amazing people a got the chance to meet and discuss with. I'm all pump-up, and I have tons of ideas and projects… more to come.

I already miss you…

Cloud


Programming



Reading Notes #317

729c5a03-6992-401b-a653-7f9b343472e0[1]Suggestion of the week


Cloud


Programming


Miscellaneous



Reading Notes #314

MVIMG_20180204_102600Suggestion of the week

  • The Modern Dev Team (Rob Conery) - What a great post. Maybe it's only me getting old ;) but I think we all have these thoughts one day or the other.

Cloud


Programming


Databases

  • My Favorite SQL Prompt Features (MarlonRibunal) - If you never try SqlPrompt and you write SQL in your day to day, stop read this, and go downloading it, or at least read this post that gives you a glimpse of it's feature.

Miscellaneous


Reading Notes #313

roy_sky_ansi2Suggestion of the week


Cloud


Programming


Databases


Miscellaneous


Reading Notes #312

IoT Hands-on Lab at Ottawa

Suggestion of the week


Cloud


Programming


Miscellaneous

Reading Notes #295

Sketch002

Cloud


Programming


Databases




Reading Notes #288

IMG_20170706_063020

Cloud




Programming