Showing posts with label Kindle. Show all posts
Showing posts with label Kindle. Show all posts

How I use Azure Logic App, API App and Function App in my life

Why should we do things over and over? I know I don't want to lose my time doing so. Every time I see myself doing something I already did, I look for optimization. Sometimes, it's just little things like trying a different path to get from the train station to the office, and the other time, it's like drinking the rainbow juice directly from the source.

In this post, I will share with you how I optimized a three hours process into a two... minutes automated customizable solution using Azure Logic App and API App.

Quick Context

For the past two hundred fifty something weeks every Monday I share my reading notes. Those are articles that I read during the previous week with a little comment; my 2 cents about it. I read all those books, articles and posts on my e-book reader; In this case a Kindle whitepaper. I use an online service call Readability that can easily clean (remove everything except the text and useful images) the article and send it to my e-book reader. All the articles are kept as a bookmark list.


When I have time I read what's on my list. Each time I'm done, I add a note ended by tags between square brackets, those will be use later for research and filter.


Every Sunday, I extract the notes, then search in Readability to find back the URL and build with all this my post for the next morning. A few years ago, I did a first optimization using a Ruby script and I blogged about it here. While it's been working very well all those years, I wanted to change it so I do not have to install Ruby and the Gems on every machine I use.

Optimization Goals

Here is the list of things I wanted for this new brew of the Reading notes post generator:
  • No install required
  • not device or service couple.
  • generate json verrsion for future purpuse
  • cheap

Optimization Plan

Azure Logic App is the perfect candidate for this optimization because it doesn't require any local installation. Since it a flow between connectors any of them can be changed to please the user. In fact, that was the primary factor that made me picked Azure Logic App. For example, i the current solution, I'm using OneDrive as an initial drop zone. If you your environment you would prefer using DropBox instead of OneDrive, you just need to switch connectors, and nothing else will be affected. Another great advantage is that Azure Logic App is part of the App service ecosystem so all those components are compatible.

Here is the full process plan, at the high level.


  1. Drop the My Clippings.txt file in an OneDrive folder.
  2. Make a copy of the file In Azure Blob Storage, using the Blob Storage built-in connector.
  3. Parse the My Clippings.txt file to extract all the clippings (notes) since last extraction, using my custom My Clippings API App.
  4. For each note,
    1. Get the URL where that post is coming from, using my custom Readability Api App.
    2. Extract the tags and category (first tag is used as a category), using my custom Azure Function.
    3. Serialize the note with all the information in json.
    4. Save to Azure Blob Storage the json file in a temporary container.
  5. Generate a summary of all json files using my last custom ReadingNotes API App. It also saved a json and Markdon files of the summary.
Note: The summary could be published directly, but I decided to keep that last step manual, so I can check for typos and Grammar.

Let's see more in details

Many great tutorials are already available about how to create Azure Logic App or Azure API App, so in this post, I won't explain how to create them, but mostly share with you some interesting snippets or gotchas I encounter during my journey.

In the next screen capture, you can see all the steps contained in my two Logic App. I decided to slip the process because many tasks needed to be done for every note. The main loop (on the left) fetches the notes collection and generates the output. For every note, it will call the Child Logic App (on the right).


The ReadingNotes Builder contains:
  1. Initial Trigger when a file is created in an OneDrive folder.
  2. Create a copy of the file in an Azure Blob storage.
  3. Delete the file in OneDrive.
  4. Get the configuration file from Azure blob storage.
  5. Call the API App responsible for parsing the file and extract the recent clippings.
  6. Call the child Logic App for every clipping returned in the previous steps (foreach loop).
    • A. Trigger of the Child Logic App.
    • B. Call the API App responsible for searching in the online bookmark collection and return the URL of the article/ post.
    • C. Call Azure Function App responsible for extracting tags from the note.
    • D. Call Azure Function App responsible for converting the object in json.
    • E. Saves the json object in a file in Azure blob storage.
    • F. Gets the updated configuration file.
    • G. Call Azure Function App responsible for keeping the latest note date.
    • H. Update the configuration with the latest date.
    • I. Return an HTTP 200 code, so the parent Logic App knows the work is done.
  7. Call the API App responsible for building the final markdown file.
  8. Save the markdown file to an Azure blob storage that was returned at the previous step.
  9. Update the config final.

As an initial trigger, I decided to use OneDrive because it's available from any devices, from anywhere. I just need to drop the MyClippings.txt file in the folder, and the Logic App will start. Here, how I configured it in the editor:


MyClippings API App

Kindle's notes and highlights are saved in a local file named My Clippings.txt. To be able to manipulate my notes, I needed a parser. Ed Ryan created the excellent [KindleClippings][KindleClippings], an open-source project available on github. In the actual project, I only wrapped that .Net Parser in an Azure API App. Wrapping the API in an API App will help me later to manage security, metrics, and it will simplify the connection in all other Azure solutions app.

I won't go into the detail on how to create an Azure API App, a lot of great documentation is available online. However, you will need at least the Azure .Net SDK 2.8.1 to create an API App project.


Swagger API metadata and UI are a must so don't forget to un-comment that section in the SwaggerConfig file.


I needed an array of notes taken after a specific date. All the heavy work is done in [KindleClippings][KindleClippings] library. The ArrayKindleClippingsAfter method gets the content of the My Clippings.txt file (previously saved in Azure blob storage), and pass it to the KindleClippings.Parse method. Then using Linq, only return the notes taken after the last ReadingNotes publication date. All the code for this My Clippings API App is available on github, but here the method:

public Clipping[] ArrayKindleClippingsAfter(string containername, string filename, string StartDate)
    var blobStream = StorageHelper.GetStreamFromStorage(containername, filename);
    var clippings = KindleClippings.MyClippingsParser.Parse(blobStream);
    var result = (from c in clippings
                    where c.DateAdded >= DateTime.Parse(StartDate)
                    && c.ClippingType == ClippingTypeEnum.Note
                    select c as KindleClippings.Clipping).ToArray<Clipping>();
    return result;

Once the Azure API App is deployed in Azure it could be easily called from the Azure Logic App.


A great thing to notice here is that because Azure Logic App is aware of the required parameters and remembers all the information from the previous steps (show in different color), it will be a very simple to configure the call.


Azure Logic App calling another Azure Logic App

Now that we have an array of note, we will be able to loop through each of them to execute other steps. When I wrote the App, the only possibility way to execute multiple steps in a loop, was to call another Azure Logic App. The child Logic App will by trigger by an HTTP POST and will return an HTTP Code 200 when it's done. A json schema is required to define the input parameters. A great tool to get that done easily is


Readability API App

Readability is an online bookmark service that I'm using for many years now. It offers many API to parse or search article and bookmarks. I found and great .Net wrapper on github called CSharp.Readability that was written by Scott Smith. I could have called directly the Readability from Logic App. However, I needed a little more so I decided to use Scott's version as my base for this API App.
From the clipping collection returned at the previous step, I only have the title and need to retrieve the URL. To do this, I added a recursive method call SearchArticle.

private BookmarkDetails SearchArticle(string Title, DateTime PublishDate, int Pass)
    var retryFactor = 2 * Pass;
    var fromDate = PublishDate.AddDays(-1 * retryFactor);
    var toDate = PublishDate.AddDays(retryFactor);
    var bookmarks = RealAPI.BookmarkOperations.GetAllBookmarksAsync(1, 50, "-date_added", "", fromDate, toDate).Result;
    var result = from b in bookmarks.Bookmarks
                    where b.Article.Title == Title
                    select b as BookmarkDetails;
    if (result.Count() > 0)
        return result.First<BookmarkDetails>();
    if (Pass <= 3)
        return SearchArticle(Title, PublishDate, Pass + 1);
    return null;

Azure Function: Extract tags

While most of the work was done in different API, I needed little different tools. Many possibilities but I decided to take advantage of the new Azure Function App. They just sit there waiting to be use! The ReadingNotes Builder uses three Azure Function App, let me share one of them: ExtractTags. An interesting part with function is that you can configure them to get triggered by some event or to act as Webhook.


To create a Function App as Webhook you can use one of the templates when you create the new Function. Or from the code editor in the Azure Portal, you can click on the Integrate tab and configure it.


Once it's done you are ready to write the code. Once again, this code is very simple. It started by validating the input. I'm expecting a Json with a node called note, then extract the tags from it and return both parts.

public static async Task<object> Run(HttpRequestMessage req, TraceWriter log)
    string tags;
    string cleanNote;
    string jsonContent = await req.Content.ReadAsStringAsync();
    dynamic data = JsonConvert.DeserializeObject(jsonContent);
    if (data.note == null ) {
        return req.CreateResponse(HttpStatusCode.BadRequest, new {
            error = "Please pass note properties in the input object"
    string note = $"{data.note}";
    int posTag = note.LastIndexOf('[')+1;
    if(posTag > 0){
        tags = note.Substring(posTag, note.Length - posTag-1);
        cleanNote = note.Substring(0,posTag-1);
        tags = "";
        cleanNote = note;
    return req.CreateResponse(HttpStatusCode.OK, new {
        tags = $"{tags}",
        cleanNote = $"{cleanNote}"

Now, to call it from the Azure Logic App, You will need first to Show Azure Function in the same region to see your Function App. And simply add the input; in this case like expected a json with a node called note.


What's next

Voila! This is a simple but real application. While I shared only a part of the code in this post, all of it is available on github. I also did a presentation at DevTeach and other details are present in those slides. Using Azure Logic App to build this application was really interesting, and easy. Now that some pieces are in place (up there), I will be able to grow my environment by adding more functionalities, an interface, more security, but that's for another post...


Quick trick: From Feedly to your eReader in one click

I’m reading most of the time in common transport and offline, on my eBook reader.  So part of my morning routine is to look for new articles to read, and send it to my reading device. Since this week, because of the new Readability updates, I can do this in only one click! Let me share this with you.

The Tooling


I’m using Readability to keep tracks of all the posts I’m reading.  It also very useful, because it cleans everything and keeps only the text of the article. Readability can also send you automatically post to your device.
post beforepost after in Readability

Readability extension for Chrome

I’m also using the Extension let you from anywhere send a page or a link to your reading list. No need to copy-paste the post’s URL. Since last week, options are also available in the context menu.
Readability extension for Chrome


To keep tracks of all the RSS I’m following I use Feedly.  The interface is very polish and the application is always improving. The application is also available on iPhone and Android in addition to being a web app.
Feedly Interface

The Trick

Finally, because Feedly is running in a web browser, you can use the Readability extension. So from now on, when I found interesting article, I can left-click and send it to my eBook reader.




Reading Notes #66 - CLOUD-FISHING, Published April 27, 2012Cloud

First step with Ruby: Kindle Clipping Extractor

In this post, I'm sharing my last (and first) Ruby script. The script is available on bitbicket here, and you can see the output result on any Reading Notes posts on my blog here: Reading Notes.  I’m still learning Ruby, so feel free to leave me a comment.  I will continue to update the script as I’m getting more comfortable with my new skill.

Starting Idea

If you are reading this post there is a good chance you already know that I’m posting every weeks a post about my weekly reading notes.  I use Instapaper to bookmark all my reading stuff and send it to my Kindle.  So order to make my weekly post I need to pass-through the “My Clippings.txt” file on my kindle, than go back on Instapaper, found the link to this article then put all this information together in a nice readable format. 
So the idea was to speedup this process.

Kindle part

First thing first I need to retrieve all my notes in the “My Clippings.txt” file.  I start trying to do it my self and the result was… not good.  Then I found the really nice gem kindleclippings that was doing exactly what I want: parsing the file and give a array of all the notes with all the information about this note well organize with properties.  So my job was to use it correctly… not to bad.

require 'kindleclippings'

parser =
clippings = parser.parse_file('My Clippings.txt')

#== Build the Html list ==
resume = "<ul>"

clippings.notes.each do |note|
  resume << "\n<li>\n<b><a href=\"#\">#{note.book_title}</a></b> - #{note.content}\n"

  resume << "#{GetBookHightlight(clippings.by_book(note.book_title))}\n</li>"

resume << "\n</ul>"

puts resume

As you can see nothing complicated, looping in the notes building a unsorted list (UL).

Instapaper part

Getting the reference link of the article is a little need more work. I need to login into Instapaper with my account, then found the matching bookmark.  Once the good bookmark is found I need to extract url to the full article.  In the same time I will move this bookmark to another folder to keep my “Unread” list short.

I'm using Watir to do my web scraping.  This nice gem is very well done and can be use for testing user interfaces, but this will be in another post.

So first thing first I need to login. Here again nothing complexes get username and password and using it to login.
def InstapaperOpen()
  browser =
  browser.goto ''
  puts "What is your Username? "
  username ||= gets.chomp
  puts "What is your Password? "
  password ||= gets.chomp

  browser.text_field(:name => 'username').set(username)
  browser.text_field(:name => 'password').set(password)
  browser.button(:type => 'submit').click
  abort("Cannot login in Instapaper.") unless => "Log out").exist?
  return browser

def InstaPaperClose(browser) => "Log out").click

Next, I need a method to search and return the bookmark matching my Kindle note.

def SearchTitle(browser, title)
  rgTitle = Regexp.escape(title)
  if, rgTitle).exist?
    anchor =, rgTitle)
    return anchor
  if, /\/u\/\d+/).span(:text => "Older items").exist?
    #puts "... Searching older...", /\/u\/\d+/).span(:text => "Older items").click
    anchor = SearchTitle(browser, title)
  return anchor

def MovePaperTo(paper, destinationFolder)
  #puts "paper: " +
  aDiv = paper.div(:class, 'secondaryControls')
  #puts "div: " +
  aLink =,'Move')
  #puts "a: " + aLink.title
  bntMoveTo =, destinationFolder)
  #puts "link: " +

2011-11-12_0808So in SearchTitle method using regex I’m looking for an anchor <a> matching the title.  If I didn’t found it I check if I found the “Older items >>” button to search deeper. I doing this calling recursively the method, until I found the matching bookmark or that no more “Older items >>” is present. 

In MovePaperTo I click on the Move link in the context menu of the bookmark.

Finally putting all this together (with some improvements) and I got my final script!  Nice way to practice the basic rules of Ruby. Of course it could be more clean more "rubist", but at least it works and I now I have something to practice my refactoring skill…

Feel free to leave me your comments or suggestions here on this blog or on Bitbuket.



Reading Notes #18