Showing posts with label docker. Show all posts
Showing posts with label docker. Show all posts

How to simplify a Docker run command

I recently wanted to create animated GIFs from videos. The idea was to get video previews, in a very lightweight file. After a quick search online, I found FFMPEG, a fantastic multimedia framework to manipulate media. There is also a few wrappers that exists in different languages (ex: C#, JavaScript) but you still need to install FFMPEG locally, and I didn't want that. In fact, I wanted a simple solution that doesn't require any installation locally and something in the cloud. In this post, I want to share how I achieved the first one.

All the code and the container are available on Github and Docker Hub.

First Contact

The ffmpeg framework is very powerful and can do so many things; therefore it's normal that it has a ton of possible parameters and extensions. After time spent on the documentation and a few trials and errors, I found how to do exactly what I needed calling it this way:

ffmpeg -r 60 -i $INPUTFILE -loop 0 -vf scale=320:-1 -c:v gif -f gif -ss 00:00:00.500 -r 10 -t 5 - > $OUTPUTFILE

This will create a five second animated GIF from a video. It speeds up the video and lowers the framerate of the GIF to keep the output lightweight. Here is an example.

Hello World episode 5 seconds preview

This is great, but this is not very friendly. How can someone who only creates a video once in a while be expected to remember all those parameters?! And even harder, when the video is vertical some parameters have different values. It was time to simplify, and here is how I did it. Note that I'm a Docker beginner and if you think there is a simpler or better way to do some steps, let me know, and let's learn together.

The Plan

The plan is simple: execute a simple Docker command like docker run fboucher/aciffmpeg -i NotInTheSky.mp4 and generate a video preview. To build our ephemeral container we will start with something lightweight like alpine, install ffmpeg and add a script that would be executed as the container runs. That sounds like an excellent plan, let's do it!

Writing the Script

The script is simple, but I learned a few things writing it. This is why it's included in this post. The goal was simple: execute the ffmpeg command using some values from the parameters: file path, and if the video is vertical. Here is the script:

#!/bin/sh

while getopts ":i:v" opt; do
  case $opt in
    i) inputFile="$OPTARG"
    ;;
    v) isVertical=true
    ;;
    \?) echo "Invalid option -$OPTARG" >&2
    exit 1
    ;;
  esac

  case $OPTARG in
    -*) echo "Option $opt needs a valid argument"
    exit 1
    ;;
  esac
done

if [ -z "$isVertical" ]; then isVertical=false; fi

# used for bash 
#IFS='.'
#read -a filePart <<< "$inputFile"
#outputFile="${filePart[0]}.gif"

# used for dash 
filename=$(echo "$inputFile" | cut -d "." -f 1)
outputFile="$filename.gif"

if $isVertical
then
  ffmpeg -r 60 -i $inputFile -loop 0 -vf scale=-1:320 -c:v gif -f gif -ss 00:00:00.500 -r 10 -t 5 - > $outputFile
else
  ffmpeg -r 60 -i $inputFile -loop 0 -vf scale=320:-1 -c:v gif -f gif -ss 00:00:00.500 -r 10 -t 5 - > $outputFile
fi

Things I learned: Parameter without values

The script needs to be as friendly as possible, therefore any unnecessary information should be removed. Most videos will be horizontal, so let's make the parameter optional. However, I don't want users to have to specify the value script.sh -i myvideo.mp4 -v true but instead script.sh -i myvideo.mp4 -v. This is very simple to do, once you know it. On the first line of code when I get the parameters: getopts ":i:v" notes that there is no ":" after the "v". This is to specify that we are not expecting any values.

Things I Learned: Bash and Dash

As mentioned earlier the container will be built from Alpine. And Alpine doesn't have bash but instead uses dash as a shell. It's mostly the same, but there are some differences. The first one will be the shebang (aka "#!/bin/sh" on the first line). And the second was the string manipulation. To generate a new file with the same name but a different extension of the script, split the file name at the ".". This can be done IFS ... read... <<< command (commented in the script) on bash but this will give syntax error: unexpected redirection and this is because there is no <<< in bash. Instead, you need to use the command cut -d "." -f 1 (where -d specifies the CHAR to use as the delimiter, and -f return only this field).

Building the image

It's now time to connect all the dots in the dockerfile.

FROM alpine:3.13
LABEL Name=aciffmpeg Version=0.0.2
RUN apk add ffmpeg
COPY ./src/myscript.sh /
RUN chmod +x /myscript.sh
ENTRYPOINT ["/myscript.sh"]

The file is not extremely complex but let’s pass through it line by line. 

  • We start FROM Alpine version 3.13 and apply a LABEL
  • RUN Will execute the command to install ffmpeg. The apk is the default utility on Alpine to install apps just like apt on Ubuntu. 
  • COPY Is copying the script from our local machine into the container at the root. 
  • The second RUN command is to make sure the script is executable. 
  • Finally, ENTRYPOINT will allow us to configure the container to run as an executable in this case as the script. All parameters passed to Docker will be passed to the script.

The only things left now are to build, tag, and push it on Docker Hub.

docker build -t fboucher/aciffmpeg .

docker tag  0f42a672d000 fboucher/aciffmpeg:2.0

docker push fboucher/aciffmpeg:2.0

The Simplified version

And now to create a preview of any video you just need to map a volume and specify the file path and optionally mention if the video is vertical.

On Linux/ WSL the command would look like this:

docker run -v /mnt/c/dev/test:/video fboucher/aciffmpeg -i /video/sample.mp4 -v

And on PowerShell like that:

docker run -v c/dev/test:/video fboucher/aciffmpeg -i /video/sample.mp4 -v

I learned a lot about Docker doing that project and now I have a very useful tool. What are the tools you built using containers that simplify your life or work?

Video Version

I recorded a video version if you are interested. 


~frank



Reading Notes #501


Another Monday, another Reading Notes. 😀

It's been five hundred weeks, this is more than nine years! Who knew I will keep doing that for so long... 

If you are new around here, welcome, The Reading Notes are a curated list of all the articles, blog posts, podcast episodes, and books that catch my interest during the week and that I found interesting. It's a mix of the actuality and what I consumed.

If you think you may have interesting content, share it!

Cloud

Programming

Miscellaneous

Miscellaneous


~Frank


Reading Notes #498


Good Monday, Already time to share my reading notes of the week. A list of all the articles, blog posts, that catch my interest during the week.

You think you may have interesting content, share it!

Cloud

Programming

Miscellaneous


~frank

Reading Notes #495


Good Monday, 
Already time to share new reading notes. It is a habit I started a long time ago where I share a list of all the articles, blog posts, podcast episodes, and books that catch my interest during the week. 


Suggestion of the week

  • Include diagrams in your Markdown files with Mermaid (Martin Woodward, Adam Biagianti) - This is very interesting. For a simple little diagram, we can now do that in text in the same document... Looking forward to trying it. Mermaid is also supported in Azure DevOps

Cloud

Programming

Book



Author: Matthew McConaughey

This autobiography is funny, light, interesting, and even weird at some moments. But you know what? I liked it. I totally kept some quotes, as I found them good. I partially read it and listen to it. The audio version is done by Matthew and it's nice to hear him telling us his story.




~frank

 

Reading Notes #491


Good Monday, already time to share new reading notes. 
It is a habit I started a long time ago where I share a list of all the articles, blog posts, podcast episodes, and books that catch my interest during the week.

You think you may have interesting content, share it!

Cloud


Programming


Databases


Miscellaneous


~frank

Reading Notes #489


Suggestion of the week

Cloud

Programming

Miscellaneous


~frank

Reading Notes #486


It's Monday (the cyber one), time to share my reading notes. Those are a curated list of all the articles, blog posts, podcast episodes, and books that catch my interest during the week and that I found interesting. It's a mix of the actuality and what I consumed. 

You think you may have interesting content, share it!

 

Cloud

Programming

~Frank

Reading Notes #485

It's Monday,
time to share my reading notes. Those are a curated list of all the articles, blog posts, podcast episodes, and books that catch my interest during the week and that I found interesting. It's a mix of the actuality and what I consumed. 

You think you may have interesting content, share it!

Cloud

Programming

Miscellaneous

  • After Microsoft Ignite, keep the learning momentum going (Jeff Sandquist) - Very happy to see this news being public.

  • How to get Windows 11 (John Cable) - It's out now and honestly, it is great! If you didn't get the automatic upgrade maybe it's because you don't meet the requirement. Following the steps, it could be a simple checkbox to check-in your current version.

  • Introduction to GitHub Actions for .Net Developers (Dave Murray) - I'm not sure why but I always felt called by automation. Maybe because it's the ultimate proactivity of any task.GitHub Action is one of those extremely powerful tools that are worth your attention.

~frank

Reading Notes #483


Already Tuesday! Time for a new Reading Notes post; a list of all the articles, blog posts, and books that catch my interest during the week and that I found interesting. It's a mix of the actuality and what I consumed.

This week is the Cloud Summit (https://azuresummit.live) is an 11-day free conference that focuses on Azure, there is surely a session that will catch your interest.

You think you may have interesting content, share it!


Cloud


Programming


Miscellaneous


~Frank


Reading Notes #481

Frank's Lego R2D2

Cloud


Programming


Podcast


~Frank


Reading Notes #480

Yes, another Monday. It's time to share my reading notes; a list of all the articles, blog posts, and books that catch my interest during the week and that I found interesting. It's a mix of the actuality and what I consumed.

You think you may have interesting content, share it!

Programming


Miscellaneous

Books


Have a nice week!


~Frank

 

Reading Notes #478

It's Monday, time to share my reading notes. Those are a curated list of all the articles, blog posts, podcast episodes, and books that catch my interest during the week and that I found interesting. It's a mix of the actuality and what I consumed.

You think you may have interesting content, share it!


Cloud

Programming

Books


Making Work Visible: Exposing Time Theft to Optimize Work & flow

Author: Dominica Degrandis, Tonianne DeMaria

This book helps define and explains how to really do and use tools like Kanban. It's not a story like in The Phoenix Project. There is a lot in this book and it is worth taking your time to read it.

A book to share.


~Frank





Reading Notes #477


Every Monday, I share my reading notes. Those are a curated list of all the articles, blog posts, podcast episodes, and books that catch my interest during the week and that I found interesting. It's a mix of the actuality and what I consumed.

You think you may have interesting content, share it!

The suggestion of the week

Cloud

Programming

Miscellaneous

~Frank

Reading Notes #472


Cloud

Programming

Podcasts

Miscellaneous


~frank


Reading Notes #465


Every Monday, I share my "reading notes". Those are a curated list of all the articles, blog posts, podcast episodes, and books that catch my interest during the week and that I found interesting. It's a mix of the actuality and what I consumed.

You think you may have interesting content, share it!

Cloud

Programming

Miscellaneous


~Frank

Reading Notes #462


Every Monday, I share my "reading notes". This is a curated list of all the articles, blog posts, and books that catch my interest during the week and that I found interesting. It's a mix of the actuality and what I consumed.


Cloud

Programming

Miscellaneous


~Frank

Reading Notes #457

Every Monday, I share my "reading notes". Those are a curated list of all the articles, blog posts, podcast episodes, and books that catch my interest during the week and that I found interesting. It's a mix of the actuality and what I consumed.

You think you may have interesting content, share it!

Cloud

Programming

Miscellaneous

Reading Notes #456


The suggestion of the week

Cloud

Programming

Podcasts

Miscellaneous


~Frank

How to Create a Continuous Integration Continuous Deployment (CI-CD) Solution for a Docker Project


I'm not a Docker master, but I understand that it's very useful and I like to use it from time to time in some projects. Another thing I like is DevOps and automation and in a project I have, I was missing that. In the previous setup, the container was built and publish to DockerHub with the date as a tag. Nice but not very easy to now with versions are "stable" and wish one are "in progress".

This post is about how I build a continuous integration and continuous deployment solution for my docker project. All the code is on GitHub and Docker Hub. I sharing my journey so others can enjoy that automation and not spend a weekend building it.

The Goal

By the end of this build, there will be two GitHub Action to build and publish a different version of the application on Docker Hub.

The release version: every time a release is published on GitHub a container tag with the matching version number will be built and published. (ex: myapp:v1)

The beta version: At every push in my branch on GitHub a container will be published with a specific tag. The tag will be matching the draft release version number with -beta. (ex: myapp:v2-beta).

In this post, the application is a Node.js Twitch chatbot. The type of application is not important the post focus on the delivery.

Publishing the release version

Every time a release is published on GitHub, the workflow will be triggered. It will first retrieve the "release version" then build and tag the container with it and finally publish (aka push) it to Docker hub. Because a "release" is also a "stable" version it will also update the container tag latest.

Let's look at the full YAML definition of the GitHub Action and I will break it down after.

name: Release Docker Image CI

on:
  release:
    types: [published]
jobs:
  update:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - name: Set outputs
      id: vars
      run: echo ::set-output name=RELEASE_VERSION::$(echo ${GITHUB_REF:10})
    - name: Publish to Registry
      uses: elgohr/Publish-Docker-Github-Action@master
      with:
        name: ${{secrets.DOCKER_USER}}/cloudbot
        username: ${{ secrets.DOCKER_USER }}
        password: ${{ secrets.DOCKER_PASSWORD }}
        tags: "latest,${{ steps.vars.outputs.RELEASE_VERSION }}"

To limit how many times the workflow is triggered I used on: release and the type: published, adjust as you like.

The next interesting part is the lines in the step vars.

- name: Set outputs
    id: vars
    run: echo ::set-output name=RELEASE_VERSION::$(echo ${GITHUB_REF:10})

Here I use the environment variable GITHUB_REF (striped of the 10 first characters contains "refs/tags/") to initialize a local variable RELEASE_VERSION. The value is available from the outputs of that step, like on the last line of the YAML.

tags: "latest,${{ steps.vars.outputs.RELEASE_VERSION }}"

From the steps identified by the id vars I retrieved from the outputs the value of RELEASE_VERSION.

In this GitHub Action, I used elgohr/Publish-Docker-Github-Action@master because it was simple and was doing what I need. You can execute docker commands directly if you prefer or use the docker/github-actions.

There are many options available from the GitHub marketplace.

Publishing the beta version

Every time a push is done on GitHub, the workflow will be triggered. It will first retrieve the "release version" of the most recent release in draft mode. The workflow will happen -beta to the retrieved version and use this to tag the container. Finally, publish (aka push) it to Docker hub.

Once more, here full YAML, I will break it down after.

name: Build Docker Images
on: [push]
jobs:
  build:
    name: cloudbot-beta
    runs-on: ubuntu-latest
    steps:
    - id: last_release
      uses: InsonusK/get-latest-release@v1.0.1
      with:
          myToken: ${{ github.token }}
          exclude_types: "release, prerelease"
          view_top: 1  
    - uses: actions/checkout@v2
    - name: Publish to Registry
      uses: elgohr/Publish-Docker-Github-Action@master
      with:
        name: ${{secrets.DOCKER_USER}}/cloudbot
        username: ${{ secrets.DOCKER_USER }}
        password: ${{ secrets.DOCKER_PASSWORD }}
        tags: "${{ steps.last_release.outputs.tag_name }}-beta"

Here the difficulty was that I wanted to create a tag from a "future" version. I decided to use the Draft Releases because those are not visible by everyone, therefore they look like the future.

If your last release is version 1 (v1.0), to make this workflow possible you will need to create a new release and save it in Draft.



Like in the Release workflow, I need to retrieve the version. Because drafts are only visible to some people we will need to get access. This is easily done by using a github.token. Those are created automatically when the GitHub Action starts.

Then using the step InsonusK/get-latest-release we will retrieve the version.

- id: last_release
    uses: InsonusK/get-latest-release@v1.0.1
    with:
        myToken: ${{ github.token }}
        exclude_types: "release, prerelease"
        view_top: 1  

This time when passing the value for the tag we will concatenate "-beta" to it.

tags: "${{ steps.last_release.outputs.tag_name }}-beta"

Wrapping Up

And voila, a very simple and easy to implement ci-cd for a container project. There are many different options, looking forward to learning how you did yours?

Reading Notes #455


The suggestion of the week


Cloud


Programming


Miscellaneous