Showing posts with label docker. Show all posts
Showing posts with label docker. Show all posts

Reading Notes #612

In this week’s Reading Notes, we explore cloud debugging, .NET Aspire, and more. Join us for insights, workshops, and podcasts covering a range of exciting topics! 🚀
 
Sharing my Reading Notes is a habit I started a long time ago, where I share a list of all the articles, blog posts, and books that catch my interest during the week.


 

Cloud

  • GalaSoft Laurent Bugnion (Laurent Bugnion) - Nice post debugging investigating a bug, that cannot be reproduce locally only in the cloud.... But with the right tools it's much easier.

Programming

Podcasts

Miscellaneous

  • What is platform engineering? (Julia Kulla-Mader, Chuck Lantz) - Platform engineering is gaining in popularity, but what ibis really. This article gives a good explanation to start our learning journey.

Reading Notes #611

Welcome to this week’s edition of Reading Notes! In this roundup, we explore a variety of topics across cloud, programming, databases, and AI. From understanding Docker’s USER instruction to styling Blazor components with CSS, I’ve got you covered. Let’s dive in!


Suggestion of the week

  • Understanding the Docker USER Instruction (Jay Schmidt) - A great post to that explains really clearly the basic usage of user when building our container. After reading this post you should feel confident to follow this best practices.

Cloud

Programming

Databases

AI

~frank

Reading Notes #609

It's reading notes time! It is a habit I started a long time ago, where I share a list of all the articles, blog posts, and books that catch my interest during the week.

You also read something you liked? Share it!

   

Cloud

Programming

AI

~frank

It turns out, it's not difficult to remove all passwords from our Docker Compose files

I used to hardcode my password in my demos and code samples. I know it's not a good practice, but it's just for demo purposes, it cannot be that dramatic, right? I know there are proper ways to manage sensitive information, but this is only temporary! And it must be complicated to remove all the passwords from a deployment... It turns out, IT IS NOT difficult at all, and that will prevent serious threats.

In this post, I will share how to remove all passwords from a docker-compose file using environment variables. It's quick to setup and easy to remember. For production deployment, it's better to use secrets, because environment variables will be visible in logs. That said, for demos and debugging and testing, it's nice to see those values. The code will be available on GitHub. This deployment was used for my talks during Azure Developers .NET Days: Auto-Generate and Host Data API Builder on Azure Static Web Apps and The most minimal API code of all... none

The Before Picture

For this deployment, I used a docker-compose file to deploy an SQL Server in a first container and Data API Builder (DAB) in a second one. When the database container starts, I run a script to create the database tables and populate them.

services:

  dab:
    image: "mcr.microsoft.com/azure-databases/data-api-builder:latest"
    container_name: trekapi
    restart: on-failure
    volumes:
      - "./startrek.json:/App/dab-config.json"
    ports:
      - "5000:5000"
    depends_on:
      - sqlDatabase

  sqlDatabase:
    image: mcr.microsoft.com/mssql/server
    container_name: trekdb
    hostname: sqltrek
    environment:
      ACCEPT_EULA: "Y"
      MSSQL_SA_PASSWORD: "1rootP@ssword"
    ports:
      - "1433:1433"
    volumes:
      - ./startrek.sql:/startrek.sql
    entrypoint:
      - /bin/bash
      - -c
      - |
        /opt/mssql/bin/sqlservr & sleep 30
        /opt/mssql-tools/bin/sqlcmd -U sa -P "1rootP@ssword" -d master -i /startrek.sql
        sleep infinity

As we can see, the password is in clear text twice, in the configuration of the database container and in the parameter for sqlcmd when populating the database. Same thing for the DAB configuration file. Here the data-source node where the password is in clear text in the connection string.

"data-source": {
 	"database-type": "mssql",
	"connection-string": "Server=localhost;Database=trek;User ID=sa;Password=myPassword!;",
	"options": {
		"set-session-context": false
	}
}

First Pass: Environment Variables

The easiest password instance to remove was in the sqlcmd command. When defining the container, an environment variable was used... Why not use it! To refer to an environment variable in a docker-compose file, you use the syntax $$VAR_NAME. I used the name of the environment variable MSSQL_SA_PASSWORD to replace the hardcoded password.

/opt/mssql-tools/bin/sqlcmd -U sa -P $$MSSQL_SA_PASSWORD -d master -i /startrek.sql

Second Pass: .env File

That's great but the value is still hardcoded when we assign the environment variable. Here comes the environment file. They are text files that holds the values in key-value paired style. The file is not committed to the repository, and it's used to store sensitive information. The file is read by the docker-compose and the values are injected. Here is the final docker-compose file:

services:

  dab:
    image: "mcr.microsoft.com/azure-databases/data-api-builder:latest"
    container_name: trekapi
    restart: on-failure
    env_file:
      - .env
    environment:
      MY_CONN_STRING: "Server=host.docker.internal;Initial Catalog=trek;User ID=sa;Password=${SA_PWD};TrustServerCertificate=True"
    volumes:
      - "./startrek.json:/App/dab-config.json"
    ports:
      - "5000:5000"
    depends_on:
      - sqlDatabase

  sqlDatabase:
    image: mcr.microsoft.com/mssql/server
    container_name: trekdb
    hostname: sqltrek
    environment:
      ACCEPT_EULA: "Y"
      MSSQL_SA_PASSWORD: ${SA_PWD}
    env_file:
      - .env
    ports:
      - "1433:1433"
    volumes:
      - ./startrek.sql:/startrek.sql
    entrypoint:
      - /bin/bash
      - -c
      - |
        /opt/mssql/bin/sqlservr & sleep 30
        /opt/mssql-tools/bin/sqlcmd -U sa -P $$MSSQL_SA_PASSWORD -d master -i /startrek.sql
        sleep infinity

Note the env_file directive in the services definition. The file .env is the name of the file used. The ${SA_PWD} tells docker compose to look for SA_PWD in the .env file. Here is what the file looks like:

SA_PWD=This!s@very$trongP@ssw0rd

Conclusion

Simple and quick. There are no reasons to still have the password in clear text in the docker compose files anymore. Even for a quick demo! Of course for a production deployment there are stronger ways to manage sensitive information, but for a demo it's perfect and it's secure.

During Microsoft Build Keynote on day 2, Julia Liuson and John Lambert talked about how trade actors are not only looking for the big fishes, but also looking at simple demos and old pieces of code, looking for passwords, keys and sensitive information.

Reading Notes #605

It's reading notes time! It is a habit I started a long time ago, where I share a list of all the articles, blog posts, and books that catch my interest during the week. 

You also read something you liked? Share it!

Cloud

Programming

AI

~Frank

Reading Notes #604

It's reading notes time! It is a habit I started a long time ago, where I share a list of all the articles, blog posts, and books that catch my interest during the week. 


You also read something you liked? Share it!

 

Cloud

Programming

DevOps

Miscellaneous

  • Introducing Plans on Microsoft Learn (Kaberi Bell) - Would it be to be a data engineer, AI specialist, app builder there's a plan for you and that's a very cool new feature on learn have a look this blog post explain all of it

  • How Do You Measure Developer Experience? (Jennifer Riggins) - An interesting article about what and how measure performance. I didn't know so many system and details concepts were that detailed.

  • Windows Terminal Preview 1.21 Release (Christopher Nguyen) - Tons of new features again and I love| seeing all those community contributions.

~Frank

Reading Notes #603

It's reading notes time! It is a habit I started a long time ago, where I share a list of all the articles, blog posts, and books that catch my interest during the week.

Having interesting content? Share it!

Cloud

Programming

Podcasts

Miscellaneous

  • DevOps Adoption for IT Managers (Chris Pietschmann) - Interesting post that shares the benefits of DevOps for your enterprise and how to approach it as a manager.

  • Cascadia Code 2404.23 (Christopher Nguyen) - I used to do ASCII art back on my C=64... Now that all those new fonts and symbols are added should I start again? Nice to have all the options available to be able to display everything we need|the console.

~Frank

Reading Notes #599

It's reading notes time! It is a habit I started a long time ago, close to 600 weeks ago in fact, where I share a list of all the articles, blog posts, and books that catch my interest during the week. 

Having interesting content? Share it!

    

Cloud

Programming

Low Code

  • Testing Your Native AOT Applications - .NET Blog (Marco Rossignoli, Jakub Jareš, Jakub Chocholowicz) - With code ahead-of-time (AOT) we still need tests but how to do it with that stripped version. of .NET? This short tutorial shows how.

Podcasts

Miscellaneous

  • How to automate tasks in Windows (David Nield) - Seeing things getting done by some automation you built is a nice feelingThis short post introduces 4 tools that will get you started with automation.
~Frank

Reading Notes #598

It's reading notes time! It is a habit I started a long time ago, close to 600 weeks ago in fact, where I share a list of all the articles, blog posts, and books that catch my interest during the week. 

If you think you may have interesting content, share it!

Cloud

Programming

DevOps

Open Source

AI


~Frank

Reading Notes #593

It is time to share new reading notes. It is a habit I started a long time ago where I share a list of all the articles, blog posts, and books that catch my interest during the week.

If you think you may have interesting content, share it!

Cloud

Programming

Miscellaneous

~Frank

Reading Notes #591

It is time to share new reading notes. It is a habit I started a long time ago where I share a list of all the articles, blog posts, and books that catch my interest during the week.

If you think you may have interesting content, share it!

Suggestion of the week

Cloud

Programming

~ Frank

Reading Notes #586

It is time to share new reading notes. It is a habit I started a long time ago where I share a list of all the articles, blog posts, and books that catch my interest during the week.


If you think you may have interesting content, share it!

Cloud

Programming

Podcasts

Miscellaneous


~Frank

Reading Notes #585

It is time to share new reading notes. It is a habit I started a long time ago where I share a list of all the articles, blog posts, and books that catch my interest during the week.


If you think you may have interesting content, share it!

Cloud

Programming

~Frank

Reading Notes #583

It is time to share new reading notes. It is a habit I started a long time ago where I share a list of all the articles, blog posts, and books that catch my interest during the week. 

If you think you may have interesting content, share it!

Suggestion of the week

  • Getting Started with Blazor’s New Render Modes in .NET 8 (Jon Hilton) - Amazing post that covers the four rendering mode for Blazor in .NET 8.There just enough code to understand the concept and see the trade-offs and advantages of each options.
    ai generated: melting snowman who love to read

Cloud

Programming

Miscellaneous

~ Frank

Database to go! The perfect database for developer

When building a new project, we don't need a big database that scales and has lots of data, but we do still need some kind of data source. Of course, it is possible to fake it and have some hardcoded value returned by an API but that takes time to create and it's not a database. In this post, I want to share a solution to have a portable, self-healing, disposable, disconnected database that doesn't require any installation.

The solution? Put the database in a container! It doesn't matter what database you are planning to use or on which OS you are developing. Most databases will have an official image available on Docker Hub and Docker runs on all platforms. If you feel uncomfortable with containers, have no fear, this post is beginner-friendly.

This post is part of a series where I share my experience while building a Dungeon crawler game. The code can be found on GitHub.


The Plan

Have a database ready at the "press of a button". By "ready", I mean up and running, with data in it, and accessible to all developer tools.

Preparation for the Database

We need a script to create the database schema and some data. There are many ways to achieve this. A beginner-friendly way is to create an empty database and use a tool like Azure Data Studio to help create the SQL scripts. Doing it this way will validate that the script works.

The Docker command to create the database's container will change a little depending on the database you are using but here what's a MySQL one look like:

docker run --name some-mysql -e MYSQL_ROOT_PASSWORD='rootPassword' -p 3306:3306 -d mysql 

Where some-mysql is the name you want to assign to your container, rootPassword is the password to be set for the MySQL root user and -d means that the container will run detached. The -p option is used to map the port 3306 of the container to the port 3306 of the host. This is required to be able to connect to the database from the host.

output of the docker run command


Now, a MySQL server is running inside the container. To connect to the server with Azure Data Studio use the extension MySQL extension for Azure Data Studio. Microsoft has a QuickStart: Use Azure Data Studio to connect and query MySQL if needed. Create a new connection in Azure Data Studio, then create a database (ex: 2d6db).

Create a new connection in Azure Data Studio

You can use the MySQL command-line tool if you prefer, but Azure Data Studio offers a lot of help when you are not that familiar with SQL. You can even use the Copilot extension and ask it to write the SQL statement for you. It's pretty good!

If you want to learn more about this, check the Open at Microsoft episode: Copilot is now in Azure Data Studio and this is how it can help you! to see it in action.

It's fantastic to generate a first draft of the create statements and to make queries.

Copilot writing SQL

Let's create two SQL scripts. The first one will be to create the schema with all the tables. The idea here is to write the script and execute it to validate the results. Here is an example creating only one table to keep the post simple.

-- schema.sql

CREATE TABLE IF NOT EXISTS 2d6db.rooms (
  id int NOT NULL AUTO_INCREMENT,
  roll int DEFAULT 0,
  level int DEFAULT 1,
  size varchar(10) DEFAULT NULL,
  room_type varchar(255) DEFAULT NULL,
  description varchar(255) DEFAULT NULL,
  encounter varchar(255) DEFAULT NULL,
  exits varchar(255) DEFAULT NULL,
  is_unique bool DEFAULT false,
  PRIMARY KEY (id)
);

Now that there are tables in the database, let's fill them with seed data. For this, the second SQL script will contain insert statement to populate the tables. We don't need all the data but only what will be useful when developing. Think about creating data to cover all types or scenarios, it's a development database so it should contain data to help you code.

-- data.sql

INSERT INTO 2d6db.rooms(roll, level, room_type, size, description, exits, is_unique)
VALUES (2,1,'Empty space', 'small','There is nothing in this small space', 'Archways',false);

INSERT INTO 2d6db.rooms(roll, level, room_type, size, description, exits, is_unique)
VALUES (3,1,'Strange Text', 'small','This narrow room connects the corridors and has no furniture. On the wall though...', 'Archways',false);

INSERT INTO 2d6db.rooms(roll, level, room_type, size, description, exits, is_unique)
VALUES (4,1,'Grakada Mural', 'small','There is a large mural of Grakada here. Her old faces smiles...', 'Archways',true);

Note: You can now stop the database's container with the command: docker stop some-mysql. We don't need it anymore.

Putting All the Pieces Together

This is when the magic starts to show up. Using a Docker Compose file, we will start the database container and execute the two SQL scripts to create and populate the database.

# docker-compose.yml

services:
  
  2d6server:
    image: mysql
    command: --default-authentication-plugin=mysql_native_password
    environment:
      MYSQL_DATABASE: '2d6db'
      MYSQL_ROOT_PASSWORD: rootPassword
    ports:
       - "3306:3306"
    volumes:
      - "../database/scripts/schema.sql:/docker-entrypoint-initdb.d/1.sql"
      - "../database/scripts/data.sql:/docker-entrypoint-initdb.d/2.sql"

The docker-compose.yml file are in YAML and usually are used to start multiple containers at once, but it doesn't need to. In this scenario, the file defines a single container named 2d6server using just like the previous Docker command and MySQL image and configuration. The last command volumes is new. It maps the path where the SQL scripts are located to /docker-entrypoint-initdb.d inside the container. When MySQL starts it will execute the files in that specific folder in alphabetic order. This is why the scripts are renamed 1.sql and 2.sql, as the table must be created first.

Do get the database up and ready, we will execute the docker compose up.


# start the database
docker compose -f /path_to_your_file/docker-compose.yml up -d 

# stop the database
docker compose -f /path_to_your_file/docker-compose.yml down -d 

By default, the command looks for a docker-compose.yml file. If you have a different name use the argument -f to specify the filename. Optionally, to free the prompt you can pass the argument -d to be in detached mode.

Docker Compose commands

When you are done coding and you don't need the database anymore, execute the docker compose down command to free up your computer. Compared to when the server is installed locally, a container will leave no trace; your computer is not "polluted".

When you need to update the database, edit the SQL script first. When the scripts are ready, execute the docker-compose restart to get the database refreshed.

To Conclude

Now, you only need to execute one simple command get a fresh database, when you want. All the developers don't need to have a database server installed and configured locally. And you don't need to be worried when deleting or modifying data, like when using a shared database. After cloning the repository all developers will have everything they need to start coding.

In a next post, I will share how I used Azure Data API Builder to generate a complete API on top of the database using the same docker compose method.

Video version!

If you prefer watching instead of reading here the video version of this post!

Reading Notes #578

It is time to share new reading notes. It is a habit I started a long time ago where I share a list of all the articles, blog posts, and books that catch my interest during the week. 


If you think you may have interesting content, share it!

Cloud

Programming

Podcasts

Miscellaneous

~ Frank Boucher

Reading Notes #575

Happy Thanksgiving to all Canadian🍁! 

It is time to share new reading notes. It is a habit I started a long time ago where I share a list of all the articles, blog posts, and books that catch my interest during the week.

If you think you may have interesting content, share it!

Suggestion of the week

Programming

Open Source

Podcasts

~Frank

Reading Notes #573

It is time to share new reading notes. It is a habit I started a long time ago where I share a list of all the articles, blog posts, and books that catch my interest during the week.

If you think you may have interesting content, share it!

Programming

Data

  • Azure Data Studio 1.46 and New Features (Erin Stellato) - I love that tool! Its fast, customizable, and its avalaible across platform meaning I don't have to use different tools depending on the computer I'm working from.

Miscellaneous

~Enjoy!


Reading Notes #567

Programming

DevOps

  • How to deploy Azure Container Apps (Shawn Sesna) - This is a grewt tutorial to get your container Apps deploy without having to care about to much infrastructure aka.kubernetes.

Podcasts

Frank

Reading Notes #561

It is time to share new reading notes. It is a habit I started a long time ago where I share a list of all the articles, blog posts, and books that catch my interest during the week.

If you think you may have interesting content, share it!

The suggestion of the week

Programming

Low Code

Miscellaneous

~Frank