Remember: Whenever you read “Docker Swarm” we are talking about “Docker in Docker Swarm mode”

Within this article, I will show and explain four services everyone should use in their Docker Swarm: traefik, Portainer, Docker-Registry, FTP.

Traefik

Makes Networking Boring Cloud-Native Networking Stack That Just Works.

One important service in any Docker Swarm is traefik. I use this for assigning domains/subdomains to every docker service. A simple docker-compose.yml can be found on my FTP here. To use this file it is necessary to update the labels for your docker swarm by executing:

export NODE_ID=$(docker info -f '{{.Swarm.NodeID}}')
docker node update --label-add traefik-public.traefik-public-certificates=true $NODE_ID

These two commands assume that you want to have traefik on your manager node! To understand what a manager node have a look at the docker documentation.

Another VERY important step is creating a file named acme.json within the folder where the docker-compose will be stored and do:

chmod 600 acme.json

Furthermore, you have to define a TRAEFIK_USERNAME, TRAEFIK_HASHED_PASSWORD, TRAEFIK_DOMAIN and TRAEFIK_SSLEMAIL:

export TRAEFIK_USERNAME=admin
export TRAEFIK_HASHED_PASSWORD=$(openssl passwd -apr1 testpassword)
export TRAEFIK_DOMAIN=dashboard.YOUR_DOMAIN.tld
export TRAEFIK_SSLEMAIL=your_email@address.de

The last command which has to be executed before you can deploy the service is creating the traefik-public network which will be used across all containers:

docker network create --driver=overlay traefik-public

After this is done it is possible to deploy traefik to your docker swarm:

docker stack deploy -c docker-compose.traefik.yml

Just test it by launching https://dashboard.YOUR_DOMAIN.tld. This domain will use an SSL certificate and you can log in with admin/testpassword if you followed the above instructions.

Portainer

Portainer is a powerful, GUI-based Container-as-a-Service solution that helps organizations manage and deploy cloud-native applications easily and securely.

My docker-compose.yml for Portainer is not special. You can find it here if you need it. Normally there are no differences between my and the one you find if you google for Portainer service.

If you use mine you have to add a label to your manager! This is very important because Portainer needs a connection to your docker socket. You can add the label with these commands:

export NODE_ID=$(docker info -f '{{.Swarm.NodeID}}')
docker node update --label-add portainer.portainer-data=true $NODE_ID

Furthermore declare the PORTAINER_DOMAIN which will be used.

export PORTAINER_DOMAIN=portainer.$PRIMARY_DOMAIN

After you have done this you can deploy Portainer:

docker stack deploy -c docker-compose.portainer.yml portainer

Timing Note: Make sure you log in and create your credentials soon after Portainer is ready, or it will automatically shut down itself for security. If you didn’t create the credentials on time and it shut down itself automatically, you can force it to restart with:

docker service update portainer_portainer --force

Docker Registry

The Registry is a stateless, highly scalable server side application that stores and lets you distribute Docker images. The Registry is open-source, under the permissive Apache license.

The docker registry is important to have in a swarm environment if you don't want to upload your code/data to a public registry. If you don't have any private registry for yourself you have to upload the resulting image to docker-hub each time you extend an official image with your application code. While this is great if you only want to extend functionality in a general way it is not advisable if you copy your website with your closed source code into the image.

Imagine you have a modified Nginx image where the HTML folder is copied into the image:

FROM nginx
COPY html /usr/share/nginx/html

In many cases the html folder will contain your website which you don't want someone to use. If you have a private registry you can build the image and upload it but if you don't have one you have to use a public registry.

That's why I think that every swarm needs to have access to a private registry. And that's why I created one. Normally you can set up a registry with a simple docker run command but I wanted to have a registry that can be reached from everywhere. Because of that, I created a registry that could be accessed by username and password.

My personal docker-compose.yml can be downloaded here. It will run within my Traefik environment and create a new registry that I can use from every node in my swarm. To deploy the service you have to declare some environment variables which are used while deploying: REGISTRY_USERNAME, REGISTRY_HASHED_PASSWORD and REGISTRY_HOST:

export REGISTRY_USERNAME=reg_adm
export REGISTRY_HASHED_PASSWORD=$(openssl passwd -apr1 regsupersecret)
export REGISTRY_HOST=reg.YOUR_DOMAIN.tld

With these variables you can deploy the service:

docker stack deploy -c docker-compose.registry.yml registry

To use the registry you have to do two more things. The first thing is a quality of life feature to easily change the domain of the registry without affecting every container which uses an image from the private registry. Add DOCKER_REGISTRY to .profile for ****root**** or whatever user you are using so it is known if docker-compose files should be pushed/downloaded. The second task is very important. You have to execute docker login on every node of your swarm so that every node is allowed to pull images.

Now you can start using your private registry in docker-compose.yml. If you created a DOCKER_REGISTRY environment variable you can use it like this in your docker-compose.yml:

myapp:
    image: ${DOCKER_REGISTRY}/simple-app
    build:
      context: ./
      dockerfile: Dockerfile

If you want to deploy a service that contains the above part you have to build and push your image to deploy it correctly:

docker-compose build
docker-compose push
docker stack deploy -c docker-compose.yml www

If you only build and deploy it other nodes in your swarm cannot pull the image and so the service cannot be deployed.

FTP

Another important service is an FTP server for saving files you want to use anywhere in any app or service you create while working with the swarm.

I decided to use “pure-ftp” because it was the one I found while googling for a nice FTP server that runs within a docker environment. Because I want to keep it simple I created an FTP server without any traefik configuration. BUT I did a simple trick in putting the FTP server together with a website in a docker-compose.yml. I have done this because I want to have the possibility to download files from the server over https which I uploaded with ftp.

After configuration, I came up with this docker-compose.yml. Within the file, you can see that I use a custom Dockerfile for the web service which contains:

FROM nginx
COPY html /usr/share/nginx/html

The trick I describe is just the defined volume within the docker-compose.yml. As you can see I defined data in both services. Within web-service I just have an extra folder within the Nginx HTML folder so that I can access it from the web. And within the ftp-service I define data as the place where user can upload their data.

Before it is possible to deploy this service you have to declare environment variables: FTP_USERNAME, FTP_PASSWORD, FTP_DOMAIN_FOR_CERT, FTP_ORG_FOR_CERT, FTP_COUNTRYCODE_FOR_CERT and WEBSERVICE_DOMAIN:

export WEBSERVICE_DOMAIN=www.MYDOMAIN.tld
export FTP_USERNAME=SUPERUSER
export FTP_PASSWORD=clearTextPW
export FTP_DOMAIN_FOR_CERT=$WEBSERVICE_DOMAIN
export FTP_ORG_FOR_CERT=mybusiness
export FTP_COUNTRYCODE_FOR_CERT=DE

Furthermore, you have to add a label to any node of your swarm. To achieve this use docker node ls to find out the ID from every node and execute:

docker node update --label-add www.ftp-data=true ID_OF_NODE_TO_USE

After this is done you can safely deploy your website with enabled FTP

docker stack deploy -c docker-compose.web.yml webandftp

Now it is possible to connect with an FTP client to your WEBSERVICE_DOMAIN and upload a file (test.txt) which then can be accessed by this URL: WEBSERVICE_DOMAIN/data/test.txt

The whole FTP docker service can be downloaded from my GitHub:

https://github.com/paulscode-de/ftp-server

It is very important to say that you only can connect with your WEBSERVICE_DOMAIN if this domain also has an A record to your manager node!

5. Closing Notes

I hope you find this article helpful and can use my provided files to set up these services within your own Docker Swarm.

In my humble opinion, these four services should be present in every Docker Swarm environment because they are mandatory (or exchanged with function-like services).

This article was originally published on my blog at https://www.paulsblog.dev/services-you-want-to-have-in-a-swarm-environment/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by frank mckenna / Unsplash

As we all probably heard Adobe has acquired the popular design tool Figma for a whopping $20 billion. Unfortunately, this "strategy" of eliminating competition by acquiring businesses is common for big tech companies.

But, luckily, there is a free and open-source design tool that also does some things better. Also, it was inspired by Figma.

Penpot: Free & Open-Source Design Tool

Penpot is an open-source project that is actively developed and totally free to use for everyone.

Here are some major features that make Penpot interesting

• Free and open-source (of course).

• Option to Self-host (will be covered here primarily).

• Cross-platform.

• Using SVG as the native format.

• Web-based.

• Featuring industry-standard features (inspired by Figma).

Watch this official Penpot video to learn about the basics of Penpot:

A very important feature of Penpot is the use of SVG as their native format instead of PNG/JPG because it enables you to be compatible with many vector graphic editing tools.

This is especially useful because you will not get locked by being forced to use a proprietary file format that no other application can use.

Also, Penpot aims to use the absolute best of open standards that already exist. The CEO of Penpot, Pablo Ruiz-Múzquiz, mentions more about it:

If you go for SVG (open standard, web, mobile, etc) at the storage level, you can suddenly integrate all your Penpot designs with your code repos. You could make changes to the actual representation of the design itself thanks to SVG and not yet another closed format. That opens the door to massive opportunities for designers AND devs. Also, SVG means we are low-code ready for free. You can pick any element in Penpot and ask for its SVG (and CSS) representation knowing it's actually what it is, no translation. That brings a more trustworthy relationship between designers and devs and allows frontend devs to try out their design skillset.

As mentioned before a very important feature of Penpot is the ability to self-host it in a Docker container on your local machine and also on any server (that runs Docker). The following steps will show how easily you can use Docker to host your own Penpot to replace Figma as your design tool.

If you want to test Penpot before installing it as a self-hosted version you can go to https://design.penpot.app/ log in with GitHub/Gitlab/Google account or create n new account to test it. But I would avoid doing this and sharing your personal data because deploying it locally is done super fast (if you already have Docker installed)

Deploy Penpot locally with Docker

Install Docker

To deploy Penpot locally (or remotely) you need to have Docker installed. To install Docker on your system follow the official tutorial on docker.com. If you are using Windows and aren't allowed to install Docker Desktop you can follow this guide:

How To Install Docker Without Docker Desktop On Windows

Deploy Penpot

If Docker is installed it is very easy to start up a Penpot instance locally. To do this you can simply retrieve the latest Compose and config file from the official Penpot Github project by download them:

    wget https://raw.githubusercontent.com/penpot/penpot/main/docker/images/docker-compose.yaml
    wget https://raw.githubusercontent.com/penpot/penpot/main/docker/images/config.env

The config file from the GitHub project is quite big and contains several variables that you will never use working locally with Penpot. It would be sufficient to use the following config as it only contains mandatory values:

    # Standard database connection parameters (only postgresql is supported):
    PENPOT_DATABASE_URI=postgresql://penpot-postgres/penpot
    PENPOT_DATABASE_USERNAME=penpot
    PENPOT_DATABASE_PASSWORD=penpot

    # Redis is used for the websockets notifications.
    PENPOT_REDIS_URI=redis://penpot-redis/0

    ASSETS_STORAGE_BACKEND=assets-fs
    PENPOT_STORAGE_ASSETS_FS_DIRECTORY=/opt/data/assets

    PENPOT_TELEMETRY_ENABLED=false

    # Enable or disable external user registration process.
    PENPOT_REGISTRATION_ENABLED=false

Now that you downloaded (or created) both files switch to the folder and start the Docker service with:

    docker-compose -p penpot up -d

If the process finishes you can open http://localhost:9001 and will see the following screen if Penpot was installed correctly:

Create a Penpot user

Although you deployed Penpot on your local machine you cannot create a user using the Web interface because you started Penpot without configuring an SMTP account to receive registration emails. However, this is no problem because you can simply use this Docker command to manually create an already-activated user:

    docker exec -it penpot-penpot-backend-1 ./manage.sh create-profile

IMPORTANT: Check if penpot-penpot-backend-1 is the correct name of your backend container. You can do this by executing docker ps.

After executing the command you could get the following output which is confusing:

    [2022-10-06 10:18:13.523] I app.config - hint="flags initialized", flags="backend-api-doc,secure-session-cookies,login,backend-worker,registration"
    [2022-10-06 10:18:14.295] I app.metrics - action="initialize metrics"
    Email:  [2022-10-06 10:18:14.334] I app.db - hint="initialize connection pool", name="main", uri="postgresql://penpot-postgres/penpot", read-only=false, with-credentials=true, min-size=0, max-size
    =30

Ignore everything and just type an email that you want to use in your local installation, then set your name and password. The resulting log will look like this:

    [2022-10-06 10:22:33.900] I app.config - hint="flags initialized", flags="backend-api-doc,secure-session-cookies,login,backend-worker,registration"
    [2022-10-06 10:22:34.636] I app.metrics - action="initialize metrics"
    Email:  [2022-10-06 10:22:34.675] I app.db - hint="initialize connection pool", name="main", uri="postgresql://penpot-postgres/penpot", read-only=false, with-credentials=true, min-size=0, max-size
    =30
    user@local.de
    Full Name:  Paul Knulst
    Password:
    User created successfully.

After your user was created you can log in to your Penpot installation at http://localhost:9001/ and start using Penpot.

Deploy Penpot remotely with Docker and Traefik

Having a running installation of Penpot locally is very useful if you are the only person working with it. Normally, you will work in teams where multiple designers, developers, or others collaborate together on different designs. To allow doing this with Penpot you can deploy Penpot on any server using Docker and Traefik.

Installing Traefik

To install Traefik on your server and use it to forward URLs to Docker service and automatically assign SSL certificates you can follow this guide:

How to setup Traefik v2 with automatic Let’s Encrypt certificate resolver

Keep in mind that you need at least one URL that points to the server where your Traefik instance will run. If you do not have any URL you have to buy one and map it to your server.

Updating Config

Deploying Penpot on a server with a specific URL where multiple users can work together needs some changes to the prior shown config file because the registration process should be enabled. Therefore, you have to add an email account that will be used by Penpot to send registration emails. A simple Google/GMX/Apple Mail account will be sufficient.

Change the PENPOT_REGISTRATION_ENABLED environment variable to true and add the following variables (with correct values) to the config file:

    PENPOT_REGISTRATION_DOMAIN_WHITELIST=""

    #Your public penpot url
    PENPOT_PUBLIC_URI=

    # Enable Email 
    PENPOT_SMTP_ENABLED=true
    PENPOT_SMTP_DEFAULT_FROM=
    PENPOT_SMTP_DEFAULT_REPLY_TO=
    PENPOT_SMTP_HOST=
    PENPOT_SMTP_PORT=
    PENPOT_SMTP_USERNAME=
    PENPOT_SMTP_PASSWORD=
    PENPOT_SMTP_TLS=
    PENPOT_SMTP_SSL=

An important setting here is PENPOT_REGISTRATION_DOMAIN_WHITELIST="" where you can add a comma-separated list of allowed email domains to register. If you leave it empty every domain is allowed. Find the full config in this gist on Github.

Adjust Compose file for Traefik usage

After updating the config file you need to update the Compose file and add the Traefik network and the labels for Traefik to automatically generate an SSL certificate after deploying Penpot. Copy the content of the following Compose file into yours:

    version: "3.5"

    networks:
      default:
        external: false
      traefik-public:
        external: true

    volumes:
      postgres_data:
      assets_data:

    services:
      frontend:
        image: "penpotapp/frontend:latest"
        hostname: frontend.penpot
        volumes:
          - assets_data:/opt/data
        env_file:
          - config.env
        depends_on:
          - penpot-backend
          - penpot-exporter
        networks:
          - default
          - traefik-public
        labels:
          - traefik.enable=true
          - traefik.docker.network=traefik-public
          - traefik.constraint-label=traefik-public
          - traefik.http.routers.penpot-http.rule=Host(`${PENPOT_URL}`)
          - traefik.http.routers.penpot-http.entrypoints=http
          - traefik.http.routers.penpot-http.middlewares=https-redirect
          - traefik.http.routers.penpot-https.rule=Host(`${PENPOT_URL}`)
          - traefik.http.routers.penpot-https.entrypoints=https
          - traefik.http.routers.penpot-https.tls=true
          - traefik.http.routers.penpot-https.tls.certresolver=le
          - traefik.http.services.penpot.loadbalancer.server.port=80
      backend:
        image: "penpotapp/backend:latest"
        hostname: backend.penpot
        volumes:
          - assets_data:/opt/data
        depends_on:
          - postgres
          - redis
        env_file:
          - config.env
        networks:
          - default

      exporter:
        image: "penpotapp/exporter:latest"
        environment:
          # Don't touch it; this uses internal docker network to
          # communicate with the frontend.
          - PENPOT_PUBLIC_URI=http://frontend.penpot
        networks:
          - default

      postgres:
        image: "postgres:13"
        restart: always
        stop_signal: SIGINT
        environment:
          - POSTGRES_INITDB_ARGS=--data-checksums
          - POSTGRES_DB=penpot
          - POSTGRES_USER=penpot
          - POSTGRES_PASSWORD=penpot
        volumes:
          - postgres_data:/var/lib/postgresql/data
        networks:
          - default

      redis:
        image: redis:6
        restart: always
        networks:
          - default

Deploy Penpot

Before you finally can deploy the Docker service and use Penpot with your coworkers/friends you have to set your URL. This has to be the same URL that you used in the config (without HTTPS). Also, the URL has to point to your server!

    export PENPOT_URL=your-website.com

Now, switch to your folder and deploy your Penpot service:

    docker-compose up -d

Switch to your URL and register a new user. You should get an email from Penpot and can use it with your coworkers.

Keep in mind that you still can deactivate registration by setting the appropriate environment variable within the config and redeploying the server. If you do this you can create new users by executing the following command and manually set username/password:

    docker exec -it penpot-penpot-backend-1 ./manage.sh create-profile

With this approach, you will not need an email account and still be able to work with others.

Closing Notes

Congratulations if you followed my approach you have just installed Penpot as your own Figma replacement on either your local machine or on a server to use with collaborators.

The full Compose file (and config) for remote deployment can be found within the GitHub Gist I created for this article.

This is the end of this tutorial. Hopefully, you are now able to set up your personal installation. If you still have questions about setting up Penpot as a replacement for Figma you can just ask in the comment section. Also, if you enjoyed reading this article consider commenting with your valuable thoughts! I would love to hear your feedback about my tutorial or Penpot in general.

Furthermore, share this article with fellow designers to help them to replace Figma with Penpot.

This article was originally published on my blog at https://www.paulsblog.dev/deploy-free-figma-alternative-penpot-with-docker/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by GuerrillaBuzz Crypto PR / Unsplash

By the end of this guide you’ll have something similar to this Codepen, so if you’re excited, keep reading and follow this tutorial step-by-step!

Setting Up the Project

Assuming you are working with UNIX system (or have Git BASH on Windows) you can create all three files that are necessary to build a video player in JavaScript with this command:

mkdir video-player
cd video-player
touch index.html script.js style.css

To add a simple video player to our application we have to add the following code to our index.html:

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1.0" />
        <title>How to build a video player in Javascript</title>
        <link rel="stylesheet" href="style.css" />
    </head>
    <body>
        <div class="player">
            <video class="video" controls>
                <source
                    src="https://ftp.f1nalboss.de/data/imgly/videoplayer/testvideo.mp4"
                    type="video/mp4"
                />
                <source
                    src="https://ftp.f1nalboss.de/data/imgly/videoplayer/testvideo.mp4"
                    type="video/webm"
                />
                <p>No HTML5 video supported</p>
            </video>
        </div>
        <script src="script.js"></script>
    </body>
</html>

Within the above code, the <video> element uses a remote video from my FTP. You can either use my default video or add any video from your local computer by adjusting the src attribute. HTML5 specification supports three different video formats and the snippet used multiple <source> tags to make the videos available in MP4 and WebM. Furthermore, the <p> tag is used to display pre-defined content to user agents that do not support the video element.

The HTML5 <video> tag accepts several native attributes. For example, the controls attribute displays the standard player controls when added or set to true. You can find out more about all video attributes here.

Before continuing you should apply all styles that are needed within this tutorial by populating your style.css with all styles from this CodePen. Save and open your index.html and load it within the browser to see the embedded video player as seen below:

Customize the Video Player With JavaScript

To customize the video player, you first have to remove the controls attribute that displays Play, Pause, Volume, etc because you will implement your own custom controls within this tutorial. Now, check your browser you will recognize that the controls are gone and you cannot play the video anymore.

Add Play and Pause

To enable play and pause the video you have to add a new button to the index.html:

<div class="controls">
    <button
            class="controls__btn playPauseBtn"
            title="Toggle Play"
            >
        ►
    </button>
</div>

Afterward, open your script.js and enable functionality by adding this code:

const videoContainer = document.querySelector(".video-container");
const playPauseBtn = document.querySelector(".playPauseBtn");
function togglePlay() {
  if (videoContainer.paused || videoContainer.ended) {
    videoContainer.play();
  } else {
    videoContainer.pause();
  }
}
function updatePlayBtn() {
  playPauseBtn.innerHTML = videoContainer.paused ? "►" : "❚❚";
}
playPauseBtn.addEventListener("click", togglePlay);
videoContainer.addEventListener("click", togglePlay);
videoContainer.addEventListener("play", updatePlayBtn);
videoContainer.addEventListener("pause", updatePlayBtn);

Within this javascript code first the video-container element and the playPauseBtn is selected (Line 1 and 2). Then two functions will are defined: togglePlay() and updatePlayBtn(). togglePlay() is used to stop and start the video based on its actual state. updatePlayBtn is used to switch between the Icon which is shown within the video player.

In the last part of the snippet, a click event listener is added to the playPauseBtn that executes the togglePlay() function. Next, three click event listeners are added to the videoContainer that executes togglePlay() on mouse click and also executes updatePlayBtn based on the video's state.

Now you can reload your index.html and should be able to play and pause the video by either clicking the video or the button:

Add A Progress Bar

Next, a progress bar will be implemented to show the current timestamp of the video when played. First, add a <div> tag to the index.html which will act as the progress bar:

<div class="controls">
    <div class="progress">
        <div class="progress__filled"></div>
    </div>
    // ..
</div>

Then open the script.js and add the following snippet:

const progress = document.querySelector(".progress");
const progressBar = document.querySelector(".progress__filled");

function handleProgress() {
  const progressPercentage = (videoContainer.currentTime / videoContainer.duration) * 100;
  progressBar.style.flexBasis = `${progressPercentage}%`;
}

function jump(e) {
  const position = (e.offsetX / progress.offsetWidth) * videoContainer.duration;
  videoContainer.currentTime = position;
}

videoContainer.addEventListener("timeupdate", handleProgress);
progress.addEventListener("click", jump);
let mousedown = false;
progress.addEventListener("mousedown", () => (mousedown = true));
progress.addEventListener("mousemove", (e) => mousedown && jump(e));
progress.addEventListener("mouseup", () => (mousedown = false));

In this snippet, the progress container and the progress__filled element will be selected and two functions will be added: handleProgress() and jump(e). handleProgress() will be responsible for updating the progress bar. The jump(e) function is used to enable clicking on the progress bar to jump to the position within the video.

The last part contains all event listeners that are needed for the progress bar. The handleProgress() will be called at every timeupdate event. Also clicking anywhere on the progress bar will call the jump(e) method and the video will jump to the position. Additionally, mousedown, mousemove, and mouseup will be used to enable sliding through the video while holding the mouse button down on the progress bar.

Closing Notes

Congratulations, if you followed the tutorial you learned how to implement your own video player and add custom controls using JavaScript. Now, you can use my CodePen and start implementing more controls like volume control, keyboard shortcuts, or skip controls to build your own customized video player.

At the moment, I am working on a follow-up tutorial where I extend this code to add keyboard controls and add additional functionality. It will be published on my personal blog and later on Dev.to/Medium.

Have any questions? No problem, just ask in the comments section. I will answer everything.

This article was published on my blog at https://www.paulsblog.dev/how-to-build-a-video-player-in-vanilla-javascript-and-html5/

Feel free to connect with me on my personal blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Image by storyset on Freepik

As an Android app developer, you should always want to implement the best code that is possible. Also, your app should be as secure as possible.

Code quality is very important for any kind of software and you should always try to optimize your implementation. This article shows and explains different quality tools that will be combined in an Android software-quality-chain to achieve better code quality, a more secure app, and improved maintainability.

tl;dr: Clone this GitHub repository into your Android project folder and execute the full chain with: sh software-quality-chain.sh. Afterward, open the URL that will be logged in to the console. Log in with admin:admin12345 and see your Android project report.

Prerequisites

Docker

To run the script Docker needs to be installed on your operating system.

Docker is a widely used platform for developing, shipping, and running all kinds of applications. It enables you to separate infrastructure from your applications to quickly deliver software from one machine to another.

Implementing software while using Docker often negates infrastructure problems like the common "works on my machine" problem:

To install Docker on your system follow the official tutorial on docker.com. If you are using Windows and aren't allowed to install Docker Desktop you can follow this guide:

https://www.paulsblog.dev/how-to-install-docker-without-docker-desktop-on-windows/

Detekt

Detekt is

a static code analysis tool for the Kotlin programming language. It operates on the abstract syntax tree provided by the Kotlin compiler.

It can be implemented in any Kotlin-based Android app and can be used with a self-defined set of rules to check an app. Detekt is a mandatory prerequisite to executing the script and has to be installed within your Android App. Fortunately, this procedure is very easy and can be done in four simple steps:

1. Add Detekt to project build.gradle

dependencies { 
    [...]
    classpath("io.gitlab.arturbosch.detekt:detekt-gradle-plugin:1.17.0")
}

2. Create a detekt.yml with a defined set of rules.
Download a sample gradle detek.yml file here and store it within project_folder/detekt/detekt.yml

3. Apply the detekt plugin in app build.gradle

apply plugin: "io.gitlab.arturbosch.detekt"

4. Add detekt block to app build.gradle

detekt {
    toolVersion = "1.17.0"
    buildUponDefaultConfig = false
    allRules = false
    config = files("../detekt/detekt.yml")
    baseline = file("../detekt/baseline.xml")
    input = files("src/main/java/com")
    debug = false
    reports {
        html {
            enabled = true
            destination = file("build/reports/detekt.html")
        }
        xml {
            enabled = true
            destination = file("build/reports/detekt.xml")
        }
        txt.enabled = false
        sarif.enabled = false
    }

Other Tools Used

MobSF

Mobile Security Framework (MobSF)

is an automated, all-in-one mobile application (Android/iOS/Windows) pen-testing, malware analysis and security assessment framework capable of performing static and dynamic analysis.

Before continuing, it is necessary to understand mobile security and how to create an android app securely. Luckily, OWASP (Open Web Application Security Project), a popular nonprofit organization that is known for community-led open-source software projects and for improving software security created the "OWASP Mobile Top Ten" to improve mobile security. These include the most critical security issues that should be avoided in any Android app:

• M1: Improper Platform Usage

• M2: Insecure Data Storage

• M3: Insecure Communication

• M4: Insecure Authentication

• M5: Insufficient Cryptography

• M6: Insecure Authorization

• M7: Client Code Quality

• M8: Code Tampering

• M9: Reverse Engineering

• M10: Extraneous Functionality

Fortunately, MobSF can help you to identify many possible security issues in your Android app in an automated way. The security framework works best for:

• local environment

• performing a quick security test

• implemented in CI/CD with mobsfscan (will be used here)

To test MobSF you can switch to https://mobsf.live and upload any Android APK that will then be analyzed for common security issues.

My advice: all developers should use MobSF to identify many security vulnerabilities during development by doing a static code analysis. This analysis will check the source code without running the application. It can be used during mobile application development and should be carried out regularly. Normally, it should be executed before every app release, update and Pull Request submission.

Keep in mind that using MobSF for static code analysis does not guarantee that your mobile application is safe! But it will help to identify the most obvious security flaws.

SonarQube

SonarQube

is an open-source platform developed by SonarSource for continuous inspection of code quality to perform automatic reviews with static analysis of code to detect bugs, code smells, and security vulnerabilities in 20+ programming languages.

For your projects, SonarQube analyzes source code (not only Android/Kotlin apps), evaluates the quality, and generates reports. It allows for the continuous monitoring of quality throughout time and combines static and dynamic analytic methods. SonarQube inspects and evaluates everything that has an impact on our code base, from inconspicuous styling decisions to serious design flaws.

As a result, developers can access and track code analysis information on everything from styling mistakes, potential bugs, and code defects to design inefficiencies, code duplication, inadequate test coverage, and excessive complexity. The Sonar platform examines source code from a variety of angles; as a result, it digs down to your code layer by layer, progressing from the module level to the class level. Additionally, it generates metrics and data at each level, highlighting problematic regions in the source that need to be examined or improved. It automatically detects what type of software you use: Kotlin-based Android app.

Other Features:
SonarQube does more than merely highlight issues. It also provides quality-management tools to proactively assist you in making corrections.

• SonarQube provides information on a dashboard on code standards, test coverage, duplications, API documentation, complexity, and architecture in addition to issues.

• It provides you with a current snapshot of the quality of your Android app code as well as trends in trailing (what has previously gone wrong) and leading (what is most likely to go wrong in the future) quality indicators.

• It offers measurements to guide you in making the best choice for your Android app.

SonarScanner

The SonarScanner CLI is a scanner that can be used every time if there is no specific scanner for your build system. Additionally, the SonarScanner CLI can be run within a Docker container to test the Android app source code.

It is a simple command line tool that scans a provided folder (the Android app folder) with SonarQube-specific roles. After a successful analysis, it uploads all data to a SonarQube server where the results can be viewed.

Prepare Dockerfiles

Now, after explaining the prerequisites and tools they will be combined in Docker Compose files to be then used in a single bash script that will result in the software-quality-chain.

SonarQube/mobsfscan Docker Compose File

As described earlier SonarQube and mobsfscan will be used within the Android software-quality-chain. To be able to use them they will be deployed with Docker Compose as a service onto your operating system.

The following Compose file contains the SonarQube server, a PostgreSQL database for SonarQube, and the mobsfscan container to scan your Android app:

version: "3"

services:
  sonarqube:
    image: sonarqube:9.6-community
    depends_on:
      sonarqube_db:
        condition: service_healthy
    environment:
      SONAR_JDBC_URL: jdbc:postgresql://sonarqube_db:5432/sonar
      SONAR_JDBC_USERNAME: sonar
      SONAR_JDBC_PASSWORD: sonar
    healthcheck:
      test: wget --no-verbose --tries=1 --spider http://localhost:9000/api/system/status || exit 1
      start_period: 10s
      interval: 30s
      timeout: 10s
      retries: 10
    volumes:
      - sonarqube_data:/opt/sonarqube/data
      - sonarqube_extensions:/opt/sonarqube/extensions
      - sonarqube_logs:/opt/sonarqube/logs
    ports:
      - "9000:9000"
  sonarqube_db:
    image: postgres:12
    environment:
      POSTGRES_USER: sonar
      POSTGRES_PASSWORD: sonar
    healthcheck:
      test: [ "CMD-SHELL", "pg_isready" ]
      interval: 10s
      timeout: 5s
      retries: 5
    volumes:
      - postgresql:/var/lib/postgresql
      - postgresql_data:/var/lib/postgresql/data
  mobsfscan:
    image: opensecurity/mobsfscan
    depends_on:
      sonarqube:
        condition: service_healthy
    volumes:
      - ./../app/src:/src
    entrypoint:
      - mobsfscan
      - --sonarqube
      - -o
      - /src/result.json
      - /src
volumes:
  sonarqube_data:
  sonarqube_extensions:
  sonarqube_logs:
  postgresql:
  postgresql_data:

Within this file three services are defined: SonarQube, PostgreSQL, and mobsfscan. While SonarQube and PostgreSQL will just use a standardized Docker service that gets started, mobsfscan will have the project root bound as a volume and an overwritten entry point that executes instantly after the SonarQube container is started correctly and marked healthy (see healthcheck).

SonarScanner Compose File

Luckily, SonarScanner utility is also available as Docker Image and can be used and configured in a Compose file:

version: "3.7"
services:
  sonarscanner:
    image: sonarsource/sonar-scanner-cli
    network_mode: "host"
    env_file:
      - .sq.env
    command:
      - -Dsonar.projectKey=YourAppName
      - -Dsonar.exclusions=**/*.java
      - -Dsonar.externalIssuesReportPaths=./output/result.json
      - -Dsonar.kotlin.detekt.reportPaths=./app/build/reports/detekt.xml
    volumes:
      - "./../:/usr/src"

As you can see in this file we define a projectKey (YourAppName), set some exclusions (comma-separated list), and also add additional reports to the execution of SonarScanner (relative path from /usr/src). Also, we add the Android project root as volume and map it to /usr/src.

Additionally, this Compose file needs an environment file that will contain the SonarQube URL and a User Token which will be used to upload the analysis:

SONAR_HOST_URL=http://localhost:9000
SONAR_LOGIN=squ_3d9c193d3699d18db4f539725090d67caef0b964

Combining the Tools

Now, all Dockerfiles will be combined within the bash script called software-quality-chain which consists of these steps:

1. Increasing the heap count of the Docker environment

2. Install, configure, and start SonarQube and mobsfscan

3. Waiting until mobsfscan finish the analysis

4. Running complete Gradle checks

5. Importing a clean PostgreSQL database.

6. Executing SonarScanner and combining results before uploading to SonarQube

This bash script contains the final solution which will execute all steps:

#!/bin/bash

echo "Removing old output"
rm -f ../output/detekt.xml
rm -f ../output/result.json

echo "Increasing heap count for SonarQube"
sysctl -w vm.max_map_count=262144

echo "Starting SonarQube, mobsfscan"
DOCKER_BUILDKIT=1 docker-compose -f docker-compose.sq.yml up -d

echo "SonarQube is started, running mobsfscan"

echo "Waiting until scan is succesfully finished"
until [ -f ../app/src/result.json ]
do
     sleep 5
done

mv ../app/src/result.json ../output/result.json

echo "finished mobsfscan"

echo "running gradle detekt"
cd ..
sed -i 's/baseline = file("..\/detekt\/baseline.xml")/\/\/baseline = file("..\/detekt\/baseline.xml")/g' app/build.gradle
./gradlew detekt
sed -i 's/\/\/baseline = file("..\/detekt\/baseline.xml")/baseline = file("..\/detekt\/baseline.xml")/g' app/build.gradle
cd docker
cp ../app/build/reports/detekt.xml ../output/detekt.xml
echo "finished gradle detekt"

echo "preparing files to import into SonarQube"
sed -i 's/\/src\/main\//app\/src\/main\//g' ../output/result.json
sed -i 's/"filePath": null/"filePath": "app\/src\/main\/AndroidManifest.xml"/g' ../output/result.json

echo "Restoring SonarQube database with default user"
docker exec -i docker-sonarqube_db-1 //bin//bash -c "PGPASSWORD=sonar psql --username sonar sonar" < dump.sql

echo "Starting SonarScanner"
DOCKER_BUILDKIT=1  docker-compose -f docker-compose.scan.yml up -d

echo "SonarScanner started, will scan code now and upload data"
docker logs -f docker-sonarscanner-1

Increasing the heap count of the Docker environment:

To execute the SonarQube server without any problems you have to increase the max_map_count of the operating system running the Docker container because the default value is not sufficient.

You can read about it in the SonarQube documentation.

Install, configure, and start SonarQube and mobsfscan:

As the Docker Compose files already contain every important setting the only command that has to be executed here is running the Compose file in the background

Waiting until mobsfscan finish the analysis:

Unfortunately, the Docker Compose service of mobsfscan is only started by running the Compose file and the analysis is executed afterward. Because of this, the script has to block until the mobsfscan analysis finishes and successfully creates a result file. This "blocking" is done in lines 16 - 21 where the script waits until the result.json is created and then moves it to the output folder.

Running complete Gradle Detekt checks:

The next step is executing a complete gradle detekt task. It's called complete because sometimes developers use a baseline.xml file in their detekt configuration to mark legacy code as "ignored". Within this step, the baseline.xml file will be removed by commenting it out with sed and then running the gradle detekt task. Afterward, the detekt report will be copied to the output folder, and the usage of baseline.xml file will be restored.

Preparing files to import into SonarQube:

There are two small bugs if combining mobsfscan utility and SonarQube while creating and importing the result.json.

1. mobsfscan also finds issues in the project that is not suited to specific source files. Unfortunately, these issues cannot be imported correctly into SonarQube because SonarQube always needs a file to add an issue. Luckily, it is easy to identify these issues because they all have filePath=null.

2. mobsfscan lists all file paths of files from the app folder while SonarQube lists all files from the project folder.

Both issues will be fixed in line 35 and 36 where sed is used to replace the wrong entries with the correct one:

• Line 35: src/main -> app/src/main

• Line 36: filePath=null -> filePath="app/src/main/AndroidManifest.xml"

Importing a clean PostgreSQL database:

Before starting the SonarScanner to analyze the source code and upload the results to the SonarQube instance a clean PostgreSQL database backup will be imported into the SonarQube instance (Line 39). This has to be done because the freshly installed SonarQube instance does not contain a User Token that can be used to upload data. The provided PostgreSQL backup contains a User (admin:admin12345) and a User Token that already is set within the SonarScanner environment variables.

Without this step, you would have to switch to your SonarQube instance and log in with the default user (admin:admin). Then you have to switch to your account configuration and create a new User Token that can be used for analysis.

Executing SonarScanner and combining results before uploading to SonarQube:

The last step will be executing the Compose file for the SonarScanner (Line 42) and opening the docker logs for this instance to be notified if the analysis is complete.

Afterward, you will get an URL within the terminal that you can open in the browser, log in with admin:admin12345, and see the results of the executed analysis.

Run the Chain

Now, that everything is set upped and explained you can start software-quality-chain. If you followed the tutorial and created the files you still need this PostgreSQL database dump that you can save as dump.sql within the docker folder. If you did not manually create every file you can clone the complete GitHub repository into your Android project root.

Then you can run it by executing:

sh software-quality-chain.sh

Grab a coffee and wait until the chain finishes. There will be an URL within the CLI that you can copy to open the SonarQube instance. Log in with admin:admin12345 and start checking your improvements, security issues, and other stuff.

Possible Errors

If you are using this software-quality-chain you maybe encounter some errors that can be fixed easily!

SonarQube does not start

As told before, to run SonarQube you have to increase the max_map_count. Normally, this will be done by the command in Line 8. If you are working with Windows or do not have permission you have to update the command:

For Windows:

wsl sysctl -w vm.max_map_count=262144

Not able because of permissions:

sudo sysctl -w vm.max_map_count=262144

Cannot run docker-compose (Windows)

If you do not have Docker Desktop (or Rancher Desktop) it could happen that docker-compose cannot be executed from your terminal.

Sometimes it happens because you have to add wsl in front of every docker-compose command (c.f. Line 11):

DOCKER_BUILDKIT=1 wsl docker-compose -f docker-compose.sq.yml up -d

Cannot import PostgreSQL backup (Windows)

If you try to run the software-quality-chain within the Powershell or Git-bash while working with windows you normally will encounter an error while the script tries to restore the PostgreSQL database.

This happens because of a problem with the file encoding of the PostgreSQL dump that is used. To fix this problem connect to your WSL and restart the software-quality-chain. Alternatively, you can log into the running SonarQube instance with default user admin:admin, switch to accounts settings, and create a new user token. Then you can update the sq.env file to use this new user token. If you do this you have to remove the part from the software-quality-chain where the PostgreSQL backup will be inserted.

Software-Quality-Chain Improvements/Changes

As this chain is mainly suited for developers that want to run a quick full test for their android apps on their own computer, this chain could also be changed in some ways.

An important feature of the described chain is that it will run on ANY device running Docker because it automatically inserts a database while executing. This means that every time you run this chain you will have a freshly installed SonarQube that only contains results from the executed analysis.

But if you want to have a curve where you can see improvements while developing you have to persist the data from every consecutive analysis. Also, you can work with several members on the same SonarQube instance and use this chain within your CI/CD pipeline to automatically check after a Pull Request is merged into your release branch.

The following chapters cover changes that have to be made to use the chain in any of these ways.

Persisting Analysis Data

To persist data between consecutive analyses you simply have to adjust the software-quality-chain after the first run.

After your first successful run of the chain switch to lines 38 and 39 where the database import takes place.

You still need the first run because it will populate the database to have a working user token and the user to log in admin:admin12345.

Use a different SonarQube instance

If you are using a different SonarQube instance within the software-quality-chain you have to adjust the two files:

docker-compose.sq.yml: Remove the SonarQube and PostgreSQL container and only keep the mobsfscan. The Compose file will look like this:

version: "3"

services:
  mobsfscan:
    image: opensecurity/mobsfscan
    volumes:
      - ./../app/src:/src
    entrypoint:
      - mobsfscan
      - --sonarqube
      - -o
      - /src/result.json
      - /src

sq.env: Replace the URL with your SonarQube instance URL and update the Token to your user token

Also, remove the PostgreSQL database backup (Line 38/39).

Use it in CI/CD chain

To use this script in a CI/CD chain you have to combine previous sections. Remove the SonarQube instance by editing the Compose file, changing the URL/Token of sq.env, and removing the PostgreSQL backup.

Additional Tools

To further improve the quality of Android apps it is possible to enhance your Android project with two additional extra tools/commands that work well in combination with the software-quality-chain.

SonarLint Plugin for Android

The SonarLint plugin can be downloaded and installed manually from the JetBrains marketplace or you can easily install it from within Android Studio by opening Settings->Plugins and installing it.

Pre-Push Hook For Executing Gradle Detekt

To guarantee that all pushed files did not contain any error that will be found with Gradle you can install a pre-push hook into your .git folder. To do this open .git/hooks, create a new file called pre-push, make it executable (chmod +x), and paste the following content:

#!/bin/bash
echo "Starting detekt check"
./gradlew detekt

Closing Notes

I hope you enjoyed reading this article and will now use my software-quality-chain to significantly improve your code quality and make your app more secure. Keep in mind that software that has high quality can be maintained and enhanced more easily!

All files that were explained in this tutorial can be found in this GitHub repository. Download the whole folder, put it into your Android project root, and execute the software-quality-chain with:

sh software-quality-chain.sh

Keep in mind that on Windows you have to be connected to the WSL!

This is the end of this tutorial. Hopefully, you are now able to use my Android software-quality-chain. If you still have questions about anything that is not fully described you can just ask in the comment section. Also, if you enjoyed reading this article consider commenting with your valuable thoughts! I would love to hear your feedback about my developed chain.

This article was originally published on my blog at https://www.paulsblog.dev/optimize-android-app-development-with-docker-sonarqube-detekt-and-mobsf/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by Jamison McAndie / Unsplash

This tutorial will cover all work to build a simple URL shortener API with NestJS and MongoDB. Additionally, I will show how to deploy it to a (Docker and Docker Swarm) production environment using Docker.

The source code is published on GitHub and can be used freely:

https://github.com/paulknulst/paulsshortener

Create the NestJS Project

First, create a NestJS project which will work as a baseline for the URL shortener. To do this, you need to install the Nest-CLI on your system which can be done by:

npm install -g @nestjs/cli

Then switch to your projects folder and use the Nest-CLI to create a new NestJS project by executing:

nest new paulsshortener

During this process, you will be asked which package manager you want to use. Within this tutorial npm will be used.

Now, you can start the NestJS project in "watch-mode" to instantly see all changes you will make during this tutorial by executing:

npm run start:dev

To test if everything is started correctly hit http://localhost:3000 which will just show "Hello World" within the browser.

Within the project open the AppService (/src/app.service.ts) and change return 'Hello World!'; to return 'This will be your URL shortener';. After reloading http://localhost:3000 the updated response will be seen.

Set Up MongoDB database

For simplicity, MongoDB will be used during this tutorial. To avoid trouble setting up the correct version, replacing an old installation, and so on you should use Docker to deploy it. Save this minimal Compose file into your project root and name it docker-compose.local.yml:

version: '3.6'

services:
  mongo:
    image: mongo
    restart: always
    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: supersafe

If you are not familiar with Docker Compose and do not want to use it to set up a MongoDB you can have a look at the official MongoDB documentation to learn how you can install it on your machine. Keep in mind that Docker is also used to deploy this URL shortener later within this tutorial!

Now, to deploy a MongoDB, switch to your project root within a terminal and execute:

docker-compose -f docker-compose.local.yml up -d

Afterward, you have successfully set up MongoDB and your machine and can start using it.

Connect Your NestJS project to MongoDB

To connect the project to MongoDB you should use Mongoose which is the most popular MongoDB object modeling tool. Start by installing the required dependencies into your project:

npm i @nestjs/mongoose mongoose

Once you have installed the dependency you can import the Mongoose module into the project by editing the AppModule (/src/app.module.ts) that it looks like this:

import { Module } from '@nestjs/common';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { MongooseModule } from '@nestjs/mongoose';

@Module({
  imports: [MongooseModule.forRoot('mongodb://localhost:12345/paulsshortener')],
  controllers: [AppController],
  providers: [AppService],
})

export class AppModule {}

The connection will now be automatically established and you can create the schema that will be used to work with the database.

Now use the Nest-CLI to create a new module within your NestJS project which will handle everything related to URLs:

nest g res url

The routine will ask two questions that you should answer.

• What transport layer do you use? -> REST API
• Would you like to generate CRUD entry points? -> No

Switch to the newly generated url folder, create a new folder schema, and add a file url.schema.ts. Then create a class (Url) add two properties (url, shortenedUrl) to the file. Also, add exports for Document and Schema. Your file should look like this:

import { Prop, Schema, SchemaFactory } from '@nestjs/mongoose';
import { Document } from 'mongoose';

export type UrlDocument = Url & Document;

@Schema()
export class Url {
  @Prop()
  url: string;

  @Prop()
  shortenedUrl: string;
}

export const UrlSchema = SchemaFactory.createForClass(Url);

Open up url.module.ts, add two new imports, and modify the @Module annotation:

import {Module} from '@nestjs/common';
import {UrlService} from './url.service';
import {UrlController} from './url.controller';
import {Url, UrlSchema} from "./schemas/url.schema";
import {MongooseModule} from "@nestjs/mongoose";

@Module({
    imports: [MongooseModule.forFeature([{name: Url.name, schema: UrlSchema}])],
    controllers: [UrlController],
    providers: [UrlService]
})
export class UrlModule {
}

Implement the URL Link Shortner Function

To shrink an URL you will be going to use the CRC32 hash algorithm which has to be added to the project to use it:

$ npm install crc-32 --save

Open the UrlService (/src/url/url.service.ts) and replace the content of the file with the following snippet:

import {Injectable} from '@nestjs/common';
import {InjectModel} from "@nestjs/mongoose";
import {Url, UrlDocument} from "./schemas/url.schema";
import {Model} from "mongoose";
import * as CRC from "crc-32";

@Injectable()
export class UrlService {

    constructor(@InjectModel(Url.name) private urlModel: Model<UrlDocument>) {
    }

    private shrink(url: string) {
        return CRC.str(url).toString(16)
    }

    async create(url: string) {
        const createdUrl = new this.urlModel({url: url, shortenedUrl: this.shrink(url)});
        await createdUrl.save();
        return createdUrl.shortenedUrl;
    }

    async find(shortenedUrl: string) { //-3c666ac
        const url = await this.urlModel.findOne({shortenedUrl: shortenedUrl}).exec();
        return url.url;
    }
}

This snippet contains three important functions:

• shrink: Converts the given URL into an 8-character string by using the CRC32 algorithm. Then returns only the 8-character string.
• create: Use the shrink function to create an 8-character string and saves a new UrlSchema document into the MongoDB
• find: Retrieves the saved URL from the MongoDB.

Add Routes to NestJS API

To use the URL shortener in a Client we need to create three REST endpoints/functions.

• GET /shrink: Create a new shortened URL using the HTTP Get method. The path parameter contains the unshortened URL. This can be done within any browser.
• POST /shrink: Create a new shortened URL using the HTTP Post method. The body contains the unshortened URL. You need an API or Postman to use this.
• GET /s: Takes the 8-character string as a path parameter and returns the unshortened URL. In the end, it will automatically forward to the unshortened URL.

As you are working with NestJS you can easily implement these endpoints by adding three new functions to the UrlController (url.controller.ts) within the URL module and annotate it with @Get and @Post. Open the UrlController and replace the content with:

import {Body, Controller, Get, Param, Post} from '@nestjs/common';
import {UrlService} from './url.service';

@Controller('')
export class UrlController {
    constructor(private readonly urlService: UrlService) {
    }

    @Get('/shrink/:url')
    getShrink(@Param('url') url: string) {
        return this.urlService.create(url)
    }

    @Post('/shrink')
    postShrink(@Body() body: { url: string }) {
        return this.urlService.create(body.url)
    }

    @Get('/s/:shortenedUrl')
    unshrink(@Param('shortenedUrl') shortenedUrl: string) {
        return this.urlService.find(shortenedUrl)
    }
}

Within this snippet, two new GET resources (/shrink/ and /s/) and one POST resource that calls the previously created functions from the UrlService are created. Also, the annotation above the class is changed from @Controller('url') to @Controller('') to further shrink the resulting URL.

Testing the URL shortener

To test the functionality you can simply use your browser because we have implemented both resources as GET calls.

Keep in mind that as you use GET for providing an URL as a path parameter you have to encode the URL! This means that an URL like this:

https://www.paulsblog.dev/manage-time-more-efficiently-with-the-pomodoro-technique/

will become this:

https%3A%2F%2Fwww.paulsblog.dev%2Fmanage-time-more-efficiently-with-the-pomodoro-technique%2F

With this information, you can create a shortened URL by opening the following URL in our browser:

http://localhost:3000/shrink/https%3A%2F%2Fwww.paulsblog.dev%2Fmanage-time-more-efficiently-with-the-pomodoro-technique%2F

With this GET call your API will return the 8-character string: -5f1a8349 (It should be the same within your project)

Append this string to your GET /s/ call to receive the unshortened version of the URL by opening: http://localhost:3000/s/-5f1a8349

The result in the browser will be the unshortened version of the previously provided URL.

Implement Forwarding to Unshortened URL

Now, that you have developed an API that can shorten URLs with help of the CRC-32 algorithm you should enable the functionality to forward the request and automatically open the URL it finds within the database.

With NestJS this is easy because you only have to adjust the unshrink function within the UrlController (src/url/url.controller.ts). Change the previously created function to this implementation:

@Get('/s/:shortenedUrl')
async unshrink(@Res() res, @Param('shortenedUrl') shortenedUrl: string) {
    const url = await this.urlService.find(shortenedUrl)
    return res.redirect(url)
}

Deploy The URL Shortener With Docker

Let's assume you want to deploy the URL shortener in a Docker environment and have to develop a Compose file that can be used to do this.

The first step to do will be to create a new Compose file (docker-compose.prod.yml) and copy the content from the previously created MongoDB file into it. Then add a new service called backend which will be used to install, compile and run the NestJS project within the Docker environment.

As you have developed a custom piece of software there will not be a suitable image on DockerHub and you have to create one from scratch. Because the project is based on NestJS, which is working in a NodeJS environment, you can create a new Dockerfile and use the latest version of node as a base image. Then simply copy the source code, install, build and run the project. The following Dockerfile will be sufficient and should be created in the project root:

FROM node:latest

WORKDIR /usr/src/app
COPY package*.json ./

RUN npm install
COPY . .
RUN npm run build

CMD [ "node", "dist/main.js" ]

This Dockerfile can now be used as the image for the backend service in the Compose file.

version: '3.6'
services:
  db:
    image: mongo
    restart: always
    environment:
      MONGODB_USER: paul
      MONGODB_DATABASE: paulsshortener
      MONGODB_PASS: paulspw
  backend:
    image: paulsshortener
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - 3001:3000
    depends_on:
      - db
    restart: unless-stopped

Unfortunately, running this Compose file will not work because the URL for the links within our project are hard-coded and will not automatically adjust. Also, the DB hostname and port within the AppModule (app.module.ts) are static.

To fix these problems, you have to add and change something within the AppModule (app.module.ts) and the UrlService (url.service.ts). To be specific, add three new variables that hold the database port, the database URL, and the base URL of the resulting service.

Switch to the AppModule (app.module.ts) and add these two variables above the class definition:

const DB_HOST = process.env.DB_HOST || 'localhost'
const DB_PORT = process.env.DB_PORT || '12345'

Additionally, change the Mongoose part within the imports from the module to correctly use these variables:

MongooseModule.forRoot('mongodb://' + DB_HOST + ':' + DB_PORT + '/paulsshortener')

Then open the UrlService (url.service.ts) and add the baseurl variable above the class definition:

const basepath = process.env.BASE_URL || 'http://localhost:3000/'

Lastly, change the create function to return the complete shortened URL using the newly introduced base variable.

async create(url: string) {
    const createdUrl = new this.urlModel({url: url, shortenedUrl: this.shrink(url)});
    await createdUrl.save();
    return basepath + "s/" + createdUrl.shortenedUrl;
}

Now, that you applied these changes you should adjust your Compose file by adding the available environment variables and adjusting them to your needs:

version: '3.6'
services:
  db:
    image: mongo
    restart: always
    environment:
      MONGODB_USER: paul
      MONGODB_DATABASE: paulsshortener
      MONGODB_PASS: paulspw
  backend:
    image: paulsshortener
    environment:
      - BASE_URL=https://locahost:3001
      - DB_HOST=db
      - DB_PORT=27017
    build:
      context: .
      dockerfile: Dockerfile
    ports:
      - 3001:3000
    depends_on:
      - db
    restart: unless-stopped

Finally, deploy it on your localhost:

docker-compose up -d --build

Your URL shortener is now correctly installed and can be used from your local environment.

Deploy to Production Using Traefik

Now that you have a running Docker Service that can be deployed anywhere you can use it to deploy it in a production environment using Traefik. To do this you will edit the Compose file, add all Traefik-related keywords (labels, networks), and adjust it to your needs.

If you are not familiar with deploying Docker Services using Traefik I can recommend the following tutorials covering basic installation on a single server and a server cluster installation using Docker Swarm:

• How to setup Traefik v2 with automatic Let’s Encrypt certificate resolver
• Deploy Any SSL Secured Website With Docker And Traefik
• Setup Docker Swarm (For Traefik)
• Install Traefik On Docker Swarm

To deploy your URL shortener using Traefik adjust the labels section. The following part will show and explain how it is done in a single server setup and additionally there will be a Docker Swarm configuration to download:

version: '3.6'
services:
  db:
    image: mongo
    restart: always
    environment:
      MONGODB_USER: paul
      MONGODB_DATABASE: paulsshortener
      MONGODB_PASS: paulspw
    volumes:
      - db:/data/db
    networks:
      - default
  backend:
    image: paulsshortener
    environment:
      - BASE_URL=https://at0m.de/
      - DB_HOST=db
      - DB_PORT=27017
    build:
      context: .
      dockerfile: Dockerfile
    networks:
      - default
      - traefik-public
    depends_on:
      - db
    restart: unless-stopped
    labels:
      - traefik.enable=true
      - traefik.docker.network=traefik-public
      - traefik.constraint-label=traefik-public
      - traefik.http.routers.pauls-shortener-http.rule=Host(`at0m.de`) || Host(`www.at0m.de`)
      - traefik.http.routers.pauls-shortener-http.entrypoints=http
      - traefik.http.routers.pauls-shortener-http.middlewares=https-redirect
      - traefik.http.routers.pauls-shortener-https.rule=Host(`at0m.de`)
      - traefik.http.routers.pauls-shortener-https.entrypoints=https
      - traefik.http.routers.pauls-shortener-https.tls=true
      - traefik.http.routers.pauls-shortener-https.tls.certresolver=le
      - traefik.http.services.pauls-shortener.loadbalancer.server.port=3000
      - traefik.http.middlewares.redirect-pauls-shortener.redirectregex.regex=^https://www.at0m.de/(.*)
      - traefik.http.middlewares.redirect-pauls-shortener.redirectregex.replacement=https://at0m.de/$${1}
      - traefik.http.middlewares.redirect-pauls-shortener.redirectregex.permanent=true
      - traefik.http.routers.blogs-knulst-https.middlewares=redirect-pauls-shortener
volumes:
  db:

networks:
  traefik-public:
    external: true

Before you can successfully deploy this service with Compose you have to adjust the Host: Use your own BASE_URL and update the traefik configuration (For me, it is at0m.de). After changing both values deploy it with:

docker-compose -f docker-compose.prod.yml up -d

Using a Docker Swarm you can use this Compose file. But, you have to adjust BASE_URL and the placement constraints for the MongoDB service. Then build, push the image to our registry, and deploy it onto your Docker Swarm:

docker-compose -f docker-compose.prod-swarm.yml build
docker-compose -f docker-compose.prod-swarm.yml push
docker stack deploy -c docker-compose.prod-swarm.yml paulsshortener

Additional Adjustments for Live Version

Because you deployed it publicly on your server you should add rate limiting by following this approach: https://docs.nestjs.com/security/rate-limiting.

tl;dr:

Install needed package within the project:

npm i --save @nestjs/throttler

In AppModule (app.module.ts) extend imports-array:

ThrottlerModule.forRoot({
    ttl: 60,
    limit: 10,
})

Then, add ThrottlerGuard to the providers array:

{
    provide: APP_GUARD,
    useClass: ThrottlerGuard,
}

Add the SkipThrottle annotation to the Get /s/ endpoint within the UrlController (url.controller.ts) to ignore rate limiting this specific call:

@SkipThrottle()

Finally, redeploy your Docker service. Don't forget to rebuild before deploying!

Closing Notes

I hope you enjoyed reading my tutorial and are now able to create, build, and deploy your URL shortener website within a Docker container.

Keep in mind that this is a very basic example without any error handling, and no URL checking. However, this tutorial should be a starting point for developing your version.

If you enjoyed reading this article consider commenting your valuable thoughts in the comments section. I would love to hear your feedback about this URL shortener. Furthermore, if you have any questions about implementing your own version, please jot them down below. I try to answer them if possible. Also, share this article with your friends and colleagues to show them how to use NestJs, MongoDB, and Docker to create their own URL shortener.

This article was originally published on my blog at https://www.paulsblog.dev/use-nestjs-mongodb-and-docker-to-create-an-url-shortener/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Image by Vectorarte on Freepik

Within this short tutorial, I will show how you can easily do this and how you can create a Git alias that you can simply reuse everywhere.

Remove A Single Local Branch

To delete a single local branch you can open your favorite Git GUI interface and just press delete on the selected branch. However, you cannot call yourself a developer if you are not able to use the terminal correctly! Switch to your favorite terminal and use the following command to delete a branch:

git branch -d YOUR_BRANCH

Unfortunately (or luckily), this command will only work if the selected branch that you want to delete is already merged. However, if you want to delete a branch that is not merged you can use the capital D:

git branch -D YOUR_BRANCH

Remove All Local Branches

When You have multiple local branches, you probably want to delete them all with one command instead of executing the delete command for every single branch. To implement a function that does this it is important to know that git branch -D can handle several files at once. With this in mind, we first write a command that finds all branches in your repository:

git branch | grep -v \*

If you execute this command in your CLI you will see that also your main and/or protected branches are included: master, main, release, ... You can adjust the find command to your needs (for me it's main/master/develop):

git branch | grep -v "main\|master\|develop"

Furthermore, you can add the --merged flag to indicate that you only want to find branches that are already merged

git branch --merged | grep -v "main\|master\|develop"

Now, you can pipe the result to the delete command introduced in the previous chapter:

git branch --merged | grep -v "main\|master\|develop" | xargs git branch -D

Executing this command in any Git repository (where main/master is the protected branch) will delete every branch that is already merged and have no local changes. You can also use -d to abort if you forgot --merged. IMPORTANT: If you execute this script while you are on a branch that has changed it does not work correctly because there will be the following error:

error: branch '*' not found.

To avoid this issue you can adjust the command by adding a \|* in the end:

git branch --merged | grep -v "main\|master\|develop\*" | xargs git branch -D

Create A Git Alias

Simple Git Aliases

Git Aliases are a feature that improves your Git experience by making your workflow simple and easy. It packs commands into an alias so you can use it wherever you want. Important aliases that you maybe already used:

• co for checkout
• ci for commit To set up these aliases you can execute:

  git config --global alias.co checkout
  git config --global alias.ci commit
  

Unfortunately, in simple Git aliases, it is not possible to use the pipe operator correctly and forward the results of a command to another one. Luckily, Git lets you shell out in aliases!

Complex Git Aliases

By using the ! (bang) operator your alias can escape to a shell and you are welcome to a new world of possibilities for all your aliases. With this technique you can use:

• Shell parameters and expansions
• Multiple git commands
• Greps, Pipes, and all Unix command-line tools that you have installed

To create a new Git alias you can use this template and add your own command:

fancy_alias = "!f() { (your complex commands here) }; f"

Within this alias, you use a simple trick by wrapping the actual git command in an anonymous bash-function (or better a function named f()). When you follow this approach you can access the command line variables and shell expansions as the following:

• $1, $2, $3 for the first, second, and third parameters passed to the Git alias
• Chain git commands with &&
• Pipe results with |
• use the entire Unix toolkit To demonstrate the functionality of Complex Git Alias You can have a look at the following 2 examples

1. Show commits since your last Git command

git config --global alias.new = !sh -c 'git log $1@{1}..$1@{0} "$@"'

Can be used by executing git new and providing the HEAD

git new HEAD

2. Add a remote to your local repository

git config --global alias.ra = "!f() { git remote add $1 $2; }; f"

Can be used by executing git ra and adding the remote name and remote URL:

git ra cool_remote git@github.com:paulknulst/nestjs-starter.git

Create An Alias To Delete Branches

Now, that you know how to create an extended Git alias we can use the previous information and wrap the command with an anonymous bash function called f():

'!f() { git branch --merged | grep -v "main\|master\|develop\|*" | xargs git branch -D; }; f'

Afterward, you can use the git config command to create a global Git alias for your current logged-in user:

git config --global alias.clean-branches '!f() { git branch --merged | grep -v "main\|master\|develop\|*" | xargs git branch -D; }; f'

Conclusion

You can have multiple local branches that you forget to delete instantly after closing a pull request. Maybe the remote branch is already deleted but your local folder keeps growing every day. Hopefully, you now know a technique to fix this problem and can use it in the CLI to completely remove every local branch that is already been merged.

Furthermore, I hope you learned an excellent way to use Git Aliases and Complex Git Alias with the ! (bang) operator to optimize your Git workflow.

If you enjoyed reading this article consider commenting your valuable thoughts in the comments section. I would love to hear your feedback about my tutorial. Furthermore, share this article with fellow developers to also optimize their Git workflow.

This article was originally published on my blog at https://www.paulsblog.dev/how-to-remove-local-git-branches-by-creating-git-aliases/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by Ujesh Krishnan / Unsplash

If you are a student or a professional, you typically work on some projects that you create in your free time for fun, for learning something new, or for your portfolio. Often, you also want that the projects could be shown to your fellow developers so that they can benefit from them. But only showing the code on GitHub is not sufficient because you want to show a running version that can be tested out by everyone before downloading your source code.

Luckily, today exist multiple services that you can use to show your work to the public. Although there are some paid options like AWS, Azure, and GCP (if you are out of the free tier), I will introduce some services that you can use without spending any money on them.

1. Github Pages⁣

GitHub Pages is a feature from GitHub that enables every user or company to host a public webpage directly from your GitHub repository. You can edit, push, and your changes will be live.

Pros of this solution:

• Widely used
• Free hosting
• Fast and easy to setup
• Great customer service
• Great documentation
• Security and reliable

Cons of this solution:

• GitHub pages are limited to static websites
• Special measures should be done for SEO
• No support for any problem. You have to rely on forums and documentation

2. Netlify

Netlify is an American cloud computing company that offers hosting and serverless backend services for web applications and static websites. Normally, you have to pay for their services but they provide a free plan for personal websites. You can build, deploy, and host any static website or app with a drag and drop interface. Also, it can automatically deploy from GitHub or Bitbucket.

Pros of this solution:

• Provides continuous integration and continuous deployment
• Drag and drop functionality is intuitive
• Customer support service is good
• Easy deploy service and use
• Support Custom domains
• SSL support
• Ability to use plugins to customize build workflow

Cons of this solution:

• Dynamic content cannot be displayed
• Supports only static websites

3. Vercel

A cloud platform for serverless deployment. It enables developers to host websites and web services that deploy instantly, scale automatically, and require no supervision, all with minimal configuration. Vercel is a tool in the Static Web Hosting category of a tech stack.

Pros of this solution:

• Simple deployment
• Easy, production-ready deployment with one command.
• Simple scaling, including auto-scaling
• SSR (Server-Side Rendering) support
• Free tier
• Free SSL

Cons of this solution:

• The platform is a bit confusing
• Lack of guides and manuals

4. Heroku

Heroku is a cloud platform with a free tier that allows up to 1000 dyno hours per month and includes deployment from Git and Docker. Normally, it is used to build, deliver, monitor, and scale all kinds of apps in different languages/frameworks: Node.js, Ruby, Python, Java, and much more.

Furthermore, with Heroku, you are allowed to use custom domains, container orchestration, and automatic OS patching.

Pros of this solution:

• Deployments with Git and Docker
• Easy to start
• Custom domains
• Free Tier
• Container orchestration
• Automatic OS patching

Cons of this solution:

• Cannot control exact configuration of applications
• Could be too overwhelming
• No SSL
• Could stop working/has some downtime

5. Firebase

Firebase from Google is primarily an app development platform that could be used as a backend for apps and games. It provides several features like Authentication, Remote configuration, A/B testing, and analytics out of the box within the free tier (and more).

Pros of this solution:

• Authentication
• Analytics
• Test Lab
• Free Tier
• Free SSL

Cons of this solution:

• Limitations of real-time database
• More complicated

Closing Notes

All of the mentioned services could be easily used to host your private project. Although some of them have more features than others you could pick the best service for a specific project. Furthermore, you can use every service if you have multiple projects that you want to show.

Use them wisely to spread your work to the public crowd!

If you enjoyed reading this article consider commenting your valuable thoughts in the comments section. I would love to hear your feedback about my tutorial. Furthermore, share this article with fellow developers to help them get hosting their own services for free!

This article was originally published on my blog at https://www.paulsblog.dev/how-to-host-your-website-for-free/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Image by 8photo on Freepik

But still: there must be a way to use the time more sensibly. But how? Let's ask the expert Ivy Lee for his advice because his method promises a lot: More productivity, better results, and at the same time less stress with almost minimal effort.

Sounds tempting, after all, most of the tasks tend to be more rather than less. To-do lists take on absurd lengths and the overview was lost some time ago. Work is being done everywhere, only to realize with horror that nothing has really been accomplished, deadlines cannot be met and the quality of the work is also suffering. The Ivy Lee Productivity Method aims to solve this problem and help you to stop working or work longer and get the important things done.

Who Was Ivy Lee and What Did He Do?

First of all, do you know Charles M. Schwab? At the beginning of the 20th century, he was one of the richest men in the world and President of Bethlehem Steel, the second-largest steel company of the time. Known initially as the world's largest shipbuilder, the steel magnate and his company pioneered the skyscraper age.

Schwab generally had a reputation for doing everything to be successful and constantly striving for improvement. It is not without reason that Thomas Edison once said of him that he is constantly looking for an advantage over his competitors. It was probably this search that prompted Schwab to hire management consultant and productivity expert Ivy Lee in 1918. He was dissatisfied with his work and the performance of his teams and sought a solution.

His task for Ivy Lee was simple: Show me how I can do more in the same amount of time. He didn't hesitate for long, but replied confidently:

I need 15 minutes with you and the executives. You can then test my method and if it works, pay me what you think is reasonable after three months.

Three months later, the two meet again and Schwab presents Ivy Lee with a check for $25,000 - the equivalent of about $400,000 today. The steel entrepreneur later said that this was the best and most profitable lesson of his life.

The enthusiasm is also reflected in the pay, which was unimaginably high for the time and hardly seems to explain given the working hours - but for Schwab it was worth every dollar.

How The Ivy Lee Productivity Method Works

And what made Ivy Lee so convincing? With the price Charles M. Schwab's paid to show his satisfaction, one could think that it must be a complex technology that enables the tasks that arise to be processed more effectively or a complex concept with which the work processes are to be optimized.

Luckily, there is nothing complicated about the Ivy Lee Productivity Method. All you need is a pen, a sheet of paper, and a few minutes. So it's no wonder that the productivity expert needed just 15 minutes to explain the method.

The whole thing can be understood and learned in just six steps.

1. Write Down Six Tasks For The Next Day

What is coming up for you the next day, which projects are pending, and which tasks have to be completed? Take a blank piece of paper and write down the six most important tasks and challenges for the coming day.

2. Arrange These Tasks In Order Of Priority

Within the second step of the Ivy Lee Productivity Method, take a close look at your task list and assign a number to each task. This number will be the priority of the task. Then put the tasks in the appropriate order so that the most important task is numbered 1 at the top.

3. Complete The First Task On The List

The next working day according to the Ivy Lee method begins with the task that has the highest priority. And without distractions, no casually checking e-mails, and also no I'll do something else first.

Concentrate fully on the most important task until it is actually completed.

4. Check The Priorities Again

Before you tackle the second task, go back to the list and see if the priorities are still right. Maybe something new has been added that urgently needs to be done or a task is no longer that important. It is important to heck the priorities again to make sure they are correct and adjust the order if necessary.

5. Get To The Next Task

The priorities are still correct or have they been put in the right order? Then it's time for the second task. According to this method, you work your way through the entire day until the end of the day: complete a task, check priorities, and move on.

6. Make A New List In The Evening

When the work day comes to an end, complete the current task and prepare the same routine for the next day. That means: Make another list of the six most important tasks - including those that are left over from today's list and are still relevant.

Pros Of The Ivy Lee Productivity Method

There are numerous tips, advice, methods, and approaches that can be used to increase productivity. Work more effectively and better - a wish of many employees and of course also of supervisors.

But what makes the Ivy Lee method so special and literally worth $400,000?

This can hardly be pinned down to a single point, but what is certain is that the method brings a whole range of advantages that speak in favor of trying out Ivy Lee's suggestion at least once to find out whether your own work benefits from it:

The Ivy Lee Productivity Method Forces Single Task Work

Multitasking is a complex, frequently discussed topic… If you try to do everything at the same time, you usually end up doing less and the results are worse. The Ivy Lee Productivity Method focuses on one task at a time. This means that every project gets the full attention it needs to perform at its best and really make progress. With this focus, problems can be identified and solved early on, instead of spending hours tinkering with them.

The Ivy Lee Productivity Method Enforces Priorities

Achieving a lot is useless if the energy is invested in the wrong places. Unfortunately, this is exactly what happens very often. It is plowed and toiled, only to realize afterward that the really important things are still unfinished.

A major advantage of the Ivy Lee method is the targeted and repeated prioritization. This is how the tasks are processed that really have to be tackled and completed.

The Ivy Lee Productivity Method Boosts Efficiency

Distractions and multitasking in particular often lead to a lot of work being done but little achieved. A little here, a little there and the day is over without anything really having happened.

Ivy Lee teaches how to use the time at work efficiently. This does not mean that no breaks are taken - quite the opposite. However, these should best be placed between the individual tasks and not prevent them from completing them.

The Ivy Lee Productivity Method Helps With Organization

The tasks and folders that still need to be looked through are now piling up on the desk, the boss brings something new every hour and you no longer know where your head is at. Organization? There is none.

With the Ivy Lee method, you automatically keep track of things and make organizing tasks a habit. By dealing with the upcoming challenges not only every evening but also between the to-dos, you always know what comes next.

The Ivy Lee Productivity Method Brings Motivation

How can fun at work arise or be maintained if the constant stress is only rewarded with the fact that the boss is dissatisfied and you yourself have doubts about your performance? Seeing you actually accomplish something, on the other hand, is a boost to motivation.

The feeling of ticking off the top priority is particularly motivating – and ideally several times a day. This gives confirmation that you are on the right track and have really achieved something that contributes to your personal and company success.

The Ivy Lee Productivity Method: So Simple And Yet So Good

One of the most frequently asked questions about the Ivy Lee Productivity Method is: It's so simple, how is it supposed to work?

True to the motto: If it's not complicated, it can't be really good either. After all, countless scientists are dealing with this topic and each one gives a different piece of advice that should solve all productivity problems.

Ironically, the simple Ivy Lee method should do particularly well? Yes, exactly that. And for good reasons:

The Ivy Lee Productivity Method Is Easy To Use

The simplicity is perhaps the strongest reason that the Ivy Lee method works. It's practical. Anyone can use them and, most importantly, maintain them without having to force themselves into strenuous or time-consuming procedures.

The Ivy Lee Productivity Method Creates Clarity

Many methods want to explain how you can complete many or, if possible, all tasks satisfactorily in a certain amount of time. The Ivy Lee Productivity Method method is different. It's not about doing everything on the list, it's about understanding what's really important and needs to be done.

The Ivy Lee Productivity Method Makes It Easy To Get Started On A Daily Basis

A major obstacle to the productivity of many employees is the start of the day. First, make coffee, read e-mails, set up at the desk, and see what needs to be done. It usually takes some time before things really get going. Thanks to the Ivy Lee method, however, you already know the evening before what the next day will be like - so you can start right away.

Conclusion

This method looks almost too simple to qualify as a specific productivity method. It just orders all tasks by priority in the evening so that you know what to do first in the morning. But this simplicity in particular makes the Ivy Lee Method so attractive and applicable to everyone. And Charles M. Schwab pays voluntarily $25.000 (the equivalent of about $400,000 today) to Ivy Lee for this simple advice.

My personal advice to you: Try it!

It would be the best way to find out if it is right for you. Try it for a week or two and if you do not see any results in your productivity you can still try another method. Luckily, the Ivy Lee Productivity Method is unbelievably simple so you don't waste time "learning" it.

If you enjoyed reading this article consider commenting your valuable thoughts in the comments section. I would love to hear your feedback about the Ivy-Lee Method. Furthermore, share this article with your co-workers and friends to increase their productivity too.

This post was initially posted on my Personal blog: https://www.paulsblog.dev/increase-your-productivity-with-the-ivy-lee-method/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by Andreas Klassen / Unsplash

Having comments on your blog is one of the most engaging features! Unfortunately, Ghost Blogging Service does not support any comments out of the box. Although they are many different services like Disqus or Discourse which are kind of "free" (they have inbuilt Ads in the "free" version. And Ads are EVIL) there also exist some really free services like Commento, Schnack, CoralProject Talk, and Isso.

I tested all of the free services for my blog and come to the conclusion that Isso is the best service to use in a Ghost Blog. Within this article, I will describe why Isso is the best software, how it can be installed in a Docker environment, and how Isso comments can be integrated into any Blog (not only Ghost Blogging software).

Why Isso?

Isso is a commenting server written in Python and JavaScript and aims to be a drop-in replacement for Disqus or Discourse.

It has several features:

• It is a very lightweight commenting system
• Works with Docker Compose
• Uses SQLite because comments are not Big Data!
• A very minimal commenting system with a simple moderation system
• Privacy-first commenting system
• Supports Markdown
• It's free
• Similar to native WordPress comments
• You can Import WordPress or Disqus
• Embed it everywhere in a single JS file; 65kB (20kB gzipped)

All of these features are good reasons to choose Isso as your backend comment system instead of one of the others. If you want to install it in a Docker (or Docker Swarm) environment you can follow my personal guide.

Install Isso Comments With Docker

Prerequisite

• Docker (optional Docker Swarm): To fully follow this tutorial about installing Isso for your blog you need to have a running Docker environment. I will also provide a Docker Swarm file at the end.
• Traefik: Traefik is the load balancer that forwards my request to the Docker container. You need to have one installed to access the comments with an URL. If you do not have a running Traefik you will learn within this tutorial how you can set one up.

If you don't want to use Docker and Traefik please follow the official Installation Guide on the Isso website.

Set Up Docker Compose File

If you prepared everything you start installing Isso. First, you have to download the latest version of Isso from the GitHub page: https://github.com/posativ/isso/. You can either download the zip file containing the master branch and extract it or clone it:

git clone git@github.com:posativ/isso.git

Then switch into the isso folder, delete the docker-compose.yml, create a new docker-compose.yml, and paste the following code snippet into it:

version: "3.4"

services:
  isso:
    build:
      context: .
      dockerfile: Dockerfile
    image: isso
    environment:
      - GID=1000
      - UID=1000
    volumes:
      - db:/db
    networks:
      - traefik-public
    labels:
      - traefik.enable=true
      - traefik.docker.network=traefik-public
      - traefik.constraint-label=traefik-public
      - traefik.http.routers.isso-http.rule=Host(`YOUR_DOMAIN`)
      - traefik.http.routers.isso-http.entrypoints=http
      - traefik.http.routers.isso-http.middlewares=https-redirect
      - traefik.http.routers.isso-https.rule=Host(`YOUR_DOMAIN`)
      - traefik.http.routers.isso-https.entrypoints=https
      - traefik.http.routers.isso-https.tls=true
      - traefik.http.routers.isso-https.tls.certresolver=le
      - traefik.http.services.isso.loadbalancer.server.port=8080
    healthcheck:
      test: wget --no-verbose --tries=1 --spider http://localhost:8080/info || exit 1
      interval: 5s  # short timeout needed during start phase
      retries: 3
      start_period: 5s
      timeout: 3s
volumes:
  db:

networks:
  traefik-public:
    external: true

Within Line 5 - 8 you can see within this Compose file that the Isso image will be built from the Dockerfile that you downloaded from the official GitHub repository and will be named isso.

In Line 12-13 (and 34-35), I defined a volume in which all comments will be saved within an SQLite DB.

The labels section in Line 16 - 27 contains important information for Traefik. Replace YOUR_DOMAIN with the domain where the Isso backend should be accessible. This is important for the moderation of comments.

In Line 28 - 33, I also added a health check that will check every 5s if the Isso backend is running correctly. To learn more about Docker health checks you should read this tutorial:

https://www.paulsblog.dev/how-to-successfully-implement-a-healthcheck-in-docker-compose/

Configuration of Isso Backend

Before you can run your Isso backend you have to adjust the Isso configuration file to your needs.

The following file shows the most important settings that you HAVE to update in order to have a working Isso backend. I will explain them afterward.

[general]
dbpath = /db/comments.db
host =
    https://www.knulst.de
notify = smtp

[admin]
enabled = true
password = yourSuperSafePasswordThatYouNeedInAdminMenu

[moderation]
enabled = true

[server]
listen = http://localhost:8080
public-endpoint = https://isso.knulst.de

[smtp]
username = YOUR_EMAIL_USER
password = YOUR_EMAIL_USERPASS
host = YOUR_EMAIL_HOST
port = 587
security = starttls
to = USER_WHO_SHOULD_RECEIVE_MAIL
from = YOUR_EMAIL_USER

[general]: Set the dbpath, the URL of your blog, and that you want to get Emails if new comments are submitted

[admin]: Enable administration and set your admin password. This is needed to login into the admin interface.

[moderation]: Enable moderation of new comments. If set to false every comment will be shown instantly. Not recommended!

[server]: Set your public endpoint for your Isso backend. Should be equal to the value within the Compose file.

[smtp]: Set your Email server settings and account data of your Email.

Run Isso Backend

After adjusting the Compose and isso.cfg file to your needs, you can start the Isso backend by executing:

docker-compose up -d --build

After some seconds your Isso backend should be running at the specified domain. You can check it by going to https://your-chosen-domain.com/info.

Integrate Isso To Ghost CMS (or any equivalent Blog)

Adjust The Source Code Snippet

Now that you have installed your Isso backend you can integrate comments into your blog. To do this, all you have to do is insert the following snippet in your source code. I will explain how it is done in Ghost Blogging software afterward.

<script data-isso="//isso.knulst.de/" src="//isso.knulst.de/js/embed.min.js"></script>
<section id="isso-thread"></section>

First of all replace isso.knulst.de with the domain, you used with your Isso bac
kend. Furthermore, you should carefully read the source code snippet because there are two important settings:

1. The URL in data-isso starts with two /.
2. The URL in src also starts with two /.

If you change the URLs to your needs keep the two / in front of the Isso backend URL. This is needed because the Isso backend is deployed on a different host than the blog.

Add Snippet To Your Page

With the adjusted source code snippet you can now add comments to your blog. If you use the Ghost Blogging platform you should first download the design that you are using on your instance and extract it into a folder.

Locate the post.hbs file and check where the article ends. The best place for comments will be below the Tag List that is often shown at the end of an article. After you found the closing </div> insert the snippet in front of that </div>.

Zip your theme, reupload it to your Ghost Blogging Software, and activate it within the Design interface.

If you use another blogging software you should also be able to edit the post html/php/js file and add the snippet where you want to see comments.

Now, switch to any article on your blog and you should see the comments section:

If you set up the Isso backend as I explained in the previous section anyone can now comment on every post and you will be informed by email about a new comment and can instantly activate or delete it within the email:

After getting some cool comments they will be displayed below your post. Because activated the Gravatar feature and set the type to robohash, I have cool robot pictures for every user that creates a comment:

Deploy In Docker Swarm

To deploy Isso into your Docker Swarm you can use my Isso Docker Compose file that has the same settings as the plain Compose file. But, this will only work as intended if you set up your Docker Swarm as I describe in this tutorial:

https://www.paulsblog.dev/docker-swarm-in-a-nutshell/

Furthermore, you need a Traefik Load Balancer in your Docker Swarm that will be used by the Compose file. I explained how to enhance your Docker Swarm with a Traefik in this tutorial:

https://www.paulsblog.dev/services-you-want-to-have-in-a-swarm-environment/#traefik

Closing Notes

While many writers argue that you do not need a comment system integrated into your blog I completely deny this argument! I personally think that comments are a great way to interact with your users, answer questions and build your community. Also, if users comment on your articles you could use this interaction to improve your content.

If you enjoyed reading this article consider commenting your valuable thoughts in the comments section. I would love to hear your feedback about my tutorial. Furthermore, share this article with fellow bloggers to help them get comments on their blogs.

This article was originally published on my blog at https://www.paulsblog.dev/how-to-add-comments-to-your-blog-with-docker/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by Volodymyr Hryshchenko / Unsplash

1. Mantine

Mantine is a fully-featured React components library that can be used to build fully functional accessible web applications. Furthermore, it includes several customizable React components and hooks with native Dark mode support to cover you in any situation. It is focused on usability, accessibility, and developer experience. Mantine is TypeScript based.

Find it on GitHub

2. react-easy-crop

react-easy-crop can be used to crop images/videos with easy interactions. It supports drag, zoom, and rotating images in any format or base64 string. Also, it supports every video format that is supported in HTML5.

Furthermore, react-easy-crop is a mobile-friendly UI component.

Find it on GitHub

3. react-colorful

react-colorful provides a tiny (only 2.8 KB gzipped) color picker component for React apps (also Preact). With absolute no dependencies, it can still be used to have a cross-browser workable, mobile-friendly color picker that is built with hooks and functional components only. Also, the color picker is written in TypeScript and has all types included.

Find it on GitHub

4. React Syntax Highlighter

React Syntax Highlighter is a React UI component that enables syntax highlighting in any React app by integrating/combining two popular projects: lowlight and refractor.

Highlighting the code is done with Prism.js and highlight.js by using inline styles instead of altering the DOM manually or using dangerouslySetInnerHTML.

Find it on GitHub

5. TipTap

TipTap is a headless, framework-agnostic rich text editor that is extendable and based on ProseMirror. It enables full control over every aspect of a text editor experience. Also, it is very customizable, comes with a lot of extensions, and is fully documented.

Furthermore, TipTap works with Vanilla JavaScript and is also integrated into several TypeScript-based frameworks (React, Next.js, Vue, etc…).

Find it on GitHub

Closing Notes

I hope you like these UI components and will use them in your next React project.

Also, if you have any questions, ideas, recommendations, or want to share your own awesome UI component, please jot them down below. I try to answer your question if possible and will test your recommendations.

This article was originally published on my blog at https://www.paulsblog.dev/advanced-react-ui-components-to-optimize-development-process/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by Lautaro Andreani / Unsplash

It doesn't care if you working with backend or frontend (or even SPACESHIPS!), JavaScript could be used everywhere because it is a quite flexible language. It has hardcore functional programming patterns as well as classes and its similarity to other "C-like" languages enables an easy transition for every developer.

If you want to level up your skills in JavaScript, I would suggest learning about, practicing, and mastering the following very important core functions that are available in the language. You will not need every function for solving most problems but they can help you to avoid heavy lifting in some cases, while in others, they will reduce the amount of code that you have to write.

1. map()

One of the most important JavaScript functions is map()! Also, it is one function that gives the most trouble to people who learn JavaScript. This happens because the way this function works is an idea taken from Functional Programming which most developers are not familiar with. It seems strange or even wrong to our biased brains.

The function map() is very simple. It attaches itself to an array and converts each item into something else, resulting in a new array. This conversion is done by an anonymous function provided in the map brackets.

That's all that map does!

At first, the syntax for map() might take some time to get used to but after mastering it will be one of your best friends working with JavaScript.

Why You Might Want To Use map()?

For instance, let's say we recorded the amount of water in a fountain for each day of the last week and stored it in an array. At the end of the week, you are told that the measurement tool was not as accurate as possible and the amount of water was 3.2 liters less than it should be.

With the map() function you can easily solve this issue by doing this:

const dailyReadings = [34.2, 35.4, 40.1, 35.2, 33.5, 36.5, 38.7];

const correctedDailyReadings = dailyReadings.map(day => day + 3.2);

console.log(correctedDailyReadings); //gives [37.4, 38.6, 43.3, 38.4, 36.7, 39.7, 41.9]

You can find another very practical example in the world of React when you create DOM element lists from arrays. This is a common pattern in React and will be achieved by this piece of code:

export default({articles}) => {
    return links.map(article => {
        return (
            <div className="link" key={article.id}>
                <div className="article-name">{article.name}</div>
                <div className="article-desc">{article.description}</div>
            </div>
        );
    });
};

Now you can argue that map() is nothing but another implementation of a for loop and you will be right but notice that maybe this is your object-oriented trained mind that makes this argument.

2. filter()

filter() is an unbelievable useful JavaScript function that you can use in many situations because it filters an array based on a given rule/logic. This rule/logic will be provided as anonymous functions.

To show how filter() is working we can reuse the previous fountain example. Assume that the array contains the amount of water stored within the fountain (This time measurement tool works as expected). Now, we want to find out how often the amount was lower than 36 liters. This can be achieved using the filter() function:

const dailyReadings = [34.2, 35.4, 40.1, 35.2, 33.5, 36.5, 38.7];

const daysWithLowAmount = dailyReadings.filter(day => {
    return day < 36
});

console.log("Days with low amount of water: " + daysWithLowAmount.length); // 4
console.log(daysWithLowAmount); // [34.2, 35.4, 35.2, 33.5]

It is important that the anonymous function that you pass to filter() has to return a Boolean value: true or false. The Boolean expression is used by the filter() function to determine if a value should be inserted in the newly generated array or not.

Within the anonymous function, you can implement any amount of complex logic to filter the data; you can read user inputs, make any API call, and so on, as long as you return a Boolean value in the end.

BEWARE: Keep in mind that filter() always returns an array. This is also the case if the filtered data is empty. Often this leads to subtle bugs within code snippets because filter() is used this way:

const dailyReadings = [34.2, 35.4, 40.1, 35.2, 33.5, 36.5, 38.7];

const daysWithCriticalAmount = dailyReadings.filter(day => {
    return day < 30
});

if(daysWithCriticalAmount) {
    console.log("The amount of water was critical");
} else {
    console.log("The amount of water was normal");
}

Did you notice something? The if condition in the end checks daysWithCriticalAmount which is actually an array that is empty because there was no day with less water than 30.

Surprisingly, this mistake is done too often if you are rushing toward a deadline and just want to implement a filter method fast. The problem with this piece of code is that especially JavaScript (Yeah, JavaScript is weird) is an inconsistent language in many ways. Sometimes the "truthiness" of things is one of them.

As you maybe know in JavaScript [] == true returns false and you may think that the above code is correct.

Unfortunately, within an if condition, [] evaluates to true! In other words, the else block will never be executed.

Fixing this problem is very easy because you only have to check if the length of the resulting array is greater or equal to 1.

3. reduce()

Compared to all the other functions in this article reduce() is competing for the first place of "confusing, weird and complex JavaScript functions". Although this function is highly important and results in elegant code in many situations, it is avoided by many JavaScript developers. They prefer to write code without the help of reduce().

One reason for this is the complexity of reduce(). It is very hard to understand the concept and the execution of this function. Furthermore, if you read its description, you have to re-read it several times, and still, you doubt yourself if you read it wrong (Happens to me until I saw it in action).

The reduce() function is used to reduce (obvious isn't it?) the given array to a single value. This could be a number, a string, a function, an object, or whatever else.

If you want to use the reduce() function to reduce an array you need to supply the needed logic by providing a function. In general, this function will be called the reducer function that transforms the first argument to reduce(). Also, it contains a second argument that is a starting value and can be a number, a string, etc. The function itself will look like this:

array.reduce(reducerFunction, startValue)

startValue: The startValue passed to reduce() function is the starting value (obvious...) for the calculation you want to process with the reducerFunction. For example, if you want to execute an addition you might start with 0 and if you want to do a multiplication you should use a value of 1.

reducerFunction: As mentioned before the reducer function is used to convert the array into a single value. It has two arguments: an accumulator and a variable to store the current value.

The accumulator is just a fancy name describing the variable that contains the result of the calculation (it is like using total to sum up all items in an array). Within the reducer function, the accumulator will be initially set to the starting value and then the calculation will be performed step by step iterating through the array while the result of each step is saved within the accumulator.

The second argument of the reducer function is currentValue and will contain the value within the array at a given position in a specific step. For example, in step 2 while executing the reducer function the variable currentValue will contain the second value from the input array.

With all this information we should look at the following example which will add every value from an input array within the reducer function:

const input = [1, 2, 3, 4];
const result = input.reduce((acc, currentValue) => acc + currentValue, 0);
console.log(result); // 10

Within the first line, we define the input array that's holding all the numbers we want to add. The next line will contain the input.reduce() call, which defines the starting value for acc to be 0 because in an addition we should start with 0 to not influence the result. The reducer functions body (acc, currentValue) => acc + currentValue does a simple addition of the current value and the value that is stored in the accumulator.

If you would do this addition with a for loop it would look like this:

const input = [1, 2, 3, 4];
let result = 0;
for (let i = 0; i < input.length; i++) {
    result += input[i]
}
console.log(total)

As you can see the simple for loop example is much longer and not as elegant as using reduce.

4. some()

Imagine you have an array of objects containing Students and their scores in an exam. Now, you want to know if there are students in your class that have a score that is greater than 90% and it does not care if there is one or multiple students that passed that exam with more than 90%.

If you want to do this in JavaScript you can solve the problem while using a loop and a variable that is set to true after the first person is found that has a score greater 90%:

const students = [{
    name: "Stud 1",
    score: 78,
}, {
    name: "Stud 2",
    score: 92
}, {
    name: "Stud 3",
    score: 90
}]

let over90 = false;

for(let i = 0; i < students.length; i++) {
    if(students[i].score > 90) {
        over90 = true;
        break;
    }
}

if(over90) {
    console.log("There are students with exceptional score!")
} else {
    console.log("Unfortunately, nothing is exceptional")
}

For me, this code looks very "ugly", "verbose", or "horrible"!

It could be optimized by using the map() function that was described earlier but the solution will still be a little bit clunky.

Luckily, JavaScript has a rather neat function called some() that is available in the core language, works with arrays, and returns a Boolean value. While providing an anonymous function it will filter the input array to solve the problem elegantly:

const students = [{
    name: "Stud 1",
    score: 78,
}, {
    name: "Stud 2",
    score: 92
}, {
    name: "Stud 3",
    score: 90
}]

if(students.some(student => {
    return student.score > 90
})) {
    console.log("There are students with exceptional score!")
} else {
    console.log("Unfortunately, nothing is exceptional")
}

You can see it gets the same input and will produce the same result but in a more elegant way. Also, the amount of code is reduced and it is much more readable than before.

5. every()

Another function that already exists in the JavaScript core language is every(). This function will return a Boolean value (like some()) but will test all items within the input array. To do this you have to provide (again) an anonymous function. As an example, we will test if every student has passed the exam by achieving an exam score of 70 or more:

const students = [{
    name: "Stud 1",
    score: 78,
}, {
    name: "Stud 2",
    score: 92
}, {
    name: "Stud 3",
    score: 90
}]

if(students.every(student => {
    return student.score > 70
})) {
    console.log("All students passed the exam!")
} else {
    console.log("Unfortunately, not all students did well")
}

Like some() this function creates elegant, readable, and maintainable code that is short.

6. includes()

Often, if I want to check an array of the existence of substrings I will use indexOf(). Unfortunately, I have to look up the docs to know how their return values are.

Luckily, there is a nice alternative in JavaScript that can be used to achieve this: includes(). The usage is very easy and it is case-sensitive (I guess you wished for it anyway). See the following code snippet for an example:

const numbers = [6, 5, 13, 26, 48, 1];
console.log(numbers.includes(5)); // true

const name = "Paul Knulst";
console.log(name.includes('paul')); // false, because first letter is in small
console.log(name.includes('Paul')); // true, as expected

However, includes can only compare simple values and no objects:

const user = {a: 6, b: 5, c: 26};
console.log(user.includes('b')); // does not work because objects does not have an "includes" method

7. shift()

If you want to remove the first element of an array shift() is the method you should use because it is easy to remember and intuitive. Notice that you also can do it with splice (see it later) but shift() is easier:

const input = [1, 2, 3, 4];
const first = input.shift();
console.log(input); // gives: Array [2, 3, 4]
console.log(first); // gives: 1

As you can see here shift alters the array it is called on and returns the removed element.

Notice: shift is extremely inefficient and should be avoided if you are working with large arrays. Too many calls of shift() on large arrays can break your applications!

8. unshift()

In contrast to shift() which removes one element this function is used to add a new element at the beginning of the array:

const input = [1, 2, 3, 4];
input.unshift(0);
console.log(input); // expected output: Array [0, 1, 2, 3, 4]

Notice: Like shift, unshift is extremely inefficient and should be avoided if you are working with large arrays. Too many calls of unshift() on large arrays can break your applications!

9. slice()

In JavaScript exist a concept that is called slicing which is a function to create a new string containing a part of an original string by providing a starting index and an end index. Now, with help of the indexOf function you can slice an input string to retrieve the searched substring:

const input = "This is a cool article about mandatory JavaScript functions that every developer shoud know";
const start = input.indexOf("cool");
const end = input.indexOf("that");
const headline = input.slice(start, end);
console.log(headline); // "cool article about mandatory JavaScript functions"

Notice that the start index is included in the resulting array but the end index is not. Also, you can see that the space between "functions" and "that" is also not sliced.

Additionally, slicing can also be used with arrays:

const animals = ['ant', 'bison', 'camel', 'duck', 'elephant'];

console.log(animals.slice(2)); // ["camel", "duck", "elephant"]
console.log(animals.slice(2, 4)); // Array ["camel", "duck"]

10. splice()

splice() sounds like slice() and could be compared with it because both functions create new arrays or strings from the original ones. The main small but very important difference is that splice() removes, changes, or adds elements but modifies the original array. This "destructuring" of the original array can lead to huge problems if you do not fully understand how deep copies and references work in JavaScript!

The following example will show how splice() can be used:

const months = ['Jan', 'March', 'April', 'June'];
const feb = months.splice(1, 0, 'Feb'); // inserts at index 1
console.log(months); // ["Jan", "Feb", "March", "April", "June"]
console.log(feb); // []

const may = months.splice(4, 1, 'May'); // replaces 1 element at index 4
console.log(months); // ["Jan", "Feb", "March", "April", "May"]
console.log(may); // ["June"]

You can see here that splice() manipulates the original array. If you remove an item it will also return the removed entry but if you add an entry the return array will be empty.

11. fill()

The last JavaScript function is fill() and is used to change several items to a single value or even reset the full array to their initial values. If you need this fill() can help you avoid loops to reset the values of an array. Furthermore, the function can be used to replace some or all entries from the array with a given value.

You can see how it works within the following example:

const input = [1, 2, 3, 4, 5, 6];
console.log(input.fill(0, 2, 4)); // [1, 2, 0, 0, 0, 0]
console.log(input.fill(5, 1)); // [1, 5, 5, 5, 5, 5]
console.log(input.fill(6)); // [6, 6, 6, 6, 6, 6]

Conclusion

This list contains functions that most JavaScript developers encounter and use in their careers. Although it is long, it is by no means complete because there are so many more minor but very useful functions in JavaScript: reverse(), sort(), find(), flat(), entries(). I would suggest that you read about these functions because you will at any point use them in your JavaScript project.

Furthermore, whenever you can you should explore the core language or famous utility libraries such as lodash to deepen your JavaScript knowledge. It will result in massive productivity and you will learn to create cleaner, compact, readable, maintainable, and usable code!

This article was originally published on my blog at https://www.paulsblog.dev/introduction-to-11-core-javascript-functions-to-improve-code-quality/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by Kyle Glenn / Unsplash

Paul Knulst in Programming • 7 min read

Last year, I have received multiple messages, mostly from newcomers that have just graduated and stepped into the world of coding. Most of the questions asked are about how to quickly improve their skills, how to become a Full Stack Developer, or how to choose a career direction.

While I try replying to everyone by one, I want to combine my own experiences and created a roadmap on how to become a Full-Stack Engineer that I want to share with everyone.

Stage 1  - HTML

Before starting with any programming language you should learn basic HTML. A good start will be the tutorial of W3Schools that teaches all the basics and also some advanced information: find it here

Stage 2  -  CSS (Grid, Flex)

With knowledge about HTML, you should learn about CSS to style your website. Again, W3Schools provides a good tutorial: find it here

Stage 3  -  JavaScript + DOM

Now it's time to learn JavaScript. JavaScript is one of the most used programming languages all over the world. To learn it you can use another W3Schools tutorial.

Also, I have written an article to learn advanced JavaScript with Gamification.

• On my blog: https://www.paulsblog.dev/learn-javascript-while-playing-games-gamify-your-learning/

• On Medium: https://medium.com/javascript-in-plain-english/learn-javascript-while-playing-games-gamify-your-learning-9f41d4d3dad9

Stage 4  -  Node.js

Node.js is a powerful JavaScript-based platform that is built on Google Chrome’s JavaScript V8 Engine and provides an event-driven, non-blocking (asynchronous) I/O and cross-platform runtime environment for building highly scalable server-side applications using JavaScript.

Start by downloading and installing the latest Node version for your OS on nodejs.dev. This is mandatory for further progression as a JavaScript (TypeScript) developer.

Afterward, you should check out their introduction to learn some basics about Node.js.

Before heading to the next stage you should make yourself familiar with the npm ecosystem. Consider answering these questions before starting with React:

• What is NPM?

• How can I install something?

• What is a package.json

• I need package XX what should I do?

Stage 5 - React

If you are proficient with JavaScript (and maybe TypeScript) you should learn React because it is the most used framework based on data from npmtrends.com:

Remember: Mastering React takes some time because it has many different concepts that all become important during development.

My personal advice would be to work on the three following lists step by step before continuing to MySQL:

Basics

1. Understand React

2. How To Set Up Your DevEnv

3. JSX

4. Components

5. State

6. Props

7. Lists/Keys

8. Lifecycle Methods

Advanced React Concepts

1. Styling

2. Form Handling

3. Data Handling

4. Reconciliation Process

5. Hooks

6. Custom Hooks

7. Context

Mastering React

1. Lazy Loading

2. Portals

3. State Management

4. Routing

5. Theming

6. Patterns

7. Anti-Patterns

This article from me here on dev.to has also come links for becoming a React developer

Stage 6  -  MySQL

After you mastered JavaScript and React to create beautiful websites the next step will be to use persistent data within these websites. Often, this is done by connecting to a database like MySQL, PostgreSQL, or SQLite.

For simplicity, you should learn MySQL over PostgreSQL!

As a developer, you have to be familiar with MySQL because it is ideal for both small and large applications (PostgreSQL is an advancement in DBMS but MySQL is basic knowledge).

To learn MySQL you can use a W3School Tutorial which will teach you all the basics to understand every MySQL query you will encounter in different applications.

One important thing is, that you have to understand database design more than queries because in future stages you will use a technique (ORM) that helps you with queries anyway. For this, you have to understand Database Normalization. This is very important if you want to work with relational databases!

Stage 7  -  MongoDB

In contrast to MySQL, MongoDB is a document-oriented database classified as NoSQL. It uses JSON-like documents with optional schemas to store data.

Get familiar with NoSQL database design and read and understand this.

Stage 8  -  The CRUD Pattern

If you are working with persistent data it is mandatory that you understand the CRUD pattern. CRUD stands for Create, Read, Update, Delete and describes the four basic operations for persistent data.

It should be sufficient to read about it on Wikipedia.

Stage 9 — ORM And ODM

ORM stands for Object-Relational Mapping; ODM stands for Object Document Mapper. Both are techniques that are used to query and manipulate data from a database using an object-oriented paradigm.

Often, when developers talk about ORM or ODM they refer to a library that implements any (or both) of these techniques.

You should learn at least one of the following supporting both Database technologies:

• TypeOrm (SQL and NoSQL)

• Prisma (SQL and MongoDB)

• Mongoose (only MongoDB)

• Typegoose (Mongoose for TypeScript)

• Knex.js (SQL)

• sequelize (SQL)

Personally, I would recommend learning TypeOrm AND Prisma!

Stage 10 - Build Side Projects To Gain Experience

Now that you know a programming language, a framework, and how to use a database you can start creating some projects.

This stage is very important in becoming a full-stack developer because you never encounter real-life problems in tutorials that you will face while working on your own project.

Here are some ideas for cool projects:

Goal Tracking App

Simple app that tracks your personal goals. Save goals in a database and show them in a nice way in the front end.

Clone A Website

Search for a nice website and try to clone it. Copy the contents and try to extrapolate the functions that the website uses. Then try to implement it for your own website.

Developer Portfolio

A very important project that you should create is a portfolio. A Portfolio is an important website that will represent yourself. If you apply for a job you will have a nice-looking website that will show your skills or introduce yourself.

Get inspiration here:

Stage 11  -  Deploy Your Side Project

The last stage is about deploying your project on a server. This stage is also mandatory because as a full-stack developer, you should know how to deploy your app in a production environment.

I would recommend learning about Docker and how it can be used to host different websites. As a starting point, you can check out an introduction from me to Docker here. If you are already into Docker you can check out this article where I explain how to deploy Traefik (a load balancer) that automatically creates SSL certificates for every service you will deploy on your server.

If you have read and understood both articles you can check out this article:

It describes how you can deploy a simple website in a Docker environment that already runs a Traefik.

Congratulations, you can call yourself a Full Stack developer!

Closing Notes

Hopefully, these resources will help you to become a Full Stack Developer.

Also, if you have any questions, ideas, or recommendations, please jot them down below. I try to answer them if possible.

Writing has always been my passion and it gives me pleasure to help and inspire people.

This article was originally published on my blog at https://www.paulsblog.dev/11-stages-to-become-a-javascript-full-stack-engineer/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by Caspar Camille Rubin / Unsplash

Within this article, I want to showcase different websites you can use to learn JavaScript while playing games. This method is called gamification and is a well-known technique nowadays.

Introduction

Often if you learn a new technology or language it could happen that you cannot keep your momentum. This is caused by the amount of technology that seems endless. It can get difficult to stay engaged with particularly complicated technology and learning gets stuck.

Gamification is a good solution to this problem. It uses a simple strategic attempt to motivate and engage users while learning something new. It is a technique of adding typical design elements from games to enhance the learning process. This is done by leveraging people’s natural desire for socializing, learning, mastery, competition, achievement, status, or self-expression. Early implementation of Gamification uses a simple reward system for players after they accomplish their tasks to engage them. The rewards include point scoring, achievement badges, or virtual currency to use.

Another approach of Gamification transforms the task itself in a game. This is accomplished by including a meaningful choice, an onboarding tutorial, or adding a narrative.

Within the next chapter, I will show different websites that can all be used to learn JavaScript by playing games or solving puzzles.

CheckiO

CheckiO is a website that teaches programming by providing an online game world with many Gamification features like point scoring, leaderboard, or socialization. It helps you to improve your coding skills. Also, it contains many coding games for beginners and advanced programmers. You can complete exercises using Python or TypeScript that cover a variety of topics, including strings, loops, objects, classes, exceptions, and problem-solving. After completing challenges you will earn points, unlock new games, and advance to higher levels. Furthermore, there is a comment section and a Forum where you can consult the help or see how other users solved the puzzle.

Here is a link to the website:

CodeCombat

CodeCombat is a hack-and-slash fantasy game that could be used to learn coding fundamentals. CodeCombat puts the player into action and uses programming to help you survive in a dungeon or battlefield. You will control your personal on-screen character through every challenge.

It contains hundreds of levels throughout different courses, including computer science, game development, and web development.

Within the game, you have the ability to control your personally created character and let him run down a hallway to taunt an ogre through programming.

Every level is set up like a puzzle and could only be solved by typing the correct code snippet. Also, after you put in a snippet the game will be executed and you will see the outcome directly.

Here is a link to the website:

TwilioQuest

In TwilioQuest, you will lead your intrepid crew on a mission to The Cloud. It is a free PC role-playing game that was inspired by the classics of the 16-bit era.

Furthermore, it is an educational game that was designed to teach programming in JavaScript or Python to new developers. The game will prepare you for real-world problems by helping you configure a local development environment and explaining tools that are used by professional programmers.

You will learn how to use your terminal, learn how to code in Python or JavaScript, and contribute to Open Source projects. All these practical software engineering skills will be covered if you play TwilioQuest

Here is a link to the website:

Elevator Saga

This is a game of programming! Your task is to program the movement of elevators, by writing a program in JavaScript.

Elevator Saga is a game where you have to use JavaScript to transport people with an elevator in an efficient manner. While progressing through the different stages you have to complete even more difficult challenges.

Only efficient programs will be able to complete all challenges.

Here is a link to the website: https://play.elevatorsaga.com

jsdares

At jsdares.com you will learn JavaScript programming by completing “dares”. These dares are short puzzles in which you minimize the number of functions used to complete the task. They are very simple in the beginning but will become more difficult as you progress.

At the moment jsdares only provide a small number of dares but they are working on a greater collection to start with. Also, you have to opportunity to create and share your own dares.

Here is a link to the website: https://jsdares.com

WarriorJS

Legend tells of a legendary sword, forgotten in the ruins of an abandoned tower. Thousands of warriors have set off on a quest for the sword, whose bearer would become enlightened in the JavaScript language.

WarriorJS is a learning platform for JavaScript that teaches you JavaScript while you playing a role-playing game. This game is designed for new or advanced JavaScript programmers and will put your skills to the test!

Here is a link to the website:

JSRobot

JSRobot is a game where you will control a robot that collects coins and should reach the end of a level. All programming is done in JavaScript and every challenge contains information about all JavaScript functions needed to complete it.

Unfortunately, it is a very simple game without a fancy UI and presentation. But it can still be used to practice your coding skills.

Here is a link to the website: https://lab.reaal.me/jsrobot/#level=1&language=en

Closing Notes

If you try any of these games they could help you to start learning JavaScript and becoming a JavaScript developer. They are encouraging funny and have a lot to offer. Besides, you will get in touch with programming and will gain deeper knowledge about syntax, algorithms, and other important concepts in programming.

Personally, I would recommend using CheckiO, TwilioQuest, CodeCombat, and WarriorJS. They are especially helpful for people who want to work as a developer.

JSRobot, ElevatorSaga, and jsdares are good to practice some basic functionality and should not be avoided but I won’t recommend starting with these games.

I hope you will test any of these sites and will find them as helpful as I have. If so I would love to hear your thoughts. Also, if you have any questions, ideas, or recommendations, please jot them down below. I try to answer them if possible.

Additionally, I would love to hear your thoughts and your feedback about these Gamification resources. Furthermore, share this article with your friends and colleagues to also learn JavaScript by Gamification.

This article was originally published on my blog at https://www.paulsblog.dev/learn-javascript-while-playing-games-gamify-your-learning/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo: Playing game vector created by pikisuperstar / www.freepik.com

What is Docker?

Docker is a platform that is used for developing, shipping, and running all kinds of applications. It enables the separation of applications from the infrastructure so that software can be delivered without concerns about the operating system of the host or the end user.

To install Docker on your machine download the installation package from the official docker page here: https://docs.docker.com/get-docker/

To work with Docker you should get familiar with the following keywords/concepts: Container, Dockerfiles, Docker-Images, Docker-Compose

Container

A container describes a unit of software that merges code and every dependency to run an application. They provide a quick and safe way to run an application on several different devices. Also, it secures the application because of different isolation capabilities.

Docker Images

A Docker image is a read-only template that is used for creating the container to run the application within the Docker platform. It packages up applications and preconfigured server environments to create the container which will be deployed. On the official DockerHub several already configured images are available to use. You can use the search function to find any image and deploy it on your system with:

    $ docker run -it "NAME_OF_IMAGE_FROM_DOCKERHUB"

Dockerfiles

A Dockerfile is a plain text document that contains every command to set up an image. It contains the base image to use (operating system) and the additional software that should be installed on the resulting image.

Every time custom content is used for a Docker container a Dockerfile has to be created and filled with the missing information.

As an example, the following two snippets will demonstrate the usage of a Dockerfile. The first one will extend an NGINX image to serve a simple HTML file that is shown in the second snippet.

    FROM nginx
    COPY index.html /usr/share/nginx/html

    <html>
    <h1>How To Deploy Docker</h1>
      <p>Learning IT. Level Up.</p>
      <p>This is a simple static website, built and deliver from a docker container</p>
      <p><a href="https://www.paulsblog.dev/">Check out my blog here</a></p>
    </html>

If you place both files within the same folder you can run it with:

    $ docker build -t nginx-sample .
    $ docker run --name test-nginx -d -p 8080:80 nginx-sample

Now open your favourite web browser and hit http://localhost:8080 (or http://your-ip:8080 if you working on a server)

Docker Compose

Docker Compose is a tool that was developed to create multi-container applications easily by using a YAML file that could contain several Docker containers.

One big advantage over simple Dockerfiles is that it enables the use of a simple “stack” file for your whole application stack. You can keep one Compose file in your repository and deploy everything with one simple command: docker-compose up -d

To install Docker Compose follow the official tutorial: https://docs.docker.com/compose/install/

Afterwards, read the official Docker Compose guide to fully understand how a Docker Compose file is created because this will not be part of this article.

What is Traefik?

Traefik Makes Networking Boring
Cloud-Native Networking Stack That Just Works.

Traefik is used to forward incoming requests to a service deployed in a Docker environment. Also, it can create Let’s Encrypt SSL certificates automatically for every domain which is managed by Traefik.

To set up a local Traefik service within your Docker environment you can download this Docker Compose file and execute the following commands:

1. Create an external network that is used for Traefik

    $ docker create network traefik-public

2. Export needed variables

    $ export PRIMARY_DOMAIN=yourdomain.de
    $ export TRAEFIK_SSLEMAIL=youremai@yourdomain.de

3. Deploy

    $ docker-compose up -d

To access the Traefik dashboard you can hit [https://dashboard.yourdomain.de](https://dashboard.yourdomain.de/) and login with:

    username: devadmin 
    password: devto

To get a deeper understanding of Traefik you can read about How To Deploy it within this article.

Set Up And Deploy Your Website

Create A Simple Website

After you learned the basics and set up a Traefik instance you can start creating a Dockerfile and a Compose file to create a Docker Container (service) for your website.

To simplify this procedure I will deploy a simple dashboard project that I forked from an awesome list on GitHub. To get a copy of that dashboard either download the latest zip file here or clone the repository:

    $ git clone git@github.com:paulknulst/personal-dashboard.git

In contrast to the original repository, I have adjusted the index.html and the CSS file to be kind of SEO optimized and have 5 items per row instead of 6.

Create a new folder (personal-dashboard) and extract (or move) the project files into a subfolder named html. Now, you should open it in your favorite IDE, copy the config.sample.json to config.json and open it.

Change it to fulfill your needs and provide your own personal links. I will use the following settings:

    {
        "title" : "Pauls Simple Dashboard",
        "items" : [
            {
                "alt" : "Github",
                "icon" : "fab fa-github",
                "link" : "https://github.com/paulknulst"
            },
            {
                "alt" : "Twitter",
                "icon" : "fab fa-twitter",
                "link" : "https://twitter.com/paulknulst"
            },
            {
                "alt" : "Portfolio",
                "icon" : "fab fa-at",
                "link" : "https://www.paulknulst.de"
            },
            {
                "alt" : "Medium",
                "icon" : "fab fa-medium",
                "link" : "https://medium.knulst.de"
            }
        ]
    }

Create Dockerfile

The Dockerfile will be created in the simple-dashboard folder and only contain the base image and one copy command to add the HTML folder into the container.

    FROM nginx
    COPY html /usr/share/nginx/html

Create Compose File

The Compose file will be created in the simple-dashboard folder too. It contains every setting to set up your personal dashboard within your Docker environment. Furthermore, it will use Traefik to enable Let’s Encrypt certificate generation.

The complete Docker Compose file will be explained afterwards:

    version: '3.4'

    services:
      web:
        image: personal-dashboard
        build:
          context: ./
          dockerfile: Dockerfile
        networks:
          - traefik-public
        healthcheck:
          test: curl --fail http://localhost || exit 1
          interval: 60s
          retries: 5
          start_period: 20s
          timeout: 10s
        labels:
          - traefik.enable=true
          - traefik.docker.network=traefik-public
          - traefik.constraint-label=traefik-public
          - traefik.http.routers.personal-dashboard-http.rule=Host(`personal-dashboard.${PRIMARY_DOMAIN}`) 
          - traefik.http.routers.personal-dashboard-http.entrypoints=http
          - traefik.http.routers.personal-dashboard-http.middlewares=https-redirect
          - traefik.http.routers.personal-dashboard-https.rule=Host(`personal-dashboard.${PRIMARY_DOMAIN}`) 
          - traefik.http.routers.personal-dashboard-https.entrypoints=https
          - traefik.http.routers.personal-dashboard-https.tls=true
          - traefik.http.routers.personal-dashboard-https.tls.certresolver=le
          - traefik.http.services.personal-dashboard.loadbalancer.server.port=80
          # - traefik.http.routers.personal-dashboard-https.middlewares=security-headers
        restart: unless-stopped


    networks:
      traefik-public:
        external: true

Within this Compose file, you can see that we define an image in Line 5 (personal-dashboard:v1). Also, we add a build section because the image does not officially exist. Within the build section, we define that the Dockerfile which is located in the same directory should be used for creating the image.

The networks part (Line 9–10) contains the external network that was created during Traefik installation. Also, within Line 33–35 we especially define the network as an external network for this Compose service.

Another very important section is healthcheck (Line 11–16). For me, this part should be contained in every Docker Compose file because it helps to know if a Docker Container is running correctly without any runtime errors. For further reading have a look at this article that explains how to implement different types of health checks. that explains how to implement different types of health checks.

The most important part of this Compose file is the labels section. This section contains several labels that are important for Traefik. First, we enable Traefik (Line 18) for this service because in our Traefik service there was an option to not be working for every Docker service. Within Line 19 we define which external network is used for traefik.

In Line 20 we set the constraint-label of this service. Normally this is an optional setting and only used if we have more than one Traefik instance deployed to our Docker environment.

From Line 21–25 we define the URL of the service for HTTP and for HTTPS. For the HTTP part, we also activate a middleware from the Traefik configuration that will always redirect HTTP requests to HTTPS (Line 23).

The labels in Line 26 and 27 are necessary to create automatic SSL certificates for this Docker service.

Line 28 is needed to “expose” the internal Docker service port to the Traefik instance. All incoming requests to the URL defined in Line 21/24 will be forwarded to the Docker service on the Port mentioned within this label.

Line 29 is commented out because it is an extra middleware that I personally use in my Docker environment. I did not include this middleware within the Traefik installation but if you want to read about it have a look at my article about hardening any Traefik service.

Keep in mind that this file only works correctly if you followed the above steps to set up Traefik. If not you have to adjust PRIMARY_DOMAIN, your Docker external network, and the entry points used for Traefik (http, https).

Deploy The Website

Now, that you finished creating the needed files you can open your CLI, switch to your project folder and start the deployment process by executing:

    $ docker-compose up -d --build

After the deployment is complete you can open your dashboard in your browser and if you followed this tutorial it should look like this:

Closing Notes

I hope you enjoyed reading my tutorial and are now able to deploy your own website within a Docker container and enhance it with Traefik and SSL certificates.

Keep in mind that you can replace the HTML folder with every other static website. Also, you can extend the Compose file with other services like a database for your website. Furthermore, you can exchange the image with another public one from Docker Hub.

To help you, I have written another tutorial on how to deploy a Kimai2 Timetracking website that also includes a Docker Hub image, Traefik settings, a database, and an NGINX.

I would love to hear your feedback about this tutorial. Furthermore, if you already run a Traefik installation and use a different approach please comment here and explain what you have done differently. Also, if you have any questions, please jot them down below. I try to answer them if possible.

This blog post was initially published on my own blog at https://www.paulsblog.dev/deploy-any-ssl-secured-website-with-docker-and-traefik/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by Arian Darvishi / Unsplash

Introduction

In the last weeks, everyone has seen a big hype surrounding the new tool called ChatGPT which can basically answer every question you might ask.

This tool can be mind-blowing and playing around with it will result in having much fun because you can ask whatever you want and ask again and again and again...

But, it can also help you with your marketing strategy and content creation if used correctly.

In this short article, you will learn about five different ways to harness the power of ChatGPT in your own content creation workflow.

1. Ask ChatGPT to come up with ideas for content

Sometimes it can be challenging for a content creator to come up with new ideas and produce interesting content. Also, if your main marketing strategy consists of creating content for your blog, it could become challenging to find new ideas.

Luckily, with ChatGPT, you can now easily ask to offer some ideas on different topics that can result in a fresh and interesting article. For example, I asked ChatGPT "Give me some ideas for articles about productivity and time management" and it produces five article ideas and a short summary:

Afterward, it is really important to read and review ChatGPT answers because they are not always correct. However, if the ideas of ChatGPT contain valid information it could be enough input you need to write your next article.

2. Let ChatGPT generate the first draft

If you have an idea about a blog post but are short on time you could use ChatGPT to generate a draft. This draft can then be used as a baseline to start with your final draft. Always keep in mind that you have to review the answer to ChatGPT before you use it!

For example, if you take one of the provided titles before you could ask ChatGPT to create a draft based on its own answer.

3. Let ChatGPT come up with a title for your content

One of the biggest benefits of tools like ChatGPT is that it could come up with some good titles easily. And as a content creator, you will know that a good title is very important. For example, you can have the best article explaining why a specific technique should be used, but if the title does not pull people, it will never be read.

Now, after you improved the draft from ChatGPT and filled it with your own researched items you can ask ChatGPT to produce some high-converting titles for your article.

You can either choose to take one of the titles ChatGPT generates or you could use them as inspiration for your own title.

4. Ask ChatGTP to help you with your research.

If you are researching information for your content you will normally use Google or Wikipedia as a primary source. With ChatGPT, you now have another tool that you can use.

You can ask ChatGPT questions about studies or find a source of information. This should be seen as a major advantage.

Given the example with the Mindfulness draft, you can ask ChatGPT, "Which Studies have shown that mindfulness practices, such as meditation, can improve our ability to concentrate and focus on tasks?". It will answer the studies and summarize the results:

Next, ask ChatGPT to state the sources for the studies and it will output where you can find them:

All these answers will be a good starting point to start your research and produce high-quality content.

5. Allow ChatGPT to shorten the text for platforms with character limits.

As you often write articles you maybe want to promote them on Instagram, LinkedIn, Twitter, and Pinterest. To do this you need to have a short summary or a resized article while maintaining the message of the post.

To do this paste the whole article into ChatGPT and ask to shorten this text to X characters while maintaining the tone.

For example, if you use the article draft from the beginning and let ChatGPT to shorten it, it will produce the following summarized sentences:

Keep in mind that the resulting text should be not posted as is. Proofreading is always needed. However, it will be a fantastic baseline for a shorter version of your article.

Closing Notes

Like many other A.I. tools that were released recently, ChatGPT is a huge step forward. You could either avoid it or use it. But if used properly, ChatGPT can be a good tool for you.

In the right hands, it will help to provide much better content because it can be used easily without any major obstacles.

But, always keep in mind: Do not use answers of ChatGPT without proofreading!

I hope this article gave you a quick and neat overview of how you can use ChatGPT to create better content more frequently.

I would love to hear your feedback! Furthermore, if you already used ChatGPT in your workflow to create some content, please comment here and explain what you have done. Also, if you have any questions, please ask them in the comments. I try to answer them if possible.

This article was originally published on my blog at https://www.paulsblog.dev/improve-your-workflow-with-chatgpt/

Feel free to connect with me on my blog, Medium, LinkedIn, Twitter, and GitHub.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Title Photo by Pixabay / Pexels

As in any other programming language, JavaScript has a number of best practices and associated bad practices. Due to its dynamic properties, JavaScript also has various pitfalls.

Callback functions are functions that are passed as parameters to other functions and called by them. They are a common design pattern in asynchronous JavaScript development. The basic (not yet optimal) structure of this design pattern is as follows:

function doStuff(callback) {
  /* ... */
  callback();
  /* ... */
}

The doStuff() function expects a function as a parameter and calls it at a specific point in time:

function justLogSomething() {
  console.log('callback called');
}
doStuff(justLogSomething);  // Output: "callback called"

Check The Callback Function Type

Due to the weak typing of JavaScript, a function that expects a (callback) function as a parameter can in principle also be passed any other value (or no value at all). However, calling this supposed function inevitably leads to a type error ("callback is not a function"). Based on the example above, the following calls would be possible, but not advisable:

doStuff(1337);              // type error -> number
doStuff('Paul Knulst');     // type error -> string
doStuff();                  // type error -> undefined

For this reason, it is important to first check the type of the callback parameter inside the function and make sure that it really is a function. This can be achieved using the typeof operator, as shown in the following listing. If this returns the value "function" for the passed parameter, it is a function and nothing stands in the way of calling it:

function doStuff(callback) {
  /* ... */
  if(typeof callback === 'function') {
    callback();
  }
  /* ... */
}

If there are several places within a function where the callback function could be called, this check would have to precede all these places:

function doStuff(callback) {
  /* ... */
  if(someBool) {
    /* ... */
    if(typeof callback === 'function') {
      callback();
    }
  } else {
    /* ... */
    if(typeof callback === 'function') {
      callback();
    }
  }
  /* ... */
}

This can be avoided if, as in the following listing, you carry out the check at the beginning of the function and, in the event that the callback parameter is not a function, simply redefine it with an anonymous empty function. This simple but efficient trick secures all the following calls to the callback function in one fell swoop:

function doStuff(callback) {
  if(!(typeof callback === 'function')) {
    callback = function() {};  // redefinition
  }
  /* ... */
  if(someBool) {
    /* ... */
    callback();
  } else {
    /* ... */
    callback();
  }
  /* ... */
}

With the help of the conditional operator, the whole thing can even be reduced to one line of code:

callback = (typeof callback === 'function') ? callback : function() {};

Parameters Of A Callback Function

Since callback functions are called asynchronously and can neither supply a direct return value nor throw errors, appropriate parameters should be provided for at least these two cases:

1. one parameter that contains information about the error that occurred in the event of an error,
2. one parameter that normally contains the result of the asynchronous calculation.

Even if the order of these two parameters does not matter in principle, it has become a convention - especially when developing Node.js modules - to list the error as the first parameter and the result as the second parameter (if there is no error, the first parameters corresponding to zero).

doStuff(function(
  error,            // first parameter: error object
  result            // second parameter: result object
  ) {
});
function doStuff(callback) {
  /* ... */
  var result = null;
  try {
    // code that throws error
  } catch(error) {
    callback(error);
  }
  callback(null, result);
}

You can use a simple if query within the callback function to find out whether an error has occurred:

doStuff(function(error, result) {
  if(error) {
    // error handling
  } else {
    // normal workflow
  }
});

Return Of The Callback Call

In some cases, calling callback functions as shown in the previous examples can lead to unintended program behavior. For example, in the penultimate listing, the callback function is called twice in case an error occurs.

Here is the code again:

function doStuff(callback) {
  /* ... */
  var result = null;
  try {
    // code that throws error
  } catch(error) {
    callback(error);
  }
  callback(null, result);
}

The problem here is that the code after the try-catch block will always be called. Even if the catch block was previously jumped due to an error and the callback function was called there. This is a fact that you can quickly overlook when just reading the source code. For this reason, you should place a "return" before each call of a callback function, which jumps directly out of the calling function.

function doStuff(callback) {
  /* ... */
  var result = null;
  try {
    // code that throws error
  } catch(error) {
    return callback(error, null);
  }
  return callback(null, result);
}

The prerequisite for this is, of course, that the calling function is structured accordingly and the call of the callback function always represents the end of the function.

For example, the following snippet shows the wrong way to do it:

function doStuff(callback) {
  /* ... */
  var result = null;
  try {
    // code that throws error
  } catch(error) {
    return callback(error, null);
  }
  return callback(null, result);
  // the code below will never be executed
  if(foo) {
    /* ... */
  }
}

The Execution Context Of The Callback Function

Particular caution is required when passing functions as callback parameters that access the execution context via this. In this case, you have to take advantage of the bind() function and use it to create a new function that is bound to the desired execution context.

Then this new function is passed as a callback parameter:

var person = {
  name: 'Paul Knulst', 
  printName: function() {
    console.log(this.name);
  }
}
doStuff(person.printName);  // WRONG

var printNameBound = person.printName.bind(person);
doStuff(printNameBound);    // CORRECT

Conclusion

When working with callback functions, several best practices should be considered, including type checking, the order of callback parameters, calling callback functions once, and setting the execution context.

The fundamental question of whether to use promises, generator functions or the async/await combination for asynchronous programming instead of callback functions will be discussed in the following articles.

This article was originally published on my blog at https://www.paulsblog.dev/patterns-and-best-practices-in-javascript-dealing-with-callback-functions/

Feel free to connect with me on my blog, Medium, LinkedIn, and Twitter.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by Carl Heyerdahl / Unsplash

Design Pattern is a term used for a general, reusable solution to a commonly occurring problem in software design. It is important to understand that Design Patterns are not finished pieces of code that can be transformed and directly applied to your software. Rather they are more like a template, a description, or a concept that can give you an idea of how to solve a problem that can be used in many different situations.

They can incredibly useful when used in the right places and for the right reasons. When used strategically, they enhance your software and make it more efficient by methods, already refined by a myriad of developers.

Why Should You Use A Design Pattern?

Using Design Patterns in software development has several benefits:

• They are proven solutions: Design patterns are often used by many different developers, so you can be sure that they work as intended. Furthermore, you can be confident that they were revised multiple times and optimization has taken place already.
• They are easily reusable: Design patterns document a reusable solution that can be modified to solve multiple particular problems because they are not tied to a specific problem.
• They are expressive: Large solutions can be explained elegantly by Design Patterns.
• They ease communication: Many developers are familiar with Design Patterns and use them easily to communicate about possible solutions to given problems with each other.
• They prevent the need for refactoring code: Often if you use Design Patterns while developing software you could avoid refactoring it later. This only applies if you use the correct one for a given problem because it describes already an optimal solution.
• They lower the size of the codebase: Because Design Patterns are usually elegant, optimized, and well-documented solutions they require less code than other solutions.

Where It All Began

In 1994 an iconic computer science book "Design Patterns: Elements of Reusable Object-Oriented Software" was first published by four authors: Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. In technology circles, you'll often see these authors named "The Gang of Four" shortened to GoF.

Although the Gang of Four wrote the book in a C++ context it still remains very relevant to Java, JavaScript, Python, or any other object-oriented programming language. The authors, through their experience in creating large-scale enterprise systems using C++, saw 23 common patterns emerge. These 23 Design Patterns are not especially unique to C++ but can be applied in any other language.

Normally, as a developer that creates small to enterprise-class applications, you will encounter the Gang of Four Design Patterns on a daily basis.

The 23 GoF Design Patterns

All 23 Design Patterns introduced by the Gang of Four

The 23 Gang of Four Design Patterns are divided into three categories:

1. Creational: For handling Object creation mechanisms. They are used to create objects, rather than having to instantiate objects directly. Your program will be more flexible in deciding which object it creates for a specific use case.
2. Structural: For identifying ways to realize relationships between objects. This concerns class/object composition with inheritance to define different ways to gain new functionality.
3. Behavioral: For handling communication between different objects.

Creational

There are 5 Design Patterns in the creational category designed by the Gang of Four.

1. Abstract Factory: Creates an instance of several families of classes without detailing concrete classes. It allows the creation of a factory for factory classes.

2. Builder: Separates object construction from its representation, and always creates the same type of object. It creates an object step by step and a method to finally get the object instance.

3. Factory Method: Moves the responsibility of instantiating an object from the class to a Factory. The Factory then makes an instance of several derived classes based on interfaced data or events.

4. Prototype: This pattern provides a mechanism to copy or clone the original object to a new object and then modify it according to your needs to avoid costly object creation.

5. Singleton: Restricts the initialization of a class to ensure that only one instance of the class can be created. Multiple initialization attempts will result in returning the same object over and over again.

Structural

There are 7 Design Patterns in the structural category designed by the Gang of Four.

1. Adapter: Provides an interface between two unrelated entities in a way that they both can work together. It wraps an interface between already existing classes to enable working with each other without modifying the source code directly.

2. Bridge: Defines a strategy to decouple the interfaces from implementation and hide the implementation details from the client program. So both can vary independently. It uses encapsulation, aggregation, and inheritance to separate responsibilities into multiple classes.

3. Composite: This pattern is used if you have to implement a part-whole hierarchy. It creates a group of objects that should be treated equally as a single instance of one object. It composes zero-or-more objects in a way that they can be manipulated as one single object.

4. Decorator: Is used for modifying the behavior of objects during runtime. It dynamically adds alternate processing to objects without affecting other objects from the same class.

5. Facade: Provides a unified interface to a set of interfaces in a subsystem. It defines a higher level that helps to easier use of the subsystem. It masks the underlying complex or structural code and serves as a "front-facing interface"

6. Flyweight: Describes a fine-grained instance used for the efficient sharing of information. It caches and reuses object instances, used with immutable objects to minimize memory usage.

7. Proxy: Provides a placeholder for another object. You should use it to control access, reduce cost, and reduce complexity.

Behavioral

There are 11 Design Patterns in the behavioral category designed by the Gang of Four.

1. Chain of Responsibility: Describes a technique to delegate commands or pass requests between a chain of objects to find the object that can handle the command or request.

2. Command: This pattern encapsulates a command request as an object to enable, log, and/or queue requests. Furthermore, it provides error handling for unhandled command requests,

3. Interpreter: Includes language-specific elements in the application to match the grammar of the intended language. It defines a grammatical representation of a language and the interpreter that works with this grammar.

4. Iterator: Describes a standardized way to traverse through a group of objects. It accesses the containing objects sequentially without exposing their underlying representation.

5. Mediator: Provides centralized communication between different objects in a system to enable loose coupling between different objects. It encapsulates how these different objects interact with each other.

6. Memento: Stores a state of an object whenever it is saved to enable restoring previous states by providing undo operations.

7. Observer: Notifies state changes to a set of subscribing classes to ensure consistency between all of them.

8. State: Enables behavioral changes based on an internal state of an object. If the state of an object changes it also changes how it reacts to requests.

9. Strategy: This pattern is used when multiple algorithms for a specific task exist and a client decides which implementation should be used at runtime. It encapsulates the algorithm inside the class separating selection from implementation.

10. Template Method: Defines an abstract skeleton of an algorithm in such a way that subclasses have to implement the concrete behavior.

11. Visitor: Separates an algorithm from an object structure on which it operates. This enables adding new operations to existing object structures without changing the structure.

Conclusion

Now that you have read this short introduction to software Design Patterns, you should have an idea about several important techniques used in nearly every programming language.

You should now hone your skills by implementing all or at least some of these Design Patterns into your side projects or just as practice. Also, you can start to contemplate how to apply them in your real projects.

Although, there are several implementations that benefit from utilizing Design Patterns keep in mind that they were never meant to be hacked together shortcuts to be applied in a 'one-size-fits-all' manner to your code. There is no perfect substitute for a genuine problem-solving ability in software engineering.

It is paramount to understand, Design Patterns provide a common language to conceptualize repeating problems and solutions while working with a team or managing large code bases.

That being said, the most important caveat is ensuring that the how and why behind each Design Pattern are fully understood by the developer.

If you enjoyed reading this article consider commenting your valuable thoughts in the comments section. I would love to hear your feedback about all the Design patterns I mentioned here. Furthermore, share this article with your friends and colleagues to also help them to know about Design Patterns.

This article was originally published on my blog at https://www.paulsblog.dev/an-introduction-to-software-design-patterns/

Feel free to connect with me on my blog, Medium, LinkedIn, and Twitter.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo generated by WordClouds.com

Within this tutorial, I will show you how to set up Ghost with Docker and run the blog with a subdomain/primary domain within your environment. Primarily I will show how to set up everything within a Docker Swarm environment but I will also provide all the necessary files to deploy it in a simple Docker environment.

Why Use Ghost?

Ghost is an open-source blogging platform that is used to create a professional blog. It was released in October 2013 as a simple alternative to WordPress because it was getting overly complex. Ghost runs within Node.js and is written in JavaScript.

Here are the main features that Ghost blogging software provides:

• Share a private link to any post. This is useful for a review. Also, you can share it with everyone and not only with people having an account on the article platform
• Ghost has a great editor that allows embedding code snippets (with Prismjs), YouTube, Twitter, Codepen, etc
• You can schedule every post so that it will be published on the exact date you selected.
• Every post/article can be optimized in terms of SEO
• You are able to add and manage authors easily
• Ghost already has more than hundreds of apps like Slack, Stripe, Shopify, etc
• There are several themes that can be customized and you are able to manually inject your personal code on the site and for every article
• Users can register for a newsletter and/or can become sponsors of your blog

Also, there are many more features that are provided by Ghost.

Prerequisite

To follow every step within this tutorial and have a running Ghost blogging Platform at the end you need to have a running Docker Swarm environment. To achieve this you should consider reading this article:

Furthermore, you need a Traefik load balancer that is used to grant Let's encrypt SSL certificates and for forwarding your services within your Docker Swarm environment. To learn about this, you can read the first chapter from this tutorial:

Additionally, I will provide files to run the Ghost blogging platform on any server running Docker with a Traefik load balancer. Please read this article to understand how Traefik is installed in a simple Docker environment:

Install Ghost

Docker Swarm

Ghost will be installed with Docker Compose. The Compose file contains the service name, settings for Traefik to have a unique URL, and an SSL certificate. Also, there are two middlewares that are used to forward requests to the main site.

To install Ghost within your Docker Swarm you can paste the following code into your docker-compose.yml which will be explained afterward.

version: "3.4"
services:
  ghost:
    image: ghost:4
    environment:
      url: https://www.${DOMAIN?Variable not set}
      mail__transport: SMTP
      mail__options__host: ${MAIL_HOST?Variable not set}
      mail__options__port: ${MAIL_PORT?Variable not set}
      mail__options__auth__user: ${MAIL_USER?Variable not set}
      mail__options__auth__pass: ${MAIL_PASS?Variable not set}
      mail__from: ${MAIL_FROM?Variable not set}
    volumes:
      - content:/var/lib/ghost/content
    networks:
      - traefik-public
    deploy:
      placement:
        constraints:
          - node.labels.blogs.knulst == true
      labels:
        - traefik.enable=true
        - traefik.docker.network=traefik-public
        - traefik.constraint-label=traefik-public
        - traefik.http.routers.blogs-knulst-http.rule=Host(`www.${DOMAIN?Variable not set}`) || Host(`${DOMAIN?Variable not set}`) || Host(`blog.${DOMAIN?Variable not set}`)
        - traefik.http.routers.blogs-knulst-http.entrypoints=http
        - traefik.http.routers.blogs-knulst-http.middlewares=https-redirect
        - traefik.http.routers.blogs-knulst-https.rule=Host(`www.${DOMAIN?Variable not set}`) || Host(`${DOMAIN?Variable not set}`) || Host(`blog.${DOMAIN?Variable not set}`)
        - traefik.http.routers.blogs-knulst-https.entrypoints=https
        - traefik.http.routers.blogs-knulst-https.tls=true
        - traefik.http.routers.blogs-knulst-https.tls.certresolver=le
        - traefik.http.services.blogs-knulst.loadbalancer.server.port=2368
        - traefik.http.middlewares.redirect-blog.redirectregex.regex=^https://blog.${DOMAIN?Variable not set}/(.*)
        - traefik.http.middlewares.redirect-blog.redirectregex.replacement=https://www.${DOMAIN?Variable not set}/$${1}
        - traefik.http.middlewares.redirect-blog.redirectregex.permanent=true
        - traefik.http.middlewares.redirect-nosub.redirectregex.regex=^https://${DOMAIN?Variable not set}/(.*)
        - traefik.http.middlewares.redirect-nosub.redirectregex.replacement=https://www.${DOMAIN?Variable not set}/$${1}
        - traefik.http.middlewares.redirect-nosub.redirectregex.permanent=true
        - traefik.http.routers.blogs-knulst-https.middlewares=redirect-blog, redirect-nosub
volumes:
  content:

networks:
  traefik-public:
    external: true

Line 4: Up to date version of Ghost v4 will be used during installation

Line 6: The main blog URL will be defined as an environment variable within the docker service

Line 7 - 12: This section can be used to set mail configuration for sending registration emails, invitations, password resets, or member login links. This email cannot be used for newsletter emails. For this feature, you have to set up a service like Mailgun. Read more about it within the Ghost documentation.

Line 13 - 14: The content of the website will be saved as a persistent volume within your Docker environment

Line 15 - 16: The main traefik network will be used here. This is important because otherwise, Traefik cannot forward requests to the service

Line 18 - 20: The service will only be deployed to a Docker Swarm node if the label blogs.knulst is true. This can be achieved by executing the following command before deploying the docker-compose.yml to the stack:

docker node update --label-add blogs.knulst=true ID_OF_NODE_TO_USE

Replace ID_OF_NODE_TO_USE with the correct ID of any worker/manager node of your Docker Swarm where the service should run.

Line 21 - 31: Set up a standard configuration for a service deployed in a Docker Swarm with Traefik and Let's Encrypt certificates. In Line 19 and 22 three URLs are registered for this service: www.${DOMAIN}, ${DOMAIN}, and blog.${DOMAIN}.

Line 32: Port used by Ghost blogging Docker container. Needed for Traefik.

Line 33 - 35: Creates a permanent Traefik middleware that forwards every request from blog.${DOMAIN} to www.${DOMAIN}

Line 36 - 38: Creates a permanent Traefik middleware that forwards every request from ${DOMAIN} to www.${DOMAIN}

Line 39: Activates the earlier created middleware for this service. This is done because I only want to have one primary website for my blog but multiple URLs to reach it.

Before deploying (or redeploying) multiple environment variables should be set with (adjust them to your needs):

export DOMAIN=knulst.de
export MAIL_HOST=smtp.your-domain.de
export MAIL_PORT=587
export MAIL_USER=blog@knulst.de
export MAIL_PASS=unbelievablehowsecurethisis
export MAIL_FROM=blog@knulst.de

Then you can deploy the Docker Swarm stack by executing:

docker stack deploy -c docker-compose.yml blog

Docker

If you do not have a running Docker Swarm you can download this Compose file.

Within this file, there are only two differences from the Docker Swarm Compose file. The first is in Line 17 where a new setting is used: restart: always. This configuration is used to automatically restart the Docker service if it is aborted. The other change is that labels are removed from the deploy - keyword and put to a higher order within the Compose file. This is done because deploy is only used in a Docker Swarm environment but labels can also be used in a simple Docker environment.

Keep in mind that this will only work if you have a running Traefik load balancer.

Before starting the docker service multiple environment variables should be set with (adjust them to your needs):

export DOMAIN=knulst.de
export MAIL_HOST=smtp.your-domain.de
export MAIL_PORT=587
export MAIL_USER=blog@knulst.de
export MAIL_PASS=unbelievablehowsecurethisis
export MAIL_FROM=blog@knulst.de

Then you can start the Docker service by executing

docker-compose up -d

Configure Ghost

If you reached this step Ghost blog is already installed on your URL with the default Casper theme and will look like this:

Screenshot of a fresh installed Ghost blog

Now you have to configure your Ghost blog instance by opening your website and appending the ghost path to open the Admin menu (should be https://your-domain.de/ghost).

If you open the admin menu you have to set the title, your name, your email address, and a secure password:

Registration Form after installation

After you create the account you should receive an email from your blog. If you do not receive any mail you can check the docker logs about email problems.

The last step will be adjusting the personal settings of your newly created blog. For this, you can open the Ghost blog settings (https://your-domain.de/ghost/#/settings) which should look like this:

Settings Dashboard

Select General and adjust everything to your needs. Afterward, you should open Design and install a theme that suits you. I can recommend Liebling by Eduardo Gómez. It has already inbuilt translations and looks very good.

Furthermore, I would suggest selecting Membership, connecting your Stripe account, and enabling paid registration as an additional Membership alternative. Within my blog, I have done this too but I do not provide extra content only to paying members. It is only activated if someone wants to sponsor me regardless of the content that is provided.

Closing Notes

Congratulations you have just installed your own Ghost blog. If you want to improve the user experience or do more with your blog you can:

• Sign up on Grammarly and install the web extension to optimize articles
• Customize the design with pictures or a theme
• Set up analytics with Google analytics
• Add code syntax highlighting
• Add comments to the posts

This is the end of this tutorial. Once you finish setting up your personal blog, share it with me on Twitter. I will love to see that!

This article was originally published on my blog at https://www.paulsblog.dev/how-to-self-host-ghost-blogging-platform-on-docker-docker-swarm/

Feel free to connect with me on my blog, Medium, LinkedIn, and Twitter.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by Damian Zaleski / Unsplash

According to the Tiobe Index, Python is the most used programming language in the world. Also, it is in high demand since it is used by all of the world’s leading firms. As of today, there is no better time to learn Python and this can be done with YouTube, books, or websites.

Within this article, I have compiled a list of the best channels that I follow, interesting books that I earn, and websites that I have experience with.

YouTube Channels:

If you want to learn Python, you will have a vast collection of resources to choose from. On YouTube, you will find plenty of channels with a simple search.

To help you find good resources to start with I come up with the following seven channels.

1. freeCodeCamp.org

• Created: 17 Dec 2014
• Creator: Quincy Larson
• Subscribers: 5.18M
• Channel Link: https://www.youtube.com/c/Freecodecamp

freeCodeCamp.org is a well-known resource among programmers. It was created by Quincy Larson back in 2014 with the aim of making programming easier and accessible for anyone without paying a fee.

This YouTube channel is one of the best resources for learning nearly every programming language including Python. In addition to tutorials for absolute beginners, they also offer Python-related video tutorials for professionals. They also cover Deep Learning or Data Analysis.

Furthermore, the freeCodeCamp website offers certification programs for a number of domains, including Python (and many more).

2. Programming with Mosh

• Created: 07 Oct 2014
• Creator: Mosh Hamedani
• Subscribers: 2.34M
• Channel Link: https://www.youtube.com/channel/UCWv7vMbMWH4-V0ZXdmDpPBA

Programming with Mosh was created in 2014 by Mosh Hamedani and provides tutorials for upcoming Python developers. He has a clear and concise way of presenting topics. His videos usually are long but they are well-structured with plenty of timestamps, making navigation through the videos painless.

3. CS Dojo

• Created: 26 Feb 2016
• Creator: YK Sugishita
• Subscribers: 1.79M
• Channel Link: https://www.youtube.com/csdojo

CS Dojo was created by YK Sugishita, who was a former Microsoft intern and an ex-Googler. Within his video tutorials, he follows a code-along approach to enable his viewers to practice writing code side-by-side with him while solving practice exercises.

Additionally, he describes the topics in a completely transparent and unclouded way to his viewers which makes the videos easy to follow.

4. Sentdex

• Created: 17 Dec 2012
• Creator: Harrison Kinsley
• Subscribers: 1.11M
• Channel Link: https://www.youtube.com/user/sentdex

Sentdex is a YouTube channel that is beginner-friendly and suitable for new developers as well as intermediate ones. It was created in 2012 by Harrison Kinsley and mainly covers topics related to Python.

In his video tutorials, Harrison follows a practical approach to the topics rather than going with the theoretical approach.

5. Corey Schafer

• Created: 01 Jun 2006
• Creator: Corey Schafer
• Subscribers: 898k
• Channel Link: https://www.youtube.com/channel/UCCezIgC97PvUuR4_gbFUs5g

Corey Schafer created his primarily Python-focused YouTube channel in 2006 and provides convenient tutorials and walkthroughs for programmers of every skill level.

It contains everything from setting up your development environment to tutorials about advanced concepts in Python.

In his tutorials, Corey is focused that a viewer will get a solid understanding of the foundational concepts in Python that then could be used in several follow-up tutorials on his channel

6. Telusko

• Created: 29 Apr 2014
• Creator: Navin Reddy
• Subscribers: 1.64M
• Channel Link: https://www.youtube.com/c/Telusko

Telusko is a YouTube channel that makes learning as fun and interesting as possible. Navin Reddy created the channel in 2014 and covers nearly every topic in Python in a playful and clear way.

This channel is a good starting point for developers who are new to programming in general and want to take simple steps in the world of Python programming.

7. Real Python

• Created: 22 May 2013
• Creator: Dan Bader
• Subscribers: 146k
• Channel Link: https://www.youtube.com/c/realpython

Real Python’s YouTube channel was created in 2013 by Dan Bader and is one of the most reputed learning resource providers for people interested in learning Python. It is your one-stop shop for learning a huge amount of topics that are beyond the foundational stuff.

It is designed for people that are already familiar with Python and are looking to strengthen their skills in different advanced topics. Also, the YouTube channel offers everything from beginner-friendly tutorials, to weekly podcasts, and coverage of Python events.

Books

1. Python Crash Course, A Hands-On, Project-Based Introduction to Programming*

Python Crash Course by Eric Matthews is a fast-paced and comprehensive introduction to Python language. It is written for beginners who want to learn Python programming to write useful programs.

This book focuses on speed to enable the reader to write real programs in no time at all. It is designed for programmers that have a basic understanding of Python and want to test their skills before creating real-life applications with Python.

You will learn libraries and tools such as Pygame, Matplotlib, Plotly, and Django and will use data to create interactive visualizations.

The book consists of two parts. Within the first part, some Python basics and concepts will be explained. You will be taught to write clean and readable code to create interactive Python programs. The second part follows a practical approach and tests your knowledge by presenting three different projects: space invaders inspired arcade game, a set of data visualizations with Python’s handy libraries, and a simple web app that you can deploy online.

You can buy the book here*

2. Automate the Boring Stuff with Python, 2nd Edition: Practical Programming for Total Beginners*

Automate the Boring Stuff with Python by Al Sweigart is a book that teaches Python3 to everyone, including technically inclined beginners, juniors, seniors, and other geeks around the world. It covers step-by-step instructions and explains each program in detail to enable you to write programs quickly and efficiently in Python.

Within the first part of the book, you will learn Python’s basics and explore several libraries to perform tasks like data scraping of websites, reading PDF or word documents, and automating typing and clicking. The second part includes input validation and tutorials on automating Gmail and Google Sheets. Also, it describes how you can automatically update CSV files. Additionally, you will learn how to automate several tasks with Python including searching for text in multiple files, creating/updating/moving/renaming files or folders, searching the web and downloading content, filling out forms on websites, and many more.

You can buy the book here*

3. Learning Python: Powerful Object-Oriented Programming*

Learning Python is a comprehensive, in-depth introduction to the core Python language by Mark Lutz and is based on his popular training course. The latest version of the book helps you to quickly write efficient, high-quality code. The book is designed for everyone who wants to learn Python including new developers, intermediates as well as professional ones.

You will explore Python’s major built-in object types such as numbers, lists, and dictionaries and create/process objects with Python statements to learn the general syntax model. Additionally, it will teach you to avoid code redundancy, organize statements/functions, and how package code for reuse.

Furthermore, you will learn about OOP concepts, the exception-handling model, and development tools. The book also covers advanced Python tools like decorators, descriptors, metaclasses, and Unicode processing.

You can buy the book here*

4. Head First Python: A Brain-Friendly Guide*

Head-First Python by Paul Barry is a book to learn Python without wrapping through counterproductive tutorials and books. The book will help you to gain a quick grasp of Python programming fundamentals and work with built-in data structures and functions. Afterward, you will create your own web application, exercise exception handling or data wrangling, explore database management, and other concepts.

This book is different than others because it uses a visually rich format to engage your mind, rather than a text-heavy approach that will put you to sleep. Additionally, it creates a multi-sensory learning experience and is designed for the way your brain really works.

You can buy the book here

5. Elements of Programming Interviews in Python: The Insiders’ Guide*

Elements of Programming is a book that contains a collection of 250 problems with detailed solutions based on interview questions at leading software companies. The problems are illustrated with 200 figures, 300 tested programs, and 150 additional variants. It provides several data structures, algorithms, and problem-solving patterns.

The book is separated into a series of chapters on basic and advanced data structures, searching, sorting, algorithm design principle, and concurrency. You will find a brief introduction, a case study, top tips, and a review of the most important library method in every chapter of the book. Then you will have a broad and thought-provoking set of problems.

Furthermore, the book summarizes many nontechnical aspects of interviewing, such as interview strategies, perspectives from the other side of the table, common interview mistakes, and tips on negotiating the best offer.

You can buy the book here*

Websites

1. The Official Python Tutorial

One of the best places to start your Python developer journey is the official Python website. The creator of the language made a Python tutorial that teaches you the basics of the language. It is designed rather for beginners than seniors who want to learn Python. It has a slow pace with clear imagery and plenty of time to explain nearly every topic.

If you already have heard about Python and are not completely new you could find the official Python tutorial too slow and should use another source for further research.

You should consider using this website if you want to learn Python tips and tricks from Python creators on a site that is created for complete beginners. Also, it is for you if you like a slow pace of learning to make sure every concept is discussed in detail before moving to the next chapter

Here is a link to the website

2. CodeWars

If you are no new Python developer or the official Python tutorial is too slow for you, CodeWars will be a good place to start. CodeWars is a set of puzzles that helps you to test your Python knowledge and learn about your strength and weaknesses while you learn several algorithms.

CodeWars is some kind of Gamification model for Python and contains many puzzles that are called katas. These catas are categorized according to their difficulty levels. After you finished a puzzle you can check solutions from other developers and learn different strategies.

You should consider using this website if you want to use a Gamification concept, love to solve coding puzzles to test your knowledge, and want to compare with other developers and see or learn from their solutions.

Here is a link to the website

3. A Byte of Python

A byte of Python is a free online e-book about Python programming that is not suited for beginners. It requires that you have at least knowledge about the basics of Python programming otherwise you will find yourself lost during reading.

Although it is an excellent site to learn Python if you want to have a book that jumps quickly into all the topics without caring about beginners. If you are a fast learner who can adapt easily or you already know the language this book is perfect for you to strengthen your Python programming skills.

Here is a link to the website

4. Real Python Tutorials

Real Python Tutorials is a website that provides interactive tutorials to you. Also, it's more structured like a blog about Python programming, where you can learn every topic by reading different blog posts one by one.

After you have read about new skills in a blog post you can practice them or jump to another topic with the search function. If you mark blog posts as completed you will be get informed about related topics that will enhance your Python knowledge based on the already completed posts.

You should consider using this website as a resource on its own. It is suitable for beginners and advanced programmers because it covers the basics and provides practical exercises to help you with Python programming. It can be used as a resource to find missing information for problems you will encounter as a Python developer.

Here is the link to the website

5. FreeCodeCamp.org

FreeCodeCamp.org is a website that contains several blog entries about Python programming (and many other programming languages). Also, it has a well-structured curriculum teaching nearly every aspect of Python programming. You are able to join several courses online that are free and allow you to earn certificates.

FreeCodeCamp is a good learning tool for beginners and seniors. You do not need any requirements to join the courses because the platform provides everything that is necessary to finish the course and receive the certification.

Here is a link to the website

Closing Notes

I hope this compilation might be useful for you and help you grow in your development career as a Python developer. If you are new to coding, the best way to learn Python is to just start with it and see how things go. Python is one of the easiest programming languages for beginners (only the basics!), so you won’t have to spend much time troubleshooting your code or learning how the syntax works. You can see results fast and start building your first small programs sooner than you may think.

If I missed your favorite Python video tutorial, the book you loved, or a website you cannot live without please leave me a comment!

I would love to hear your ideas and thoughts about these sources.

If you have any questions, please jot them down below. I try to answer them if possible.

This article was originally published on my blog at https://www.paulsblog.dev/how-to-start-with-python-programming/

Feel free to connect with me on Medium, LinkedIn, and Twitter.

🙌 Support this content

If you like this content, please consider supporting me. You can share it on social media or buy me a coffee! Any support helps!

Furthermore, you can sign up for my newsletter to show your contribution to my content. See the contribute page for all (free or paid) ways to say thank you!

Thanks! 🥰

Photo by Hitesh Choudhary / Unsplash

Everyone knows it’s really important to have a good security score on several websites. Within this tutorial, I will explain how I used traefik to get one.

Important: I moved the website in the screenshots from https://www.f1nalboss.de to https://ftp.f1nalboss.de after I wrote this article.

Precondition

If you want to apply the content from this tutorial you need to have a Docker environment in Docker Swarm Mode. If you want to know how to set up one for yourself try following this tutorial that I wrote.

Also, you need to have traefik installed as a load balancer because it is used for adding security headers to your website requests. I have written another tutorial that shows how to integrate traefik in your Docker Swarm environment.

If you did not have a Docker Swarm and you don't want to set up one you can try to install traefik as a load balancer on a single machine. I explain how this is done in this tutorial. But keep in mind that for non-Docker Swarm environments you have to adjust the docker-compose files. If you have questions about that just ask in the comments and I will answer them if possible.

How I Added Security Headers

There is a very important website published by Mozilla: The Mozilla Observatory

The Mozilla Observatory has helped over 240,000 websites by teaching developers, system administrators, and security professionals how to configure their sites safely and securely.

I ran a check for my site and got the following result:

First test on Mozilla Observatory with the result: F

After quick research at traefik documentation, I came up with these important headers which I want to implement in my environment:

accesscontrolallowmethods=GET, OPTIONS, PUT, POST, PATCH
accesscontrolmaxage=100
addvaryheader-true
hostsproxyheaders=X-Forwarded-Host
sslredirect=true
X-Forwarded-Proto=https
stsseconds=63072000
stsincludesubdomains=true
stspreload=true
forcestsheader=true
framedeny=true
contenttypenosniff=true
browserxssfilter=true
referrerpolicy=same-origin
featurepolicy=camera 'none'; geolocation 'none'; microphone 'none'; payment 'none'; usb 'none'; vr 'none';
customresponseheaders.X-Robots-Tag=none,noarchive,nosnippet,notranslate,noimageindex,

To add these headers in my traefik installation I added a new middleware to my traefik docker-compose.yml:

    [...]
    - traefik.http.middlewares.security-headers.headers.accesscontrolallowmethods=GET, OPTIONS, PUT
    - traefik.http.middlewares.security-headers.headers.accesscontrolmaxage=100
    - traefik.http.middlewares.security-headers.headers.addvaryheader=true
    - traefik.http.middlewares.security-headers.headers.hostsproxyheaders=X-Forwarded-Host
    - traefik.http.middlewares.security-headers.headers.sslredirect=true
    - traefik.http.middlewares.security-headers.headers.sslproxyheaders.X-Forwarded-Proto=https
    - traefik.http.middlewares.security-headers.headers.stsseconds=63072000
    - traefik.http.middlewares.security-headers.headers.stsincludesubdomains=true
    - traefik.http.middlewares.security-headers.headers.stspreload=true
    - traefik.http.middlewares.security-headers.headers.forcestsheader=true
    - traefik.http.middlewares.security-headers.headers.framedeny=true
    - traefik.http.middlewares.security-headers.headers.contenttypenosniff=true
    - traefik.http.middlewares.security-headers.headers.browserxssfilter=true
    - traefik.http.middlewares.security-headers.headers.referrerpolicy=same-origin
    - traefik.http.middlewares.security-headers.headers.featurepolicy=camera 'none'; geolocation 'none'; microphone 'none'; payment 'none'; usb 'none'; vr 'none';
    - traefik.http.middlewares.security-headers.headers.customresponseheaders.X-Robots-Tag=none,noarchive,nosnippet,notranslate,noimageindex
    [...]

After adding this middleware I updated my traefik service that runs within a Docker Swarm (cf. this article)

docker stack deploy -c docker-compose.traefik.yml traefik

This addition made it possible to use this middleware within every service deployed to my swarm and managed by traefik. To do this I had to add the following line to the labels section within the docker-compose.yml:

    [...]
    - traefik.http.routers.simpleweb-https.middlewares=security-headers
    [...]

After restarting the simpleweb service I ran another test and got a B score because of this error:

Content Security Policy Error on Mozilla Observatory

I researched and found the correct traefik header for CSP. Additionally, I created a very strict CSP directive and added it to the header value:

    contentsecuritypolicy=default-src 'none'; img-src 'self' https://i.postimg.cc; script-src 'self'; style-src 'self'"

I exclusively allow access to https://i.postimg.cc for img-src because the only picture I use at https://ftp.f1nalboss.de is hosted there.

Because this header is very strict I did not create a global middleware for it. I upgraded the service in which I want to use this CSP directive by adding this line to the simpleweb service and adjusting the middleware:

    [...]
    - traefik.http.routers.simpleweb-https.middlewares=security-headers, simpleweb-csp
    - traefik.http.middlewares.simpleweb-csp.headers.contentsecuritypolicy=default-src 'none'; img-src 'self' https://i.postimg.cc; script-src 'self'; style-src 'self'
    [...]

I restarted the service and ran another check to finally achieve an A+!

The final result on Mozilla Observatory after using security headers in Traefik; result: A+

How I Harden My Website

Because I also want to harden my server I ran a check at hardenize.com and got non-satisfying results for TLS and HSTS.

Due to already having adjusted the traefik headers for HSTS beforehand, the only thing left to do was submit my domain name at hstspreload to fix the HSTS issue.

THIS PART IS VERY IMPORTANT: If you want to submit your website to hstspreload.org think about it very carefully. There can be problems if you cannot fulfill everything: read here for information

Solving the TLS issue was more involved. I had to adjust the traefik configuration so that the minimum version of TLS is 1.2 and NOT 1.0, as is the default. To set a minimum TLS version I added a file provider to my traefik installation within the command section of my traefik docker-compose.yml:

   command:
      - --providers.docker
      [...]
      - --providers.file.directory=/configuration/
      - --providers.file.watch=true

Then I created a new configuration file (called tls.toml) which contains an entry [tls-options] and should be used to set the minimum TLS version to 1.2. Additionally, I added excellent cipher suites (read this to know about cipher suites).

[tls.options]
  [tls.options.mintls12]
    minVersion = "VersionTLS12"

    cipherSuites = [
        "TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
        "TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
        "TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305",
        "TLS_AES_128_GCM_SHA256",
        "TLS_AES_256_GCM_SHA384",
        "TLS_CHACHA20_POLY1305_SHA256"
    ]

    sniStrict = true

This configuration guarantees that traefik will use at least TLS 1.2 with the provided cipher suites. I chose these six cipher suites because I want to have three for TLS 1.2 and three for TLS 1.3 which are declared safe!

I saved the file within the ./configuration/ folder and updated the volume section within the traefik docker-compose.yml:

    volumes:
      [...]
      - ./configuration:/configuration

After that, I restarted the traefik instance.

The last step was activating the min TLS version within the simpleweb service by adding a new label:

    [...]
    - traefik.http.routers.simpleweb-https.tls.options=mintls12@file
    [...]

After a restart of the simpleweb service I retried the test on Hardenize and Mozilla and got the wanted results:

• https://www.hardenize.com/report/f1nalboss.de/1628027450
• https://observatory.mozilla.org/analyze/ftp.f1nalboss.de

There is also an informative test at ssl-labs which checks certificates. With the new configuration, I also got an A+

• https://www.ssllabs.com/ssltest/analyze.html?d=ftp.f1nalboss.de

Closing Notes

To simplify everything you can use this docker-compose.yml and run it within any Docker Swarm as www-stack service:

docker stack deploy -c docker-compose-simpleweb.example.yml www-stack

I hope you find this tutorial helpful and are now able to secure your website and increase trust by adding security headers and optimizing SSL usage.

I would love to hear your ideas and thoughts. If you already run a traefik installation and use different headers/middleware or you have other cipher suites please comment here and explain. Also, if you have any questions, please jot them down below. I try to answer them if possible.

This article was originally published on my blog at https://www.paulsblog.dev/harden-your-website-with-traefik-and-security-headers/

Feel free to connect with me on my personal blog, Medium, LinkedIn, Twitter, and GitHub.

Did you find this article valuable? Want to support the author? (... and support development of current and future tutorials!). You can sponsor me on Buy Me a Coffee or Ko-Fi. Furthermore, you can become a free or paid member by signing up to my website. See the contribute page for all (free or paid) ways to say thank you!

Photo by Peter Conrad / Unsplash