Containers and Docker

Hello readers this is a blog regarding containers in DevOps and software development.

How it all started (monolithic -> VMs -> Containers)

Before starting with containers you must have a brief knowledge of monolithic architecture and virtual machines(VMs), as it will help you to understand the concept of containers in and out.

Monolithic Architecture:- This was the architecture of software in the very early stages around in the '80s and '90s which follows one server one application architecture, where each server was created to handle only one application.

The companies had no idea about running multiple applications on a server, every time they needed to run a new application they needed to buy a new server.

The arrival of VMs:-

VMware solved this problem of unwanted expenditure on the servers by the introduction of virtual machines.

virtual machines are an advancement to the physical servers that help to run multiple applications on the same server.

Virtualization in the physical server is attained through the help of a hypervisor.

Each virtual machine has its guest operating system that needs a hypervisor to run.

Although virtual machines solved many problems of the physical server-based architecture still there were chances for improvement because each VM was using its very own guest OS and hence they were heavier and consumed more compute resources.

Problem with VM and the arrival of containers:-

Let us say we have a physical server for our organization with 100GB of RAM and 100 CPUs. We created 4 VMs using our server and hypervisor each with an equal share of resources.

now if we run VM1 which at its full capacity uses 10 GB of RAM and 6 CPUs then also we are wasting our resources as 15GB of RAM and 19 CPUs are still unused. Although it is better than one server - one application architecture still there is wastage of resources.

Another problem includes dependencies and licenses that is the virtual machine must have the required dependencies and licenses to run an application.

To solve this problem containerization comes into the picture.

Virtual machines were an advancement to the physical server and containers are an advancement to the virtual machines.

Virtual machines have solved some problems of physical servers and containers have solved some problems of virtual machines.

Containers - what are they

Containers in general language can be defined as a box in which we can store all application-related stuff like source code, dependencies such as application dependencies and system dependencies and libraries to run that application.

VM is hardware virtualization whereas containerization is OS virtualization that is in simple words VMs are created on top of physical servers which are hardware and containers are created on top of the host operating system and they dont have their own operating system.

Containers are lightweight because they don't have full-fledged operating systems within them but they have minimal system dependencies (minimal OS).

Minimal OS is sometimes the Base Image.

Containers are the bundle or package of application, application dependencies, libraries and system dependencies and they use all other resources from the kernel of the host operating system.

Containerization - Definition and overview

Containerization is an efficient method for deploying applications. A container encapsulates an application with its own operating environment. It can be placed on any host machine without any special configuration, thus removing issues of dependencies.

containerized applications are easy to transfer from one system to another.

Container help application to run in an isolated environment it is better than VM because we do need not to install a new OS for each application.

How Containers solve the problem with VM

Containers as discussed earlier are very lightweight and also require minimal resources so they save resources.

As they do not require guest OS each for one application they never make wasteful use of resources because in VM a guest OS with required compute resources was created and it took up the physical resources even when not in use but in the case of container they do not use any resources when they are not in use.

another way they are better than VM is that they are created only if they are required and vanish after the job is done and free the resources acquired by them to be used by another container basically they follow the concept of shared resources.

EX:- suppose we have a VM to take its backup we create a snapshot whose size may range in GBs but incase of containers it might come in MBs.

Architecture of Containers

Containers can be created in two different ways

On top of a virtual machine.
On top of a physical server.

To create containers we need a containerization platform like we need a hypervisor to create VMs.

In current scenarios, model 1 which is on top of a virtual machine is used mostly because every company has shifted to a cloud provider and they don't want maintainance overhead of their data centers.5

Files and folders in containers - where they come from

containers are lightweight and use resources from kernal of host os only when they are running

containers need a minimal system dependency because it separates two containers from one another logically.

If there is no logical separation any hacker who has access to one container or has access to Kubernetes cluster may hack all containers

Since we know that base image works as minimal OS the files and folders in containers base image are:-

  /bin: contains binary executable files, such as the ls, cp, and ps commands.

  /sbin: contains system binary executable files, such as the init and shutdown commands.

  /etc: contains configuration files for various system services.

  /lib: contains library files that are used by the binary executables.

  /usr: contains user-related files and utilities, such as applications, libraries, and documentation.

  /var: contains variable data, such as log files, spool files, and temporary files.

  /root: is the home directory of the root user.

These files and folders create logical separation between containers.

Files and folders that the container uses from the host OS

 The host's file system: Docker containers can access the host file system using bind mounts, which allow the container to read and write files in the host file system.

    Networking stack: The host's networking stack is used to provide network connectivity to the container. Docker containers can be connected to the host's network directly or through a virtual network.

    System calls: The host's kernel handles system calls from the container, which is how the container accesses the host's resources, such as CPU, memory, and I/O.

    Namespaces: Docker containers use Linux namespaces to create isolated environments for the container's processes. Namespaces provide isolation for resources such as the file system, process ID, and network.

    Control groups (cgroups): Docker containers use cgroups to limit and control the amount of resources, such as CPU, memory, and I/O, that a container can access.

It's important to note that while a container uses resources from the host operating system, it is still isolated from the host and other containers, so changes to the container do not affect the host or other containers.

Containerization tools or platforms

There are multiple popular containerization tools such as

Docker
Buildah
Podman etc.

Each of these tools have different pros and cons we can choose any one according to our requirements.

In this blog we will discuss everything in respect to docker.

Docker - an introduction

Docker is a containerization platform that allows us to build, test and deploy applications quickly.

It is a containerization platform that provides easy way to containerize your application.

A developer defines all the applications and their dependencies in the docker file which is then used to build docker images that define the docker container.

Doing this will ensure that your application will run in any environment.

Using docker we can build container images, run the images to create containers and also push the containers to container registries such as dockerhub, quay.io etc.

The life cycle of docker container

step 1:- Create a docker file

step 2:- execute build command

step 3:- place it in docker engine (happens automatically after executing build command)

step 4:- Docker engine will create the image of the application specified in dockerfile.

step 5:- Execute the run command to create a container out of the image.

this is the overall life cycle of the docker container.

Why Docker

Docker is very popular and has a huge open-source support community it is easy to use because it came up with many commands and GUI making it easy to use.

Problems with Docker

The Docker engine is the single point from where all the containers are executed so docker engine is the single point of failure for docker. Once the docker engine is down no container can work.
Docker builds images in layers which take up much of disk memory.

Docker Architecture

Docker deamon is the heart of the docker like kernel is the heart of OS.

Docker deamon is the process which accepts the command from cli(command line interface) and perform the docker tasks.

we can create images and push them to registry using docker deamon.

Terminologies in Docker

Docker daemon

The Docker daemon (dockerd) listens for Docker API requests and manages Docker objects such as images, containers, networks, and volumes. A daemon can also communicate with other daemons to manage Docker services.

Docker client

The Docker client (docker) is the primary way that many Docker users interact with Docker. When you use commands such as docker run, the client sends these commands to dockerd, which carries them out. The docker command uses the Docker API. The Docker client can communicate with more than one daemon.

Docker Desktop

Docker Desktop is an easy-to-install application for your Mac, Windows or Linux environment that enables you to build and share containerized applications and microservices. Docker Desktop includes the Docker daemon (dockerd), the Docker client (docker), Docker Compose, Docker Content Trust, Kubernetes, and Credential Helper. For more information, see Docker Desktop.

Docker registries

A Docker registry stores Docker images. Docker Hub is a public registry that anyone can use, and Docker is configured to look for images on Docker Hub by default. You can even run your own private registry.

When you use the docker pull or docker run commands, the required images are pulled from your configured registry. When you use the docker push command, your image is pushed to your configured registry. Docker objects

When you use Docker, you are creating and using images, containers, networks, volumes, plugins, and other objects. This section is a brief overview of some of those objects.

Dockerfile

Dockerfile is a file where you provide the steps to build your Docker Image.

Images

An image is a read-only template with instructions for creating a Docker container. Often, an image is based on another image, with some additional customization.

A docker file is used to build a docker image and a docker image is used to create containers.

For example, you may build an image that is based on the Ubuntu image but installs the Apache web server and your application, as well as the configuration details needed to make your application run.

Docker runtime

It basically allows us to start and stop the container.

there are two runtimes for docker

run c
containerd

low-level runtime is called runc.

another type of runtime is container d. This lies on higher level of the hierarchy and its role is to manage run c

it also helps in getting data from the network i.e; generally communication between the container and the interne.

Docker engine

it is used to interact with docker.

ex:- docker daemon

we can write commands using docker CLI which uses REST API to communicate with docker daemon to perform specific tasks.

it uses a client-server architecture.

OCI - OPEN CONTAINER INITIATIVE

There were multiple runtime like runc, containerd, rkt etc. which leads to confusion so folks developed OCI

The OCI is an open governance structure for the express purpose of creating open industry standards around container format and runtime.

it has two specifications

runtime
images

OCI runtime spec defines how to run the OCI image bundle as container.

OCI image spec defines how to create an OCI image, which includes an image manifest, a file system (layer) serialization, and an image config.

Docker Commands - just important once

these are only some important of improtant commands that everyone must know

docker run <image-name>
docker run -it <image-name>
docker pull <image-name>
docker container ls
docker images
docker ps
docker stop <image_id>
docker rm <image_id>
docker inspect <image-id>
docker logs <id> etc.

there are many more commands in docker which i will cover in next blog and also explain these commands.

This is the basic information about the concept of containers and docker i have covered all important topics yet some of the topics like volumes , networking etc i will cover in next blog thankyou for reading . Have a good day