Docker Learning Notes — Part 1: Containers, Images, Storage, Networking and More

Introduction

This article is part of a series based on the notes I took while learning Docker. As I continue exploring Docker and container technologies, I plan to organize and publish these notes in several parts, with each article focusing on a different group of concepts.

While learning Docker, I realized that memorizing commands alone was not enough. To use containers confidently, I also needed to understand what happens behind the Docker CLI: how containers differ from virtual machines, how images are built from layers, how Docker Engine, containerd, and runc work together, and how storage, networking, logging, resource limits, and environment variables affect running containers.

These articles are reorganized versions of my personal learning notes. The series will begin with the fundamental concepts of containerization and gradually move toward practical Docker CLI usage, storage, networking, logging, and other important Docker components.

Some sections also include short notes about Linux, operating systems, and system-level concepts. Topics such as processes, namespaces, cgroups, filesystems, standard streams, shells, and environment variables are not exclusive to Docker, but they are essential for understanding how containers work behind the scenes.

The purpose of this series is not to replace the official Docker documentation. Instead, it aims to provide a structured and beginner-friendly reference for people who want to understand both Docker commands and the underlying system concepts.

1. Containers and Core Concepts

1.1. What Is a Container?

A container is the executable form of an application in which all dependencies required for the application to run are packaged together. It performs operating-system-level virtualization. It directly uses the kernel of the underlying operating system, but runs in complete isolation from other processes.

1.2. Container Characteristics

• Because containers do not include a complete operating system within themselves, they occupy space at the megabyte level on disk and their memory and CPU consumption is very low.
• Each container has its own file system, process hierarchy, and network interface. The crash or resource consumption of one container does not affect the others.
• Containers are infrastructure-independent. An application runs exactly the same way on test and production servers as it does in the development environment.
• Because a container does not boot its own operating system, starting a container is as fast as executing a standard terminal command.
• From a software perspective, a container is simply a standard Linux process running on the host operating system. The only difference from a standard process is that the operating-system kernel applies strict limitations and isolation rules to that process. A process inside a container cannot see the other processes in the system, is confined to a restricted file directory, and can use only the hardware resources allocated to it.

1.3. Linux Isolation Foundations

1.3.1. Namespaces

A namespace limits what a process can see. It provides logical isolation. The Linux kernel contains six fundamental namespaces: PID, NET, MNT, UTS, IPC, and USER.

1.3.2. Cgroups

While namespaces restrict what a process can see, cgroups restrict what a process can use on the hardware: CPU, memory, and block I/O.

Docker is a high-level interface that automatically configures these low-level and complex tools provided by the Linux kernel.

1.4. Brief History

Container technology existed long before Docker. In 2006, Google added cgroups to the Linux kernel, and LXC emerged when cgroups were combined with namespaces. However, using LXC was a very difficult process that required expert-level manual configuration. Solomon Hykes was working at a PaaS company called dotCloud. The company used LXC to run applications in isolation on servers, but managing these processes was very demanding. Hykes developed an internal tool to make LXC easier to use, package, and transport. In March 2013, he introduced this tool to the world under the name “Docker” at the PyCon conference and released the project as open source.

1.5. Container Lifecycle

• By design, a container initially has only one main command that triggers it. This command creates the container’s main process. The container remains alive as long as this process is running. In other words, at startup, the only thing that brings the container up is the triggering of a single application. However, this can be changed. The application that starts by default is specified in the Dockerfile, but the default application can be overridden by writing a new command in the arguments section at the end of the docker run command. The ideal approach, however, is to run different parts of an application in different containers.
• Technically and theoretically, multiple applications can run inside a container. However, this is not recommended and makes management more difficult.
• The lifetime of a container depends strictly on the lifetime of its main process. The container runs while the main process is running; when the main process stops or crashes, the container shuts down.

2. Docker Images and Layered File Systems

2.1. Docker Image

• A Docker image is a read-only template in which an application and all libraries, configuration files, environment variables, and the runtime environment required for that application are packaged together. It contains all instructions and files required to create and start a container.
• Once an image has been created, its contents cannot be modified. If a code change is made in the application, the existing image is not edited; a new image is built.
• Images are not a single monolithic file. They consist of different file-system layers stacked on top of one another.
• Images are named with tags. This makes it easy to switch between different versions of the same application.
• Images can be stored in centralized repositories called registries, such as Docker Hub, and can be downloaded over the internet and used on different servers.

2.2. UnionFS and the Layered File-System Structure

Docker images do not consist of a single monolithic file block. Instead, they consist of read-only file blocks stacked on top of one another, each containing file changes made relative to the previous version. This architecture is called a layered file-system structure. Every instruction in a Dockerfile that affects the file system physically occupies space.

UnionFS is a technology that hierarchically overlays different physical directories and file systems and presents them to a process as a single root file system. The read-only layers are all the layers that form the image itself, and they are read-only. UnionFS stacks them one under another. When a container is started, UnionFS adds a completely empty, thin, writable read/write layer on top of these layers. This layer is where all new log files, user uploads, or updated data produced while the container is running are stored. The image files in the lower layers remain safe and unchanged. If the container only performs a read operation, it reads files directly from the lower layers. However, when a file in a lower layer is to be deleted or modified, UnionFS takes the original version of the file from the lower layer and copies it into the read/write layer. The file is not modified directly in the lower layer. The original file in the lower layer remains unchanged; it is only hidden from the container by UnionFS.

2.3. Layer Structure

a) Base Layer: This is the lowest starting layer. It usually contains a simplified operating-system core environment or a runtime directly.

b) Instruction Layers: While an image is being built, each command in the Dockerfile creates a new layer. This allows application requirements to be installed step by step.

The layered structure saves disk space. For example, if there are images for different projects but all of them use the same base layer, Docker does not save that base layer to disk multiple times. It downloads the layer once and shares it among the images.

2.4. Difference Between a Container and an Image

• A Docker image is static. It is a collection of files and metadata stored on disk. It does not run by itself. A Docker container is dynamic. It is a running process that uses RAM and CPU resources.
• A Docker image can exist on its own and does not require any container. A Docker container always requires an image and cannot exist without one.
• A Docker image is read-only. A Docker container adds a thin read/write-capable layer on top of the read-only image.
• A Docker image is created by building it and is stored in a repository. It remains persistent until it is manually deleted. A Docker container is started from an image and can be stopped, restarted, or deleted.

3. Docker Engine Architecture

3.1. Docker Engine

Docker Engine is the main software package installed on a computer to create, manage, and run containers. It is an application with a client-server architecture. In the past, Docker Engine was a single massive program. Over time, it was divided into components to make it more secure, standardized, and manageable. Docker Engine currently consists mainly of three principal layers:

1. Docker CLI: The Docker tool used in a terminal or command line. It does not contain any ability to run containers or download images; it is only a client tool. It receives the commands you write, converts them into REST API requests in an HTTP format that the computer can understand, and sends them to the Docker daemon running in the background.
2. Docker REST API: The communication bridge between the CLI and the main service running in the background. It works over the standard HTTP protocol. It allows not only your terminal but also different software applications to communicate with Docker. It forwards commands to the daemon in a standard data format.
3. Docker Daemon: The main service that continuously runs in the background of the operating system. It is the brain of Docker. It is the primary unit that manages images, containers, networks, and storage areas. It listens for requests coming through the REST API and fulfills them. However, in the current architecture, it does not run containers directly. It performs high-level management tasks such as configuring images and determining network settings, and delegates the actual execution operation to the containerd component beneath it.

3.2. containerd

• containerd is the software that runs beneath the Docker daemon and manages the lifecycle of a container. It can operate independently of the daemon. If a container crashes, it does not affect the daemon.
• When it receives a “run” command from the Docker daemon, it downloads the image from the internet if the image is not available on the computer, combines the image layers on disk, prepares the root file system for the container, connects the network interfaces required by the container, and, after all preparations are complete, connects to the operating-system kernel and instructs the runc tool to start the container in practice. While the container is running, it collects its logs and monitors its status.

3.3. runc

runc is a very small, low-level program that communicates with the operating-system kernel. It has one task: starting the container process. It does not download images and cannot configure networks. When it is called by containerd, it uses the namespaces and cgroups features in the Linux kernel to create boundaries and isolation. It places the target application inside this isolated environment and runs it. As soon as the application starts running, its task is complete and control remains with containerd.

Note: Virtualization History and Techniques

Bare Metal

Bare metal is the traditional method in which no virtualization layer is used. The operating system is installed directly on the physical hardware. All resources belong to a single operating system and the applications running on it. There is no intermediary between the hardware and the application, so latency is minimal. However, it causes resource waste, capacity remains unused, and resources cannot be divided for another application. When the hardware fails, the entire system crashes. Installation and configuration times are long.

Virtual Machine

Virtual machines perform hardware-level virtualization. Special software called a hypervisor is installed on the physical server. The hypervisor divides the physical hardware into logical parts. Each virtual machine created has its own full operating system. This provides very strong security and isolation. Even if one virtual machine completely crashes or root access is compromised, the other machines on the same physical server are not affected. However, virtual machines are very heavy.

Container

Containers perform operating-system-level virtualization. Instead of installing a complete operating system in the lower layer, they share the host computer’s own kernel. They do not contain hardware drivers or an operating-system kernel. They contain only the code, libraries, and dependencies of the application to be run. They are extremely lightweight and fast. They occupy space at the megabyte level on disk and do not have a boot process. Their isolation level is not as strict as that of virtual machines. Because they share the same kernel, a security vulnerability in the operating-system kernel can put all containers at risk. In addition, a Windows-based container cannot be run on a computer that uses a Linux kernel.

4. Docker CLI Fundamentals

The Docker CLI is the tool that receives commands written in the terminal and converts them into API requests that the Docker daemon can understand. It has a hierarchical and object-oriented structure. Commands are logically grouped according to the targeted Docker object.

The standard syntax is as follows:

docker [GENERAL OPTIONS] [MANAGEMENT COMMAND] [SUBCOMMAND] [SPECIFIC OPTIONS] [ARGUMENTS]

• docker is the primary keyword that invokes the Docker CLI tool.
• General options are settings that apply to the entire CLI.
• The management command specifies the object on which an operation will be performed, such as container, image, or network.
• The subcommand specifies what will be done to that object, such as run, ls, or rm.
• Specific options are parameters that modify how the command works.
• Arguments specify the name or ID of the target on which the command will operate.

4.1. Management Commands

• docker container …: Manages running or stopped processes.
• docker image …: Manages read-only images stored on disk.
• docker volume …: Manages persistent data-storage areas.
• docker network …: Manages network configurations between containers.
• docker system …: Performs operations related to the Docker infrastructure as a whole.

4.2. Base Commands

In the first versions of Docker, management commands did not exist; every operation was written directly next to the docker keyword. To preserve backward compatibility and speed up frequently used operations, these commands are still valid: docker run, docker ps, docker build, docker pull, and docker rmi.

Docker CLI includes highly detailed built-in documentation. Without needing the internet, you can see what a command does and which parameters it accepts directly from the terminal. The documentation has a hierarchical structure: the deeper you go, the more specific the help becomes.

There are three fundamental levels of use:

a) General Help: Used to see a list of all management commands and base commands.

docker — help

b) Management Command Help: Lists which operations can be performed for a particular object group.

docker image — help
docker container — help

c) Specific Command Help: After finding a command, shows in detail the specific options it can accept and its usage format.

docker run — help
docker container run — help

5. Container Management Commands

5.1. docker container create

This command creates a new container from an image but does not start it. It prepares the file system required for the container, configures its network settings, and assigns an ID and a name. The container remains in the created state.

5.2. docker container run

This command is the most commonly used method in Docker. In the background, it combines the create and start operations in sequence. It first creates the container and then immediately starts it by triggering the main process. If the specified image is not available on disk, it automatically downloads the image from the internet.

General syntax:

docker run [OPTIONS] IMAGE_NAME [COMMAND] [ARGUMENTS]

The syntax of both docker container create and docker container run is completely the same.

• The options section is where properties such as hardware, network, name, or identity are assigned to the container.
• The image name specifies the template from which the container will be produced.
• The command and arguments section is used when the default startup command is to be overridden.

Basic Parameters

• — name <name>: Prevents a random name from being assigned to the container and gives it the specified fixed name.
• -d: Runs the container in the background and does not occupy the terminal.
• -p <host_port>:<container_port>: Performs port forwarding so that a port inside the container can be accessed through the host computer.
• -e <key=value>: Sends environment variables into the container.
• -v <host_directory>:<container_directory>: Connects a directory on the host computer to a directory inside the container. It allows the data to remain persistent even if the container is deleted.

docker run [DOCKER_SETTINGS] IMAGE_NAME [APPLICATION_SETTINGS]

Everything written before the image name is an instruction given to Docker Engine. Everything written after the image name bypasses Docker completely and is forwarded directly to the operating system or application inside the container.

5.3. docker container ls

This is the fundamental information-retrieval command used to list objects associated with Docker containers and display their states. Its main purpose is to print a summary of the current states of objects in the system as a table. When the command is executed for containers, the table contains the following columns:

• CONTAINER ID: The unique alphanumeric identification number assigned to the container.
• IMAGE: The template from which the container was created.
• COMMAND: The main command triggered when the container starts.
• CREATED: How long ago the container was created.
• STATUS: The current state of the container.
• PORTS: Port mappings made between the host machine and the container.
• NAMES: The readable name assigned to the container.

Parameters

• -a: By default, only running containers are displayed. With the -a (all) parameter, all containers, including those in Exited and Created states, are displayed.
• -q: Hides the detailed information in the table and prints only the container ID numbers, one per line.
• -n: If the system contains the history of hundreds of containers, the list becomes difficult to read. This parameter is used to list a specified number of the most recently created containers.
• -f: Narrows the output according to specific rules. It works in the Key=Value format. A search can be performed by status, name, or exit code.

The usage logic of the ls command also applies to other objects in Docker.

5.4. docker container stop container_name/ID

This command is used to stop a container from actually running while continuing to keep its file system and data on disk.

5.5. docker container rm container_name/ID

The rm command is used to completely remove a stopped container from disk, free the space it occupies, and release the name assigned to it. By default, a running container cannot be deleted.

Parameters

• -f: Forces the deletion of a running container.
• -v: When deleting the container, also removes from disk the unnamed data areas created for that container. If this parameter is not used, the anonymous storage areas attached to the container remain on disk even after the container is deleted, causing “dead data” to accumulate over time.

5.6. docker container exec [options] container_name/ID command [Args]

This command is used to send a new process or command into an already-running container. The run command creates and starts a new container from scratch. exec, on the other hand, enters an existing running container. It allows operations to be performed inside without disrupting the existing state, files, or database.

Parameters

• -i (Interactive) and -t (TTY): These are generally combined as -it. They are required to open a terminal session inside the container. This allows you to write commands inside the container as if you were using your own computer. To leave, it is sufficient to type exit in the terminal.

5.7. docker container prune

The prune command is used to collectively delete unused and idle objects that occupy unnecessary disk space in the Docker environment. This command can operate on an object basis, such as container, or on a system basis, such as system.

6. Image Management

6.1. docker pull

docker pull downloads a read-only Docker image from a remote repository such as Docker Hub to the computer.

Get ortatepeyusuf’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

docker image pull [options] image_name:label

By default, Docker automatically searches for the :latest tag. If a specific version is required, it is specified with :.

7. Persistent Storage

7.1. Docker Volume

Containers are temporary by nature. When a container is deleted, all data accumulated in its read/write layer is permanently deleted together with the container. A Docker volume is the fundamental data-storage mechanism used to make the data produced and used by containers independent of the container lifecycle and to store it persistently. Volumes are physically stored on the host operating system’s disk in a special and isolated directory managed directly by Docker.

The data in a volume is not deleted even if containers stop, crash, or are completely removed. A volume works completely independently of the image structure and the UnionFS mechanism. Therefore, disk I/O is higher than in the container’s own read/write layer. The same volume can be attached to multiple containers at the same time. In this way, different containers can simultaneously read from and write to a shared data pool. Although the data is located on the host machine, it is difficult for a normal user or another program to intervene directly. It is isolated so that it is managed only by Docker. Volumes can easily be listed, backed up, or moved to another server through Docker CLI.

A volume is created with:

docker volume create [options] volume_name

7.2. Connecting a Volume to a Container

A volume that has been created, or that will be created automatically, is attached to a container during the docker run command. There are two different parameters for this operation:

a) -v

-v <VOLUME_NAME>:<CONTAINER_DIRECTORY>:<ACCESS_TYPE>

If no value is entered in the access-type section, the volume is automatically attached as read/write. If the ro parameter is added, the container is guaranteed to be able to read the data in the volume but not modify it.

b) — mount

This method was introduced in newer versions of Docker. Its syntax is more explicit and easier to read. It is written as Key=Value pairs, and its sections are separated by commas. It is preferred as the standard in production environments and complex scripts because it improves readability.

— mount type=volume,source=<VOLUME_NAME>,target=<CONTAINER_DIRECTORY>[,ADDITIONAL_SETTINGS]

7.3. Volume-Mount Scenarios

When a volume and a directory are connected in Docker, Docker applies very specific rules according to the initial data state of both sides.

• Empty Volume → Non-empty Directory: Docker copies all original files and directories located inside the container into the empty volume. No data loss occurs.
• Non-empty Volume → Non-empty Directory: The container sees and uses only the files coming from the volume. The container’s original files remain underneath and become inaccessible until the mount operation is removed.
• Non-empty Volume → Empty Directory: The data in the volume is placed directly into the empty directory.
• Empty Volume → Empty Directory: A clean connection is established. From the moment the container starts running, all new data it produces and writes into this empty directory is immediately transferred to the volume on the host machine and becomes persistent.

7.4. What Is a Bind Mount?

A bind mount is a method of connecting a specific directory or a single file in the host machine’s file system directly into a container. With a Docker volume, data is stored in an isolated area under Docker’s own control. With a bind mount, control belongs entirely to you: any full file path you choose on the host machine is directly mapped to a path inside the container.

Bind mounts are used especially during development processes and in system configurations that require external intervention. When a bind mount is applied, the -v or — mount parameters are used just as they are with volumes. However, the most fundamental and mandatory rule is that the path on the host machine must be written as an absolute path.

8. Docker Networking

Docker Network is the virtual network infrastructure that allows containers to communicate with one another, with the host computer on which they run, and with the outside world. Just as a physical computer needs a network card, a cable, and a router to connect to the internet, Docker uses operating-system kernel features to create entirely software-based virtual network cards, virtual switches, and routing rules.

In the Docker architecture, networks are independent first-class objects, just like images or containers. In other words, a network can be created on Docker Engine and exist on its own even if no container is attached to it. When a container runs, it is attached externally to this network object in the same way that a configuration file is attached. Docker provides five network drivers for different needs and use cases.

8.1. Bridge

Bridge is Docker’s default network driver. It is the software representation of a hardware network bridge modeled entirely in the operating-system kernel. Its primary purpose is to allow containers running on the same physical or virtual server to communicate securely with one another and to access the outside world through the host machine. Bridge is only a driver type. Just as dozens of different containers can be created from the same Ubuntu image, dozens of completely independent network objects with different IP blocks can be created by using the Bridge driver.

Docker has two types of Bridge network, and they have different characteristics:

1. Default Bridge Network (docker0): It is automatically created when Docker is installed. If you do not specify a custom network, every newly created container is automatically attached to it. It does not provide automatic DNS resolution. Therefore, containers cannot communicate with one another by using their names; they can communicate only by using IP addresses.
2. User-Defined Bridge Networks: Containers attached to this network do not need to know one another’s IP addresses. They can communicate with one another without problems by using container names directly. Containers in different Bridge networks cannot communicate with one another even if they can see each other’s IP blocks or names. Traffic is blocked at the operating-system level.

Bridge networks are preferred in single-server architectures, in development environments, and in situations where security and isolation need to be provided.

A network can be created with:

docker network create [network_name]

To attach a container directly to a Bridge network that you created while creating the container, add the — network parameter to the run command:

docker run — network custom_project_network …

A running container can be added to a new network or removed from an existing network without stopping it. A container can be attached to more than one Bridge network at the same time.

docker network connect custom_project_network existing_database
docker network disconnect custom_project_network existing_database

Containers on a Bridge network are not completely closed to the outside world. Containers can access the outside world by using the host machine’s internet connection. In the background, the host machine performs network address translation and sends the container’s request outward as if it were its own request. An external user cannot directly reach the container’s IP address. For a request coming from outside to reach the container, an open “door” must be defined on the host machine.

Docker assigns each container a virtual Ethernet card as if it were a physical computer. The containers are located in the same network block. When these two containers communicate with each other, they use the virtual network switch. This operation is not reflected on the host machine’s physical external network; it takes place entirely inside the computer in a secure and isolated manner. If an external user wants to access one of these containers, traffic is forwarded inward through the router and the bridge.

8.2. Host

Host is the driver that completely removes the network-isolation wall between the container and the host operating system. The container does not create its own private network namespace. Docker does not assign a private IP address to this container. Instead, the container directly shares the network stack and physical or virtual network interface of the machine on which it runs. When an application inside the container is started, the application can directly open and listen on a port of the host machine. Port forwarding or network address translation is not required. This driver is preferred when high performance and the use of many ports are required.

Syntax:

docker run — network host image_name

When an external user sends a request to port 80 of the address 192.168.0.11, the packet does not pass through any NAT operation, routing rule, or virtual bridge. As soon as the packet reaches the eth0 interface, it is directly received and processed by the web service inside the container. At the network layer, the blue box and the green box have exactly the same authority and position in the eyes of the operating system.

8.3. None

As its name suggests, None is the driver that completely isolates the container from every type of network connection. Docker does not assign a virtual Ethernet card or an IP address that would allow a container started with this network type to communicate with the outside world. The container contains only a loopback interface so that it can perform operations within itself. The container cannot access the internet, cannot send data to the host machine over the network, and cannot communicate with any other container.

docker run — network none image_name

Host and None networks can only be defined when the container is created and cannot be disconnected afterward.

8.4. macvlan

Instead of assigning containers a virtual IP address, the macvlan driver assigns them a MAC address that is valid directly on the physical network. In standard Bridge or Host networks, containers cannot be seen directly by devices on the physical network; physical devices see only the host machine. When macvlan is used, physical routers, switches, and firewalls on the network see the container as a separate physical computer connected to the network by cable, completely independent of the host machine.

8.5. Overlay

Overlay is a distributed network driver that combines Docker daemons running on multiple different physical or virtual servers under a single logical network. It allows containers running on servers in different cities or data centers to communicate securely and in an encrypted manner as if they were inside the same computer and connected to the same local network. It is mainly integrated with Docker Swarm.

9. Container Interaction and Terminal Access

9.1. docker attach

This command connects directly to the main process of a container that is already running. It does not start a new process or a new terminal session inside the container. Instead, it connects the current terminal to the standard input, standard output, and error streams that the container’s main process has kept open in the background since the container started running.

9.2. Terminal Parameters and Shells

The functions of the flags and shell types commonly encountered when establishing a terminal connection to containers are as follows.

Terminal Parameters

These parameters are used with the docker run or docker exec commands to configure the terminal environment.

• -i: Keeps the container’s standard input channel open.
• -t (tty): Creates a virtual TTY inside the container.

Shell Types

To access the container terminal, the shell program to be run is written at the end of the command.

• /bin/bash: An advanced, full-featured terminal shell that is included by default in Linux distributions.
• /bin/sh: A very basic, lightweight, and simple terminal shell that complies with POSIX standards.

10. Logging and Monitoring

10.1. docker logs

This command displays in the terminal the standard output and error messages sent to the operating system by a running or stopped process. It is used to understand why an application crashed and to watch incoming requests or performed operations live through the terminal. Logs are associated with processes. Therefore, they cannot be used with static objects. They are used with two different object types: container and service (docker container logs, docker service logs).

Note: Standard Streams in Linux and Unix

Every program run on Linux- and Unix-based operating systems is automatically started by the operating system with three fundamental communication channels. A program receives data from the outside world through these channels and sends the data it produces to the outside world through these channels.

• STDIN: The channel through which a running program listens for data from outside. On a standard computer, the default source of the STDIN channel is the keyboard.
• STDOUT: The channel through which a successfully running program sends its normal results and informational messages outward. Its default destination is the terminal screen.
• STDERR: A separate channel through which a program sends error messages and diagnostic warnings when an unexpected situation occurs, an error is encountered, or a crash happens. Like STDOUT, the default destination of STDERR is the terminal screen.

Modern containerized applications do not write their logs to static text files such as /var/application/application.log, as older systems did. Instead, the main application inside the container writes normal logs directly to STDOUT and error logs directly to STDERR. When Docker Engine runs the container, it hooks into the process’s STDOUT and STDERR channels. Whatever the process sends through these channels is immediately captured by Docker and stored on disk in the specified format. When the logs are inspected, Docker prints a combined dump of the STDOUT and STDERR data to the screen.

10.2. Log Drivers

Log drivers are plugins that receive the standard output and error messages produced by a container and transmit them to a specified destination in a specified format. Captured logs do not have to be written only to the local disk; they can be directed to enterprise centralized log-management systems. The most commonly used types are:

• json-file: Docker’s standard log driver. It formats logs produced by the container as JSON and physically stores them on the host machine.
• syslog: Sends logs to the Syslog service of the host machine or of a remote server on the network. Syslog is a standard protocol used to collect all server logs in one place.
• journald: Sends logs to systemd-journald, the core logging service of the Linux operating system.
• splunk: Sends logs directly through HTTP Event Collector to the Splunk server, an enterprise data-analysis and logging platform.
• fluentd: Sends logs to the open-source Fluentd data-collector service.

Without changing global settings, the — log-driver and — log-opt parameters are used during the docker run command to send the logs of only a specific container to a different location. To use a log driver, the target service must be running, and Docker must connect to that service with the required authentication and address information. Locally stored logs can be accessed with:

docker logs container_name

10.3. docker stats

The docker stats command is a live monitoring tool that shows how much CPU, memory, network, and disk resources containers are currently consuming. It creates a continuously updating table in the terminal and reflects the metrics second by second.

docker stats [OPTIONS] [CONTAINER_NAME_OR_ID…]

When used without parameters, it prints the metrics of all containers currently running in the system as a live stream.

10.4. docker top

This is the container-based equivalent of the classic top or ps command on Linux systems. It lists in the terminal all processes that are currently active inside a specified container.

docker top CONTAINER_NAME_OR_ID [ps OPTIONS]

It operates on only one container at a time. With this command, without entering the container through exec, you can immediately see whether the main application inside the container and the child processes it creates are running, under which user’s privileges they were started, and which PID numbers they have.

11. Resource Limits

By default, Docker does not apply any hardware limitation to a container that it runs. A single container is free to consume all resources of the host machine on which it runs. However, this can cause a malfunctioning or heavily loaded application to lock the entire system. Hardware limits are used to prevent this. These limits are normally configured by adding parameters to the run and create commands while the container is being created and started.

11.1. Memory Limits

Memory limits determine how much RAM and swap space the container can actively use before writing to disk.

— memory=x, — memory-swap=y

11.2. CPU Limits

CPU limits control at what intensity and on which physical processor cores the container can use the host machine’s CPU cores.

— cpus=’floating_point_value’, — cpuset-cpus=’comma_separated_cpus’

12. Environment Variables

Note: General Linux and System Environment Variables

Environment variables are dynamically named references that store data as key-value pairs defined at the operating-system level or within a runtime environment. Every process and program running on a computer or inside a container can access this list of variables stored in the operating system’s memory and shape its own configuration according to these values. There are two fundamental and critical reasons for using them:

1. Security: Writing critical information such as database passwords, API keys, or private access tokens directly into an application’s source code creates a major security vulnerability. When the code is uploaded to a version-control system, these passwords are exposed. With environment variables, only the variable name appears in the source code; the actual password is injected into the application externally by the environment in which the code runs.
2. Portability and Configuration Management: An application needs to connect to different databases or ports in development, test, and production environments. Instead of changing the code for each environment, the application remains the same and only the environment variables are changed. This allows the same image to be used repeatedly with different configurations, especially in container architectures consisting of multiple services.

Environment variables in the system can be listed with:

env
printenv

To read the value of one specific variable instead of the entire list, the echo command can be used in the terminal:

echo $PATH

This prints only the directory paths stored by the PATH variable.

A new variable can be created in the terminal by using the export command:

export API_PORT=8080

As soon as this command is executed, a new variable named API_PORT with the value 8080 is created in that terminal session. However, it is temporary; when the terminal is closed or the container is restarted, the variable is deleted. The value of a previously defined variable can be changed in the same way.

12.1. Docker Environment Variables

Docker environment variables are the mechanism used to inject configuration settings, API keys, or network ports from outside into applications running inside containers. They are the fundamental tool that allows an application to run under different conditions without changing its source code or rebuilding the Docker image. Docker environment variables can be defined at two different stages. These stages determine the persistence of the variable and whether it can be overridden.

a) Build Time

While a Docker image for an application is being prepared, default environment variables can be embedded in the configuration file named Dockerfile by using the ENV instruction. This is used to define default settings that are absolutely necessary for the application to run and that will be used if the user does not provide a value externally.

b) Run Time

This is the most common use case. When a container is started from an image, values are provided with docker run or docker create to override the default values in the Dockerfile or to send completely new settings. Sensitive data such as database passwords or API keys, which would create a security vulnerability if embedded in the image, is transferred only into the temporary memory of the container that will run at that moment. There are two methods:

1. Single Pass (-e or — env): Used when only one or a few environment variables need to be defined. They are written directly on the command line.

docker run -e KEY=VALUE [IMAGE_NAME]

1. Multiple Pass ( — env-file): Used when dozens of environment variables need to be transferred into the container at once and securely. In this method, all variables are stored in a physical .env file on the host machine, and only the path of this file is provided to Docker.

docker run — env-file [FILE_PATH] [IMAGE_NAME]

Sources I Used

While preparing and reviewing these notes, I mainly referred to the following resources:

• Docker Official Documentation: https://docs.docker.com/
• containerd Documentation: https://containerd.io/docs/
• Open Container Initiative: https://opencontainers.org/
• Linux Kernel Documentation: https://docs.kernel.org/