We’ve all copy-pasted docker run or docker exec a thousand times. It’s the standard developer reflex. But for the longest time, I treated Docker as this magical black box that just “ran lightweight VMs.”

Then I had to debug a weird file-lock bug on a production database container and realized my mental model was completely wrong. Docker containers aren’t virtual machines. They are just standard Linux processes running inside custom boundaries.

If you want to move from blindly typing commands to actually understanding what your server is doing, you need to know what happens to the Linux kernel when you hit Enter. Here is my practical guide to daily Docker commands, explained in simple steps through the lens of namespaces, cgroups, and OverlayFS.

---

1. The Creation: docker build

Before you can run a container, you have to build an image.

docker build -t my-app .

Here is how Docker builds your image layers step-by-step:

1. Send the Context: Docker compresses the current directory (excluding files in .dockerignore) and sends it to the Docker daemon.
2. Execute Instructions: For each line in your Dockerfile (like RUN apt-get update), Docker spins up a temporary container.
3. Commit the Layer: Once the instruction finishes, Docker captures the modifications in the temporary container’s Read-Write layer. It freezes this layer, saves it as a new read-only layer with a unique content ID, and discards the temporary container.
4. Cache the Result: If you run the build again and the instruction hasn’t changed, Docker simply reuses the cached read-only layer, saving precious compilation time.

---

2. The Startup: docker run

When you type docker run -d -p 8080:80 --name web nginx, Docker doesn’t boot an operating system. Instead, the Docker daemon performs a sequence of system calls on your host kernel.

docker run -d -p 8080:80 --name web nginx

Here is exactly what that command does in simple steps:

Step 1: Pull and Stack the Image (OverlayFS)

If the nginx image isn’t local, Docker fetches it from the registry. An image is just a stack of read-only tarballs (layers) created during the build phase. Docker mounts these layers on top of each other using OverlayFS (a union mount filesystem). It then slaps a thin, empty Read-Write layer (often called the “container layer”) right on top. Any file changes you make while the container runs go into this top layer, leaving the base image untouched.

Step 2: Slice the Kernel namespaces

Docker asks the Linux kernel to clone a new process but isolate it using namespaces. Namespaces partition kernel resources so the process thinks it’s the only one on the machine:

• PID namespace: The process gets assigned PID 1 inside the container, even though it might be PID 34521 on the host.
• NET namespace: It gets its own virtual loopback interface and a private IP address.
• MNT namespace: It gets a private root filesystem (/) rooted in the OverlayFS mount.
• UTS namespace: It gets its own hostname (the container ID).

Step 3: Apply the Limits (Cgroups)

Docker creates a control group (cgroup) folder on the host (usually under /sys/fs/cgroup/). If you passed limit flags like --memory="512m", Docker writes 536870912 (in bytes) to the cgroup configuration file. The Linux kernel will now aggressively throttle or kill the container process if it tries to consume more than that limit.

Step 4: Map the Ports

The -p 8080:80 flag tells Docker to configure network address translation (NAT) rules in the host’s routing table (using iptables or nftables). This redirects any incoming traffic on host port 8080 straight to port 80 inside the container’s private network namespace.

---

3. Listing and Status Checking: docker ps and docker images

Listing what you’ve got running is the most common muscle-memory action in Docker.

# List active containers
docker ps

# List all containers (including stopped ones)
docker ps -a

# List cached images
docker images

Here is what’s happening under the hood:

docker ps & docker ps -a

The Docker daemon keeps a local database of all containers in /var/lib/docker/containers/. When you run ps, Docker doesn’t do a full system scan.

1. It queries its internal metadata list.
2. For active containers, it verifies that the container’s main PID is still running on the host.
3. For ps -a, it includes stopped processes whose namespaces have been torn down, but whose read-write OverlayFS layers and metadata files are still stored on the disk.

docker images

This simply reads from the local image store metadata. Under /var/lib/docker/image/overlay2/imagedb/, Docker stores JSON files describing the stack of layers that make up each image. Running docker images parses these JSON files and presents the list of tag names and sizes.

---

4. Entering the Container: docker exec

When a container goes sideways, your first instinct is to shell into it:

docker exec -it web /bin/bash

(A quick note: The -it flag stands for interactive and pseudo-TTY. It hooks your terminal’s standard input/output directly to the process inside the container.)

Here is the magic behind exec:

1. Locate the target: Docker finds the PID of the main container process running on the host.
2. Call setns: Docker spawns your /bin/bash shell on the host, but calls the Linux setns() system call. This system call tells the host kernel: “Shove this new bash process into the namespaces of the target container PID.”
3. Run inside: Now, your host shell is looking through the same isolated window. When you type ps aux or ls, you see the container’s processes and files, not the host’s.

It’s a beautiful illusion: you aren’t “logged in” to a VM; your local bash process was simply tricked into sharing the container’s isolation boundaries.

---

5. Reading Logs and Checking Ports: docker logs and docker port

Getting output and checking network maps are essential debugging steps.

# Fetch container logs
docker logs --tail 100 web

# Verify port mappings
docker port web

docker logs

When processes run inside a container, their standard output (stdout) and standard error (stderr) streams are intercepted by the container runtime. The runtime writes these streams directly to a JSON file on the host disk. On a standard Linux install, you can find the raw log files here:

sudo cat /var/lib/docker/containers/<container-id>/<container-id>-json.log

When you run docker logs, Docker is literally just reading that text file from the host filesystem, formatting it, and printing it to your terminal.

docker port

This command looks up the NAT tables. Docker queries the virtual network bridge (like docker0) and the IP forwarding rules (iptables) it configured on startup to tell you exactly how traffic is routed into the container’s network namespace.

---

6. Monitoring in Real-Time: docker stats and docker top

When the server feels sluggish, you need to see who is eating the RAM.

# Real-time resource stream
docker stats

# Process list inside container
docker top web

docker stats

How does Docker calculate memory and CPU usage in real-time? It reads the cgroup files directly! On the host, the kernel exposes metrics inside virtual files. For example, Docker reads memory usage by loading:

/sys/fs/cgroup/memory/docker/<container-id>/memory.usage_in_bytes

It does some quick math with the limits and presents a beautiful live terminal dashboard. It’s a lightweight wrapper around kernel cgroup statistics.

docker top

This command lists processes running inside the container. Under the hood, Docker queries the host’s standard ps process table, but filters the output to only show processes sharing the target container’s PID namespace. No processes are hidden from the host system.

---

7. Moving Files: docker cp

Copying configuration files or log archives in and out of containers is a daily necessity:

# Copy from host to container
docker cp ./config.conf web:/etc/nginx/nginx.conf

# Copy from container to host
docker cp web:/var/log/nginx/access.log ./local-access.log

Here is why docker cp is so fast: Under the hood, Docker doesn’t run a file transfer server or require SSH to be active inside the container. Because the container filesystem is just a directory on the host (mounted via OverlayFS), docker cp is literally a normal file copy (cp) behind the scenes from/to the path /var/lib/docker/overlay2/<merged-dir>/.

Docker resolves the container path on the host filesystem and uses standard file write operations to move the data instantly.

---

8. Storage and Volume Management: docker volume

Because container filesystems are ephemeral (wiped out when the container is deleted), we use volumes and bind mounts to persist data.

# Create a named volume
docker volume create db-data

# Mount it during run
docker run -d -v db-data:/var/lib/mysql --name database mysql

What actually happens on the host disk?

Step 1: Create the Directory

When you run docker volume create db-data, Docker creates a physical directory on the host’s hard drive at:

/var/lib/docker/volumes/db-data/_data

Step 2: Bind Mount at Startup

When the container starts, Docker calls the Linux kernel mount() function with the MS_BIND flag (a bind mount). This bypasses the OverlayFS layer completely. The host directory is mapped directly into the container’s Mount (MNT) namespace at /var/lib/mysql. When database engines write data inside the container, they write directly to the host disk at native speeds, totally ignoring the container’s overlay system.

---

9. Communication: docker network

Containers need to talk to each other and to the outside world.

# Create a network
docker network create my-net

# Connect container to network
docker network connect my-net web

Here is how Docker bridges containers in simple steps:

1. Create the virtual bridge: When you create a network, Docker configures a virtual Ethernet bridge interface on the host (named something like br-xxxxxx, visible when you run ip link on the host).
2. Create the virtual wire: When a container connects, Docker creates a veth pair (virtual ethernet cable).
3. Plumb the interfaces: Docker plugs one end of the virtual cable into the host’s bridge interface, and places the other end inside the container’s private Network (NET) namespace.
4. Assign IPs: Docker runs a mini DHCP/DNS server inside the daemon to assign a private IP (e.g. 172.18.0.2) to the container’s interface, allowing direct, isolated container-to-container communication.

---

10. Multi-Container Orchestration: docker compose

Managing ten separate containers with complex networks, volumes, and environmental variables via the CLI gets messy. That’s why we use Docker Compose.

# Start the stack in the background
docker compose up -d

# Stop and clean up the stack
docker compose down

What is Compose actually doing? It’s not a different runtime. It is simply a smart orchestrator that runs on your local machine and uses the Docker API.

Here is what happens when you run docker compose up -d:

1. Parse the YAML & Build the Startup Graph: Compose reads docker-compose.yml. It builds a dependency tree using the depends_on keys to determine the exact start order (e.g., Database must boot before API backend).
2. Create the Project Network: By default, Compose creates a custom project-wide virtual bridge network (e.g. app_default). This enables automatic container service discovery—your backend container can reach your database container by simply typing host: database instead of hardcoding IP addresses.
3. Send API Requests: Compose connects to the host’s Unix socket (/var/run/docker.sock) and makes sequential HTTP API requests to the Docker Engine to build, run, and link the containers.
4. Enforce State Declarations: Compose compares the declared config in your YAML against the active containers. If a container’s environment variables or image tag changed, Compose stops and rebuilds only that specific container, leaving the rest of the stack untouched.

When you run docker compose down, Compose reverses the process: it gracefully stops the containers, tears down the custom network bridge, but safely leaves your named volumes intact.

---

11. Stopping and Killing: docker stop vs docker kill

When it’s time to shut down a container, you have two choices:

docker stop web
# or
docker kill web

They do very different things under the hood:

docker stop (The Polite Request)

When you run stop, Docker sends a SIGTERM (Signal 15, terminate) to the main process (PID 1) inside the container. This tells the application: “Hey, we are shutting down. Finish your current database queries, close open connections, and exit cleanly.” Docker then waits patiently. By default, it waits 10 seconds. If the process is still running after that grace period, Docker loses patience and sends a SIGKILL to force-terminate it.

docker kill (The Hostile Takeover)

docker kill bypasses the pleasantries. It immediately sends SIGKILL (Signal 9) to the container process. The kernel instantly tears down the process, leaving database transactions half-written and network connections abruptly severed. Use this only when your application has locked up completely.

---

12. Cleanups: docker rm and docker rmi

Over time, your host gets cluttered with dead containers and old images.

docker rm web
docker rmi nginx

Here is what happens to the files:

• docker rm: Tries to delete the container. This destroys the thin Read-Write layer associated with that container ID. Any logs, temp files, or application data written during its execution are permanently deleted from the host disk. If you mounted a persistent volume, that data survives because it lives outside the OverlayFS container layer.
• docker rmi: Tries to delete the image. It checks if any running or stopped containers are still referencing the image layers. If not, it deletes the read-only layer files from /var/lib/docker/overlay2/.

If you just want a clean slate without thinking, this is the ultimate GC command:

docker system prune -a --volumes

This identifies any container that is stopped, any network not in use, any dangling image layers, and wipes them out. It’s a satisfying way to recover 20GB of disk space in three seconds.

---

13. The Host-Level Internals Cheatsheet

If you want to prove to yourself that Docker is just normal Linux processes, run a container and execute these commands directly on your Linux host terminal.

First, spin up a detached sleep container:

docker run -d --name test-sleep alpine sleep 3600

Now, let’s inspect it from the host’s perspective:

Find the Host PID

Every container process has a standard PID on the host. Let’s find it:

# Query the container metadata and format the output to get the PID
PID=$(docker inspect -f '{{.State.Pid}}' test-sleep)
echo "The host PID is: $PID"

Inspect the Namespaces

Look at the symbolic links in the host /proc directory for that PID. These point to the specific kernel namespaces the process belongs to:

sudo ls -l /proc/$PID/ns/

You’ll see distinct descriptors for net, mnt, pid, and others. If you compare these IDs with your host’s own namespace IDs (ls -l /proc/self/ns/), you’ll see they are completely different. That’s the isolation in action.

Inspect the Cgroups

To prove that the container’s memory is restricted by the kernel, look at the cgroup configuration:

cat /proc/$PID/cgroup

This shows the path to the cgroup slices. You can navigate into /sys/fs/cgroup/system.slice/ (on modern systemd distros) and inspect the exact CPU shares and memory limits allocated to that container.

---

The View from Here

Docker’s abstractions are incredibly convenient. The CLI hides all the complex system calls and namespace plumbing behind a clean interface.

But hiding the complexity has a cost. When a container runs out of memory, or when a port mapping breaks, or when disk space vanishes, the Docker abstraction layer stops being helpful. You’re forced to look at the host level.

Understanding that containers are just processes isolated by namespaces, limited by cgroups, and reading/writing to OverlayFS changes how you write software. You stop treating containers like fragile, heavy virtual machines, and start treating them for what they actually are: highly portable, cleanly packaged, vanilla Linux processes.