Docker network problem – Linux – The Next Generation

While creating new docker instances I recently often got the following error message:

linux # docker compose up
[+] Running 1/1
 ✘ Network roundcube_default  Error                                                                                                                                                      0.0s 
failed to create network roundcube_default: Error response from daemon: all predefined address pools have been fully subnetted

In the beginning it helped to prune old network configs:

linux # docker network prune
WARNING! This will remove all custom networks not used by at least one container.
Are you sure you want to continue? [y/N] y

But now I’m at a point where this does no longer help.

Maybe the number of subnets is limited?

linux # docker network ls -q | wc -l
33

So I currently got 33 subnets configured for docker … sounds close enough to 32 …

linux # docker network inspect $(docker network ls -q) | jq -r '.[] | "\(.IPAM.Config[0].Subnet // "N/A"): \(.Name)"' | sort -n
N/A: host
N/A: none
172.17.0.0/16: bridge
172.18.0.0/16: toolXYZ01_default
172.19.0.0/16: toolXYZ02_default
172.20.0.0/16: toolXYZ03_default
172.21.0.0/16: toolXYZ04_default
172.22.0.0/16: toolXYZ05_default
172.23.0.0/16: toolXYZ06_default
172.24.0.0/16: toolXYZ07_default
172.25.0.0/16: toolXYZ08_default
172.26.0.0/16: toolXYZ09_default
172.27.0.0/16: toolXYZ10_default
172.28.0.0/16: toolXYZ11_default
172.29.0.0/16: toolXYZ12_default
172.30.0.0/16: toolXYZ13_default
172.31.0.0/16: toolXYZ14_default
192.168.1.0/24: macvlan1
192.168.16.0/20: tool101XYZ_default
192.168.32.0/20: tool102XYZ_default
192.168.48.0/20: tool103XYZ_default
192.168.64.0/20: tool104XYZ_default
192.168.80.0/20: tool105XYZ_default
192.168.96.0/20: tool106XYZ_default
192.168.112.0/20: tool107XYZ_default
192.168.128.0/20: tool108XYZ_default
192.168.144.0/20: tool109XYZ_default
192.168.160.0/20: tool110XYZ_default
192.168.176.0/20: tool111XYZ_default
192.168.192.0/20: tool112XYZ_default
192.168.208.0/20: tool113XYZ_default
192.168.224.0/20: tool114XYZ_default
192.168.250.0/24: docker1

In the above list there are 2 networks that I created manually: “macvlan1” and “docker1”.

And there are 2x 14 subnets:

14 x 172.x.x.x/16
14 x 192.168.x.x/20

So where do those subnets come from? I couldn’t find any clue about them in the available configs. But I guess I’m not the first to look for this information: Looking for more information I found “The definitive guide to docker’s default-address-pools option” (Thanks to Matthew!).

He explains in detail, that the default “local” networks used are:

172.17.0.0/12 (with a default size of /16)
192.168.0.0/16 (with a default size of /20) (used when the above range is exhausted)

That exactly matches the observed subnets above 🙂

And it creates a conflict with my self-defined macvlan1 (192.168.1.0/24) that I wasn’t aware of till now.

But using /16 subnets (65.534 IPs) for every docker compose I fire up seems a little exaggerated, and even the /20 subnet (still 4.094 IPs) is far from what I need.

So I followed Matthew’s advice and reduced the size of the created subnets (and removed the 192.168.x.x range completely):

linux # vi /etc/docker/daemon.json
{
    "default-address-pools" : [
    { 
      "base" : "172.17.0.0/12",
      "size" : 24
    }
    ]
}

After that I restarted the system.

However that wasn’t sufficient:

Several network configs survived the restart, so I had to manually remove lots of networks and do docker compose down/docker compose up to get all networks to use the new settings … there’s surely a better way to do that, I’ll let you know if I find it 🙂

Leave a Reply Cancel reply