Share volume in docker swarm for many nodes

Question

I'm facing a big challenge. Trying run my app on 2 VPS in docker swarm. Containers that use volumes should use shared volume between nodes.

My solution is:

1. Use plugin glusterFS and mount volume on every node using nfs. NFS generate single point of failure so when something go wrong my data are gone. (it's not look good maybe im wrong)

2. Use Azure Storage - store data as blob ( Azure Data Lake Storage Gen2 ). But my main is problem how can i connect to azure storage using docker-compose.yaml? I should declarate volume in every service that use volume and declare volume in volume section. I don't have idea how to do that.

Docker documentation about it is gone. Should be here https://docs.docker.com/docker-for-azure/persistent-data-volumes/.

Another option is use https://hub.docker.com/r/docker4x/cloudstor/tags?page=1&ordering=last_updated but last update was 2 years ago so its probably not supported anymore.

Do i have any other options and which share volume between nodes is best solution?

Answer 1

There are a number of ways of dealing with creating persistent volumes in docker swarm, none of them particularly satisfactory:

First up, a simple way is to use nfs, glusterfs, iscsi, or vmware to multimount the same SAN storage volume onto each docker swarm node. Services just mount volumes as /mnt/volumes/my-sql-workload On the one hand its really simple, on the other hand there is literally no access control and you can easilly accidentally load services pointing at each others data.

Next, commercial docker volume plugins for SANs. If you are lucky and possess a Pure Storage, NetApp or other such SAN array, some of them still offer docker volume plugins. Trident for example if you have a NetApp.

Third. if you are in the cloud, the legacy swarm offerings on Azure and Aws included a built in "cloudstor" volume driver but you need to dig really deep to find it in their legacy offering.

Four, there are a number of opensource or free volume plugins that will mount volumes from nfs, glusterfs or other sources. But most are abandoned or very quiet. The most active I know of is marcelo-ochoa/docker-volume-plugins

I wasn't particularly happy with how those plugins mounted pre-existing volumes, but made operations like docker volume create hard, so I made my own, but really

Swarm Cluster Volume Support with CSI Plugins is hopefully going to drop in 2021¹. Which hopefully is a solid rebuttal to all the problems above.

¹Its now 2022 and the next version of Docker has not yet gone live with CSI support. Still we wait.

Answer 2

In my opinion, a good solution could be to create a GlusterFS cluster, configure a single volume and mount it in every Docker Swarm node (i.e. in /mnt/swarm-storage).

Then, for every Container that needs persistent storage, bind-mount a subdirectory of the GlusterFS volume inside the container.

Example:

services:
  my-container:
    ...
    volumes:
      - type: bind
        source: /mnt/swarm-storage/my-container
        target: /a/path/inside/the/container

This way, every node shares the same storage, so that a given container could be instantiated indifferently on every cluster node.

You don't need any Docker plugin for a particular storage driver, because the distributed storage is transparent to the Swarm cluster.

Lastly, GlusterFS is a distributed filesystem, designed to not have a single point of failure and you can cluster it on as many node you like (contrary to NFS).