GlusterFS
From Bashlinux
Facilitates distributed filesystems across the network, it runs on user space.
Setup
How to install GlusterFS with 2 masters
- Install glusterfs and dependencies
# apt-get -y install glusterfs-server glusterfs-client ntp
- Make peers to look for each other, by running on each node at the same time
# gluster peer probe $PEER
- On the primary do:
- Create the volume
# gluster volume create $VOL replica 2 transport tcp $PRI:$VOL_SERVER $SEC:$VOL_SERVER $FORCE
- Start the volume
# gluster volume start $VOL
- Grant access to any client on the same subnet
# gluster volume set $VOL auth.allow $NET.*
- Check the volume status on any node
# gluster volume info
How to replicate a filesystem across two storage servers (Legacy)
The input file must be from any mount point in the client side, from there the data is split up and send to each storage server.
Server side of the paired master
# file: /etc/glusterfs/glusterfsd.vol volume posix type storage/posix option directory /data/export end-volume volume locks type features/locks subvolumes posix end-volume volume brick type performance/io-threads option thread-count 8 subvolumes locks end-volume volume server type protocol/server option transport-type tcp option auth.addr.brick.allow * subvolumes brick end-volume
Client side of the paired master
# file: /etc/glusterfs/glusterfs.vol volume remote1 type protocol/client option transport-type tcp option remote-host storage1.example.com option remote-subvolume brick end-volume volume remote2 type protocol/client option transport-type tcp option remote-host storage2.example.com option remote-subvolume brick end-volume volume replicate type cluster/replicate subvolumes remote1 remote2 end-volume volume writebehind type performance/write-behind option window-size 1MB subvolumes replicate end-volume volume cache type performance/io-cache option cache-size 512MB subvolumes writebehind end-volume
How to mount a volume on a client
- Create an fstab entry
<PEER_IP>:<VOL> <LOCAL_MOUNT> glusterfs defaults,_netdev,backupvolfile-server=<LOCAL_IP> 0 0
- Create the proper direcectory and mount it
# mkdir -p <LOCAL_MOUNT> # mount <LOCAL_MOUNT>
How to mount a volume on a client (Legacy)
The fstab entry for glusterfs client must be: /etc/glusterfs/glusterfs.vol /dest/folder glusterfs defaults 0 0
Troubleshooting
- How deal with a volume that can't be created because a previous attempt failed
- ERROR MESSAGE:
volume create: <VOL>: failed: /var/lib/glusterfs-server/<VOL> or a prefix of it is already part of a volume
- Remove the extended attributes to the directory created by gluster
# setfattr -x trusted.glusterfs.volume-id /var/lib/glusterfs-server/<VOL> # setfattr -x trusted.gfid /var/lib/glusterfs-server/<VOL> # rm -rf /var/lib/glusterfs-server/<VOL>/.glusterfs