GlusterFS

Facilitates distributed filesystems across the network, it runs on user space.

Setup

How to install GlusterFS with 2 masters

• Install glusterfs and dependencies

# apt-get -y install glusterfs-server glusterfs-client ntp

• Make peers to look for each other, by running on each node at the same time

# gluster peer probe $PEER

• On the primary do:

• Create the volume

# gluster volume create $VOL replica 2 transport tcp $PRI:$VOL_SERVER $SEC:$VOL_SERVER $FORCE

• Start the volume

# gluster volume start $VOL

• Grant access to any client on the same subnet

# gluster volume set $VOL auth.allow $NET.*

• Check the volume status on any node

# gluster volume info

How to replicate a filesystem across two storage servers (Legacy)

The input file must be from any mount point in the client side, from there the data is split up and send to each storage server.

Server side of the paired master

# file: /etc/glusterfs/glusterfsd.vol 
volume posix
  type storage/posix
  option directory /data/export
end-volume

volume locks
  type features/locks
  subvolumes posix
end-volume

volume brick
  type performance/io-threads
  option thread-count 8
  subvolumes locks
end-volume

volume server
  type protocol/server
  option transport-type tcp
  option auth.addr.brick.allow *
  subvolumes brick
end-volume

Client side of the paired master

# file: /etc/glusterfs/glusterfs.vol
volume remote1
  type protocol/client
  option transport-type tcp
  option remote-host storage1.example.com
  option remote-subvolume brick
end-volume

volume remote2
  type protocol/client
  option transport-type tcp
  option remote-host storage2.example.com
  option remote-subvolume brick
end-volume

volume replicate
  type cluster/replicate
  subvolumes remote1 remote2
end-volume

volume writebehind
  type performance/write-behind
  option window-size 1MB
  subvolumes replicate
end-volume

volume cache
  type performance/io-cache
  option cache-size 512MB
  subvolumes writebehind
end-volume

How to mount a volume on a client

• Create an fstab entry

<PEER_IP>:<VOL>    <LOCAL_MOUNT>    glusterfs defaults,_netdev,backupvolfile-server=<LOCAL_IP>    0 0

• Create the proper direcectory and mount it

# mkdir -p <LOCAL_MOUNT>
# mount <LOCAL_MOUNT>

How to mount a volume on a client (Legacy)

The fstab entry for glusterfs client must be:
/etc/glusterfs/glusterfs.vol /dest/folder glusterfs defaults 0 0 

Troubleshooting

• How deal with a volume that can't be created because a previous attempt failed

ERROR MESSAGE:

volume create: <VOL>: failed: /var/lib/glusterfs-server/<VOL> or a prefix of it is already part of a volume

• Remove the extended attributes to the directory created by gluster

# setfattr -x trusted.glusterfs.volume-id /var/lib/glusterfs-server/<VOL>
# setfattr -x trusted.gfid /var/lib/glusterfs-server/<VOL>
# rm -rf /var/lib/glusterfs-server/<VOL>/.glusterfs