Just to give a bit of background - I have a 3 node GlusterFS cluster running, and I made sure to test it works by mounting a few directories on some servers. I also tested a mount on the Kubernetes cluster (currently just 1 node), to make sure it's not an issue there, and that the gluster-client package is installed.

Been using Rancher as the management overlay for Kubernetes. I manually created a Persistent Volume and selected Gluster plugin. For the endpoint, I had to create custom Endpoints and Service objects.

Here's the error I have been getting:

  Type     Reason       Age                  From                  Message
  ----     ------       ----                 ----                  -------
  Warning  FailedMount  7m8s (x34 over 86m)  kubelet, ssk8smstd01  Unable to mount volumes for pod "hello-7f66d8f588-whxwz_default(6a45cd5e-7259-11e9-ab24-005056aa5a40)": timeout expired waiting for volumes to attach or mount for pod "default"/"hello-7f66d8f588-whxwz". list of unmounted volumes=[pvc1]. list of unattached volumes=[pvc1 default-token-n7tlv]
  Warning  FailedMount  3m2s (x44 over 86m)  kubelet, ssk8smstd01  (combined from similar events): MountVolume.SetUp failed for volume "gluster1" : mount failed: mount failed: exit status 1
Mounting command: mount
Mounting arguments: -t glusterfs -o backup-volfile-servers=ip1:ip2:ip3,log-file=/var/lib/kubelet/plugins/kubernetes.io/glusterfs/gluster1/hello-7f66d8f588-whxwz-glusterfs.log,log-level=ERROR ip1:gv0 /var/lib/kubelet/pods/6a45cd5e-7259-11e9-ab24-005056aa5a40/volumes/kubernetes.io~glusterfs/gluster1
Output: Mount failed. Please check the log file for more details.

 the following error information was pulled from the glusterfs log to help diagnose this issue: 
[2019-05-09 14:19:04.005668] E [MSGID: 100026] [glusterfsd.c:2307:glusterfs_process_volfp] 0-: failed to construct the graph
[2019-05-09 14:19:04.005900] E [graph.c:982:glusterfs_graph_destroy] (-->/usr/sbin/glusterfs(mgmt_getspec_cbk+0x12e) [0x5589853fc38e] -->/usr/sbin/glusterfs(glusterfs_process_volfp+0x148) [0x5589853f6dc8] -->/usr/lib/x86_64-linux-gnu/libglusterfs.so.0(glusterfs_graph_destroy+0x6b) [0x7fa9c305268b] ) 0-graph: invalid argument: graph [Invalid argument]

Has anyone seen something similar to this or have any ideas?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.