we have two independent kubernetes clusters that work like Main and it's high available one. and we have some stateful applications that store data within cluster's persistent volumes. my question is .. can I share a PV between those two clusters, so I can have a final result of two pods running on the two clusters reading/writing on the same volume as if they're running on one same cluster. I'm focusing on sharing data between the pods regardless how the pods would handle the data writing.
1 Answer
Can you share a PV, between clusters: in theory yes. You can mirror one PV object into another cluster, given you have similar storage classes and accesses to your storage backend.
Sharing PV between Pods, within one or more clusters: depends. First, on your storage backend. There may be some locking mechanism that would prevent you from attaching the same volume to more than one client at a time.
In addition of which: if your storage backend provides with block devices, your volumes would involve some file system, that may not support being mounted on two systems at once.
To work around those two points above, we could consider something like NFS, CephFS, EFS (aws), a samba share, ... Next, it depends on your application: what are the risks of those two Pods of yours writing the same file at once, and corrupting your data?
Short answer: in theory yes, you could. Although if possible: try to question this requirement. Sounds unusual, may be something wrong with design/architecture.
-
I got your point. I'm planning to have one master pod to write and the other pods sharing the storage would only read the data so I'm breaking the reading process which is the main big fat one over multiple pods on multiple clusters. but to conclude it's only about mirroring the storage and not pointing to the same storage. did I get you correctly? Commented Aug 28, 2022 at 13:29
-
You could point your PV to the same storage, if your storage backend allows it (cephfs, nfs, ...). Although won't scale for ever: if you're struggling with reads, a single NFS server would eventually have the same limitation. Depending on your application: adding new processes on the same filesystem might not help at all.– SYNCommented Aug 28, 2022 at 13:36
-
@SherifHamdy If you're planning for one pod to write to this storage, what happens when the cluster with that pod becomes unavailable and your High Availability architecture switches to the other cluster? Will your architecture stop writing to the storage until the writer pod is launched on the other cluster? Commented Aug 29, 2022 at 4:41
-
@Sotto then the other pod is promoted to be the master and start the write and the gtm record will failover to the HA cluster in the writing ingress Commented Aug 29, 2022 at 5:59