2

We're using ArgoCD to manage deployments, and I'm in the process of sorting out the config repository. I'm planning on having pull requests for any change to the config, and I want to validate the configuration to ensure it's not breaking anything. I've done some looking around, and it looks like the main options are kubeval, kubeconform or using --dry-run with kubectl.

Due to kubectl actually connecting to the cluster, and have the cluster perform the validation I prefer this approach as it should catch every possible error, however I'm running into a problem.

One of the resources uses generateName which is not compatible with kubectl apply, so if I try and validate using kubectl apply -f manifest.yaml --dry-run=server I get the error cannot use generate name with apply. To get around this, I tried to use kubectl create -f manifest.yaml --dry-run=server but instead I get a load of errors about resources already existing (understandable).

So how can I do this? I can't use apply, and I can't use create. Are there any other options? Does anyone know what Argo uses to validate, because if I push something invalid it presents an error before it is told to sync.

Improve this question
4
  • dealing with generateNames, we might use BeforeHookCreation ( argo-cd.readthedocs.io/en/stable/user-guide/resource_hooks/… ). Although this would not answer your question regarding dry-runs ... leaving ArgoCD context/tip testing kube code: use k3s. You may start a k3s instance inside your Kubernetes cluster (or on your laptop), then work with that disposable API, and delete that container once you're done testing.
    – SYN
    Commented Oct 29, 2022 at 8:26
  • here's a sample, using Tekton, testing a Kubernetes controller, authenticating against some disposable k3s sidecar: gitlab.com/synacksynack/opsperator/docker-operator/-/blob/… . For sure it could be simplified testing kubernetes yamls, helm charts, ... Keep in mind k3s is a lightweight version of kubernetes / might need to customize your k3s installation, or just create fake Custom Resources Definitions, depending on which objects you expect to test
    – SYN
    Commented Oct 29, 2022 at 8:29
  • Reading back your question, I'm having doubts ... regarding the create/apply/generateName thing, abstracting from your stack, probably something like: kubectl create -f manifest.yaml --dry-run -o yaml | kubectl apply --dry-run=server -f-
    – SYN
    Commented Oct 29, 2022 at 8:37
  • 1
    So far I've got close by actually creating a namespace, then running the create command with a dry run flag, then remove the namespace after. I tested it locally and it seemed to work so I'm just plumbing it into the CI at the moment and will report back.
    – PaReeOhNos
    Commented Nov 1, 2022 at 15:37

1 Answer 1

Reset to default
1

After a lot of playing around, I came to a working solution that I briefly mentioned in a comment in the original question. The CI is now creating a namespace on the cluster, running the dry run apply and then deleting the namespace when finished. Not sure if this is the perfect solution but it's working as I hoped.

helm template . \
  --values common/values-common.yaml \
  --values variants/$VARIANT/values-$VARIANT.yaml \
  --name-template=github-actions-test \
  --set image.tag=github-actions-test \
  --namespace $NAMESPACE \
  --debug > dry-run.yaml

kubectl create namespace $NAMESPACE --dry-run=client -o yaml | kubectl apply -f -
echo "errors=$(kubectl create -f dry-run.yaml -n $NAMESPACE --dry-run=server -o yaml 2>&1 > /dev/null)" >> $GITHUB_OUTPUT
kubectl delete namespace $NAMESPACE
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.