After starting a long running kubernetes job we've found that the final result will fail to upload to its final location. We would like to force the pod to stay open after the main process fails so we can exec in and manually process the final results. If the main process fails and the pod exits before uploading the final result we will lose a large amount of time to re-process the job.

Is there a way to ensure the pod stays alive manually?

2 Answers 2


Keeping your pod alive after its main process exited is an anti-pattern and not possible in K8s.

The entire concept of a K8s Job is to run till completion. So what you are looking to do is neither ideal nor recommended.

It would be better to change the entrypoint command of your container to include a retry/error handling mechanism in case of failure.

With that said, you do have ways in which you can debug the failed pod.

If your Pod's container process exits due to error, the Pod will be marked with Status: Failed

But you can still debug the failed pod by attaching an Ephemeral container: https://kubernetes.io/docs/tasks/debug-application-cluster/debug-running-pod/#debugging-with-ephemeral-debug-container

Note: Ephemeral containers are available as K8s alpha feature only from v1.16

  • If you're using Docker Desktop as your local Kubernetes cluster, you can set the flag by modifying the kubeadm manifest files, as seen here.
    – Venryx
    Commented Aug 29, 2021 at 9:44

I want to add a quick and dirty solution to this. I often add sleep 600 to the end of a command that's failing unexpectedly so that I can exec in and poke around to see what happened. Yaml example (note: - >- enables multi-line concatenation, just a convenience):

        command: ["bash", "-c"]
          - >-
            python myscript.py; sleep 600;

With that said, Karthick's answer is better, this is just a simple solution that doesn't follow best practices. I encourage you not to use sleep or sleep infinity, because it's very easy to forget that your pod is running and holding onto resources indefinitely.

  • 1
    This is the better solution. The above is just the usual rubbish about getting Kubernetes unhelpful status messages
    – MikeKulls
    Commented Sep 7, 2022 at 8:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.