Check out the eks versions faq at the bottom of this page:
Q: What happens on the end of support date?
A: On the end of support date, [...]. Existing control planes are automatically updated by Amazon EKS to the earliest supported version through a gradual deployment process after the end of support date. [...]
End of support date for 1.23 is October 11, 2023
Also about timing:
Q: When exactly is my control plane automatically updated after the end of support date?
A: Amazon EKS can't provide specific time frames. Automatic updates can happen at any time after the end of support date.
so you should do it yourself to keep control. Otherwise it could happen any time after the EOL date.
They also state why they do it like that:
Q: Can I leave my control plane on a Kubernetes version indefinitely?
A: No. Cloud security at AWS is the highest priority. [...]
But the worker nodes are under your domain
Q: Are Amazon EKS managed node groups automatically updated along with the cluster control plane version?
A: No. A managed node group creates Amazon EC2 instances in your account. [...] The Kubernetes version that's on your managed node group might be more than one version earlier than your control plane. [...]
so your dockershim use-cases should be save.