This problem is caused when the person (admin) that tried to execute the pipeline replay had insufficient access to the repository they replayed it on. This can happen when the for example the pipeline is private and they're not member, but an admin hits the replay button. The admin's replay does not have the ability to download from the cache. This bug is reported as issue 21700 upstream.
For a temporary workaround, you may opt to make the repository public if possible. Then the admin can replay the job.