I am running my application (4 services) in AWS Fargate. Containers are stored in AWS ECR. Last month I ended up with a huge bill for transferred data because of constant download container from ECR into Fargate tasks (more than 1 TB last month due to issues inside my containers, so they were not able to start properly). So I have decided to tweak networking somehow, but I am stuck.
According to the topology picture below, I have split connections into 3 groups (subnet-to-subnet, subnet-to-AWS, subnet-to-net).
In my initial network topology, I have used NAT Gateway between subnet with Fargate containers and the rest of the world (also ECR), which lead to massive costs (NAT GW runtime + transfer), but covered connection (2) and (3). So I have removed NAT GW.
I have tried to implement VPC Endpoint between Fargate and ECR (2) -> and I think this works fine.
However I do not know how to solve (3), maybe here is NAT necessary?
Also, I can observer some issues with a connection from subnet-to-subnet (1), also endpoint has no usage there, so this should be solved using basic AWS Routing?
Summ: How to improve networking for all three categories, avoid downloading images from ECR through NAT GW and keep the connection to the remote server?
Thank you very much