Configure VPC for ECR access

Question

I am creating my infrastructure with Ansible. I create a VPC and an ECS cluster where I use Fargate to run my docker containers. When the tasks are started I am prompted that the task was not able to pull the Image from the ECR. STOPPED (CannotPullContainerError: Error response from daem)

I know for a fact that the URI to the image and the IAM permissions are correct. Also I don't think the issue is in the AMI or docker image, because when I create the same scenario over the AWS UI, it works fine.

Therefore, I assume my VPC is not configured correctly in a way that it can not connect to the ECR.

This is how I create my VPC and Subnet:

- name: "2) Create VPC"
  ec2_vpc_net:
    name: "{{vpc_name}}"
    state: present
    cidr_block: 10.10.0.0/16
    region: "{{ region_name }}"
  register: vpc_net

- name: "3) Create VPC Subnet"
  ec2_vpc_subnet:
    state: present
    map_public: yes
    vpc_id: "{{vpc_net.vpc.id}}"
    cidr: 10.10.0.0/24
    tags:
      name: "test-subnet"
  register: vpc_subnet

How can I ensure my VPC and subnet is configured correctly so that the docker daemon can connect and pull the image from the ECR repo? Am I missing an important attribute?

Answer 1

This error CannotPullContainerError usually occurs when you have no access to the Internet from ECS and thus it doesn't able to pull an image from registry.

Make sure you have networking configured in such a way to have an access to Internet https://stackoverflow.com/questions/48226547/aws-fargate-cannotpullcontainererror-500

Manually you can confirm everything with such steps

1. Find route table. Check whether it belongs to your VPC.
2. Find proper NAT (you need NAT Gateway ID) (IMPORTANT: NAT should be launched in PUBLIC subnet)

3. Create a route for that

   $ aws ec2 create-route --route-table-id "rtb-037148d7b5967a231" --destination-cidr-block "0.0.0.0/0" --nat-gateway-id "nat-0c137ae8a2b409088"

You can also use internet gateway instead of NAT but the main idea is the same - to create connectivity from service to registry. If you have that in place and verified you can easily rewrite that to ansible tasks.

Update To start an ECS task you want to use ecs_task module https://docs.ansible.com/ansible/latest/modules/ecs_task_module.html

To pull image from repository and start it you can use these tasks

- name: docker login
  shell: "$(aws ecr get-login --no-include-email --region {{ default_region }})"
  args:
    executable: "/bin/bash"

- name: pull latest app image
  docker_image:
    name: "{{ ecr_repository }}/myapp:{{ image_tag }}"
    force: yes

- name: run application with docker compose
  docker_service:  # module is called docker_service before Ansible 2.8
    project_name: "myapp"
    project_src: "{{ app_dir }}"  # path to a directory containing a docker-compose.yml
    pull: yes
    recreate: always  # run scheduler from a clean state
    remove_orphans: yes  # remove containers for services not defined in the compose file
    state: present  # specifying present is the same as running docker-compose up
    restarted: yes  # use with state present to restart all containers
  register: output

but in case of Fargate the above is not needed as everything is handled by task definition

Answer 2

ECR needs authentication and authorization before you can pull an image.

In your ansible ecs_taskdefinition, please make sure you have parameter task_role_arn or execution_role_arn pointing to a role (i.e. you need to create a role my-task-role1).

Attach the AWS managed policy AmazonECSTaskExecutionRolePolicy to the role (for Fargate tasks, the policy AmazonEKSFargatePodExecutionRolePolicy might be a better choice).

This should provide you with the necessary permissions to download your image from ECR.

---

Policy descriptions from AWS Console:

• AmazonECSTaskExecutionRolePolicy

  Provides access to other AWS service resources that are required to run Amazon ECS tasks

• AmazonEKSFargatePodExecutionRolePolicy

  Provides access to other AWS service resources that are required to run Amazon EKS pods on AWS Fargate

---

Side-note: AWS Console creates ecsTaskExecutionRole automatically if you create a Task Definition there.

Description from "Create new Task Definition" in AWS Console:

This role is required by tasks to pull container images and publish container logs to Amazon CloudWatch on your behalf. If you do not have the ecsTaskExecutionRole already, we can create one for you.