2

I am creating my infrastructure with Ansible. I create a VPC and an ECS cluster where I use Fargate to run my docker containers. When the tasks are started I am prompted that the task was not able to pull the Image from the ECR. STOPPED (CannotPullContainerError: Error response from daem)

I know for a fact that the URI to the image and the IAM permissions are correct. Also I don't think the issue is in the AMI or docker image, because when I create the same scenario over the AWS UI, it works fine.

Therefore, I assume my VPC is not configured correctly in a way that it can not connect to the ECR.

This is how I create my VPC and Subnet:

- name: "2) Create VPC"
  ec2_vpc_net:
    name: "{{vpc_name}}"
    state: present
    cidr_block: 10.10.0.0/16
    region: "{{ region_name }}"
  register: vpc_net

- name: "3) Create VPC Subnet"
  ec2_vpc_subnet:
    state: present
    map_public: yes
    vpc_id: "{{vpc_net.vpc.id}}"
    cidr: 10.10.0.0/24
    tags:
      name: "test-subnet"
  register: vpc_subnet

How can I ensure my VPC and subnet is configured correctly so that the docker daemon can connect and pull the image from the ECR repo? Am I missing an important attribute?

  • 1
    If you check on ECR repository you see your image? – Weyder Ferreira Jan 9 at 1:48
  • 1
    Yeah, the image is there and the URI is correct. @WeyderFerreira – Kyu96 Jan 9 at 1:49
  • 1
    Are you sure that you Ansible role/playbook has access on AWS ECR? – Weyder Ferreira Jan 9 at 1:54
  • 2
    is it possible to login to SSH to system, manually login Docker to the ECR and manually pull the image? – Peter Muryshkin Jan 9 at 8:21
  • 1
    Can you edit your question to include the code that creates the Fargate service and task definition? – ydaetskcoR Jan 14 at 18:00
4
+25

This error CannotPullContainerError usually occurs when you have no access to the Internet from ECS and thus it doesn't able to pull an image from registry.

Make sure you have networking configured in such a way to have an access to Internet https://stackoverflow.com/questions/48226547/aws-fargate-cannotpullcontainererror-500

Manually you can confirm everything with such steps

  1. Find route table. Check whether it belongs to your VPC.
  2. Find proper NAT (you need NAT Gateway ID) (IMPORTANT: NAT should be launched in PUBLIC subnet)
  3. Create a route for that

    $ aws ec2 create-route --route-table-id "rtb-037148d7b5967a231" --destination-cidr-block "0.0.0.0/0" --nat-gateway-id "nat-0c137ae8a2b409088"

You can also use internet gateway instead of NAT but the main idea is the same - to create connectivity from service to registry. If you have that in place and verified you can easily rewrite that to ansible tasks.

Update To start an ECS task you want to use ecs_task module https://docs.ansible.com/ansible/latest/modules/ecs_task_module.html

To pull image from repository and start it you can use these tasks

- name: docker login
  shell: "$(aws ecr get-login --no-include-email --region {{ default_region }})"
  args:
    executable: "/bin/bash"

- name: pull latest app image
  docker_image:
    name: "{{ ecr_repository }}/myapp:{{ image_tag }}"
    force: yes

- name: run application with docker compose
  docker_service:  # module is called docker_service before Ansible 2.8
    project_name: "myapp"
    project_src: "{{ app_dir }}"  # path to a directory containing a docker-compose.yml
    pull: yes
    recreate: always  # run scheduler from a clean state
    remove_orphans: yes  # remove containers for services not defined in the compose file
    state: present  # specifying present is the same as running docker-compose up
    restarted: yes  # use with state present to restart all containers
  register: output

but in case of Fargate the above is not needed as everything is handled by task definition

| improve this answer | |
  • Yeah, I do want to use Fargate. Why do I have to create two separate subnets, a private and a public one? I'll have a look and see if I can solve it with the link you referenced. – Kyu96 Jan 16 at 17:29
  • That's a common practice and a more secure way. Obviously you can create everything in a public subnet and in Fargate configuration choose Auto assign public IP address. Here's nice explanation with diagrams to help you sort out the things aws.amazon.com/blogs/compute/task-networking-in-aws-fargate – Most Wanted Jan 17 at 9:47
  • Do you mind expanding your reply with an Ansible script to set up the networking components correctly, as these still confuse me. For this example, security is not a concern. I just try to have a minimal viable example that runs on ECS with Fargate and is reachable from the outside, all setup via Ansible. – Kyu96 Jan 17 at 18:45
1

ECR needs authentication and authorization before you can pull an image.

In your ansible ecs_taskdefinition, please make sure you have parameter task_role_arn or execution_role_arn pointing to a role (i.e. you need to create a role my-task-role1).

Attach the AWS managed policy AmazonECSTaskExecutionRolePolicy to the role (for Fargate tasks, the policy AmazonEKSFargatePodExecutionRolePolicy might be a better choice).

This should provide you with the necessary permissions to download your image from ECR.


Policy descriptions from AWS Console:

  • AmazonECSTaskExecutionRolePolicy

    Provides access to other AWS service resources that are required to run Amazon ECS tasks

  • AmazonEKSFargatePodExecutionRolePolicy

    Provides access to other AWS service resources that are required to run Amazon EKS pods on AWS Fargate


Side-note: AWS Console creates ecsTaskExecutionRole automatically if you create a Task Definition there.

Description from "Create new Task Definition" in AWS Console:

This role is required by tasks to pull container images and publish container logs to Amazon CloudWatch on your behalf. If you do not have the ecsTaskExecutionRole already, we can create one for you.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.