3

Assume I have two clusters devcluster and qacluster on ECS. How to remove an EC2 instance (ecs-optimized amazon-linux) from qacluster and add to devcluster?

6

You need to follow these steps:

To make sure it uses correct cluster, check logs - docker logs ecs-agent. It should have a message like:

[INFO] Registration completed successfully. I am running as 'arn:aws:ecs:eu-west-1:ACCOUNT_ID:container-instance/CLUSTER_ID' in cluster 'devcluster'
3
  • 1
    You should avoid publishing any ressource arn without masking your account id and the ressource id in my opinion. I've edited and cleared the the revision history. – Tensibai Jan 24 '18 at 15:54
  • @Tensibai you're totally right. But it was actually masked as I have replaced last 6 digits with zeros ;) But this change would not harm. – lagivan Jan 24 '18 at 22:31
  • There's a small chance it match someone else account number, prefer replacing with xxxx than with 0 to avoid bots scanning the internet launching attacks :) – Tensibai Jan 24 '18 at 23:13
2

The way a cluster becomes aware of the EC2 instances associated to it, is a configuration file used by the ECS agent. You can modify this file, located at /etc/ecs/ecs.config, and name a different cluster. Then restart the ECS agent. This will effectively "move" the EC2 instance to the other cluster.

More information about ecs.config in the documentation - http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-config.html

After the modification, it is possible to verify by sending an HTTP request to the ECS Agent metadata service and see the new cluster displayed. http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-update.html

The ECS cluster itself also needs to be notified. Can use the Register/Deregister API calls for this, for example using the AWS CLI:

More housekeeping might be required on the ECS agent itself. Like removing cached checkpoint files, etc. https://github.com/aws/amazon-ecs-agent/issues/139#issuecomment-123061463

2
  • i modified /etc/ecs/ecs.config to ECS_CLUSTER=devcluster and restarted ec2 instance. but the instance still appear on qacluster – sith Sep 18 '17 at 5:19
  • @sith I updated the answer with details about registration of instances in the ECS Cluster metadata. Might need to be updated as well, or else the cluster does not know about a new instance or about a removed one. – Evgeny Zislis Sep 18 '17 at 7:06
0

Updated for ECS optimized AMI image as of May 2021. The sudo start command is not present and the checkpoint file /var/lib/ecs/data/ecs_agent_data.json changed to agent.db.

  1. Edit /etc/ecs/ecs.config as previously described (add ECS_CLUSTER=devcluster).
  2. Rename the agent state file with mv /var/lib/ecs/data/agent.db /var/lib/ecs/data/agent.db.old
  3. Restart the ECS agent using sudo systemctl restart ecs

Check the status of the agent with systemctl status ecs, you want it to display Active: active (running).

This was a bit frustrating to figure out because the Amazon online documentation I found still displays the old information. Couldn't have done it without devops!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.