13

I am trying to automate my AWS application deployment with Jenkins.

Right now if we want to update the application in any of the environments, say UAT, we build our docker images, find the current ECS Task and update it with the new image, find the running ECS Cluster, and update the tasks.

Broadly, what is best practice for keeping track of cloud resource Id's (ECS Cluster Id's, ECS Task Id's, EC2 Id's, etc) in your Continuous Integration environment?

8

A judicious usage of one of the cloud "orchestration" tools, such as Terraform or possibly Fugue seems to be the best way.

You can start small, pick a less important and not very extensive environment, carefully decipher it into automatable code and proceed from there.

Broadly this is referred to as infrastructure-as-code, for googling and other buzzword-oriented media applications.

More specifically, say, if you had your environment described in Terraform's HCL files, you would have required ID's exported as "outputs" and could operate on those from scripts/Jenkins jobs/etc

  • 1
    Thanks, looks like terraform might be an alternative to what i am using. Doing some research. – Chris Mar 2 '17 at 4:21
  • Using a cloud orchestration tool to do my deployments and updates is the answer. I am actually using CloudFormation to automate the initial build, I just didn't realize I could use it to update the piece I was looking for yet. – Chris Mar 3 '17 at 15:30
  • That is one way to handle the visibility issue. I am currently using TotalCloud [totalcloud.io]. It provides a visual topological view of the resources that are present on my cloud accounts. – Veer Abheek Singh Manhas Feb 6 '18 at 6:34
1

Netflix have presented their method of dealing with the problem of resource sprawl. They created the Janitor Monkey which is responsible to clean-up resources when it is apparent that those are not being used.

To track the created resources in Amazon AWS, generate an audit log, and allow to search through time (and history) they also created Edda. Edda allows to store and search through your AWS resources.

There is some mention that Janitor Monkey might be integrated with Edda, but they have not published much information about these since.

Today with AWS CloudTrail and AWS Config Rules it is possible to achieve similar results without third-party software. When Edda and Janitor Monkey were announced (2012-2013) both of these services were not yet ready to solve these kinds of problems.

With CloudTrail and Config, the tracking of resources is already in place. All to do is write a script that will decide what to do with these by going over the list from time to time.

  • 1
    Thanks, looking at your links, aws may have a facility for this, so i may be looking in the wrong spot. – Chris Mar 2 '17 at 4:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.