Create multiple EC2 instances using python script (in loop iteration)

Question

I Have JSON file in which email address and count of instances are specified.I'm using python script to iterate through that file and get username/count number (count number will be used as number of EC2 instances), output from JSON file:

DJukes
1
JWilson
2
eflame
3

So, for DJukes one instance will be created,JWilson 2,eFlame 3, also these names will be usernames for new instances.

Idea is to create as many instances as it's specified in JSON file based on username and count values

When creating manually i got

Terraform will perform the following actions:

-/+ aws_instance.win-example (new resource required)
      id:    

                   "i-073bda5b30f10d3ba" => <computed> (forces new resource)

I passed these variables to python-terraform wrapper (https://github.com/beelit94/python-terraform)

If only one user/count exists in JSON file, no issues, but as soon i add more entries in JSON, then machine for DJukes is created then terminated,same for JWilson, only last created machine remains running. Can someone point out what's wrong with this code ?

#!/bin/python
import json
from pprint import pprint
from python_terraform import *




def myfunc():

  tf = Terraform(working_dir='/home/ja/terraform-course/demo-2b', variables={'count':count,'INSTANCE_USERNAME':user})
  tf.plan(no_color=IsFlagged, refresh=False, capture_output=False)
  approve = {"auto-approve": True}
  print(tf.plan())
  print(tf.apply(**approve))
  return

    json_data=open('./my.json')
    data = json.load(json_data)

    json_data.close()

    for i in range (0, len (data['customers'])):
        #print data['customers'][i]['email']
        k=data['customers'][i]['email']
        #print(k.split('@')[0])
        user=k.split('@')[0]
        #print(user)
        count=data['customers'][i]['instances']
        #print(count)
        #enter = int(input('Enter number of instances: '))
        myfunc()

windows.tf:

resource "aws_key_pair" "mykey" {
  key_name = "mykey"
  public_key = "${file("${var.PATH_TO_PUBLIC_KEY}")}"
}

resource "aws_instance" "win-example" {
  ami = "${lookup(var.WIN_AMIS, var.AWS_REGION)}"
  instance_type = "t2.medium"
  count="${var.count}"

  key_name = "${aws_key_pair.mykey.key_name}"
  user_data = <<EOF
<powershell>
net user ${var.INSTANCE_USERNAME} '${var.INSTANCE_PASSWORD}' /add /y
net localgroup administrators ${var.INSTANCE_USERNAME} /add

winrm quickconfig -q
winrm set winrm/config/winrs '@{MaxMemoryPerShellMB="300"}'
winrm set winrm/config '@{MaxTimeoutms="1800000"}'
winrm set winrm/config/service '@{AllowUnencrypted="true"}'
winrm set winrm/config/service/auth '@{Basic="true"}'

netsh advfirewall firewall add rule name="WinRM 5985" protocol=TCP dir=in localport=5985 action=allow
netsh advfirewall firewall add rule name="WinRM 5986" protocol=TCP dir=in localport=5986 action=allow

net stop winrm
sc.exe config winrm start=auto
net start winrm
</powershell>
EOF

  provisioner "file" {
    source = "test.txt"
    destination = "C:/test.txt"
  }
  connection {
    type = "winrm"
    timeout = "10m"
    user = "${var.INSTANCE_USERNAME}"
    password = "${var.INSTANCE_PASSWORD}"
  }

tags {
Name="${var.INSTANCE_USERNAME}"
}
}

Answer 1

The main issue I see is that you are calling Terraform at the end of every python loop. This is creating new "state" file. That will make it really hard to manage what you are creating. This is how I would do it.

users = []
count = 0
for customer in data['customers']:
    username = customer['email'].split('@')[0]
    instances = customer['instances']
    count += instances
    users.extend([username] * instances)
    #print(Added {} to build list with {} instances.".format(username, instances))
myfunc(count, users)

then I would change...

def myfunc(count, users):

  tf = Terraform(working_dir='/home/ja/terraform-course/demo-2b', variables={'count':count,'INSTANCE_USERS':users})
  tf.plan(no_color=IsFlagged, refresh=False, capture_output=False)
  approve = {"auto-approve": True}
  print(tf.plan())
  print(tf.apply(**approve))
  return

and finally the Terraform.

resource "aws_key_pair" "mykey" {
  key_name = "mykey"
  public_key = "${file("${var.PATH_TO_PUBLIC_KEY}")}"
}

resource "aws_instance" "win-example" {
  ami = "${lookup(var.WIN_AMIS, var.AWS_REGION)}"
  instance_type = "t2.medium"
  count="${var.count}"

  key_name = "${aws_key_pair.mykey.key_name}"
  user_data = <<EOF
<powershell>
net user ${var.INSTANCE_USERS[count.index]} '${var.INSTANCE_PASSWORD}' /add /y
net localgroup administrators ${var.INSTANCE_USERS[count.index]} /add

winrm quickconfig -q
winrm set winrm/config/winrs '@{MaxMemoryPerShellMB="300"}'
winrm set winrm/config '@{MaxTimeoutms="1800000"}'
winrm set winrm/config/service '@{AllowUnencrypted="true"}'
winrm set winrm/config/service/auth '@{Basic="true"}'

netsh advfirewall firewall add rule name="WinRM 5985" protocol=TCP dir=in localport=5985 action=allow
netsh advfirewall firewall add rule name="WinRM 5986" protocol=TCP dir=in localport=5986 action=allow

net stop winrm
sc.exe config winrm start=auto
net start winrm
</powershell>
EOF

  provisioner "file" {
    source = "test.txt"
    destination = "C:/test.txt"
  }
  connection {
    type = "winrm"
    timeout = "10m"
    user = "${var.INSTANCE_USERNAME[count.index]}"
    password = "${var.INSTANCE_PASSWORD}"
  }

tags {
Name="${var.INSTANCE_USERS[count.index]}"
}
}

What I did was change your code so we now create a list of the users, that match the length of the count. That way we can call Terraform just once, and use the count.index on our users list to make sure we create the correct amount of instances per user.

You can read more about count.index here. I have always found reading and understanding more complicated code makes me better. Try reading some of the modules from the Terraform Registry. The vpc module gives good examples of using count.

If you still need help drop me a comment.