How to Manage AWS EC2 Infrastructure with Python

In this article, we'll take a look at how system administrators can use Python to manage their Amazon Web Services (AWS)  Elastic Compute Cloud (EC2) infrastructure. You’ll learn to query existing EC2 instances, change the state of the virtual machines and their configurations, or terminate any instances that are no longer required.

Before we can proceed, make sure to setup your Python scripting environment for the first use, by referring my previous article here.

List All AWS EC2 Instances

Now that we've set up the scripting environment and configured our credentials, let’s begin by importing the boto3 library and using it to create an EC2 resource, which acts like a connection to access to AWS EC2 console. I’ll use this EC2 resource to iterate through all EC2 instances, then access the individual properties of each virtual machine and print the list of all EC2 instances and their respective properties on the console.

import boto3
 ec2 = boto3.resource('ec2')
  
 for instance in ec2.instances.all():
     print(
         "Id: {0}\nPlatform: {1}\nType: {2}\nPublic IPv4: {3}\nAMI: {4}\nState: {5}\n".format(
         instance.id, instance.platform, instance.instance_type, instance.public_ip_address, instance.image.id, instance.state
         )
     )

You can save the above Python script under the python file name: 'list_ec2.py' and call it to get the list of AWS EC2 instances, as demonstrated in the following image. 

Starting, Stopping, and Rebooting EC2 Instances

To change the state of an EC2 instance you have to target it using the EC2 instance id, by passing the id to the 'Instance()' method and then using the 'stop()' method to stop the EC2 instance. 

import boto3
ec2 = boto3.resource('ec2')
ec2.Instance('i-00434b87058703892').stop()

When you want to start the machine, use the 'start()' method to start the instance. 

ec2.Instance('i-00434b87058703892').start()

Similarly, instances can be rebooted using the 'reboot()' method. 

ec2.Instance('i-00434b87058703892').reboot()

NOTE: There is a difference between reboot and stop/start here. With Amazon EC2 instances a reboot is exactly like restarting your local physical computer, but when you stop an EBS boot instance, you are releasing the physical hardware was running on. This means the released instance could be used by EC2 to start someone else's instance on that hardware. Whereas, the EBS boot volume and other attached EBS volumes are still preserved. 

Modifying EC2 Instance Properties

Oftentimes you will be required to modify the EC2 instance properties. For example, you can also use a python script to scale EC2 instances up or down. To modify an EC2 instance, first you need to stop the instance, and wait until it is properly stopped. Then, using the instance id, you can specify the attribute and a new value with the 'modify_instance_attribute()' method, which will change the EC2 instance attribute. In our case, that attribute is 'instanceType'. Once that is done, start the instance with the modified properties. 

import boto3
ec2 = boto3.client('ec2')

# choose an EC2 instance with id
instance_id = 'i-05ca5f05f965b3a4b'

# Stop the instance
ec2.stop_instances(InstanceIds=[instance_id])
waiter=ec2.get_waiter('instance_stopped')
waiter.wait(InstanceIds=[instance_id])

# Change the instance type
ec2.modify_instance_attribute(InstanceId=instance_id, Attribute='instanceType', Value='t2.small')

# Start the instance
ec2.start_instances(InstanceIds=[instance_id])

Terminating an EC2 Instance 

Not only can we create and modify an EC2 instance—boto3 also provided a method to terminate instances that are no longer required. If following code sample is saved in a file named terminate_ec2.py,

import boto3, sys
ec2 = boto3.resource('ec2')

# iterate through instance IDs and terminate them
for id in sys.argv[1:]:
 instance = ec2.Instance(id)
 print(instance.terminate())

Then, from a terminal, we can pass instance ID(s) as a command line argument, to terminate these EC2 instances in the AWS console, using the following command:

python terminate_ec2.py i-0e42d934a8c2d8337

To wrap up everything we've learned so far: In this article we covered how to list all existing EC2 instances, and how to change the state of these virtual machines. We also learned to modify EC2 instances properties to scale the virtual machines size up or down, and to terminate any instances that were no longer required in Python.