Simplify and Automate AWS EC2 Management using Python

Introduction

Managing AWS EC2 instances manually can be time-consuming and inefficient, especially when dealing with multiple instances. However, by leveraging the power of automation, you can significantly simplify and streamline this process. In this blog post, we will explore how to automate the management of AWS EC2 instances using AWS Lambda, EventBridge, and the Boto3 Python library.

Creating the Lambda Function

AWS Lambda is a serverless compute service provided by Amazon Web Services (AWS). It allows you to run your code without provisioning or managing servers. With Lambda, you can focus solely on writing your application logic while AWS handles the underlying infrastructure, including scaling, patching, and availability.

To create an AWS Lambda function for checking and starting EC2 instances, follow these steps:

Sign in to the AWS Management Console.
Open the AWS Lambda service.
Click on "Create function" to create a new Lambda function.
Choose the "Author from scratch" option.
Provide a suitable name for your function (e.g., "EC2InstanceAutomation").
Select the desired runtime. In this case, choose "Python 3.10" or any other compatible runtime.
Click on "Create function" to create the function.

Let's create a new function:

Successfully created the function EC2InstanceAutomation

Setting the Lambda permissions

In the function's configuration page, scroll down to the "Permissions" section.

Under "Execution role," click on the role name next to the "View the <role_name> role" link. This will open the IAM console in a new tab.

Click on the "Attach policies" button to add the necessary permissions to the role.

In the policy details page, click on the "Edit policy" button.

The policy document editor will open, displaying the JSON representation of the policy.

Modify the policy document by adding the necessary EC2 permissions to the existing policy.

{
      "Effect": "Allow",
      "Action": "ec2:*",
      "Resource": "*"
}

Update the policy document as per your specific requirements, ensuring you include the necessary actions and any additional conditions or resource restrictions as needed.

Once you have made the required changes, click on the "Review policy" button to validate the policy syntax.

After reviewing the policy, set this as a new version(check the box) and click on the "Save changes" button to update the policy attached to the execution role.

Note: It's crucial to follow the principle of least privilege and grant only the specific permissions required by the Lambda function. Regularly review and audit the permissions granted to ensure they align with your security requirements.

Now that the execution role has been updated with the necessary EC2 permissions, now let's go back to the Lambda function.

Writing Lambda with Python and Boto3

In the function's configuration page, scroll down to the "Function code" section.
Choose the inline code editor option.

Better refer to this doc for a better understanding of how to use Boto3 to manage AWS services

https://boto3.amazonaws.com/v1/documentation/api/latest/index.html

Replace the existing code in the editor with the following code snippet:

 import boto3

 def lambda_handler(event, context):

     # create EC2 client
     ec2 = boto3.resource('ec2')

     for instance in ec2.instances.all():
         print(instance.state)

Click on "Deploy" or "Save" to save the Lambda function code.

In the function's configuration page, scroll down to the "Function overview" section.

Click on the "Select a test event..." dropdown and choose "Configure test events".

In the test events page, click on the "Create new test event" button.

But we don't have EC2 instances to test, let's create some EC2 instances

Let's stop 2 instances and will do the test,

Now go back to the code and test it,

This is the test output:

{'Code': 80, 'Name': 'stopped'}
{'Code': 80, 'Name': 'stopped'}
{'Code': 16, 'Name': 'running'}
{'Code': 16, 'Name': 'running'}
{'Code': 16, 'Name': 'running'}
{'Code': 80, 'Name': 'stopped'}

If the instance is stopped, let's start it again:

import boto3

def lambda_handler(event, context):

    # create EC2 client
    ec2 = boto3.resource('ec2')

    for instance in ec2.instances.all():
        state = instance.state['Name']
        if state == 'stopped':
            instance.start()

Now refresh the EC2 management console tab, all stopped instances are started:

import boto3

def lambda_handler(event, context):

    # create EC2 client
    ec2 = boto3.resource('ec2')

    for instance in ec2.instances.all():
        state = instance.state['Name']
        if state == 'stopped':
            try:
                instance.start()
            except:
                print("Something went wrong")

That's it our code is done 🎉

The Lambda function is done, we use boto3, and we can talk to the EC2 instance, now we need to trigger that lambda every hour.

Creating the EventBridge cron

To write Cron expression you can refer to this site:

https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/ScheduledEvents.html

And now we have a trigger that executes on every hour

Remember to delete your EC2 instances

Conclusion

In conclusion, we have discussed the process of automating AWS EC2 instance management using AWS Lambda, Python, and Boto3. Here's a summary of the steps involved:

Creating an AWS Lambda Function:
- Use the AWS Management Console to create a new Lambda function.
- Choose the appropriate runtime, such as Python 3.8.
- Configure the function's name and execution role.
Writing Lambda Function Code:
- Use the inline code editor in the AWS Management Console to write the Python code.
- Utilize the Boto3 library to interact with the EC2 service.
- Implement logic to check for stopped instances and start them if necessary.
Setting the Lambda Permissions:
- Configure the execution role associated with the Lambda function.
- Use the AWS Management Console's IAM service to edit the role's permissions.
Creating a Test Event:
- Use the AWS Management Console to create a test event for the Lambda function.
- Simulate a scheduled event or provide a sample payload.
- Modify the event payload to include stopped instances for testing purposes.

By following these steps, you can automate the process of checking for stopped EC2 instances and starting them using an AWS Lambda function triggered by EventBridge. This automation helps ensure that your EC2 instances remain in the desired state and can be efficiently managed.

And don't forget to connect with us on social media to stay updated with the latest tips, tutorials, and guides:

Connect with us on LinkedIn: shaikahmadnawaz
Follow us on Twitter: shaikahmadnawaz

We also encourage you to check out our GitHub repository for more code samples and projects:

Explore our GitHub: shaikahmadnawaz

Simplify and Automate AWS EC2 Management using Python and Boto3

Table of contents