Autoscaling Lifecycle Hooks

AWS Auto Scaling watches your applications and automatically adapts capacity to keep steady, predictable performance at the cheapest potential cost. Utilizing AWS Auto Scaling, it’s convenient to set up application scaling for various resources over multiple services in minutes. Auto Scaling provides us with several features and benefits. For today we will see how to take advantage of the Auto Scaling Lifecycle Hooks.

Amazon EC2 Auto Scaling gives the capability to append lifecycle hooks to your Auto Scaling groups. These hooks allow an Auto Scaling group to be conscious of events in the Auto Scaling instance lifecycle and then execute a custom response when the corresponding lifecycle event happens. In addition, a lifecycle hook gives a stipulated amount of time (60 minutes by default) to perform the lifecycle action before the instance turns to the following state.

Why use Auto Scaling Lifecycle Hooks?

Lifecycle Hooks resolves a big issue in my scenario: Save the Logs from the EC2 before terminating it. Sometimes, bad things happen with our application, and our instances start to fail on the health check, and this makes the Load Balance detects that the instance is not in a healthy state anymore, then change the status from the EC2. 

Later the Load Balance contacts Auto Scaling asking to replace the bad instance. So, whatever the problem with the instance, we will never know the root cause because we will lose the logs on the server.

So, to resolve that issue, we will use the Lifecycle Hook to trigger an event that will save the instance logs to an S3 Bucket, where later you can download the logs and make our analysis.

Before we create the solution, let’s see a summary of which services and resources from AWS it’s required to build the whole solution:

1 – Auto Scaling Template and Group.

2 – AIM Role for the Instance.

3 – CloudWatch 

4 – Lambda Function

5 – IAM Role for Lambda

6 – Document (on SSM, Systems Manager)

You have noticed that we don’t need too many services and resources. And to make this whole solution only requires a few minutes.

How it works Auto Scaling Lifecycle Hooks?

Let’s see how this solution work-flow works:

Autoscaling Lifecycle Hooks
Auto Scaling Lifecycle Hooks

There are two representations for the health status of an Auto Scaling instance: healthy or unhealthy. All instances in your Auto Scaling group begin in a healthy state. Instances are expected to be healthy unless Amazon EC2 Auto Scaling gets a warning that they are unhealthy. This warning can come from one or more subsequent sources: Amazon EC2, Elastic Load Balancing (ELB), or a custom health check.

So, after Auto Scaling receives an unhealthy status, the CouldWatch will catch that event from Auto Scaling. Then, the CloudWatch Rule will trigger a Lambda Function. 

Our Lambda Function was written using Python. And the main goal for this Lamdba function is to start a script (Document) from SSM (Systems Manager, Run Command) that contains the steps we defined to execute in EC2. So, in other words, the steps that we need to do before terminating the instance, in our case, will zip and upload all logs files that it’s interesting for us.

Before terminating the instance, our script uses the SNS to send a notification every time one instance is terminated. However, this step is optional, also requires an IAM Role to allow the Lambda function to call the SNS service.

It’s straightforward. Now let’s see all steps to do it on the AWS Console:

Create the IAM Role

If you already have one existing EC2 with Role, you can add a new inline policy. Like this:

{
 "Version": "2012-10-17",
 "Statement": [
   {
     "Action": [
       "autoscaling:CompleteLifecycleAction",
       "sns:Publish"
     ],
     "Effect": "Allow",
     "Resource": "*"
   }
 ]
}

Also, add the AWS Policy AmazonEC2RoleforSSM, to allow the EC2 instance to act for SSM.

The Role will look like this one:

Autoscaling Lifecycle Hooks
Auto Scaling Lifecycle Hooks

Also, remember to give access to your EC2 instance on the S3 bucket, where you plan to save your logs.

Create IAM Role for the Lambda

Autoscaling Lifecycle Hooks
Auto Scaling Lifecycle Hooks

We need to create a Role for our Lambda function, and we must add the policies, AWSLambdaBasicExecutionRole and AmazonSSMFullAccess. Furthermore, if you need to send emails using Lambda, we need also create an inline policy:

{
 "Version": "2012-10-17",
 "Statement": [
   {
     "Action": [
       "sns:Publish"
     ],
     "Effect": "Allow",
     "Resource": "*"
   }
 ]
}

Create a Lifecycle Hook on the Auto Scaling Group

We need to configure our existing Auto Scaling Group to enable our new Lifecycle and then make it available for the CloudWatch Rule.

Navigate on AWS Console and select your exiting Auto Scaling Group, and on the details screen, go to the “Instance Management” tab.

You will see something like this:

Autoscaling Lifecycle Hooks
Auto Scaling Lifecycle Hooks

Later, click on “Create lifecycle hook,” and then this screen will show up:

Autoscaling Lifecycle Hooks
Auto Scaling Lifecycle Hooks

Give an excellent name for the Lifecycle Hook. Also, selecting the value “Instance Terminate” on the “Lifecycle transition” option means appending one listener to all events that will terminate any EC2 from this Auto Scaling Group. Also, select “CONTINUE” for the “Default Result” option.

Now we have our Auto Scaling prepared and visible on the CloudWatch. It means that this Auto Scaling group will be listed to be used in one Rule from the CloudWatch.

So, when Auto Scaling raises the intention to terminate an instance, CloudWatch needs to be aware that such an event will happen. So, we need to create one Rule on the CloudWatch, then CloudWatch will trigger our Lambda. But, we will create the CloudWatch Rule later.

Create an SSM Document:

An AWS Systems Manager document (SSM document) represents the Systems Manager’s operations on your managed instances. Systems Manager covers more than 100 pre-configured documents that you can reuse by defining parameters at Runtime. You can find Pre-configured documents in the Systems Manager Documents console by picking the Owned by Amazon. Documents utilize JavaScript Object Notation (JSON) or YAML, and they introduce steps and parameters that you define.

In our scenario, we will create our version of the SSM Document. You can download our script from the GitHub account:

Let’s see the steps that we need to do on AWS Console to create our SSM Document.

Under “Management & Governance,” you will find the “Systems Manager” service.

Autoscaling Lifecycle Hooks
Auto Scaling Lifecycle Hooks
Autoscaling Lifecycle Hooks
Auto Scaling Lifecycle Hooks

Give one Name for your SSM Document, and on the content, copy and paste our document.json file from our GitHub repository. Later, click on “Create document“.

Easy, right? Let’s see the details from our Document, and it should look like this one:

Autoscaling Lifecycle Hooks
Auto Scaling Lifecycle Hooks

and on the details, we can see all required parameters that this Document needs:

Autoscaling Lifecycle Hooks
Auto Scaling Lifecycle Hooks

It’s important to highlight that our CloudWatch will provide the parameters ASGNAME, LIFECYCLEHOOKNAME, from this Document. And the other variables our Lambda function will provide. 

Create Lambda Function

The AWS Lambda function utilizes modules incorporated in Python 3.7 and the Python module (boto3). The function executes the following steps:

1. Checks whether the SSM document exists, which has the script that runs on your EC2 Instances.

2. Sends the command to the EC2 instance that is designated for termination.

3. Checks for the status of EC2 Run Command, and if it fails, the Lambda function finishes the lifecycle hook.

Let’s create the lambda function:

  1. Log in to the AWS Lambda console, select Create Lambda function.
  2. For Name, type any name, and for Runtime, choose Python version 3.7.

To simplify the Lambda function code, paste the Lambda function (lambda_ssm_call.py) from the GitHub repository.

Autoscaling Lifecycle Hooks
Auto Scaling Lifecycle Hooks

Later, we need to define the environment variables for our Lambda function. So, we need to specify:

1 – S3BUCKET -> it’s the S3 Bucket name to store the EC2 Logos.

2 – SNSTARGET -> It’s the ARN for your topic on SNS (it’s optional). If you don’t want that, remove line 60 from our SSM Document.

3 – SSM_DOCUMENT_NAME -> it’s the SSM Document that we created before.

Will looks like this one:

Autoscaling Lifecycle Hooks
Auto Scaling Lifecycle Hooks

We are almost there! I promise you! 

Our final step it’s to create the CloudWatch, to link all services and resources together.

Create CloudWatch Rule

Go to the CloudWatch and later, select the Rule:

Autoscaling Lifecycle Hooks
Auto Scaling Lifecycle Hooks

Now, let’s specify all configurations that we need:

1 – Service Name: Auto Scaling

2 – Event Type: Instance Launch and Terminate

3 – Only select the event “EC2 Instance-Terminate Lifecycle Action”.

4 – Select your Auto Scaling Group.

And for the Targets session:

Select your Lambda function created in the previous step.

Autoscaling Lifecycle Hooks
Auto Scaling Lifecycle Hooks

Finally! We are done will all configurations.

You may be asking yourself how the best way to test it. Let me explain. 

From the Auto Scaling Console, you can adjust the Desired and the Minimum for your testing on your Auto Scaling Group to 0. Next, AWS EC2 Auto Scaling Group tags EC2 Instances for termination, that Instance lifecycle status turned to Termination: Wait, which you can observe from the Instances tab. Later the AWS Lambda function gets triggered by AWS CloudWatch Event.

You can examine your CloudWatch logs to examine the Lambda function output. From the CloudWatch Console, select Logs and then /aws/lambda/{function_name} to view the execution result.

Autoscaling Lifecycle Hooks
Auto Scaling Lifecycle Hooks

Also, you can go to your S3 bucket and check the uploaded files. You can also check AWS SSM Command history from the EC2 Console to see if SSM accurately executed the command.

Conclusion

Now that you’ve observed an example of how you can link several AWS services to automate the backup of your EC2 files by relying solely on AWS services, I hope you are encouraged to design your solutions.

Also, you may notice that the Auto Scaling lifecycle hooks, Lambda, and EC2 Run Command are essential tools because they enable you to react to Auto Scaling events automatically, such as when an instance is terminated. Nevertheless, you can also apply the same approach for other solutions like exiting processes appropriately before an EC2 instance is terminated, deregistering your EC2 from service managers, and scaling stateful services by moving state to other running instances. Your imagination only restricts the possible use cases.

What's your reaction?

Excited
3
Happy
0
In Love
0
Not Sure
0
Silly
0

You may also like

Leave a reply

Your email address will not be published. Required fields are marked *

More in:AWS