6

I have my Beanstalk environment with a "Scaling Trigger" using "CPUUtilization" and it works well.

The problem is that I can not combine this with a system that automatically reboots (or terminate) instances that have been considered "OutOfService" for a certain amount of time.

Into the "Scaling > Scaling Trigger > Trigger measurement" there is the option of "UnHealthyHostCount". But this won't solve my problem optimally, because it will create new instances as far there is one unhealthy, this will provoque my environment to grow until the limit without a real reason. Also, I can not combine 2 "Trigger measurements" and I need the CPU one.

The problem becomes crucial when there is only one instance in the environment, and it becomes OutOfService. The whole environment dies, the Trigger measurement is never triggered.

2
  • Is this a single instance env or does it have an ELB attached to it?
    – strongjz
    Jun 4, 2017 at 0:07
  • @strongjz it is not a single instance, but the solution should work for single instance environments too
    – fguillen
    Jun 4, 2017 at 15:15

4 Answers 4

8

If you use Classic Load Balancer in your Elastic Beanstalk.

You can go to EC2 -> Auto Scaling Groups.

Then change the Health Check Type of the load balancer from EC2 to ELB.

By doing this, your instances of the Elastic Beanstalk will be terminated once they are not responding. A new instance will be created to replace the terminated instance.

3

AWS Elastic Beanstalk uses AWS Auto Scaling to manage the creation and termination of instances, including the replacement of unhealthy instances.

AWS Auto Scaling can integrate with the ELB (load balancer), also automatically created by Elastic Beanstalk, for health checks. ELB has a health check functionality. If the ELB detects that an instance is unhealthy, and if Auto Scaling has been configured to rely on ELB health checks (instead of the default EC2-based health checks), then Auto Scaling automatically replaces that instance that was deemed unhealthy by ELB.

So all you have to do is configure the ELB health check properly (you seem to have it correctly configured already, since you mentioned that you can see the instance being marked as OutOfService), and you also have to configure the Auto Scaling Group to use the ELB health check.

For more details on this subject, including the specific steps to configure all this, check these 2 links from the official documentation:

This should solve the problem. If you have trouble with that, please add a comment with any additional info that you might have after trying this.

Cheers!

2
  • I'm currently moving this into a production stage test and will have to wait for issues to arise before I can completely verify this solution works but the documentation and explanation appear spot on. I will comment once I have confirmed expected behaviour.
    – Techmag
    Oct 22, 2018 at 13:55
  • Only works with .ebextensions, not with namespaces: github.com/aws/elastic-beanstalk-roadmap/issues/35
    – JRichardsz
    Feb 17, 2022 at 0:18
0

You can setup a CloudWatch alarm to reboot the unhealthy instance using StatusCheckFailed_Instance metric.

For detailed information on each step, go through the Adding Reboot Actions to Amazon CloudWatch Alarms section in the following AWS Documentation.

0

If you want Auto Scaling to replace instances whose application has stopped responding, you can use a configuration file to configure the Auto Scaling group to use Elastic Load Balancing health checks. The following example sets the group to use the load balancer's health checks, in addition to the Amazon EC2 status check, to determine an instance's health.

Example .ebextensions/autoscaling.config

Resources:
  AWSEBAutoScalingGroup:
    Type: "AWS::AutoScaling::AutoScalingGroup"
    Properties:
      HealthCheckType: ELB
      HealthCheckGracePeriod: 300

See: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/environmentconfig-autoscaling-healthchecktype.html

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.