4

I have an app with a bunch of Lambdas. Instead of adding one CloudWatch alarm for each is there a way to combine all of them into one alarm that goes off if any of the Lambdas error out?

So far I tried:

  • Adding each function as a dimension (CloudFormation complains about duplicate FunctionNames)
  • Adding each function as a dimension with a different key than FunctionName, CloudFormation doesn't complain, but it doesn't really work either.
1
  • 3
    What did you try so far? Commented Dec 24, 2019 at 13:23

4 Answers 4

1

One possible solution is to use Metrics and combine all dimensions using Expression: MAX([d1,d2,...]). Just make sure to pass 'ReturnData: false' on the other metrics.

1
1

You have your metrics per lambda function but you also have overall metrics that include data for all functions. You can just alarm on that.

You can find them in CloudWatch Console Metrics view by selecting All > Lambda > Across All Functions

These metrics don't have any dimensions, just the namespace and metric name, example source of a graph would be:

{
    "metrics": [
        [ "AWS/Lambda", "Errors" ]
    ],
    "view": "timeSeries",
    "stacked": false,
    "region": "eu-west-1",
    "stat": "Sum",
    "period": 300
}
1
  • 2
    That's helpful, but I needed only a subset of the lambdas, not one, not all. Commented Dec 25, 2019 at 1:56
1

You should create Alarm using metric math expression on the metrics (plural) required.

In this way you can control the metrics you need and not use "All > Lambda > Across All Functions.".

https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/Create-alarm-on-metric-math-expression.html

5
  • Exactly what I ended up doing, but then you lose the granularity of knowing which function errored out. Not that big of a problem if you have two or three, but checking 10ish is a bit of a pain... Commented Dec 25, 2019 at 15:15
  • @FábioDias so any solution for it? and what expression you used for it?
    – Tula
    Commented May 20, 2020 at 8:09
  • @Tula one alarm per lambda function, to keep granularity. Templated that out into the same default file and moved on Commented May 21, 2020 at 23:29
  • @FábioDias Instead, you can directly configure sns as a lambda function destination right? In this way, you'll get the error message too
    – Tula
    Commented Jun 2, 2020 at 12:30
  • @Tula, I always had an SNS as AlarmActions of the alarm, but the point is that I didn't want to create one alarm per lambda, but I wanted the alarm itself to tell me which lambda errored out, which afaik, it doesn't Commented Jun 3, 2020 at 1:18
1

I came across this page while looking for a solution and learned about composite alarms from a comment above. So to pay it forward, here's some sample code in Python that I used to get the composite alarm working. I don't know if this is the best way to do this, but it works for my use case.

from aws_cdk import aws_cloudwatch as cw

lambda_alarms = {}
for lambda_def in lambda_defs:
    lambda_name = lambda_def['name']
    lambda_object = _lambda.Function()
    
    error_metric = lambda_object.metric_errors(
        period=aws_cdk.Duration.minutes(5),
        statistic="Sum",
        label="Total errors past five minutes"
    )

    lambda_alarms[lambda_name] = cw.Alarm(self, f"{lambda_name}-alarm",
        comparison_operator=cw.ComparisonOperator.GREATER_THAN_THRESHOLD,
        threshold=10,
        evaluation_periods=1,
        metric=error_metric
    )
        
alarm_rule = cw.AlarmRule.any_of(*lambda_alarms.values())

# the name here will show up in the email subject
composite_alarm = cw.CompositeAlarm(self, "LambdasCompositeAlarm",
    alarm_rule=alarm_rule
) 

topic_name = "Lambda Alarm Notifications"
error_topic = sns.Topic(self, topic_name)

email_subscription = subs.EmailSubscription('[email protected]')
error_topic.add_subscription(email_subscription)

alarm_action = cw_actions.SnsAction(
    topic=error_topic
)
composite_alarm.add_alarm_action(alarm_action)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.