19

How do you connect an aws cloud watch alarm to a lambda function invocation?

I am programmatically adding a cloud watch alarm to the ELBs that we create as part of a cloud formation stack via AWS CloudFormation Templates. I want to have the alerts sent to a lambda function that will post the message to Slack. Although the alert works, and the SNS config seems correct to me, the lambda function is never invoked.

The lambda function follows these examples:

https://medium.com/cohealo-engineering/how-set-up-a-slack-channel-to-be-an-aws-sns-subscriber-63b4d57ad3ea#.x2j9apedu

http://inopinatus.org/2015/07/13/hook-aws-notifications-into-slack-with-a-lambda-function/

The lambda function works, and I can send it test data via the aws console resulting in a message posted to Slack.

The load balancer is created with a correct-looking cloud watch alarm:

enter image description here

The alarm appears to be configured to send alerts to the correct SNS topic:

enter image description here enter image description here

There is an SNS subscription to that topic, with the lambda function as it's endpoint:

enter image description here

Alarms are triggered and messages sent to the correct topic when the alarm fires:

enter image description here

But the lambda function is never invoked:

enter image description here

However, if I manually add the SNS topic as an "event source" on the lambda function, it is invoked when the alarm fires and Slack messages are posted.

enter image description here

Am I misunderstanding how to connect a cloud watch alarm to a lambda function? Or is there a small detail I am missing?

If this approach cannot work, and the only way to connect a lambda function to a cloud watch alarm is to add the SNS topic as an "event source", what is the appropriate way to do that via AWS CloudFormation Templates? I don't see an obvious way to modify an existing resource such as a fixed lambda function.

Here is my CloudFormation Template:

"GenericSlackAlertSNSTopic" : {
    "Type" : "AWS::SNS::Topic",
    "Properties" : {
        "Subscription" : [ {
            "Endpoint" : "arn:aws:lambda:us-east-1:[...]:function:snsToSlack",
            "Protocol" : "lambda"
        } ]
    }
},
"ELBNoTrafficAlarm": {
    "Type": "AWS::CloudWatch::Alarm",
    "Properties": {
        "Namespace" : "AWS/ELB",
        "AlarmDescription": "Alarm for no apparent traffic on an ELB.",
        "AlarmActions": [{
            "Ref": "GenericSlackAlertSNSTopic"
        }],
        "InsufficientDataActions": [{
            "Ref": "GenericSlackAlertSNSTopic"
        }],
        "MetricName": "RequestCount",
        "Statistic": "Sum",
        "Dimensions" : [ {
            "Name" : "LoadBalancerName",
            "Value" : { "Ref" : "ElasticLoadBalancer" }
        } ],
        "Period": "60",
        "EvaluationPeriods": "3",
        "Threshold" : "10",
        "ComparisonOperator": "LessThanOrEqualToThreshold"
    }
}

Thanks!

-neil

7
  • I was setting this up myself today so can confirm it does work. I don't see why what you have done doesn't though. Jan 14, 2016 at 22:31
  • Thanks. It's not a CloudFormation creation hiccup, I have spun up a stack ~10 times, and the result is the same (i.e., no lambda function invocation.) Jan 14, 2016 at 22:39
  • Go into CloudFormation in the console and verify that the SNS Topic is set up. If an error occurred it should show up there. Aside from that this looks correct to me... Jan 14, 2016 at 23:16
  • No error was logged in the CloudFormation "Events" tab. 14:53:34 UTC-0800 CREATE_COMPLETE AWS::SNS::Topic GenericSlackAlertSNSTopic Physical ID:arn:aws:sns:us-east-1:[...]:v[...]-GenericSlackAlertSNSTopic-[...]8ZEX Jan 14, 2016 at 23:34
  • @JamesOgden If you either of you have a working cloudformation template that you can share, would you mind redacting and posting it? TIA! Jan 15, 2016 at 2:15

6 Answers 6

8

AWS released (~3 days ago) a blueprint for the slack integration with AWS Cloudwatch using lambda both in python and nodejs: https://aws.amazon.com/blogs/aws/new-slack-integration-blueprints-for-aws-lambda/

Being said that, I also had the same problem as you, following the steps mentioned in the blueprint, I do not get the alarms until I manually add the SNS topic as an "event source" on the lambda function. Further investigation lead me to this question: Can't create a SNS Event source on a Lambda function using CloudFormation

And finally reading the AWS documentation: 1) http://docs.aws.amazon.com/lambda/latest/dg/intro-core-components.html

Amazon SNS maintains the event source mapping via topic subscription configuration (there is no AWS Lambda API to configure this mapping).

2) http://docs.aws.amazon.com/sns/latest/dg/sns-lambda.html

Configuring Amazon SNS with Lambda Endpoints with the AWS Management Console

Concluded that the subscription at the moment should be done through the AWS Management console

Summary: at the moment the only way to configure Amazon SNS with Lambda Endpoints is through the AWS Management Console

Bonus: similar question with the same answer: AWS Lambda scheduled event source via cloudformation

3

You can now use AWS EventBridge (a renamed and extended version of CloudWatch events) to do this with a little less faff! See here for information on how.

It's similar to SNS, but seems to be a bit simpler to work with.

It also has some built in options on filtering of which events trigger your lambda.

2

At writing time, in order to connect a cloudwatch alarm to lambda you need a SNS topic.

The cloudwatch alarm will have the SNS topic as alarm action and the SNS topic requires to have lambda:Invoke permission in addition to have the lambda subscribed to the specified topic.

CloudFormation Template

Provided that you have the cloudwatch alarm in a template of cloudformation and the lambda arn (either the arn string or the resource in the same template) you can connect them with the resources AWS::Lambda::Permission and AWS::SNS::Topic.

For example, with this example (add the default lambda arn value):

  • aws cloudformation create-stack --stack-name MyTest --template-body file://mytest.yaml
AWSTemplateFormatVersion: '2010-09-09'

Parameters:

  LambdaArn:
    Type: String
    Description: The lambda arn, if lambda resource is in template change !Ref LambdaArn with !GetAtt LogicName.Arn
    Default: "[YOUR ARN]"

Resources:

  DummyTopic:
    Type: "AWS::SNS::Topic"
    Description: "sns topic to complete this dummy template, remove it when changing the cloudwatch alarm"
    Properties:
      TopicName: "mytest"

  CloudWatchAlarmInvocationAlarm:
    Type: "AWS::CloudWatch::Alarm"
    Description: "DummyAlarm change it (mainly Namespace, MetricName and Dimensions) to achieve your goal"
    Properties:
      Namespace: "AWS/SNS"
      AlarmDescription: "Dummy alarm"
      AlarmActions:
        - !Ref NotificationTopic
      MetricName: NumberOfMessagesPublished
      Statistic: "Sum"
      Dimensions:
        - Name: TopicName
          Value: !GetAtt DummyTopic.TopicName
      Period: 60
      EvaluationPeriods: 1
      Threshold: 1
      ComparisonOperator: "GreaterThanOrEqualToThreshold"
      TreatMissingData: notBreaching   # missing data points will not trigger the alarm => maintain

  NotificationTopic:
    Type: "AWS::SNS::Topic"
    Description: "Sns topic that communicates directly with the lambda"
    Properties:
      TopicName: "MyNotificationTopic"
      Subscription:
        - Endpoint: !Ref LambdaArn
          Protocol: "lambda"

  LambdaInvokePermission:
    Type: "AWS::Lambda::Permission"
    Properties:
      FunctionName: !Ref LambdaArn
      Action: "lambda:InvokeFunction"
      Principal: "sns.amazonaws.com"
      SourceArn:  !Ref NotificationTopic

1

CloudWatch Scheduled event now has Lambda an a native target. enter image description here

Also you can add a scheduled event to cloudformation for lambda

EventListFunction:
  Type: 'AWS::Serverless::Function'
  Properties:
    ...
    Events:
      Schedule1:
        Type: Schedule
        Properties:
          Schedule: rate(1 day)
1

Make sure you give the SNS topic permission to invoke your Lambda function. The CloudFormation for the permission will look something like this:

"LambdaInvokePermission": {
    "Type": "AWS::Lambda::Permission",
    "Properties": {
        "FunctionName" : "arn:aws:lambda:us-east-1:[...]:function:snsToSlack",
        "Action": "lambda:InvokeFunction",
        "Principal": "sns.amazonaws.com",
        "SourceArn": { "Ref": "GenericSlackAlertSNSTopic" }
    }
}
0

This issue is not specific to Cloudwatch, but is a SNS-topic/Lambda permissions thing.

See also:

AWS SNS ought to trigger my lambda, but does not

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.