is there ANY way to execute aws-cli inside AWS Lambda? It doesn't seem to be pre-installed. (I've checked with "which aws" via Node.js child-process, and it didn't exist.)
-
3Any progress on this? I'm looking for the same question, I need to use command "aws opsworks register". It sucks that they don't provide this ability in all of Lambda languages– CasperFeb 26, 2016 at 22:59
-
Why would you want to use the CLI? What is it that you're trying to do that cannot be achived using the AWS SDK?– st.huberJan 10, 2022 at 11:30
9 Answers
Now we can use Layers inside Lambda. Bash layer with aws-cli is available at https://github.com/gkrizek/bash-lambda-layer
handler () {
set -e
# Event Data is sent as the first parameter
EVENT_DATA=$1
# This is the Event Data
echo $EVENT_DATA
# Example of command usage
EVENT_JSON=$(echo $EVENT_DATA | jq .)
# Example of AWS command that's output will show up in CloudWatch Logs
aws s3 ls
# This is the return value because it's being sent to stderr (>&2)
echo "{\"success\": true}" >&2
}
-
2Omg!!! Thank you! I was literally fighting to get something like this working for 4 hours! Thanks :D :D– Jose AMar 16, 2019 at 19:20
-
I'm trying to use the aws cli from this layer with my python code. But I'm getting nothing back. May 27, 2019 at 12:25
-
This is bash layer. I am not sure why you need it in python. You may just use the python SDK.– JaiMay 27, 2019 at 16:05
-
1
Not unless you include it (and all of its dependencies) as part of your deployment package. Even then you would have to call it from within python since Lambda doesn't allow you to execute shell commands. Even if you get there, I would not recommend trying to do a sync in a Lambda function since you're limited to a maximum of 5 minutes of execution time. On top of that, the additional spin-up time just isn't worth it in many cases since you're paying for every 100ms chunk.
So you can, but you probably shouldn't.
EDIT: Lambda does allow you to execute shell commands
-
5See alestic.com/2016/11/aws-lambda-awscli for a good overview on how to accomplish bundling the aws-cli python package for use in lambda– NoahDec 27, 2016 at 19:20
-
Actually using aws cli IS a good idea when using Lambda for a Cloudformation Custom resource for example– Yves M.Oct 24, 2018 at 9:55
aws-cli is a python package. To make it available on a AWS Lambda function you need to pack it with your function zip file.
1) Start an EC2 instance with 64-bit Amazon Linux;
2) Create a python virtualenv:
mkdir ~/awscli_virtualenv
virtualenv ~/awscli_virtualenv
3) Activate virtualenv:
cd ~/awscli_virtualenv/bin
source activate
4) Install aws-cli and pyyaml:
pip install awscli
python -m easy_install pyyaml
5) Change the first line of the aws python script:
sed -i '1 s/^.*$/\#\!\/usr\/bin\/python/' aws
6) Deactivate virtualenv:
deactivate
7) Make a dir with all the files you need to run aws-cli on lambda:
cd ~
mkdir awscli_lambda
cd awscli_lambda
cp ~/awscli_virtualenv/bin/aws .
cp -r ~/awscli_virtualenv/lib/python2.7/dist-packages .
cp -r ~/awscli_virtualenv/lib64/python2.7/dist-packages .
8) Create a function (python or nodejs) that will call aws-cli:
For example (nodejs):
var Q = require('q');
var path = require('path');
var spawn = require('child-process-promise').spawn;
exports.handler = function(event, context) {
var folderpath = '/folder/to/sync';
var s3uel = 's3://name-of-your-bucket/path/to/folder';
var libpath = path.join(__dirname, 'lib');
var env = Object.create(process.env);
env.LD_LIBRARY_PATH = libpath;
var command = path.join(__dirname, 'aws');
var params = ['s3', 'sync', '.', s3url];
var options = { cwd: folderpath };
var spawnp = spawn(command, params, options);
spawnp.childProcess.stdout.on('data', function (data) {
console.log('[spawn] stdout: ', data.toString());
});
spawnp.childProcess.stderr.on('data', function (data) {
console.log('[spawn] stderr: ', data.toString());
});
return spawnp
.then(function(result) {
if (result['code'] != 0) throw new Error(["aws s3 sync exited with code", result['code']].join(''));
return result;
});
}
Create the index.js file (with the code above or your code) on ~/awscli_lambda/index.js
9) Zip everything (aws-cli files and dependencies and your function):
cd ~
zip -r awscli_lambda.zip awscli_lambda
Now you can simply run it as Docker container within lambda along with AWS CLI.
-
To add some context - AWS Lambda now supports container images - aws.amazon.com/blogs/aws/…– SashiSep 20, 2021 at 21:29
You can use the AWS node.js SDK which should be available in Lambda without installing it.
var AWS = require('aws-sdk');
var lambda = new AWS.Lambda();
lambda.invoke({
FunctionName: 'arn:aws:lambda:us-west-2:xxxx:function:FN_NAME',
Payload: {},
},
function(err, result) {
...
});
As far as I can tell you get most, if not all the cli functionality. See the full documentation here: http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Lambda.html
-
4Thanks Ryan, but I want some functions which do NOT exit in node.js SDK and do exist in aws-cli, because it seems easier to use aws-cli in some occasions, such as s3 sync across buckets. Nov 4, 2015 at 6:23
-
Ah, I see now. In that case it seems you would need to install it on every run which sounds kind of painful and inefficient.– RyanNov 4, 2015 at 15:09
-
you cant install it, yum and pip install is not available in nodejs environment– halilAug 26, 2016 at 19:47
-
you can try this. I got this working for me.
1- add the AWS CLI layer
https://harishkm.in/2020/06/16/run-aws-cli-in-a-lambda-function/
2- add a lambda and run the following commands to run any AWS CLI command line.
https://harishkm.in/2020/06/16/run-bash-scripts-in-aws-lambda-functions/
function handler () {
EVENT_DATA=$1
DATA=`/opt/awscli/aws s3 ls `
RESPONSE="{\"statusCode\": 200, \"body\": \"$DATA\"}"
echo $RESPONSE
}
If you are provisioning your Lambda using code, then this is the most easiest way
lambda_function.add_layers(AwsCliLayer(scope, "AwsCliLayer"))
This is a feasible approach, It can still be used now.
https://bezdelev.com/hacking/aws-cli-inside-lambda-layer-aws-s3-sync/
The macos shell is working well
# Creates a directory for virtual environment
mkdir awscli-virtualenv
# Initializes a virtual environment in the virtual environment directory
virtualenv -p python3.10 awscli-virtualenv
# Changes current dir to the virtual env directory
cd awscli-virtualenv/bin/
# Activate virtual environment
source activate
pip3.10 install awscli
sed -i '' "1s/.*/\#\!\/var\/lang\/bin\/python/" aws
deactivate
# Changes current directory back to where it started
cd ../..
# Creates a temporary directory to store AWS CLI and its dependencies
mkdir awscli-lambda-layer
# Changes the current directory into the temporary directory
cd awscli-lambda-layer
# Copies aws and its dependencies to the temp directory
cp ../awscli-virtualenv/bin/aws .
cp -r ../awscli-virtualenv/lib/python3.10/site-packages/ .
# Zips the contents of the temporary directory
zip -r ../awscli-lambda-layer.zip *
# Goes back to where it started
cd ..
# Removes virtual env and temp directories
rm -r awscli-virtualenv
rm -r awscli-lambda-layer
I think that you should separate your trigger logic from the action. Put a container with aws cli on another ec2 And use aws lambda to trigger that into an action.
-
2This means that you need to have an ec2 instance sitting around at all times rather than implementing the usefulness of lambdas - small scope & run only when needed. Dec 18, 2018 at 18:40