24

I am using API gateway to call lambda function that imports a mpeg file (10 mb) from s3 and saves in /tmp folder of lambda and the lambda uploads it to external API (youtube etc) Recently the API gateway call to lambda is failing intermittently with error

[Errno 28] No space left on device

Here is how i am downloading the file

urllib.urlretrieve (s3_mpeg_url, '/tmp/{}'.format(mpeg_filename))

If i create a new version of that same lambda function and assign to alias API gateway pointing to , it starts to work and again at some point it keeps getting the same error

When i test that lambda function from lambda console it always works

Any idea ?

1
  • 3
    Are you deleting the file once you're done with it? Lambda functions can be reused.
    – kichik
    Commented Jan 19, 2018 at 18:35

2 Answers 2

26

Your lambda function has approximately 500MB of disk space available on /tmp. However, for performance reasons, AWS Lambda might retain and reuse instances of your function on subsequent calls. There are two possible paths you can take here:

  1. If your function is totally stateless (i.e. you don't need the mpeg file after uploading it to the external API), just delete it from the /tmp folder after the upload.
  2. If you need to keep the data around, consider storing it on another media, like S3 or DynamoDB after processing.
4
  • Thanks .. I am looking into option 1 you mentioned but it didnt helped...please pardon my lack of knowledge on deleting files on aws servers... i tried os.remove(/tmp/*.mpeg) but its not working(i am using python in lambda). Can you let me know how we can clear tmp folder Commented Jan 19, 2018 at 19:01
  • Don't believe that os.remove() supports wildcards. Delete the file by name, or use some feature of glob.
    – jarmod
    Commented Jan 19, 2018 at 20:23
  • that may be it.it wont support wildcards.I used following to get it work def emptyDir(folder): fileList = os.listdir(folder) for f in fileList: filePath = folder + '/'+f if os.path.isfile(filePath): os.remove(filePath) elif os.path.isdir(filePath): newFileList = os.listdir(filePath) for f1 in newFileList: insideFilePath = filePath + '/' + f1 if os.path.isfile(insideFilePath):\n os.remove(insideFilePath) Commented Jan 19, 2018 at 20:51
  • Right now, you can configure your ephemeral storage between 512MB and 10,240MB. Have in mind that the ephemeral storage is available in each function’s /tmp directory.
    – robe007
    Commented Jun 21, 2022 at 19:44
4

You can increase the storage from 512MB (default) up to 10,240MB.

GUI: how to change your storage

CLI:

 aws lambda update-function-configuration --function-name <function_name> \
          --ephemeral-storage '{"Size": 10240}'

Note that you are charged for any additional size you incremented.

See full article here: https://aws.amazon.com/blogs/aws/aws-lambda-now-supports-up-to-10-gb-ephemeral-storage/

1
  • It is not free and AWS charges you for every MB.
    – tom10271
    Commented 2 days ago

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.