59

I'm using Python/Django on Heroku (Cedar Stack) and I've got a management command that I need to write that will pull a file out of an S3 bucket and process it. I'm not sure I understand how to use the ephemeral filesystem. Are there only certain directories that are writeable? I found an other article that implied that there were only certain folders that were writable (but, it doesn't seem to apply to the Cedar stack). I found this dev article but it doesn't go into much detail (note: I do understand that it's just temporary. I only need to unzip the file and process the file). Can I just create a folder anywhere under the application's root? And how would I get that? It seems like I could probably just use $HOME. I did a bit of testing by connecting to via

$ heroku run bash

and running:

$ echo #HOME

returns:

/app

and running:

$ mkdir $HOME/tmp

creates a folder in the app's root and gives with the same user and group as the other files and folders.

So... anything I'm missing here? A better way to do it? Is there an OS environment variable for this? I've run "env" and I don't see a better one.

2
  • 4
    Note that, whenever you execute heroku run, this spins up a new one-off dyno on Heroku that will be charged. You can check the status of your dynos afterwards via heroku ps to see if they're still running. When using heroku run bash, the dyno is tied to your terminal, i.e. it should stop when you close your terminal session (e.g. with Ctrl+C). Furthermore, the one-off dyno doesn't have access to files from your other dynos written at run-time, so ls etc. won't work to see what's on your other dynos (except for your app files).
    – caw
    Oct 12, 2014 at 15:01
  • 5
    @MarcoW. I would like to highlight the imporatnce of your comment! This is so unobvious that heroku run spins up a new dyno. Took me a while to understand why I can't access via HTTP the files created via heroku run "echo test > /app/web/test.html"... It turns out they're created at one-off dyno and never were facing web...
    – Dimitry K
    Aug 28, 2015 at 8:01

2 Answers 2

79

To really understand the ephemeral filesystem, you need to understand what a dyno is. You can read more about how dynos work. In a nutshell, though, a process runs on Heroku in a virtual machine with its own filesystem. That virtual machine can stop for a number of reasons, taking the filesystem along with it.

The underlying filesystem will be destroyed when an app is restarted, reconfigured (e.g. heroku config ...), scaled, etc. For example, if you have two web dynos, write some files to the ephemeral filesystem, and scale to three dynos, those files will be destroyed because your app is running on new dynos.

In general, the ephemeral filesystem works just like any filesystem. Directories you have permission to write to, such as $HOME and /tmp, you can write files to. Any files that require permanence should be written to S3, or a similar durable store. S3 is preferred as Heroku runs on AWS and S3 offers some performance advantages. Any files that can be recreated at will can be stored on the dyno's ephemeral store.

1
  • 1
    " S3 is preferred as Heroku runs on AWS and S3 offers some performance advantages" - Good to know! ! :D @Naaman Oct 27, 2016 at 8:03
58

You can create a file under the '/tmp' directory, and that file will be destroyed after the request is complete. I'm doing this on Cedar, and I haven't had any problems.

2
  • 5
    in fact, anywhere is writable on the cedar stack - /tmp only was back from the Bamboo days. You will loose anything uploaded though when the dyno is restarted/scaled etc. Sep 14, 2012 at 8:00
  • 2
    In case it's helpful for others - I had to create a directory called "tmp" in my root directory. When I first came across this answer it sounded as though maybe tmp already exists as part of Heroku - in my case I created the directory.
    – Dan
    Aug 20, 2017 at 23:51

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.