56

I ran a ruby script from Heroku bash that generates a CSV file on the server that I want to download. I tried moving it to the public folder to download, but that didn't work. I figured out that after every session in the Heroku bash console, the files delete. Is there a command to download directly from the Heroku bash console?

1
  • did you have scp, ftp, sftp or mailx command in heroku bash?
    – BMW
    Jan 30, 2014 at 19:37

11 Answers 11

75

If you manage to create the file from heroku run bash, you could use transfer.sh.

You can even encrypt the file before you transfer it.

cat <file_name> | gpg -ac -o- | curl -X PUT -T "-" https://transfer.sh/<file_name>.gpg

And then download and decrypt it on the target machine

curl https://transfer.sh/<hash>/<file_name>.gpg | gpg -o- > <file_name>
4
  • 4
    I always struggled to take any data dump from heroku bash. Thanks. No more pain
    – mohitmun
    Nov 24, 2016 at 7:49
  • Can upload the files in heroku using filezilla
    – Subha
    May 3, 2018 at 8:00
  • Josh Justice mentioned this below, but transfer.sh is no more. file.io is a great alternative
    – Matt Dodge
    Dec 17, 2020 at 5:33
  • I don't know if it is a different transfer.sh now, but I just used this answer. Thank you!
    – Brandon
    Jun 19, 2022 at 22:15
41

There is heroku ps:copy:

#$ heroku help ps:copy 
Copy a file from a dyno to the local filesystem

USAGE
  $ heroku ps:copy FILE

OPTIONS
  -a, --app=app        (required) app to run command against
  -d, --dyno=dyno      specify the dyno to connect to
  -o, --output=output  the name of the output file
  -r, --remote=remote  git remote of app to use

DESCRIPTION
  Example:

       $ heroku ps:copy FILENAME --app murmuring-headland-14719

Example run:

#$ heroku ps:copy app.json --app=app-example-prod --output=app.json.from-heroku
Copying app.json to app.json.from-heroku
Establishing credentials... done
Connecting to web.1 on ⬢ app-example-prod... 
Downloading... ████████████████████████▏  100% 00:00 

Caveat

This seems not to run with dynos that are run via heroku run.

Example

#$ heroku ps:copy tmp/some.log --app app-example-prod --dyno run.6039 --output=tmp/some.heroku.log
Copying tmp/some.log to tmp/some.heroku.log
Establishing credentials... error
 ▸    Could not connect to dyno!
 ▸    Check if the dyno is running with `heroku ps'

It is! Prove:

#$ heroku ps --app app-example-prod
=== run: one-off processes (1)
run.6039 (Standard-1X): up 2019/08/29 12:09:13 +0200 (~ 16m ago): bash

=== web (Standard-2X): elixir --sname dyno -S mix phx.server --no-compile (2)
web.1: up 2019/08/29 10:41:35 +0200 (~ 1h ago)
web.2: up 2019/08/29 10:41:39 +0200 (~ 1h ago)

I could connect to web.1 though:

#$ heroku ps:copy tmp/some.log --app app-example-prod --dyno web.1 --output=tmp/some.heroku.log
Copying tmp/some.log to tmp/some.heroku.log
Establishing credentials... done
Connecting to web.1 on ⬢ app-example-prod... 
 ▸    ERROR: Could not transfer the file!
 ▸    Make sure the filename is correct.

So I fallen back to using SCP scp -P PORT tmp/some.log user@host:/path/some.heroku.log from the run.6039 dyno command line.

1
  • TYSM this helped me out soo much! Jan 5, 2021 at 11:24
19

Now that https://transfer.sh is defunct, https://file.io is an alternative. To upload myfile.csv:

$ curl -F "file=@myfile.csv" https://file.io

The response will include a link you can access the file at:

{"success":true,"key":"2ojE41","link":"https://file.io/2ojE41","expiry":"14 days"}

I can't vouch for the security of file.io, so using encryption as described in other answers could be a good idea.

16

Heroku dyno filesystems are ephemeral, non-persistant and not shared between dynos. So when you do heroku run bash, you actually get a new dyno with a fresh deployment of you app without any of the changes made to ephemeral filesystems in other dynos.

If you want to do something like this, you should probably either do it all in a heroku run bash session or all in a request to a web app running on Heroku that responds with the CSV file you want.

12

I did as the following:

  • First I entered heroku bash with this command:

    heroku run 'sh'

  • Then made a directory and moved the file to that

  • Made a git repository and commited the file
  • Finally I pushed this repository to github

Before commiting, git will ask you for your name and email. Give it something fake!

If you have files bigger than 100 Mg, push to gitlab.

If there is an easier way please let me know!

Sorry for my bad english.

4
  • How did you push the repository to GitHub without credentials? Sep 19, 2015 at 18:17
  • You pushed to github with fake credentials? How is that possible? Jan 5, 2016 at 9:02
  • i was referring to git settings not github credentials! you DO need a github account. @tommy7xo Jan 6, 2016 at 18:30
  • If you want to get more than a few files, this is an excellent way of doing it. I just downloaded the entries dyno like this!
    – cyberspy
    Jul 6, 2020 at 8:59
12

Another way of doing this (that doesn't involve any third server) is to use Patrick's method but first compress the file into a format that only uses visible ASCII charaters. That should make it work for any file, regardless of any whitespace characters or unusual encodings. I'd recommend base64 to do this.

Here's how I've done it:

  1. Log onto your heroku instance using heroku run bash
  2. Use base64 to print the contents of your file: base64 <your-file>
  3. Select the base64 text in your terminal and copy it
  4. On your local machine decompress this text using base64 straight into a new file (on a mac I'd do pbpaste | base64 --decode -o <your-file>)
6

I agree that most probably your need means a change in your application architecture, something like a worker dyno. But by executing the following steps you can transfer the file, since heroku one-off dyno can run scp:

  1. create vm in a cloud provider, e.g. digital ocean;
  2. run heroku one-off dyno and create your file;
  3. scp file from heroku one-off dyno to that vm server;
  4. scp file from vm server to your local machine;
  5. delete cloud vm and stop heroku one-off dyno.
4

I see that these answers are much older, so I'm assuming this is a new feature. For all those like me who are looking for an easier solution than the excellent answers already here, Heroku now has the capability to copy files quite easily with the following command: heroku ps:copy <filename>

Note that this works with relative paths, as you'd expect. (Tested on a heroku-18 stack, downloading files at "path/to/file.ext"

For reference: Heroku docs

1
  • I get this I want to save the db, Do i press y? Copying db.sqlite3 to db.sqlite3 Running this command for the first time requires a dyno restart. Do you want to continue? [y/n]: Feb 13, 2022 at 3:58
2

Heroku dyno's come with sftp pre-installed. I tried git but was too many steps (had to generate a new ssh cert and add it to github every time), so now I am using sftp and it works great.

You'll need to have another host (like dreamhost, hostgator, godaddy, etc) - but if you do, you can:

sftp username@ftp.yourhostname.com

Accept the server fingerprint/hash, then enter your password.

Once on the server, navigate to the folder you want to upload to (using cd and ls commands).

Then use the command put filename.csv and it will upload it to your web host.

To retrieve your file: Use an ftp client like filezilla or hit the url if you uploaded to a folder in the www or website folder path.

This is great because it also works with multiple files and binaries as well as text files.

1

For small/quick transfers that fit comfortably in the clipboard:

  1. Open a terminal on your local device
  2. Run heroku run bash
  3. (Inside your remote connection, on the dyno) Run cat filename
  4. Select the lines in your local terminal and copy them to your clipboard.
  5. Check to ensure proper newlines when pasting them.
0

Now i created shell script to upload some files from to git backup repo (for example, my app.db sqlite file is gitignored and every deploy kills it)

## upload dyno files to git via SSH session
## https://devcenter.heroku.com/changelog-items/1112
# heroku ps:exec
git config --global user.email 'dmitry.cheva@gmail.com'
git config --global user.name 'Dmitry Cheva'
rm -rf ./.gitignore
git init
## add each file separately (-f to add git ignored files)
git add app.db -f
git commit -m "backup on `date +'%Y-%m-%d %H:%M:%S'`"
git remote add origin https://bitbucket.org/cheva/appbackup.git
git push -u origin master -f

The git will reboot after the deploy and does not store the environment, you need to perform the first 3 commands. Then you need to add files (-f for ignored ones) and push into repo (-f, because the git will require pull)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy