233

I'm working on a simple tool that transfers files to a hard-coded location with the password also hard-coded. I'm a python novice, but thanks to ftplib, it was easy:

import ftplib

info= ('someuser', 'password')    #hard-coded

def putfile(file, site, dir, user=(), verbose=True):
    """
    upload a file by ftp to a site/directory
    login hard-coded, binary transfer
    """
    if verbose: print 'Uploading', file
    local = open(file, 'rb')    
    remote = ftplib.FTP(site)   
    remote.login(*user)         
    remote.cwd(dir)
    remote.storbinary('STOR ' + file, local, 1024)
    remote.quit()
    local.close()
    if verbose: print 'Upload done.'

if __name__ == '__main__':
    site = 'somewhere.com'            #hard-coded
    dir = './uploads/'                #hard-coded
    import sys, getpass
    putfile(sys.argv[1], site, dir, user=info)

The problem is that I can't find any library that supports sFTP. What's the normal way to do something like this securely?

Edit: Thanks to the answers here, I've gotten it working with Paramiko and this was the syntax.

import paramiko

host = "THEHOST.com"                    #hard-coded
port = 22
transport = paramiko.Transport((host, port))

password = "THEPASSWORD"                #hard-coded
username = "THEUSERNAME"                #hard-coded
transport.connect(username = username, password = password)

sftp = paramiko.SFTPClient.from_transport(transport)

import sys
path = './THETARGETDIRECTORY/' + sys.argv[1]    #hard-coded
localpath = sys.argv[1]
sftp.put(localpath, path)

sftp.close()
transport.close()
print 'Upload done.'

Thanks again!

4
  • 4
    Thanks ! Got an SFTP upload script working in 5 minutes :) Commented Mar 11, 2010 at 0:03
  • 2
    Just a general note on original question that python ftplib has also support for FTPS - ftp over TLS en.m.wikipedia.org/wiki/FTPS . FTPS servers are arguably less used in Unix world, partly due to omnipresence of ssh/sftp,however, sftp servers are much less present in Windows environment, where FTPS is more common.
    – Gnudiff
    Commented Oct 27, 2017 at 8:51
  • Looks like FTPS support was added in Python 3.2 with an extended class source: class ftplib.FTP_TLS(host='', user='', passwd='', acct='', keyfile=None, certfile=None, context=None, timeout=None, source_address=None)
    – mgrollins
    Commented Jun 5, 2019 at 21:23
  • I followed this exactly and I'm getting a file not found error. I've checked using os.path.abspath and os.path.isfile but still getting errors when running this script. Whats going on? Commented Mar 3, 2021 at 15:23

11 Answers 11

135

Paramiko supports SFTP. I've used it, and I've used Twisted. Both have their place, but you might find it easier to start with Paramiko.

6
105

You should check out pysftp https://pypi.python.org/pypi/pysftp it depends on paramiko, but wraps most common use cases to just a few lines of code.

import pysftp
import sys

path = './THETARGETDIRECTORY/' + sys.argv[1]    #hard-coded
localpath = sys.argv[1]

host = "THEHOST.com"                    #hard-coded
password = "THEPASSWORD"                #hard-coded
username = "THEUSERNAME"                #hard-coded

with pysftp.Connection(host, username=username, password=password) as sftp:
    sftp.put(localpath, path)

print 'Upload done.'
8
  • 7
    Vote up for with in the example Commented Jul 24, 2014 at 14:07
  • 3
    pip install pysftp
    – Bob Stein
    Commented Jan 20, 2016 at 12:33
  • 2
    Is there an option to automatically add new sftp host to known hosts?
    – user443854
    Commented Jul 27, 2016 at 18:39
  • 11
    WARNING: Is seems like pysftp is no longer very active. The link to the issue tracker on their site is broken (says its not setup). I found found a bug and am trying to figure out the best way to document it and fix it but doesn't seem like it has been updated for over 4 years
    – JD D
    Commented Aug 25, 2020 at 20:47
  • 8
    I think it would have been nice if this answer disclosed that the answerer was the author and sole maintainer of the software being recommended. But that user seems to have stopped coming to SO as well as stopped working on pysftp.
    – John Y
    Commented Mar 1, 2022 at 2:23
18

Here is a sample using pysftp and a private key.

import pysftp

def upload_file(file_path):

    private_key = "~/.ssh/your-key.pem"  # can use password keyword in Connection instead
    srv = pysftp.Connection(host="your-host", username="user-name", private_key=private_key)
    srv.chdir('/var/web/public_files/media/uploads')  # change directory on remote server
    srv.put(file_path)  # To download a file, replace put with get
    srv.close()  # Close connection

pysftp is an easy to use sftp module that utilizes paramiko and pycrypto. It provides a simple interface to sftp.. Other things that you can do with pysftp which are quite useful:

data = srv.listdir()  # Get the directory and file listing in a list
srv.get(file_path)  # Download a file from remote server
srv.execute('pwd') # Execute a command on the server

More commands and about PySFTP here.

4
  • srv.get(file_path) # Download a file from remote server can you explain where does it download the file into? Commented Jun 29, 2017 at 10:04
  • Did you try the local for you executed from?
    – radtek
    Commented Jul 4, 2017 at 17:19
  • Yeah but where on the file system? Everything goes through successfully but I can't seem to find the file from anywhere. Commented Jul 5, 2017 at 6:45
  • Sorry I meant local dir. Try executing the script from your home dir and seeing if the file is there.
    – radtek
    Commented Jul 5, 2017 at 15:39
17

If you want easy and simple, you might also want to look at Fabric. It's an automated deployment tool like Ruby's Capistrano, but simpler and of course for Python. It's build on top of Paramiko.

You might not want to do 'automated deployment' but Fabric would suit your use case perfectly none the less. To show you how simple Fabric is: the fab file and command for your script would look like this (not tested, but 99% sure it will work):

fab_putfile.py:

from fabric.api import *

env.hosts = ['THEHOST.com']
env.user = 'THEUSER'
env.password = 'THEPASSWORD'

def put_file(file):
    put(file, './THETARGETDIRECTORY/') # it's copied into the target directory

Then run the file with the fab command:

fab -f fab_putfile.py put_file:file=./path/to/my/file

And you're done! :)

1
  • I followed this and got the message "Can't find any collection named fabfile!" when i run the python file. Commented Mar 3, 2021 at 11:55
9

fsspec is a great option for this, it offers a filesystem like implementation of sftp.

from fsspec.implementations.sftp import SFTPFileSystem
fs = SFTPFileSystem(host=host, username=username, password=password)

# list a directory
fs.ls("/")

# open a file
with fs.open(file_name) as file:
    content = file.read()

Also worth noting that fsspec uses paramiko in the implementation.

5

With RSA Key then refer here

Snippet:

import pysftp
import paramiko
from base64 import decodebytes

keydata = b"""AAAAB3NzaC1yc2EAAAADAQABAAABAQDl""" 
key = paramiko.RSAKey(data=decodebytes(keydata)) 
cnopts = pysftp.CnOpts()
cnopts.hostkeys.add(host, 'ssh-rsa', key)


with pysftp.Connection(host=host, username=username, password=password, cnopts=cnopts) as sftp:   
  with sftp.cd(directory):
    sftp.put(file_to_sent_to_ftp)
3

Twisted can help you with what you are doing, check out their documentation, there are plenty of examples. Also it is a mature product with a big developer/user community behind it.

1
3

Paramiko is so slow. Use subprocess and shell, here is an example:

remote_file_name = "filename"
remotedir = "/remote/dir"
localpath = "/local/file/dir"
    ftp_cmd_p = """
    #!/bin/sh
    lftp -u username,password sftp://ip:port <<EOF
    cd {remotedir}
    lcd {localpath}
    get {filename}
    EOF
    """
subprocess.call(ftp_cmd_p.format(remotedir=remotedir,
                                 localpath=localpath,
                                 filename=remote_file_name 
                                 ), 
                shell=True, stdout=sys.stdout, stderr=sys.stderr)
3
  • The question is about "Python", which generally implies sticking within that - especially when there are so many options for doing so. More importantly though, it says "Platform independent". I can't say if your answer works faster or not though. Perhaps?
    – BuvinJ
    Commented Apr 29, 2020 at 0:40
  • 1
    Seeing that shabang... yeah this only works on *nix. Not to mention that lftp isn't installed by default on a lot of *nix computers. Commented May 28, 2021 at 15:04
  • this is a great idea in general, the use of templates to build commands on the local filesystem
    – Joe
    Commented Jul 30 at 1:50
1

There are a bunch of answers that mention pysftp, so in the event that you want a context manager wrapper around pysftp, here is a solution that is even less code that ends up looking like the following when used

path = "sftp://user:p@[email protected]/path/to/file.txt"

# Read a file
with open_sftp(path) as f:
    s = f.read() 
print s

# Write to a file
with open_sftp(path, mode='w') as f:
    f.write("Some content.") 

The (fuller) example: http://www.prschmid.com/2016/09/simple-opensftp-context-manager-for.html

This context manager happens to have auto-retry logic baked in in the event you can't connect the first time around (which surprisingly happens more often than you'd expect in a production environment...)

The context manager gist for open_sftp: https://gist.github.com/prschmid/80a19c22012e42d4d6e791c1e4eb8515

1

PyFilesystem with its sshfs is one option. It uses Paramiko under the hood and provides a nicer paltform independent interface on top.

import fs

sf = fs.open_fs("sftp://[user[:password]@]host[:port]/[directory]")
sf.makedir('my_dir')

or

from fs.sshfs import SSHFS
sf = SSHFS(...
0

Here's a generic function that will download any given sftp url to a specified path

from urllib.parse import urlparse
import paramiko

url = 'sftp://username:password@hostname/filepath.txt'

def sftp_download(url, dest):
    url = urlparse(url)
    with paramiko.Transport((url.hostname, 22)) as transport:
        transport.connect(None,url.username,url.password)
        with paramiko.SFTPClient.from_transport(transport) as sftp:
            sftp.get(url.path, dest)

Call it with

sftp_download(url, "/tmp/filepath.txt")

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.