2

I am working on a task to back up (copy) about 100 Gb of data (including a thousand files and sub folders in a directory) to another server. Normally, for the smaller scale, I can use scp or rsync instead. However, as the other server is not on the same LAN network, it could easily take hours, even days, to complete the task. I can't just leave my computer there with the terminal running. I don't think that's the best choice, and again, I have another good reason to use Python :)

Is there any library, or best practice for me to start with? As, it's just for in-house project, we don't need any fancy features, just some fundamental things such as logging, error tolerance, etc.

3
  • "I can't just leave my computer there with the terminal running." - what's the problem? Can you start something and lock the screen? Or do you not want to leave your computer wherever "there" is? Aug 10, 2012 at 4:02
  • Hi Greg, there were 2 problems, yes, I am running scp/rsync over the terminal on the laptop, and I need to move eventually. Secondly, with hours of running the progress, it's very easy to get the connection error if there's anything with my home network. That's why I try to find a good solution in Python.
    – mrblue
    Aug 10, 2012 at 4:06
  • 3
    I would certainly recommend rsync, since it can easily be restarted if something goes wrong with the connection. You can't restart scp from where it left off as easily. Aug 10, 2012 at 4:09

1 Answer 1

2

I think your best bet is to use scp or rsync from within screen. That way you can detach the screen session and logout and the transfer will keep going.

man screen

2
  • Hi pizen, that means I can use screen then login to the server, then start the rsync to the other one, and turn off the terminal windows without stopping the progress?
    – mrblue
    Aug 10, 2012 at 4:19
  • You would log into the server, then use screen. You would start your transfer from within screen, detach the session, and logoff. Then later you would log back on, resume the screen session, and check the progress of your transfer.
    – pizen
    Aug 10, 2012 at 4:53

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.