2

I am using scp to transfer files between two linux systems - when the transfer finishes, the final print to screen from SCP shows the wrong file size:

vmtbackup.zip                                       100% 2613MB  17.5MB/s   02:29 .   

As you can see here, the final reported size is 2613MB and the transfer has completed. However the actual size of the file is 2741MB

When I check on the target system, the file has the correct size (so the transfer has completed successfully).

I am using the output from SCP as part of a script, and the check condition to finish the script is when the size in SCP = the size of the file (as the 100% is equally unreliable) - it's part of an expect script, which looks like this:

function Transfer {
/usr/bin/expect <<EOD
#transfers the vmtbackup.zip file from source system to target system

spawn scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no /tmp/vmtbackup.zip $user@$ip:/tmp/
expect {
#handles all unexpected outcomes. Primarily used to capture wrong 

password, as no remote connection should be caught earlier in migration.sh
default { 
send_user "\nERROR: I was unable to connect to $ip for some reason.  Please try again.\n"
exit 1
}
#passes the password defined in migration.sh to scp password prompt"
-re "root@$ip's \[pP\]assword:" {
send -- "$password\r"
exp_continue
}
-re "\[pP\]assword:" {
send -- "$password\r"
exp_continue
}
-re "ETA" {
exp_continue
}
#when file transfer reaches 100%, exits script
-re "$si" {
exit 0
}
EOD
}

I'm aware of the security implications of using password in plaintext and that I should use SSL, but for now that's not part of the problem.

Can anyone tell me why SCP behaves in this way?

1 Answer 1

2

scp incorrectly shows MB, which means the factor is 1000 * 1000. It should show MiB, which is factor 1024 * 1024:

2613 MiB == 2613 * (1024 * 1024) / (1000 * 1000) == 2739.929088 MB

See IEEE 1541-2002 for details.

8
  • 1
    Filled a bug for OpenSSH: bugzilla.mindrot.org/show_bug.cgi?id=2701
    – Jakuje
    Mar 30, 2017 at 11:31
  • 1
    @Jakuje They might reply that scp predates IEEE 1541-2002 and changing the output units may break existing applications. It may make sense to add a new command line option --si-units or something like that. Mar 30, 2017 at 11:33
  • 2
    Yes, but the same code is used in the SFTP, which is "quite modern" and has the same issue. Also we need to move forward somehow. And better keep it tracked than sorry.
    – Jakuje
    Mar 30, 2017 at 11:37
  • @Maxim Thanks very much! This is exactly what it is. I grab the size of the file using ls -l --block-size=MB /tmp | grep -i vmtbackup.zip | awk '{ print $5 }' so I will just change it to ls -l --block-size=M /tmp | grep -i vmtbackup.zip | awk '{ print $5 }' and that should fix my problem? Mar 30, 2017 at 14:43
  • I would mention though that there is a discrepancy of about 0.04 percent between MiB and MB which on larger files can still potentially cause an issue on my side (SCP reports 2613 MiB, ls reports 2614MiB) but it's closer to the solution! Mar 30, 2017 at 14:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.