I'm using Google Console to upload a large file (~20 GB). It appears to create a large temp file on my system disk while it's uploading, which fills up the system disk. Is it supposed to do that? Why? Is there some way I can put the temp file on another disk? I have a disk mounted with several hundred GB free, but it apparently is not used.
I'm new at using Google Cloud Storage, so if this is common knowledge, then I apologize. I did look in the Google Docs and on Stackoverflow [google-cloud-storage]. I'm on MacOS 10.12.1. Here's what happens:
- On console.cloud.google.com, I open Cloud Storage and click on a bucket.
- I click on "Upload Files:" and select a sparseimage of about 20 GB. The "Uploading" msg appears.
- Things sit for maybe 15-20 minutes, and then MacOS says that the system is low on space. If I watch my system disk during the upload, its free space will go from about 19 GB down to a few hundred MB, then I get the system message.
- After a while longer, Google Console will say that the upload is done. However, file has 0 bytes. Safari is almost unusable at this point, but I can cancel the upload (eventually), and Safari will come back to life (eventually), and I can quit it.
- However, the temp file will remain until I restart MacOS. It will be something like this (the random characters vary): /private/var/folders/t9/wjkgn8c11bb0h5ncfxgwfj3h0000gn/T/WebKitGeneratedFileITn3fU
I have not installed gsutil yet; I wanted to try the simple way first.
- Herman
More info:
I uploaded my 20 GB sparseimage using gsutil, and that worked just fine. It was MUCH faster than the console, by almost an order of magnitude, and it didn't create any big temp files on my system disk.
However, I had to add the -r flag and upload it as a directory. The way MacOS stores sparse image files, they look like single files on the Mac side, but they are folders, metadata, and files on the Unix side.
I looked at the 2 GB file that I uploaded previously. The type was 'application/zip', so I guess the console recognized it as a spase image and zipped it for me. I didn't tell it to. Perhaps that's what the big temp file is used for.
I can't find anything right away about automatic zipping, but more experimentation is in order.
gsutil's cp
command for this task? EDIT: OK, just noticed theI have not installed gsutil yet; I wanted to try the simple way first.
part... I'd say that using a browser to upload 20gb of data isn't the "simple way" anyway...