1

I'm using go in order to interact with cloud storage .

I can't use gsutil from the app engine and delete with the rm command ?

I can delete one object with DeleteObject or iterate over a range of objects and delete each one , but i`m looking for another solution something like DeleteMulti in Datastor.

Do you have a better solution for multi deletion ?

1 Answer 1

0

Each object that is deleted requires one call to GCS. Iterating over each object and calling delete is the easiest and likely best solution. If you need faster performance, you may want to send multiple delete requests to GCS at a time using multiple threads.

If this is a significant performance issue for your app, there is, however, another way, which I hesitate to mention because it adds significant complexity and doesn't buy much extra performance. GCS supports batching calls together into a single connection. It likely won't be much faster than sending delete requests over several threads, but it does behave more like a DeleteMulti call.

Effectively, batch calls work by sending a multipart HTTP request to the /batch path, each part of which representing an HTTP call. A request to delete several objects would look like this:

POST /batch HTTP/1.1
Host: www.googleapis.com
Content-Length: content_length
Content-Type: multipart/mixed; boundary="===============7330845974216740156=="
Authorization: Bearer oauth2_token


--===============7330845974216740156==
Content-Type: application/http
Content-Transfer-Encoding: binary
Content-ID: <b29c5de2-0db4-490b-b421-6a51b598bd22+1>

DELETE /storage/v1/b/example-bucket/o/obj1 HTTP/1.1
accept: application/json


--===============7330845974216740156==
Content-Type: application/http
Content-Transfer-Encoding: binary
Content-ID: <b29c5de2-0db4-490b-b421-6a51b598bd22+2>

DELETE /storage/v1/b/example-bucket/o/obj2 HTTP/1.1
accept: application/json


--===============7330845974216740156==--

There's more documentation on it here: https://cloud.google.com/storage/docs/json_api/v1/how-tos/batch

But, again, I recommend just sending individual delete requests. Batch calls are not atomic, which means some deletes might succeed while others fail. In the event one of the batch delete operations fails, you'll need to parse the batch response message to figure out which call failed so that you can retry it, which is very likely not worth the effort for the gain you'll get back.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.