0

I store pictures as Blob properties in Datastore, I am not using Blobstore, just Blob properties.

Now I want to migrate my app to a different platform and I'd like to get the pictures out as JPEG files. Is there a way of doing this with the bulkloader tool or using the mapper tool that appengine provides?

Thanks!

Update

I tried using the blob_to_file transform helper but could not make it work.

This is my bulkloader.yaml file

python_preamble:
- import: base64
- import: re
- import: google.appengine.ext.bulkload.transform
- import: google.appengine.ext.bulkload.bulkloader_wizard
- import: google.appengine.ext.db
- import: google.appengine.api.datastore
- import: google.appengine.api.users

transformers:

- kind: File
  connector: csv
  connector_options:
      encoding: utf-8
      columns: from_header

  property_map:

      - property: fileId
          external_name: fileId

      - property: bytes_content
          external_name: local_filename
          export_transform: transform.blob_to_file('fileId', 'pictures')

And this is my File model:

class File(db.Model):
    fileId = db.IntegerProperty()
    bytes_content = db.BlobProperty()

I run the bulkloader with the following command:

appcfg.py download_data --application=XXXX --kind=File --url=XXXXX/_ah/remote_api --filename=File.csv --config_file=bulkloader.yaml

At the end of download all I get is a big binary file and the folder pictures is empty. Thanks for your help!

2 Answers 2

1

You can use the "Bulkloader Transform Helper function" blob_to_file(). General information can be found at http://code.google.com/appengine/docs/python/tools/uploadingdata.html and Nick Johnson's write-up: http://blog.notdot.net/2010/04/Using-the-new-bulkloader.

For BlobProperty export in particular, check out the bulkloader sample app: http://bulkloadersample.appspot.com/

When you look at the 'bulkloader.yaml' file you will find this:

python_preamble:
- import: google.appengine.ext.bulkload.transform

and this:

# A sample writing a BlobProperty to an external file.
# (Note: not a BlobReferenceProperty.)
- model: models.Attachment
  connector: csv
  connector_options:
    encoding: utf-8
    columns: from_header
  property_map:
    - property: __key__
      external_name: ID
      export_transform: datastore.Key.name
    - property: filename
      external_name: filename
    - property: filecontents
      external_name: local_filename
      export_transform:
        transform.blob_to_file('filename', 'AttachmentBlobs')
1
1

You have two other alternatives : 1) Send the data to the other system with URL Fetch 2) If the number of images is not that big, you can download them one by one. I extracted some working code as an example :

    kwis = KindWithImages.get_by_key_name(file_name)              
    if kwis: 
        (content_type, encoding) = mimetypes.guess_type(kwis.file_ext)
        logging.info('download : %s%s content-type %s encoding %s' % (file_name, kwis.file_ext, content_type, encoding))
        if content_type == None :
            self.response.out.write('Content_type unknown for : %s' % (kwis.file_ext))
        else :
            self.response.headers['Content-Type'] = content_type
            self.response.headers['Content-Disposition'] = 'attachment; filename=%s%s' % (file_name, kwis.file_ext)
            self.response.out.write(kwis.content)  

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.