4

I am trying, server-side, to take an image from the web by it's url (i.e. http://www.skrenta.com/images/stackoverflow.jpg) and save this image to my AWS S3 bucket using Meteor, the aws-sdk meteorite package as well as the http meteor package.

This is my attempt, which indeed put a file in my bucket (someImageFile.jpg), but the image file is corrupted then and cannot be displayed by a browser or a viewer application.

Probably I am doing something wrong with the encoding of the file. I tried many combinations and none of them worked. Also, I tried adding ContentLength and/or ContentEncoding with different encodings like binary, hex, base64 (also in combination with Buffer.toString("base64"), none of them worked. Any advice will be greatly appreciated!

This is in my server-side-code:

var url="http://www.skrenta.com/images/stackoverflow.jpg";
HTTP.get(url, function(err, data) {
    if (err) {
        console.log("Error: " + err);
    } else {
        //console.log("Result: "+JSON.stringify(data));
        //uncommenting above line fills up the console with raw image data
        s3.putObject({
                ACL:"public-read",
                Bucket:"MY_BUCKET",
                Key: "someImageFile.jpg",
                Body: new Buffer(data.content,"binary"),
                ContentType: data.headers["content-type"], // = image/jpeg
                //ContentLength: parseInt(data.headers["content-length"]),
                //ContentEncoding: "binary"
            },
            function(err,data){ // CALLBACK OF HTTP GET
                if(err){
                    console.log("S3 Error: "+err);
                }else{
                    console.log("S3 Data: "+JSON.stringify(data));
                }
            }
    );
    }
});

Actually I am trying to use the filepicker.io REST API via HTTP calls, i.e. for storing a converted image to my s3, but for this problem this is the minimum example to demonstrate the actual problem.

2 Answers 2

6

After several trial an error runs I gave up on Meteor.HTTP and put together the code below, maybe it will help somebody when running into encoding issues with Meteor.HTTP.

Meteor.HTTP seems to be meant to just fetch some JSON or text data from remote APIs and such, somehow it seems to be not quiet the choice for binary data. However, the Npm http module definitely does support binary data, so this works like a charm:

var http=Npm.require("http");

url = "http://www.whatever.com/check.jpg";

var req = http.get(url, function(resp) {
    var buf = new Buffer("", "binary");
    resp.on('data', function(chunk) {
        buf = Buffer.concat([buf, chunk]);
    });
    resp.on('end', function() {
        var thisObject = {
            ACL: "public-read",
            Bucket: "mybucket",
            Key: "myNiceImage.jpg",
            Body: buf,
            ContentType: resp.headers["content-type"],
            ContentLength: buf.length
        };
        s3.putObject(thisObject, function(err, data) {
            if (err) {
                console.log("S3 Error: " + err);
            } else {
                console.log("S3 Data: " + JSON.stringify(data));
            }
        });
    });
});
0

The best solution is to look at what has already been done in this regard:

https://github.com/Lepozepo/S3

Also filepicker.so seems pretty simple:

Integrating Filepicker.IO with Meteor

1
  • Hey FredericoC, thank you for the input, these sources are really good, but do not address the case. I'll update my post to make it more clear that it's a server side get and push, no clients involved. Sep 26, 2013 at 4:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.