30

I'm trying to simplify code with async / await

But have problems making https.get with async / await structure.

I am aware of how to do this with third-party modules but prefer the native node.js https module.

Below code doesn't work for me:

async function get_page() {

    const https = require('https')
    const url = 'https://example.com'

    const util = require('util')
    const https_get = util.promisify(https.get)

    const data = await https_get(url)

    do_awesome_things_with_data(data)
}

This code working fine:

function get_page() {

    const https = require('https')
    const url = 'https://example.com'

    let data = ''

    https.get(url, res => {

        res.on('data', chunk => { data += chunk }) 

        res.on('end', () => {

           do_awesome_things_with_data(data)

        })
    }) 
}
2
  • 1
    twilio.com/blog/…
    – iLuvLogix
    Commented Dec 15, 2020 at 13:21
  • I think you call do_awesome_things_with_data(data) at the moment when data is not resolved. Just for debugging could you do a setTimeout of 1sec between the function call and the promise
    – Aalexander
    Commented Dec 15, 2020 at 13:21

3 Answers 3

50

https.get doesn't return something that can be promisified as the signature of the callback doesn't match (err, value), so you can't await it.

However, you can wrap the https.get call within a Promise, like so, then await when calling get_page

const https = require('https')

async function get_page() {
    const url = 'https://example.com'

    return new Promise((resolve) => {
        let data = ''

        https.get(url, res => {

            res.on('data', chunk => { data += chunk }) 

            res.on('end', () => {

               resolve(do_awesome_things_with_data(data));

            })
        }) 
    })
}

// usage

(async () => await get_page())()

Edits

I've updated my answer to include the note of https.get not being able to be promisified and moved the require('https') outside of the function call.

7
  • 1
    The problem is that get does not return something that promisify can wrap. The OP was aware that a promise was not returned. Commented Dec 15, 2020 at 13:41
  • 2
    require() shouldn’t be used in an async function as it’s sync. It’s also bad practice to require() inside a function that would get called repeatedly, generally, the same goes for util.promisify(). They both should be moved outside the function.
    – peteb
    Commented Dec 15, 2020 at 14:10
  • 1
    Yup, both comments here are correct. I'll update my answer to reflect.
    – steadweb
    Commented Dec 15, 2020 at 15:27
  • This is an elegantly minimalistic answer, but I don’t understand why the get_page function needs to have the async prefix. When I implement this pattern without that prefix, it works the same. My understanding is that async permits await prefixes, but you don’t have any, and Promisifies non-Promise return values, but you don’t return any. Commented May 24, 2021 at 15:32
  • 1
    The code is missing declaration of 'data' variable, like so data = '' either before https.get function or before the whole Promise thing. Otherwise the resolve will complain about the data not being defined.
    – Igor
    Commented Aug 30, 2022 at 15:38
16

Instead of promisify, roll your own function, or use a 3rd party library. Promisify cannot wrap what https.get returns.

// generic promise method for https
const requestPromise = ((urlOptions, data) => {
  return new Promise((resolve, reject) => {
    const req = https.request(urlOptions,
      (res) => {
        let body = '';
        res.on('data', (chunk) => (body += chunk.toString()));
        res.on('error', reject);
        res.on('end', () => {
          if (res.statusCode >= 200 && res.statusCode <= 299) {
            resolve({statusCode: res.statusCode, headers: res.headers, body: body});
          } else {
            reject('Request failed. status: ' + res.statusCode + ', body: ' + body);
          }
        });
      });
    req.on('error', reject);
    req.write(data, 'binary');
    req.end();
  });
});

Then call it like this:

async function get_page() {
    const url = 'https://example.com'
    const data = await requestPromise({url, method:'GET'})
    do_awesome_things_with_data(data)
}

Or simply use a library such as axios to return a native promise, and also handle additional boilerplate cases.

1
  • Thanks Steven. I cannot make it work, would you check what I did wrong at runkit.com/embed/df2eku6jnxix please? Better yet, post a runkit link that works. thanks.
    – xpt
    Commented Feb 2, 2023 at 18:20
0

Here is how I did it:

async myFunc = function {
    let url = 'http://your.data/file';
    let promise = new Promise((resolve, reject) => {
        var data = '';
        https.get(url, res => {
            res.on('data', chunk => { data += chunk }) 
            res.on('end', () => {
               resolve(data);
            })
        }) 
    });

    let result = await promise; // wait until the promise resolves
    doStuffWithResult(result);
};

In my case, I was fetching a .json file, so I actually used resolve(JSON.parse(data)) to return cleanely JSON object.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.