64

I am using Puppeteer to try to take a screenshot of a website after all images have loaded but can't get it to work.

Here is the code I've got so far, I am using https://www.digg.com as the example website:

const puppeteer = require('puppeteer');

(async () => {
    const browser = await puppeteer.launch();
    const page = await browser.newPage();
    await page.goto('https://www.digg.com/');

    await page.setViewport({width: 1640, height: 800});

    await page.evaluate(() => {
        return Promise.resolve(window.scrollTo(0,document.body.scrollHeight));
    });

    await page.waitFor(1000);

    await page.evaluate(() => {
        var images = document.querySelectorAll('img');

        function preLoad() {

            var promises = [];

            function loadImage(img) {
                return new Promise(function(resolve,reject) {
                    if (img.complete) {
                        resolve(img)
                    }
                    img.onload = function() {
                        resolve(img);
                    };
                    img.onerror = function(e) {
                        resolve(img);
                    };
                })
            }

            for (var i = 0; i < images.length; i++)
            {
                promises.push(loadImage(images[i]));
            }

            return Promise.all(promises);
        }

        return preLoad();
    });

    await page.screenshot({path: 'digg.png', fullPage: true});

    browser.close();
})();
2

5 Answers 5

106

There is a built-in option for that:

await page.goto('https://www.digg.com/', {"waitUntil" : "networkidle0"});

networkidle0 - consider navigation to be finished when there are no more than 0 network connections for at least 500 ms

networkidle2 - consider navigation to be finished when there are no more than 2 network connections for at least 500 ms.

Of course it won't work if you're working with endless-scrolling-single-page-applications like Twitter.

Puppeteer GitHub issue #1552 provides explanation for the motivation behind networkidle2.

6
  • In case of digg.com some of the images are loaded only when you scroll down, do you know of a way to wait for the images to load after scrolling? Sep 15, 2017 at 7:59
  • 2
    I guess your solution will work, but - after studying how digg's home page works - I'll say you have to scroll little by little, whereas in your code you jump by almost a full page. Look in the source - there are lots of lazy-loading images that will only load if in the viewport.
    – Vaviloff
    Sep 15, 2017 at 9:49
  • 3
    I think it should be: { waitUntil: "networkidle" } instead of {"waitUntil" : "networkidle"}
    – standac
    Sep 30, 2017 at 0:47
  • 1
    In the latest puppeteer builds networkidle is deprecated and replaced with networkidle0 & networkidle2 github.com/GoogleChrome/puppeteer/blob/master/docs/…
    – Levi
    Dec 16, 2017 at 1:17
  • Hi, everytime i click something it would load stuff, how can i wait for the next network idle , but there isn't any goto you see cuz it's a button click.
    – CodeGuru
    Feb 16, 2019 at 3:25
30

Another option, actually evaluate to get callback when all images were loaded

This option will also work with setContent that doesn't support the wait networkidle0 option

await page.evaluate(async () => {
  const selectors = Array.from(document.querySelectorAll("img"));
  await Promise.all(selectors.map(img => {
    if (img.complete) return;
    return new Promise((resolve, reject) => {
      img.addEventListener('load', resolve);
      img.addEventListener('error', reject);
    });
  }));
})
9
  • 1
  • @BenjaminGruenbaum yea but it's event emitter, npm that promisify it won't do exactly the same?, +thanks for the good edit Apr 21, 2018 at 9:09
  • You can't promisify EventTargets automatically yet as far as I know - but the rest doesn't need new Promise :) Apr 21, 2018 at 9:28
  • Note that unlike networkidle this will wait for all the images based on tags present in the DOM when the evaluate is called. So if scripts add more images asynchronously this won't work (you can in theory call it recursively but... meh). Apr 21, 2018 at 9:30
  • 1
    FYI this answer is out of date. setContent supports waitUntil now, which is very helpful.
    – brainbag
    Mar 22, 2019 at 21:07
12

Wait for Lazy Loading Images

You may want to consider scrolling down first using a method such as Element.scrollIntoView() to account for lazy loading images:

await page.goto('https://www.digg.com/', {
  waitUntil: 'networkidle0', // Wait for all non-lazy loaded images to load
});

await page.evaluate(async () => {
  // Scroll down to bottom of page to activate lazy loading images
  document.body.scrollIntoView(false);

  // Wait for all remaining lazy loading images to load
  await Promise.all(Array.from(document.getElementsByTagName('img'), image => {
    if (image.complete) {
      return;
    }

    return new Promise((resolve, reject) => {
      image.addEventListener('load', resolve);
      image.addEventListener('error', reject);
    });
  }));
});
1
0

I found a solution which is applicable to multiple sites using the page.setViewPort(...) method as given below:

const puppeteer = require('puppeteer');

async(() => {
    const browser = await puppeteer.launch({
        headless: true, // Set to false while development
        defaultViewport: null,
        args: [
            '--no-sandbox',
            '--start-maximized', // Start in maximized state
        ],
    });

    const page = await = browser.newPage();
    await page.goto('https://www.digg.com/', {
        waitUntil: 'networkidle0', timeout: 0
    });

    // Get scroll width and height of the rendered page and set viewport
    const bodyWidth = await page.evaluate(() => document.body.scrollWidth);
    const bodyHeight = await page.evaluate(() => document.body.scrollHeight);
    await page.setViewport({ width: bodyWidth, height: bodyHeight });

    await page.waitFor(1000);
    await page.screenshot({path: 'digg-example.png' });
})();
1
-1

I'm facing the exact same issue. I have a feeling the solution will involve using:

await page.setRequestInterceptionEnabled(true);

page.on('request', interceptedRequest => {
    //some code here that adds this request to ...
    //a list and checks whether all list items have ...
    //been successfully completed!
});

https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#pagesetrequestinterceptionenabledvalue

1
  • It should be possible to achieve it using promises only. Sep 13, 2017 at 11:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.