15

I try to remove elements by classname but it doesn't work.

This is the code i used:

await page.screenshot({path: 'pic.png'});   //for testing purposes
    let div_selector_to_remove= ".xj7.Kwh5n";
    var thingToRemove = document.querySelectorAll(div_selector_to_remove);
    var number_of_elements = thingToRemove.length;
    for (var i = 0; i < number_of_elements.length; i++) {
        thingToRemove[i].parentNode.removeChild(thingToRemove);
    }

The browser loads and i get a screenshot with the elements loaded. The nothing happens. The elements remain there

3
  • How do you test that they are not removed? You're making a screenshot before. Are you sure that the elements were there when the code is evaluated? Did you try to debug thingToRemove ? Jun 14, 2018 at 23:10
  • I used a non headless version of puppeteer. I could see the elements were still there in chromium. Jun 14, 2018 at 23:18

5 Answers 5

33

Run document.querySelector inside page.evaluate. Here's my answer:

await page.goto('<url_here>');
let div_selector_to_remove= ".xj7.Kwh5n";
await page.evaluate((sel) => {
    var elements = document.querySelectorAll(sel);
    for(var i=0; i< elements.length; i++){
        elements[i].parentNode.removeChild(elements[i]);
    }
}, div_selector_to_remove)
1
  • Once again... You saved me... Thank you so much. I am new to puppeteer and i hadn't really grasped page.evaluate(). Jun 19, 2018 at 23:05
5

Probably easier...

Remove the first node matching selector:

await page.$eval(selector, el => el.remove());

Remove all nodes matching selector:

await page.$$eval(selector, els => els.forEach(el => el.remove()));

Wait for a selector, then remove it:

const el = await page.waitForSelector(selector);
await el.evaluate(el => el.remove());

If for some reason you need to select and remove in console browser-land:

const selector = ".foo";

// single
await page.evaluate(`
  document.querySelector("${selector}").remove()
`);

// multiple
await page.evaluate(selector =>
  document.querySelectorAll(selector).forEach(el => el.remove()),
  selector
);

You can hardcode the selector into the eval string/function as well but I figured it'd be useful to show it coming from a variable in two different ways just in case.

As with anything in Puppeteer, understanding which code runs in Node/Puppeteer-land and which code runs in browser/console-land is extremely important. The rule of thumb is: if it's a callback or stringified function body, it's in the browser and you can only use browser/DOM concepts like HTMLElement.remove(), window and document, otherwise it runs in Node and you can only call Puppeteer API functions. In OP's case, it looks like we're outside of a callback in Node-land, so document isn't a thing, only Puppeteer page. functions.

2

First, number_of_elements is number.

But you call number_of_elements.length.

Next, thingToRemove[i].parentNode.removeChild(thingToRemove), thingToRemove[i].parentNode is parent of thingToRemove[i], not thingToRemove,

so you can't remove thingToRemove from thingToRemove[i].parentNode.

I think this code may be useful.

await page.screenshot({path: 'pic.png'});   //for testing purposes
let div_selector_to_remove= ".xj7.Kwh5n";
var thingToRemove = document.querySelectorAll(div_selector_to_remove);
var number_of_elements = thingToRemove.length;
for (var i = 0; i < number_of_elements; i++) {
    thingToRemove[i].parentNode.removeChild(thingToRemove[i]);
}
2
    //Wait for the element with id="xe7COe" to show up
    await page.waitForSelector('#xe7COe');
    //delete the element with id="xe7COe"  
    await page.evaluate(() => document.querySelector('#xe7COe').remove());
1
  • waitForSelector returns the node it found, so you can simplify your code to const el = await page.waitForSelector('#xe7COe'); and then await el.evaluate(el => el.remove()) rather than re-querying it. If you do want to re-query it, or don't need to wait for it, prefer using page.$eval over page.evaluate(). page.$eval is a shorthand for the common pattern of running document.querySelector as the first thing in an evaluate.
    – ggorlen
    May 24 at 19:19
1

Best Solution Run document.querySelector inside page.evaluate. Here's my answer

    await page.goto('<url_here>');
let div_selector_to_remove= ".xj7.Kwh5n";
await page.evaluate((sel) => {
    var elements = document.querySelectorAll(sel);
    for(var i=0; i< elements.length; i++){
        elements[i].parentNode.removeChild(elements[i]);
    }
}, div_selector_to_remove)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.