9

I am web scraping a page where with various numbers appears also images of small price charts.

If I click on this images inside the browser I can save that chart as a .png image.

When I look at the source code that element looks like this when inspected:

<div class="performance_2d_sparkline graph ng-isolate-scope ng-scope" x-data-percent-change-day="ticker.pct_chge_1D" x-sparkline="watchlistData.sparklineData[ticker.ticker]">
  <span class="inlinesparkline ng-binding">
    <canvas width="100" height="40" style="display: inline-block; width: 100px; height: 40px; vertical-align: top;">
    </canvas>
  </span>
</div>

Is there any way I can save through web scraping the same images that I can save manually through the browser?

0

1 Answer 1

11

If you are using Selenium for your web scraping, you can get the canvas element and save it to the image file using the following code snippet:

# get the base64 representation of the canvas image (the part substring(21) is for removing the padding "data:image/png;base64")
base64_image = driver.execute_script("return document.querySelector('.inlinesparkline canvas').toDataURL('image/png').substring(21);")

# decode the base64 image
output_image = base64.b64decode(base64_image)

# save to the output image
with open("image.png", 'wb') as f:
   f.write(output_image)
5
  • thanks, but I am really struggling to understand where I should modify with my details. where to enter the tag name in which the image resides? I manage to open the page with Selenium but not sure on how to use your code -- WebDriverException: Message: unknown error: Cannot read property 'toDataURL' of null -- Jun 12, 2017 at 19:45
  • It does work is just that if I launch your code at the same time I am launching the code that login in to the page, your code execute too quickly before the page opened. is there a 'sleep' command? there is also the issue that I have many tags 'inlinesparkline canvas' and in that case it does not work Jun 12, 2017 at 21:13
  • it does only save the first canvas not the others Jun 12, 2017 at 21:21
  • I tried with 'document.querySelectorAll' but it says is not a function Jun 12, 2017 at 21:29
  • sorted with this: for i in range(0,3): base64_image = browser.execute_script("var i = {a}; return document.querySelectorAll('.inlinesparkline canvas')[i].toDataURL('image/png').substring(21);".format(a=i)) # decode the base64 image output_image = base64.b64decode(base64_image) # save to the output image with open("image{a}.png".format(a=i), 'wb') as f: f.write(output_image) Jun 12, 2017 at 22:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.