2

The example I'm looking at is morningstar.com. Most of the contents are loaded through this javascript function. When I use wget, I don't get the data that actually show up in the browser (because wget doesn't interprets javascript code).

Is there some way to evaluate the javascript and download the full page (with cmdline or code)?

SRT_stocFund.LoadAComponent("sfcontent", "C", "is", "en-us","USA", "", "USA", null,{showSubTab: false});

2 Answers 2

3

For that specific page, maybe you could just wget the URL of the "Export" button? Obviously this will only work if you are trying to use the actual data that the site is displaying (since that is all that is in the extract).

Here is a link directly to the .csv file: http://financials.morningstar.com/ajax/exportKR2CSV.html?t=GOOG&culture=en_us&region=USA&order=asc&r=765325

4
  • how did you get this csv link?
    – kefeizhou
    Feb 4, 2011 at 4:46
  • I clicked the morningstar link that you supplied and there is a link right on that page (just do a find on the page for "export"). It's in the top-right of the table.
    – Trevor
    Feb 4, 2011 at 7:00
  • Thanks. not the automated way I was hoping for but it'll work.
    – kefeizhou
    Feb 4, 2011 at 15:53
  • It's looking like if you know the symbol for the company (GOOG or AAPL), you can sort of automate the process. Just replace "GOOG" in that URL with the correct symbol. To grab the information for Apple, for example, this would be the URL: financials.morningstar.com/ajax/…
    – Trevor
    Feb 4, 2011 at 16:03
2

Install Firebug for Firefox, and check out what's going in the 'Net' tab. That'll show you all the network requests Firefox puts out for a page. From that you can copy the URL of any Ajax requests (in the XHR tab), and see the response. Feed the Ajax URL to wget (unless its a POST, in which case its a bit trickier).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.