How do i screen scrape a html page generated by javascript? Ive tried something like this:
HtmlAgilityPack.HtmlDocument doc = new HtmlAgilityPack.HtmlDocument();
HtmlWeb hw = new HtmlWeb();
doc = hw.Load("http://stats.nba.com/scores.html?gameDate=04/11/2014");
HtmlNodeCollection nodes = doc.DocumentNode.SelectNodes("//div[@id='scoreboards']");
This doesn't work however because the content is generated by javascript. Is there a way to scrape the page after the javascript has generated the html? Maybe using something other then agility pack?