2

I'd like to download web page as html file. Before I save the web page source code in html file, I'd like to edit some of the page content first. I assume I can edit the content using Javascript. Unfortunately I have little experience with Javascript. I guess I have to inject my script into the web page so that the browser can execute them together. How should I write my script? Should I write a standalone script and pass the page url to my script so that they can be executed at the same time? Or there are other ways to inject my script?

EDIT: To make my problem clear, see this post and this post

7
  • search jquery site for .getScript();
    – Jai
    Jan 11, 2013 at 6:49
  • Your question is not very clear. Where do you want to download this webpage? This task seems to be more suited to a server side script, not javascript. Jan 11, 2013 at 6:50
  • @DarinDimitrov From browser
    – Terry Li
    Jan 11, 2013 at 6:51
  • Saving a HTML file from the browser does only save the initial markup, not any of the JavaScript changes to the DOM.
    – Bergi
    Jan 11, 2013 at 6:51
  • @Bergi I'm not sure. I can edit the source code and save the edited code using Firebug though.
    – Terry Li
    Jan 11, 2013 at 6:54

3 Answers 3

3

As you are only doing this once, starting your script from the browsers JavaScript console should be enough. Open the developer tools, navigate to the console tab, paste your script content, and press enter.

To get the edited HTML, evaluate the expression document.documentElement.outerHTML in the console. Copy the output to a text editor of your choice, prepend it with a doctype, and save it as html.

4
  • To make my problem clear, see this post
    – Terry Li
    Jan 11, 2013 at 7:00
  • Safing the "injected" source from javascript is not an easy task. What is that "further processing" you were talking of? If you told us more about your actual problem, we might can suggest better ways to achieve your aims.
    – Bergi
    Jan 11, 2013 at 7:05
  • I'd like to have millions of web pages annotated with predefined tags so that they will have semantics and thus be processable by machines.
    – Terry Li
    Jan 11, 2013 at 7:10
  • See this post as well
    – Terry Li
    Jan 11, 2013 at 7:13
1

If you want to save modified source as html you can use different aproaches, depends on what you want to mainupulate. Sadly with javascript saveing file is tricky and depends on many things, so you could use option to copy paste file source manually or write your browser and settings specific file saver. I would prefer javascript+php combo solution. Or if there is no need to manipulate someting with javascript i would do it entirely in php.

Step 1 - open browser with console, in chrome and firefox CTRL+SHIFT+J And allow popups. Step 2 - open webpage you want Step 3 - Copy next code to console

//Script loading function
function load_script( source ) {
    var new_script  = document.createElement('script');
    new_script.type = 'text/javascript';
    new_script.src = source;
    new_script.className = 'MyInjectedScript';
    document.getElementsByTagName('head')[0].appendChild(new_script);
}
function escapeHtml(unsafe) {
  return unsafe
      .replace(/&/g, "&")
      .replace(/</g, "&lt;")
      .replace(/>/g, "&gt;")
      .replace(/"/g, "&quot;")
      .replace(/'/g, "&#039;");
}
//Load jQuery, if page do not have it by default
if (typeof(jQuery) != 'function') load_script('http://code.jquery.com/jquery-latest.js');

Step 4 - Do your manipulations in console

Step 5 - Copy next code to console

//In the end remove your injected scripts
$('.MyInjectedScript').remove(); //Or jquery script will be in source
//get Document source
var doc_source = $('html',document).html();
doc_source = '<html>'+doc_source+'</html>';


var new_window = window.open('', '', 'scrollbars=yes,resizable=yes,location=yes,status=yes');
$(new_window.document.body).html('<textarea id="MySource">'+escapeHtml(doc_source)+'</textarea>');

STEP 6 - copy paste code from opened window textarea

If you want to do it with PHP you can easily download page with curl and manipulate content and save file as desired.

-1

You can use a browser extension like Requestly to Inject custom Javascript/CSS on web pages.

This is how you can do it.

  1. Download Requestly and Open Rules Page
  2. Create New Rule and Select Insert Custom Script/CSS Rule Type
  3. Enter your domain (or Page URL Pattern) and define your Script

Screenshot - Insert Script Rule

Requestly Insert Script feature

If you are looking for a Cross-Browser solution then you can use Requestly Desktop App and Configure your rule similarly.

In your particular case, you can choose an option like to run the script after page load so that all DOM Elements are present on the page before you modify/annotate them.

Disclaimer - I built Requestly

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.