31

Background:

I want to make an "app" that uses JavaScript/HTML only and can be opened by a browser directly from the filesystem. This app must be able to read data from another file. I'll then use JS to parse it and render pages. As a simplified example, imagine I have a CSV file (download here):

Mark Rodgers,[email protected],Accounting
[...]
Melissa Jones,[email protected],CEO

I want to be able to read the file using JS and use data in it to generate my page.

What I've accomplished so far:

Demo (right-click -> "Save As" to save HTML to your computer). It's also available on jsfiddle in semi-broken fashion (layout is broken, but it should still be functionally correct).

Simply drag and drop the CSV text file into the drag and drop box, or select the text file using the file menu, and JavaScript will read, parse the file and populate the table.

This relies on the FileReader API; most of the heavy lifting is done by this function:

function handleFileSelect(evt) {
    evt.stopPropagation();
    evt.preventDefault();

    var files = evt.target.files || evt.dataTransfer.files; // FileList object.
    var file = files[0];

    // this creates the FileReader and reads stuff as text
    var fr = new FileReader();
    fr.onload = parse;
    fr.readAsText(file);

    // this is the function that actually parses the file
    // and populates the table
    function parse()
    {
        var table = document.getElementById('emps');
        var employees = fr.result.split('\n'); var c = 0;
        for (var i in employees)
        {
            var employee = employees[i].split(',');
            if (employee.length == 3)
            {
                var row = document.createElement('tr');
                row.innerHTML = "<td>" + employee.join("</td><td>") + "</td>";
                table.appendChild(row);
                c++;
            }
        }
        document.getElementById('result').innerHTML = '<span>Added ' + c + ' employees from file: ' + file.name + '</span>';
    }
}

This is almost OK, but it inconveniences the user into manually loading a file. Ideally it should be able to load it automatically, but for security reasons no browser will allow that... yet.

Solution Requirements:

  • Must work offline; ie: it can't rely on any online service. This also includes HTTP servers running on the local machine. The idea is to have this run on any computer with just a browser installed.

  • Must work when the page is opened using the file:/// protocol (ie: a HTML page on the hard drive).

  • Should not rely on third party add ons (eg: Flash, Java, shudders ActiveX). I'm pretty sure these probably wouldn't work anyways if the page is in file:///

  • It must be able to accept arbitrary data. This rules out loading a file in a well-behaved format that's ready for consumption like JSON.

  • If it works on either (ideally both) Firefox or Chrome it's fine. It's also OK to rely on experimental APIs

I know what the file name is beforehand, so it could be coded in the HTML itself. Any solution that enables me to read a file from disk is fine, it doesn't have to use the FileReader API.

So if there's a clever hack to load a file into a page that's fine too (maybe load it into an invisible iframe and have JS retrieve the contents); that's OK too.

12
  • 2
    In Chrome you should specify command line parameter --allow-file-access-from-files in order to allow access to file:// scheme. I'm not sure Chrome will allow to do this in any other, more convenient way, because of security considerations.
    – Stan
    Nov 25, 2012 at 21:37
  • 3
    <script id="file" type="text/html" src="file.txt"></script> Nov 26, 2012 at 6:33
  • 1
    Not sure, but the node-webkit project might be interesting to you. github.com/rogerwang/node-webkit Nov 27, 2012 at 13:39
  • if you only need mozilla (firefox/seamonkey) or chrome, you can use an XMLHttpRequest on file:// (though chrome will require that "feature" to be enabled) as long as the webpage is also on file:// - I updated my answer with an example (no check for browser though, there are already plenty of examples for that). I'm pretty sure it won't work if the page itself is not also on file:// (at least it shouldn't, if it does its a huge security hole that should be reported immediately) Nov 27, 2012 at 22:18
  • Is there a reason you are not just using html5 offline api? Basically the same thing except you don't even have to save anything on the computer. Also, with FileSystem API the input file needs to be prompted only once.
    – Esailija
    Nov 29, 2012 at 21:51

6 Answers 6

4
+50

Here is the code I used for Firefox, which is not portable, but works:

As OP commented, enablePrivilege() has been deprecated, this should be considered usable. But as my Firefox using previous profile still work with my code, so I dig a little into the prefs.js (as about:config is hiding these settings,) And here is the settings you need you get it work.

user_pref("capability.principal.codebase.p0.granted", "UniversalXPConnect");
user_pref("capability.principal.codebase.p0.id", "file://");  // path to the html file.
user_pref("capability.principal.codebase.p0.subjectName", "");

And here goes the code:

var File = function(file) {
  netscape.security.PrivilegeManager.enablePrivilege('UniversalXPConnect');
  var ios = Components.classes["@mozilla.org/network/io-service;1"]
                            .getService(Components.interfaces.nsIIOService);
  if (!File.baseURI) {
    File.baseURI = ios.newURI(location.href.substring(0, location.href.lastIndexOf('/')+1), null, null);
    File.baseFolder = File.baseURI.QueryInterface(Components.interfaces.nsIFileURL).file.path;
  }
  var URL = ios.newURI(file, null, File.baseURI);
  this.fptr = URL.QueryInterface(Components.interfaces.nsIFileURL).file;
}

File.prototype = {
  write: function(data) {
    netscape.security.PrivilegeManager.enablePrivilege('UniversalXPConnect');
    var foStream = Components.classes["@mozilla.org/network/file-output-stream;1"]
                             .createInstance(Components.interfaces.nsIFileOutputStream);
    foStream.init(this.fptr, 0x02 | 0x08 | 0x20, 0666, 0);
    var converter = Components.classes["@mozilla.org/intl/converter-output-stream;1"]
                              .createInstance(Components.interfaces.nsIConverterOutputStream);
    converter.init(foStream, null, 0, 0);
    converter.writeString(data);
    converter.close();
  },
  read: function() {
    netscape.security.PrivilegeManager.enablePrivilege('UniversalXPConnect');
    var fstream = Components.classes["@mozilla.org/network/file-input-stream;1"]
                            .createInstance(Components.interfaces.nsIFileInputStream);
    var cstream = Components.classes["@mozilla.org/intl/converter-input-stream;1"]
                            .createInstance(Components.interfaces.nsIConverterInputStream);
    fstream.init(this.fptr, -1, 0, 0);
    cstream.init(fstream, null, 0, 0);
    var data = "";
    // let (str = {}) { // use this only when using javascript 1.8
    var str = {};
      cstream.readString(0xffffffff, str);
      data = str.value;
    // }
    cstream.close();
    return data;
  }
};
5
  • What argument does File() take? Nov 17, 2012 at 7:18
  • @NullUserException the path to the file, in relative to current file. like var file = new File('xxx.csv');
    – xiaoyi
    Nov 17, 2012 at 7:21
  • Hmmm Firefox is choking on this line: let (str = {}) { Nov 17, 2012 at 7:28
  • @NullUserException I have updated my solution already. It was in Javascript 1.8 specs. Just use my latest code.
    – xiaoyi
    Nov 17, 2012 at 7:29
  • Damn, now FF tells me Error: A script from "file://" was denied UniversalXPConnect privileges. I will try to monkey with the settings later, but it's not looking good from what I'm reading here: bugzilla.mozilla.org/show_bug.cgi?id=790913 Nov 17, 2012 at 7:36
4
+100

Here is an example that uses JSON data in an external file that works locally or on a server. This example just uses the browser's language setting to load a < script > with localized html and then processes its json object to reset the data in the indicated tags with localized content

<html><meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<head>
<script>
    function setLang(){
        for (var i=0;i<items.length;i++){
            term=document.getElementById(items[i].id)
            if (term) term.innerHTML=items[i].value
        }
    }
    var lang=navigator.userLanguage || navigator.language;
    var script=document.createElement("script");
    script.src=document.URL+"-"+lang.substring(0,2)+".js"
    var head = document.getElementsByTagName('head')[0]
    head.insertBefore(script,head.firstChild)
</script>
</head>
<body onload='setLang()'>
<div id="string1" class="txt">This is the default text of string1.</div> 
<div id="string2" class="txt">This is the default text of string2.</div>
</body></html>

The data files for this look like:

items=[
{"id":"string1","value":"Localized text of string1."},
{"id":"string2", "value":"Localized text of string2."}
];

but you can use any parameter to conditionally load the appropriate file (it will be inserted as the first tag in < head >, so it will be usable in anywhere) and the JSON format is capable of handling a large variety of data. You may want to rename the function setLang to something more appropriate and modify it to meet your needs such as ... for each i add a row, then add fields with the data (it looks like you already have a handle on that part) and your JSON would look like:

items=[
{"fname":"john","lname":"smith","address":"1 1st St","phone":"555-1212"},
{"fname":"jane","lname":"smith","address":"1 1st St","phone":"555-1212"}
];

if you need to preprocess your data, awk is pretty handy - it would be something like: (untested guestimate)

awk 'BEGIN{FS=",";print "items=[\n"}
{printf "{\"fname\":\"%s\",\"lname\":\"smith\",\"address\":\"1 1st St\",\"phone\":\"555-1212\"},\n", $1, $2, $3, $4}
END{print "];"}' file.csv > file.js

Edit: now that OP is more clear, only mozilla browsers allow XMLHttpRequest on file:// out of the box and chrome (possibly other webkit based browsers) can be configured to allow it. Knowing that it may NOT work on IE<10, you can:

var filePath = "your_file.txt";
xmlhttp = new XMLHttpRequest();
xmlhttp.open("GET",filePath,false);
xmlhttp.overrideMimeType('text/plain');
xmlhttp.send(null);
//maybe check status !=404 here
var fileContent = xmlhttp.responseText;
var fileArray = fileContent.split('\n')
var n = fileArray.length;
//process your data from here probably using split again for ','

I'm leaving the initial json-p variation for others that may have a similar issue, but have some control of their data format, since it will work on all javascript capable browsers. However, if anyone knows a way to make it work for IE (other than running a small web server), please edit.

Edit 2:

With mozilla browsers you can also use iframes

<html>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<head>
<script>
function showContents(frameObject){
    alert(frameObject.contentDocument.body.innerHTML);
    //replace with your code
}
</script>
</head>
<body onload='showContents()'>
<iframe id="frametest" src="data.txt" onload="showContents(this);" 
    style="visibility:hidden;display:none"></iframe>
</body></html>
6
  • 1
    I am sorry, but I have no control over the format of the files, nor will I be able preprocess the files in any way before loading them with JS. Otherwise I wouldn't be asking this question, because this is the easy way out. It must be able to handle arbitrary data Nov 27, 2012 at 6:09
  • Actually this is not a JSON file, but a JSONP script.
    – Bergi
    Nov 27, 2012 at 6:18
  • true, I only said it was a file that contained json data because I didn't wan't to imply that it needed a json-p style callback function wrapper. It basically acts as an include file that could contain variables, functions, etc... but since the OP updated that the format is fixed as csv, XMLHttpRequest is really the only way and not portable, unless there is some hack to get the innerHTML of an "external" script ... iframe method only works by default in mozilla browsers too Nov 27, 2012 at 22:55
  • +1 This was the first thing I tried and it failed on me, thanks to Chrome's overprotective security policies. Worked out of the box on FF. Works on Chrome if you use --allow-file-access-from-files, and it works on IE10 as well (there's a pop up asking if you want to allow blocked content). Unlike FileReader, Ajax can get a bit messy when you handle binary files but I think I can live with it. This seems to be the best solution so far. Nov 29, 2012 at 23:01
  • @NullUserException "Ajax can get a bit messy when you handle binary files". That's caused by normalization. If you're expecting binary responses, use an appropriate responseType attribute of a XHR instance, e.g. "arraybuffer" or "blob".
    – Rob W
    Dec 11, 2012 at 13:21
3

Assuming the csv file is in the same directory as the app, I would load the file with AJAX. As far as I know, one can get the file in text format, and then parse it. This should work in IE and Firefox, but does not work in Chrome (unless one runs chrome with the --allow-file-access-from-files command line setting).

2

This can be done quite easily using javascript XMLHttpRequest() class:

function FileHelper()
{}
{
    FileHelper.readStringFromFileAtPath = function(pathOfFileToReadFrom)
    {
        var request = new XMLHttpRequest();
        request.open("GET", pathOfFileToReadFrom, false);
        request.send(null);
        var returnValue = request.responseText;

        return returnValue;
    }
}

...

var text = FileHelper.readStringFromFileAtPath ( "mytext.txt" );
1

As I understand, the content of the file is completely under your control, and it doesn't have to be a specific format? And you only need a way to read?

You could declare a global function "handleFile". In your external file the content would have to be like this:

handleFile('Mark Rodgers,[email protected],Accounting');

To "read" the file, just add a script element with a corresponding src attribute. In your function "handleFile", you get your contents.

The location of the file would probably initally have to be set by the user, but after that you can save the location in localStorage or something like that.

1
  • 1
    Regrettably, the format of the file is not under my control. Nov 26, 2012 at 15:12
1

Ensuring that the file is in the same directory or in a sub directory, load the file with AJAX.

Unlike a script tag, you will get access to the contents.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.