45

I am running into a parsing problem when loading JSON files that seem to have the TAB character in them.

When I go to http://jsonlint.com/, and I enter the part with the TAB character:

{
    "My_String": "Foo bar.  Bar foo."
}

The validator complains with:

Parse error on line 2:
{    "My_String": "Foo bar. Bar foo."
------------------^
Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '['

This is literally a copy/paste of the offending JSON text.

I have tried loading this file with json and simplejson without success. How can I load this properly? Should I just pre-process the file and replace TAB by \t or by a space? Or is there anything that I am missing here?

Update:

Here is also a problematic example in simplejson:

foo = '{"My_string": "Foo bar.\t Bar foo."}'
simplejson.loads(foo)

JSONDecodeError: Invalid control character '\t' at: line 1 column 24 (char 23)

5 Answers 5

61

From JSON standard:

Insignificant whitespace is allowed before or after any token. The whitespace characters are: character tabulation (U+0009), line feed (U+000A), carriage return (U+000D), and space (U+0020). Whitespace is not allowed within any token, except that space is allowed in strings.

It means that a literal tab character is not allowed inside a JSON string. You need to escape it as \t (in a .json-file):

{"My_string": "Foo bar.\t Bar foo."}

In addition if json text is provided inside a Python string literal then you need double escape the tab:

foo = '{"My_string": "Foo bar.\\t Bar foo."}' # in a Python source

Or use a Python raw string literal:

foo = r'{"My_string": "Foo bar.\t Bar foo."}' # in a Python source
2
  • 1
    Thanks! I am a bit confused about the statement "Whitespace is not allowed within any token" and your quote in general. The string "Foo bar.\t Bar foo." has spaces in it. How do you infer from that quote, that literal spaces are allowed, but not literal tabs?
    – Josh
    Commented Nov 5, 2013 at 21:47
  • 7
    @Josh: read the full sentence: "except that space is allowed in strings".
    – jfs
    Commented Nov 5, 2013 at 21:49
15

Tabs are legal as delimiting whitespace outside of values, but not within strings. To get a tab inside a JSON string you need to use the sequence \t instead.

But beware multiple levels of interpretation. This Python string from your update:

foo = '{"My_string": "Foo bar.\t Bar foo."}'

is not valid JSON, because the Python interpreter turns that \t sequence into an actual tab character before the JSON processor ever sees it.

You can tell Python to put a literal \t in the string instead of a tab character by doubling the backslash:

foo = '{"My_string": "Foo bar.\\t Bar foo."}'

Or you can use the "raw" string syntax, which doesn't interpret any special backslash sequences:

foo = r'{"My_string": "Foo bar.\t Bar foo."}'

Either way, the JSON processor will see a string containing a backslash followed by a 't', rather than a string containing a tab.

2
  • Hmm. Interesting. Replacing the offending tab by \t passes the JSONlint validator online, but simplejson complains with: JSONDecodeError: Invalid control character '\t' at: line XXX column YYY (char ZZZ)
    – Josh
    Commented Nov 5, 2013 at 21:13
  • 2
    The JSONDecodeError is misleading. It found a literal tab character which is invalid. For the sake of presenting the error message, it translated to '\t' rather than displaying a literal tab on the screen. Commented Nov 5, 2013 at 21:28
5

You can include tabs within values (instead of as whitespace) in JSON files by escaping them. Here's a working example with the json module in Python2.7:

>>> import json
>>> obj = json.loads('{"MY_STRING": "Foo\\tBar"}')
>>> obj['MY_STRING']
u'Foo\tBar'
>>> print obj['MY_STRING']
Foo    Bar

While not escaping the '\t' causes an error:

>>> json.loads('{"MY_STRING": "Foo\tBar"}')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 365, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 381, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Invalid control character at: line 1 column 19 (char 18)
7
  • Thanks mtitan8. Do you know if this issue is mentioned/discussed in an official JSON spec? Or is it specific to how libraries parse the file?
    – Josh
    Commented Nov 5, 2013 at 21:20
  • 3
    Josh, your question implies there's an issue, but there isn't one. If you use simple "\t" then Python interprets that to a literal tab character, which isn't valid in JSON values. So you escape it as "\\t" which results in the Python value "\t" which is then valid JSON. The valid characters are on the right side of json.org Commented Nov 5, 2013 at 21:24
  • Python raw literal could be used: foo = r'{"My_string": "Foo bar.\t Bar foo."}'
    – jfs
    Commented Nov 5, 2013 at 21:30
  • 1
    Thank you @AronGriffis. Just for me to clarify: Are you saying that having unescaped \t characters in a JSON file is legal in JSON? Someone else wrote these files to disk, and I am trying to see if this problem is a result of them not writing the JSON files correctly, or a problem in the way I parse the files. I can see that there is an easy fix anyway.
    – Josh
    Commented Nov 5, 2013 at 21:32
  • 1
    The literal tab character is ASCII 9. This is similar to newline which is ASCII 10 but typically represented in strings (and JSON) as \n. Whoever wrote these files wrote them with literal tabs in the string values. Commented Nov 5, 2013 at 21:39
0

Just to share my experience:

I am using snakemake and a config file written in Json. There are tabs in the json file for indentation. TAB are legal for this purpose. But I am getting error message: snakemake.exceptions.WorkflowError: Config file is not valid JSON or YAML. I believe this is a bug of snakemake; but I could be wrong. Please comment. After replacing all TABs with spaces the error message is gone.

0

In node-red flow i facing same type of problem:

flow.set("delimiter",'"\t"');

error:

{ "status": "ERROR", "result": "Cannot parse config: String: 1: in value for key 'delimiter': JSON does not allow unescaped tab in quoted strings, use a backslash escape" }  

solution:

i added in just \\t in the code.

 flow.set("delimiter",'"\\t"');
1
  • This is for node-red flow and JSON fomat. Commented Nov 4, 2016 at 14:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.