Is there a (Unix) shell script to format JSON in human-readable form?
Basically, I want it to transform the following:
{ "foo": "lorem", "bar": "ipsum" }
... into something like this:
{
"foo": "lorem",
"bar": "ipsum"
}
Join Stack Overflow to learn, share knowledge, and build your career.
Is there a (Unix) shell script to format JSON in human-readable form?
Basically, I want it to transform the following:
{ "foo": "lorem", "bar": "ipsum" }
... into something like this:
{
"foo": "lorem",
"bar": "ipsum"
}
With Python 2.6+ you can do:
echo '{"foo": "lorem", "bar": "ipsum"}' | python -m json.tool
or, if the JSON is in a file, you can do:
python -m json.tool my_json.json
if the JSON is from an internet source such as an API, you can use
curl http://my_url/ | python -m json.tool
For convenience in all of these cases you can make an alias:
alias prettyjson='python -m json.tool'
For even more convenience with a bit more typing to get it ready:
prettyjson_s() {
echo "$1" | python -m json.tool
}
prettyjson_f() {
python -m json.tool "$1"
}
prettyjson_w() {
curl "$1" | python -m json.tool
}
for all the above cases. You can put this in .bashrc
and it will be available every time in shell. Invoke it like prettyjson_s '{"foo": "lorem", "bar": "ipsum"}'
.
Note that as @pnd pointed out in the comments below, in Python 3.5+ the JSON object is no longer sorted by default. To sort, add the --sort-keys
flag to the end. I.e. ... | python -m json.tool --sort-keys
.
pygmentize -l javascript
to get syntax colored output in your command line. Edit: If you have pygments installed that is.
– Shrikant Sharat
Dec 3 '11 at 3:24
alias pretty='python -mjson.tool | pygmentize -l json
so that I can just run: command params | pretty
. Hope this helps. PS: Should anyone manages to extend this to a) remove the curl-output I'm seeing every time and/or b) NOT sort the json keys; please do let me know, I will be highly thankful.
– Clint Eastwood
Feb 25 '15 at 17:42
You can use: jq
It's very simple to use and it works great! It can handle very large JSON structures, including streams. You can find their tutorials here.
Usage examples:
$ jq --color-output file1.json file1.json | less -R
$ command_with_json_output | jq .
$ jq # stdin/"interactive" mode, just enter some JSON
$ jq <<< '{ "foo": "lorem", "bar": "ipsum" }'
{
"bar": "ipsum",
"foo": "lorem"
}
Or use jq
with identity filter:
$ jq '.foo' <<< '{ "foo": "lorem", "bar": "ipsum" }'
"lorem"
--sort-keys
option, which is helpful in some cases.
– Matthew Flaschen
Nov 4 '14 at 1:17
curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5' | jq '.'
– Hover Ruan
Apr 21 '15 at 7:44
<<<
operator – exactly what I was looking for.
– jchook
Dec 28 '16 at 20:10
I use the "space" argument of JSON.stringify
to pretty-print JSON in JavaScript.
Examples:
// Indent with 4 spaces
JSON.stringify({"foo":"lorem","bar":"ipsum"}, null, 4);
// Indent with tabs
JSON.stringify({"foo":"lorem","bar":"ipsum"}, null, '\t');
From the Unix command-line with Node.js, specifying JSON on the command line:
$ node -e "console.log(JSON.stringify(JSON.parse(process.argv[1]), null, '\t'));" \
'{"foo":"lorem","bar":"ipsum"}'
Returns:
{
"foo": "lorem",
"bar": "ipsum"
}
From the Unix command-line with Node.js, specifying a filename that contains JSON, and using an indent of four spaces:
$ node -e "console.log(JSON.stringify(JSON.parse(require('fs') \
.readFileSync(process.argv[1])), null, 4));" filename.json
Using a pipe:
echo '{"foo": "lorem", "bar": "ipsum"}' | node -e \
"\
s=process.openStdin();\
d=[];\
s.on('data',function(c){\
d.push(c);\
});\
s.on('end',function(){\
console.log(JSON.stringify(JSON.parse(d.join('')),null,2));\
});\
"
node -p "JSON.stringify(JSON.parse(process.argv[1]), null, '\t');"
also outputs the result to STDOUT.
– Julian D.
Nov 5 '14 at 16:26
I wrote a tool that has one of the best "smart whitespace" formatters available. It produces more readable and less verbose output than most of the other options here.
This is what "smart whitespace" looks like:
I may be a bit biased, but it's an awesome tool for printing and manipulating JSON data from the command-line. It's super-friendly to use and has extensive command-line help/documentation. It's a Swiss Army knife that I use for 1001 different small tasks that would be surprisingly annoying to do any other way.
Latest use-case: Chrome, Dev console, Network tab, export all as HAR file, "cat site.har | underscore select '.url' --outfmt text | grep mydomain"; now I have a chronologically ordered list of all URL fetches made during the loading of my company's site.
Pretty printing is easy:
underscore -i data.json print
Same thing:
cat data.json | underscore print
Same thing, more explicit:
cat data.json | underscore print --outfmt pretty
This tool is my current passion project, so if you have any feature requests, there is a good chance I'll address them.
[32m
, [33m
, [39m
along with some non-printable before each of them, which makes JSON not valid. However, underscore print alone doesn't add anything to a file and does its formatting job perfectly.
– tiurin
Jan 24 '15 at 12:18
jq
but this worked great for my "JSON" that didn't have double quoted keys.
– Bluu
Feb 27 '15 at 0:37
I usually just do:
echo '{"test":1,"test2":2}' | python -mjson.tool
And to retrieve select data (in this case, "test"'s value):
echo '{"test":1,"test2":2}' | python -c 'import sys,json;data=json.loads(sys.stdin.read()); print data["test"]'
If the JSON data is in a file:
python -mjson.tool filename.json
If you want to do it all in one go with curl
on the command line using an authentication token:
curl -X GET -H "Authorization: Token wef4fwef54te4t5teerdfgghrtgdg53" http://testsite/api/ | python -mjson.tool
| prettyjson
– Iolo
Feb 16 '15 at 9:47
Thanks to J.F. Sebastian's very helpful pointers, here's a slightly enhanced script I've come up with:
#!/usr/bin/python
"""
Convert JSON data to human-readable form.
Usage:
prettyJSON.py inputFile [outputFile]
"""
import sys
import simplejson as json
def main(args):
try:
if args[1] == '-':
inputFile = sys.stdin
else:
inputFile = open(args[1])
input = json.load(inputFile)
inputFile.close()
except IndexError:
usage()
return False
if len(args) < 3:
print json.dumps(input, sort_keys = False, indent = 4)
else:
outputFile = open(args[2], "w")
json.dump(input, outputFile, sort_keys = False, indent = 4)
outputFile.close()
return True
def usage():
print __doc__
if __name__ == "__main__":
sys.exit(not main(sys.argv))
dict
objects do not have a defined order. Try json.dumps(json.loads('{"b": 1, "a": 2}'), sort_keys=False)
and you'll find they're swapped anyway. To fix it, import OrderedDict
and load
passing object_pairs_hook=OrderedDict
.
– icktoofay
May 31 '13 at 2:58
inputFile = sys.stdin
. This lets you pipe stuff to the script like so: curl http://somewhere.com/foo.json | pp_json.py
– Gabe Johnson
Oct 31 '14 at 18:37
from collections import OrderedDict
.
– Hugo
Sep 22 '15 at 9:30
com! FormatJSON %!python -c "from collections import OrderedDict; import sys; import json; j = json.load(sys.stdin, object_pairs_hook=OrderedDict); json.dump(j, sys.stdout, sort_keys=False, indent=4, separators=(',', ': '))"
Note that the separators must be set as (',', ': ') to avoid trailing whitespace being added: bugs.python.org/issue16333
– blindsnowmobile
Jun 13 '16 at 17:05
sort_keys = True
instead, because I want to use this to compare json files and it works like a charm.
– JL Peyret
Jan 25 '17 at 20:40
If you use npm and Node.js, you can do npm install -g json
and then pipe the command through json
. Do json -h
to get all the options. It can also pull out specific fields and colorize the output with -i
.
curl -s http://search.twitter.com/search.json?q=node.js | json
jq . file.json
; but also cat file.json | jq
(without the filter .
) also works. (on ubuntu/linux; ymmv across platforms)
– michael
Dec 1 '17 at 7:33
With Perl, use the CPAN module JSON::XS
. It installs a command line tool json_xs
.
Validate:
json_xs -t null < myfile.json
Prettify the JSON file src.json
to pretty.json
:
< src.json json_xs > pretty.json
If you don't have json_xs
, try json_pp
. "pp" is for "pure perl" – the tool is implemented in Perl only, without a binding to an external C library (which is what XS stands for, Perl's "Extension System").
-t null
gave me null: not a valid toformat... but leaving it off worked splendidly. Thanks.
– Lucas
Mar 4 '17 at 20:53
On *nix, reading from stdin and writing to stdout works better:
#!/usr/bin/env python
"""
Convert JSON data to human-readable form.
(Reads from stdin and writes to stdout)
"""
import sys
try:
import simplejson as json
except:
import json
print json.dumps(json.loads(sys.stdin.read()), indent=4)
sys.exit(0)
Put this in a file (I named mine "prettyJSON" after AnC's answer) in your PATH and chmod +x
it, and you're good to go.
fileinput.input()
reads from stdin if there are no files given at a command-line. Example
– jfs
May 19 '11 at 6:41
The JSON Ruby Gem is bundled with a shell script to prettify JSON:
sudo gem install json
echo '{ "foo": "bar" }' | prettify_json.rb
Script download: gist.github.com/3738968
json.tool
. However, it also seems to have nesting depth limitations (nesting of 20 is too deep (JSON::NestingError)
).
– a3nm
May 30 '11 at 6:40
sudo apt-get install ruby-json-pure
instead of gem install
– Janus Troelsen
Mar 27 '12 at 17:45
~/bin
folder (make sure it's in your PATH) rename prettify_json.rb
to ppj
and run chmod +x ppj
. Now you can do something like curl www.jsonsring.com/something.json | ppj
– Uri
Oct 2 '12 at 14:39
a simple bash script for pretty json printing
json_pretty.sh
#/bin/bash
grep -Eo '"[^"]*" *(: *([0-9]*|"[^"]*")[^{}\["]*|,)?|[^"\]\[\}\{]*|\{|\},?|\[|\],?|[0-9 ]*,?' | awk '{if ($0 ~ /^[}\]]/ ) offset-=4; printf "%*c%s\n", offset, " ", $0; if ($0 ~ /^[{\[]/) offset+=4}'
Example:
cat file.json | json_pretty.sh
%*c
notation so I changed the printf
to c=0; while (c++<offset) printf " "; printf $0;
. And my awk has different regex escaping and backslash does not work in []
. I changed the regexes to /^[[{]/
and /[]}]/
.
– Zaboj Campula
Oct 6 '16 at 10:21
grep
that was erased by mistake ;)
– Blag
Feb 21 '18 at 6:47
UPDATE I'm using jq
now as suggested in another answer. It's extremely powerful at filtering JSON, but, at its most basic, also an awesome way to pretty print JSON for viewing.
jsonpp is a very nice command line JSON pretty printer.
From the README:
Pretty print web service responses like so:
curl -s -L http://<!---->t.co/tYTq5Pu | jsonpp
and make beautiful the files running around on your disk:
jsonpp data/long_malformed.json
If you're on Mac OS X, you can brew install jsonpp
. If not, you can simply copy the binary to somewhere in your $PATH
.
python -mjson.tool
(from other answer here) and it took 10-20sec...
– volker
Apr 28 '15 at 19:27
json_pp
- which does format json nicely, although despite the similarity in naming, I believe this to be an entirely different project from the jsonpp
mentioned here
– davidpricedev
Jul 29 '16 at 11:24
That's how I do it:
curl yourUri | json_pp
It shortens the code and gets the job done.
$ echo '{ "foo": "lorem", "bar": "ipsum" }' \
> | python -c'import fileinput, json;
> print(json.dumps(json.loads("".join(fileinput.input())),
> sort_keys=True, indent=4))'
{
"bar": "ipsum",
"foo": "lorem"
}
NOTE: It is not the way to do it.
The same in Perl:
$ cat json.txt \
> | perl -0007 -MJSON -nE'say to_json(from_json($_, {allow_nonref=>1}),
> {pretty=>1})'
{
"bar" : "ipsum",
"foo" : "lorem"
}
Note 2: If you run
echo '{ "Düsseldorf": "lorem", "bar": "ipsum" }' \
| python -c'import fileinput, json;
print(json.dumps(json.loads("".join(fileinput.input())),
sort_keys=True, indent=4))'
the nicely readable word becomes \u encoded
{
"D\u00fcsseldorf": "lorem",
"bar": "ipsum"
}
If the remainder of your pipeline will gracefully handle unicode and you'd like your JSON to also be human-friendly, simply use ensure_ascii=False
echo '{ "Düsseldorf": "lorem", "bar": "ipsum" }' \
| python -c'import fileinput, json;
print json.dumps(json.loads("".join(fileinput.input())),
sort_keys=True, indent=4, ensure_ascii=False)'
and you'll get:
{
"Düsseldorf": "lorem",
"bar": "ipsum"
}
to_json
doesn't seem to accept options. But this works: perl -MJSON -nE 'say JSON->new->pretty->encode(from_json $_)' text.json
– Rörd
Dec 16 '11 at 15:12
python -m json.tool
.
– dwlz
Mar 17 '14 at 2:28
json.tool
code examples. 1. this version allows you to change some parameters e.g., indent
2. At the time of the posting (2008) Python 2.4 was still used that doesn't support -mjson.tool
– jfs
Mar 17 '14 at 7:21
Try pjson
. It has colors!
Install it with pip
:
âš¡ pip install pjson
And then pipe any JSON content to pjson
.
I use jshon to do exactly what you're describing. Just run:
echo $COMPACTED_JSON_TEXT | jshon
You can also pass arguments to transform the JSON data.
You can use this simple command to achieve the result:
echo "{ \"foo\": \"lorem\", \"bar\": \"ipsum\" }"|python -m json.tool
Or, with Ruby:
echo '{ "foo": "lorem", "bar": "ipsum" }' | ruby -r json -e 'jj JSON.parse gets'
to_json
methods; Kernel#jj
only pretty-prints arrays and hashes of the same (or numbers/strings/booleans).
– Phrogz
Jun 27 '11 at 15:59
echo { "foo": "lorem", "bar": "ipsum" } | ruby -r json -e 'jj JSON.parse gets'
– Ross Attrill
Feb 14 '16 at 22:43
Check out Jazor. It's a simple command line JSON parser written in Ruby.
gem install jazor
jazor --help
curl
and this is the only one that did it for me.
– Leo Cassarani
Nov 23 '11 at 0:32
JSONLint has an open-source implementation on GitHub that can be used on the command line or included in a Node.js project.
npm install jsonlint -g
and then
jsonlint -p myfile.json
or
curl -s "http://api.twitter.com/1/users/show/user.json" | jsonlint | less
npx
instead: curl -s "http://api.twitter.com/1/users/show/user.json" | npx jsonlint | less
(i.e. no npm install
necessary)
– Adrien Joly
Jun 18 '20 at 12:50
Simply pipe the output to jq .
.
Example:
twurl -H ads-api.twitter.com '.......' | jq .
cat <file_name.txt> | jq . > <output_name.txt>
– Pramit
Oct 7 '16 at 21:50
jq .
for pretty-printing has one potential drawback: all extant versions of jq insist on treating JSON numbers as IEEE numbers, so precision is easily lost, e.g. for very large integers.
– peak
Oct 21 '16 at 6:00
cat file |
is invariably a waste of a process; just do jq . <file_name.txt >output_name.txt
(with literal <
and >
characters).
– Mark Reed
Jan 31 '17 at 13:08
With Perl, if you install JSON::PP from CPAN you'll get the json_pp command. Stealing the example from B Bycroft you get:
[pdurbin@beamish ~]$ echo '{"foo": "lorem", "bar": "ipsum"}' | json_pp
{
"bar" : "ipsum",
"foo" : "lorem"
}
It's worth mentioning that json_pp
comes pre-installed with Ubuntu 12.04 (at least) and Debian in /usr/bin/json_pp
I combine Python's json.tool with pygmentize:
echo '{"foo": "bar"}' | python -m json.tool | pygmentize -g
There are some alternatives to pygmentize which are listed in my this answer.
Here is a live demo:
python-pygments
, i.e. apt-get install python-pygments
– jan
Jan 5 '18 at 11:56
You can simply use standard tools like jq or json_pp.
echo '{ "foo": "lorem", "bar": "ipsum" }' | json_pp
or
echo '{ "foo": "lorem", "bar": "ipsum" }' | jq
will both prettify output like the following (jq even more colorful):
{
"foo": "lorem",
"bar": "ipsum"
}
The huge advantage of jq is that it can do A LOT more if you'd like to parse and process the json.
I recommend using the json_xs command line utility which is included in the JSON::XS perl module. JSON::XS is a Perl module for serializing/deserializing JSON, on a Debian or Ubuntu machine you can install it like this:
sudo apt-get install libjson-xs-perl
It is obviously also available on CPAN.
To use it to format JSON obtained from a URL you can use curl or wget like this:
$ curl -s http://page.that.serves.json.com/json/ | json_xs
or this:
$ wget -q -O - http://page.that.serves.json.com/json/ | json_xs
and to format JSON contained in a file you can do this:
$ json_xs < file-full-of.json
To reformat as YAML, which some people consider to be more humanly-readable than JSON:
$ json_xs -t yaml < file-full-of.json
jj is super-fast, can handle ginormous JSON documents economically, does not mess with valid JSON numbers, and is easy to use, e.g.
jj -p # for reading from STDIN
or
jj -p -i input.json
It is (2018) still quite new so maybe it won’t handle invalid JSON the way you expect, but it is easy to install on major platforms.
bat
is a cat
clone with syntax highlighting:
Example:
echo '{"bignum":1e1000}' | bat -p -l json
-p
will output without headers, and -l
will explicitly specify the language.
It has colouring and formatting for JSON and does not have the problems noted in this comment: How can I pretty-print JSON in a shell script?
Install yajl-tools with the command below:
sudo apt-get install yajl-tools
then,
echo '{"foo": "lorem", "bar": "ipsum"}' | json_reformat
json
library, but I added pygments as well to get syntax highlighting. – exhuma Nov 9 '12 at 13:40