I would like to convert a list into JSON array. I'm looking at jq
for this but the examples are mostly about parsing JSON (not creating it). It would be nice to know proper escaping will occur. My list is single line elements so the new line will probably be the best delimiter.
5 Answers
I was also trying to convert a bunch of lines into a JSON array, and was at a standstill until I realized that -s
was the only way I could handle more than one line at a time in the jq
expression, even if that meant I'd have to parse the newlines manually.
jq -R -s -c 'split("\n")' < just_lines.txt
-R
to read raw input-s
to read all input as a single string-c
to not pretty print the output
Easy peasy.
Edit: I'm on jq
≥ 1.4, which is apparently when the split
built-in was introduced.
-
Looks easy .. I can't verify though. I get an error on ubuntu jq version 1.3
error: split is not defined
Commented Jan 18, 2015 at 16:18 -
1Unfortunately, the
jq
documentation doesn't say when each built-in function was introduced, nor doesjq --version
tell me what version I've got, but it's at least 1.4, installed withbrew install jq --HEAD
. Good point, though, I'll edit the answer.– chbrownCommented Jan 18, 2015 at 20:15 -
6I like this answer except that for a multi-line output that has a trailing newline(\n), the
split
built-in function will yield an extra empty string item in the array :( Anyway to fix that?– DevyCommented Oct 21, 2016 at 15:46 -
3My current hack is to repipe it into
jq
to use its slicing feature ¯_(ツ)_/¯echo $(jq -R -s -c 'split("\n")' < urls.txt) | jq '.[:-1]'
Commented Feb 18, 2018 at 6:03 -
3Consider
'split("\n") | map(select(length > 0))'
to remove empty lines, too. Commented Nov 5, 2019 at 21:11
--raw-input, then --slurp
Just summarizing what the others have said in a hopefully quicker to understand form:
cat /etc/hosts | jq --raw-input . | jq --slurp .
will return you:
[
"fe00::0 ip6-localnet",
"ff00::0 ip6-mcastprefix",
"ff02::1 ip6-allnodes",
"ff02::2 ip6-allrouters"
]
Explanation
--raw-input/-R:
Don´t parse the input as JSON. Instead, each line of text is passed
to the filter as a string. If combined with --slurp, then the
entire input is passed to the filter as a single long string.
--slurp/-s:
Instead of running the filter for each JSON object in the input,
read the entire input stream into a large array and run the filter
just once.
You can also use jq -R .
to format each line as a JSON string and then jq -s
(--slurp
) to create an array for the input lines after parsing them as JSON:
$ printf %s\\n aa bb|jq -R .|jq -s .
[
"aa",
"bb"
]
The method in chbrown's answer adds an empty element to the end if the input ends with a linefeed, but you can use printf %s "$(cat)"
to remove trailing linefeeds:
$ printf %s\\n aa bb|jq -R -s 'split("\n")'
[
"aa",
"bb",
""
]
$ printf %s\\n aa bb|printf %s "$(cat)"|jq -R -s 'split("\n")'
[
"aa",
"bb"
]
If the input lines don't contain ASCII control characters (which have to be escaped in strings in valid JSON), you can use sed
:
$ printf %s\\n aa bb|sed 's/["\]/\\&/g;s/.*/"&"/;1s/^/[/;$s/$/]/;$!s/$/,/'
["aa",
"bb"]
-
When pipeing from
jq
tojq
pretty printing is not necessary:jq -cR . | jq -s .
– cevingCommented Oct 27, 2016 at 10:00 -
1Rather than slurping the inputs in a separate call, just throw the raw inputs into an array.
jq -Rn '[inputs]' input.json
. These are both equivalent:jq -s '.' input.json
andjq -n '[inputs]' input.json
Commented Aug 10, 2017 at 3:18
Update: If your jq has inputs
you can simply write:
jq -nR [inputs] /etc/hosts
to produce a JSON array of strings. This avoids having to read the text file as a whole.
-
Adding
-n
makes it not skip the first line – it avoids consuming the first input separately. Commented Jan 3, 2020 at 3:47 -
This also gets around the issue with an empty element if the input ends with a newline character. Commented Apr 2, 2022 at 2:11
I found in the man page for jq and through experimentation what seems to me to be a simpler answer.
$ cat test_file.txt | jq -Rsc '. / "\n" - [""]'
["aa","bb"]
The -R is to read without trying to parse json, the -s says to read all of the input as one string, and the -c is for one-line output - not necessary, but it's what I was looking for.
Then in the string I pass to jq, the '.' says take the input as it is. The '/ \n' says to divide the string (split it) on newlines. The '- [""]' says to remove from the resulting array any empty strings (resulting from an extra newline at the end).
It's one line and without any complicated constructs, using just simple built in jq features.
jq
man pages. Thanks for the feedback, that up-vote was me.