241

I can't seem to get jq to behave "normally" in a shell pipeline. For example:

$ curl -s https://api.github.com/users/octocat/repos | jq | cat

results in jq simply printing out its help text*. The same thing happens if I try to redirect jq's output to a file:

$ curl -s https://api.github.com/users/octocat/repos | jq > /tmp/stuff.json

Is jq deliberately bailing out if it determines that it's not being run from a tty? How can I prevent this behavior so that I can use jq in a pipeline?

Edit: it looks like this is no longer an issue in recent versions of jq. I have jq-1.6 now and the examples above work as expected.


* (I realize this example contains a useless use of cat; it's for illustration purposes only)

0

2 Answers 2

394

You need to supply a filter as an argument. To pass the JSON through unmodified other than the pretty printing jq provides by default, use the identity filter .:

curl -s https://api.github.com/users/octocat/repos | jq '.' | cat
10
  • 26
    I left it in because the OP had it; I assume it's just a placeholder to indicate that jq is both reading from a pipe and writing to another pipe. If the desire is to simply see the output of jq, then cat itself is unnecessary.
    – chepner
    Oct 21, 2015 at 3:36
  • 4
    (1) jq version 1.5 has been enhanced so that the . is unnecessary in cases such as this. (2) There is no need to quote the ".".
    – peak
    Oct 23, 2015 at 3:27
  • 9
    @peak It may not be necessary for a pipeline, but I have 1.5 and had to provide the "." to redirect the output to a file.
    – MHarris
    Apr 5, 2016 at 14:15
  • 4
    I quote the . out of habit, since my standard practice is to first see what the JSON looks like, then go back and start adding to the filter, which more often than not will need to be quoted.
    – chepner
    Apr 5, 2016 at 14:16
  • 11
    @WalterTross Yes, and I explained in a comment 18 months ago why I quote it.
    – chepner
    Sep 14, 2017 at 23:38
18

One use case I have found myself doing frequently as well is "How do I construct JSON data to supply into other shell commands, for example curl?" The way I do this is by using the --null-input/-n option:

Don’t read any input at all! Instead, the filter is run once using null as the input. This is useful when using jq as a simple calculator or to construct JSON data from scratch.

And an example passing it into curl:

jq -n '{key: "value"}' | curl -d @- \
  --url 'https://some.url.com' \
  -H 'Content-Type: application/json' \
  -H 'Accept: application/json'
3
  • 15
    That's cool, but not sure how it's relevant to this question?
    – mgalgs
    May 20, 2017 at 17:51
  • 5
    @mgalgs I found myself wanting to generate/use jq at the start of a shell pipeline rather than in the middle/end for filtering down on some data. The curl example is a basic one, but I found myself frequently hand typing in JSON data to curl and trying to get quoting right, so I thought it might be useful to others as well.
    – mkobit
    May 20, 2017 at 19:08
  • The question title also being "How to use jq in a shell pipeline?" and this being a highly voted question also leads towards this being relevant for those who stumble upon it.
    – mkobit
    Mar 3 at 2:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.