177

I'm trying to get jq to parse a JSON structure like:

{
  "a" : 1,
  "b" : 2,
  "c" : "{\"id\":\"9ee ...\",\"parent\":\"abc...\"}\n"
}

That is, an element in the JSON is a string with escaped json.

So, I have something along the lines of $ jq [.c] myFile.json | jq [.id]

But that crashes with jq: error: Cannot index string with string

This is because the output of .c is a string, not more JSON. How do I get jq to parse this string?

My initial solution is to use sed to replace all the escape chars (\":\", \",\" and \") but that's messy, I assume there's a way built into jq to do this?

Thanks!

edit: Also, the jq version available here is:

$ jq --version
jq version 1.3

I guess I could update it if required.

1
  • This question also helps if you are looking for: "How to unescape json string using jq?"
    – k0pernikus
    Oct 18, 2017 at 14:37

3 Answers 3

315

jq has the fromjson builtin for this:

jq '.c | fromjson | .id' myFile.json

fromjson was added in version 1.4.

5
  • 2
    Thank you. This works. I'll accept this answer, as it's more 'idiomaitic' I feel. Cheers. Feb 2, 2016 at 14:04
  • @ColinGrogan please do.
    – vbence
    Oct 20, 2017 at 12:51
  • 1
    @ColinGrogan: I don't see any reason to change the accepted answer since you clearly wrote in your question that you used the version 1.3 of jq in which the fromjson feature isn't available. In other words, even if this answer is interesting, it doesn't answer the question. Oct 20, 2017 at 15:57
  • Is it possible to use this but on an entire json file (not specifying the .id property)? Mar 15, 2019 at 10:01
  • 2
    @FlorianCastelain yes, either omit it or use dot: jq 'fromjson | .' myfile, where myfile contains "{\"key\":1, \"word\":\"cat\"}"
    – user3456014
    Sep 30, 2019 at 8:21
78

You can use the raw output (-r) that will unescape characters:

jq -r .c myfile.json | jq .id

ADDENDUM: This has the advantage that it works in jq 1.3 and up; indeed, it should work in every version of jq that has the -r option.

2
  • 1
    -r is much easier than the accepted answer. You got my upvote. I use jq to view json from my clipboard. So when I run into this I do: pbpaste | jq -r
    – Noah Gary
    Oct 27, 2021 at 15:30
  • bonus: if you want to escape quotes you can use sed: pbpaste | sed -e 's/\\\"/\"/g'. this helps when you have escaped json that contains escaped json. Take out the first level of escaped quotes with sed and then jq can remove the last level. e.g pbpaste | sed -e 's/\\\"/\"/g' | jq -r
    – Noah Gary
    Oct 27, 2021 at 15:33
1

Motivation: you want to parse JSON string - you want to escape a JSON object that's wrapped with quotes and represented as a String buffer, and convert it to a valid JSON object. For example:

some JSON unescaped string :

"{\"name\":\"John Doe\",\"position\":\"developer\"}"

the expected result ( a JSON object ):

{"name":"John Doe","position":"developer"}

Solution: In order to escape a JSON string and convert it into a valid JSON object use the sed tool in command line and use regex expressions to remove/replace specific characters:

cat current_json.txt | sed -e 's/\\\"/\"/g' -e 's/^.//g' -e 's/.$//g'

s/\\\"/\"/g replacing all backslashes and quotes ( \" ) into quotes only (")

s/^.//g replacing the first character in the stream to none character

s/.$//g replacing the last character in the stream to none character

3
  • I had a similar problem. Used the same sed command verbatum. Bravo! Pipe that result to jq -r and you can process escaped json inside your json.
    – Noah Gary
    Oct 27, 2021 at 15:37
  • Boo, hiss re: trying to parse JSON with sed. Any tool that tries to match just a few of the most common escapings is going to miss others (for example, you're handling " but not \t) Jun 14, 2022 at 23:44
  • I had to also replace double quote characters: cat a.json | sed -e 's/\\\"/\"/g' -e 's/^.//g' -e 's/.$//g' -e 's/""/"/g' | jq Jul 21, 2022 at 20:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.