158

Consider the curl command below, is it possible to allow newline in JSON (without the minify) and execute directly in bash (Mac/Ubuntu)

curl -0 -v -X POST http://www.example.com/api/users \
-H "Expect:" \
-H 'Content-Type: text/json; charset=utf-8' \
-d \
'
{
    "field1": "test",
    "field2": {
        "foo": "bar"
    }
}'

When I run the command above, seems error occurred at the second { How to fix the above command?

Updated: actually I was able to run the command without issue previously, not sure why problem happen recently.

4
  • 2
    Can you tell us more about the error? Your example works "as is" on my system. mymac > bash --version GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin15) Copyright (C) 2007 Free Software Foundation, Inc. Jan 27, 2016 at 21:00
  • Yup, works for me as well: GNU bash, version 4.3.42(1)-release
    – miken32
    Jan 29, 2016 at 4:43
  • 3
    Also check out ANSI C-like string syntax: echo $'here is a newline:\nand here is a tab:\t'
    – miken32
    Jan 29, 2016 at 19:50
  • 1
    application/json is the correct media type for JSON data -- see RFC4627
    – Pocketsand
    Jan 10, 2019 at 19:02

6 Answers 6

215

I remembered another way to do this with a "Here Document" as described in the Bash man page and detailed here. The @- means to read the body from STDIN, while << EOF means to pipe the script content until "EOF" as STDIN to curl. This layout may be easier to read than using separate files or the "echo a variable" approach.

curl -0 -v -X POST http://www.example.com/api/users \
-H "Expect:" \
-H 'Content-Type: application/json; charset=utf-8' \
--data-binary @- << EOF
{
    "field1": "test",
    "field2": {
        "foo": "bar"
    }
}
EOF

NOTE: Use the --trace <outfile> curl option to record exactly what goes over the wire. For some reason, this Here Document approach strips newlines. (Update: Newlines were stripped by curl -d option. Corrected!)

8
  • 6
    This is clean, no extra quoting, no escaping and it works very well. Thanks.
    – Seth
    Dec 20, 2016 at 17:53
  • 1
    Can we use pipes with such a syntax? Mar 8, 2018 at 14:31
  • 3
    Yes you can pipe the output to another command, although the placement is right in the middle. Add the stdout redirect after the stdin redirect. Example using word count: -d @- << EOF | wc May 4, 2018 at 15:15
  • 2
    It's not Here Document that strips new lines, but curl -d option: curl.haxx.se/docs/manpage.html#-d. Use --data-binary to keep newline and carriage return characters.
    – dzieciou
    Jun 28, 2019 at 12:30
  • 1
    You most certainly can insert variables! See what happens when you replace "foo": "bar" with "$SHELL": "$(date)". Mar 14, 2021 at 21:07
58
+50

Along the lines of Martin's suggestion of putting the JSON in a variable, you could also put the JSON in a separate file, and then supply the filename to -d using curl's @ syntax:

curl -0 -v -X POST http://www.example.com/api/users \
  -H "Expect:" \
  -H 'Content-Type: text/json; charset=utf-8' \
  -d @myfile.json

The disadvantage is obvious (2 or more files where you used to have one.) But on the plus side, your script could accept a filename or directory argument and you'd never need to edit it, just run it on different JSON files. Whether that's useful depends on what you are trying to accomplish.

3
  • 1
    Note: make sure you've valid json content through jsonlint.com
    – vikramvi
    Feb 20, 2018 at 14:10
  • This approach is clean and easy to debug compared to others.
    – vikramvi
    Feb 20, 2018 at 14:22
  • Great solution. Be careful of PATH tho ! Oct 23, 2018 at 15:30
36

For some reason, this Here Document approach strips newlines

@eric-bolinger the reason the Heredoc strips newlines is because you need to tell your Heredoc to preserve newlines by quoting the EOF:

curl -0 -v -X POST http://www.example.com/api/users \
-H "Expect:" \
-H 'Content-Type: text/json; charset=utf-8' \
-d @- <<'EOF'

{
    "field1": "test",
    "field2": {
        "foo": "bar"
    }
}
EOF

Notice the single-ticks surrounding EOF the first time it's defined, but not the second.

30

You should use outer double quotes, and the escape all inner quotes like this:

curl -0 -v -X POST http://www.example.com/api/users \
-H "Expect:" \
-H 'Content-Type: text/json; charset=utf-8' \
-d \
"
{
    \"field1\": \"test\",
    \"field2\": {
        \"foo\": \"bar\"
    }
}"
28

You could assign your json to a var:

json='
{
    "field1": "test",
    "field2": {
        "foo": "bar"
    }
}'

Now you can forward this to curl using stdin:

echo $json | curl -0 -v -X POST http://www.example.com/api/users \
-H "Expect:" \
-H 'Content-Type: text/json; charset=utf-8' \
-d @-
2
  • 1
    Using single quotes to surround the block means that you cannot use variables (e.g. ${username}) in the JSON.
    – Air
    Jan 6, 2017 at 3:10
  • 1
    Yup, but using double quotes means you can't use $ signs in your data. Pick which one is right for you. Jan 6, 2017 at 6:13
10

I think this can be an answer

curl -0 -v -X POST http://www.example.com/api/users \
-H "Expect:" \
-H 'Content-Type: text/json; charset=utf-8' \
--data-raw '
{
    "field1": "test",
    "field2": {
        "foo": "bar"
    }
}'
1
  • I like this because it works well even with indentation and if one uses "" quotes, even variables can be used.
    – lnksz
    Aug 5, 2022 at 16:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.