7

I need to sign a JSON but I noticed that unmarshaling/marshaling can change JSON's order which might make the signature invalid.

Is there anyway to produce the same hash from a JSON string despite its order?

I've had a look at JOSE but couldn't find the function that actually hashes JSON.

10
  • 2
    You should be able to preserve the order if you avoid using maps.
    – mkopriva
    Mar 20, 2019 at 8:30
  • so if I marshal and unmarshal and marshal again it will produce the same order? I'll test
    – mrclx
    Mar 20, 2019 at 8:31
  • which version of go are you using?
    – Matteo
    Mar 20, 2019 at 8:34
  • @mkopriva Output is deterministic even if you use maps: "The map keys are sorted and used as JSON object keys..." So all in all, output should be deterministic if you don't use custom marshaler in which you screw up the order.
    – icza
    Mar 20, 2019 at 8:35
  • 7
    To verify a signed message you do not need to perform the unmarshall. The signature is made on the JSON string encoded in base64url, and the verification is done directly on the base64. Only if the verification is correct the message is decoded and the unmarshall performed. This is how JWS (Json Web Signature) standard works. Once validated, the order of the nodes is not significant
    – pedrofb
    Mar 20, 2019 at 9:06

1 Answer 1

5

JOSE JWS will absolutely do what you want at the cost of having to manage keys for signatures and verification.

But let's assume that you don't really need the whole key management stuff and general crypto functionality in JOSE and you're not SUPER concerned about performance (so a little string mangling in this process is OK).

You could dumbly unmarshal your JSON and re-marshal it, then just hash that:

package main

import (
    "crypto/sha256"
    "encoding/hex"
    "fmt"
    json "encoding/json"
)

// NB These docs are strictly-speaking the same.
const DOCA = "{ \"foo\": 1.23e1, \"bar\": { \"baz\": true, \"abc\": 12 } }"
const DOCB = "{ \"bar\": { \"abc\": 12, \"baz\": true }, \"foo\": 12.3 }"

func hash(doc string) string {
    // Dumb af, but it's a cheap way to specific the most generic thing
    // you can :-/
    var v interface{}
    json.Unmarshal([]byte(doc), &v) // NB: You should handle errors :-/
    cdoc, _ := json.Marshal(v)
    sum := sha256.Sum256(cdoc)
    return hex.EncodeToString(sum[0:])
}

func main() {
    fmt.Println(DOCA)
    fmt.Printf("Hash: %s\n", hash(DOCA))
    fmt.Println(DOCB)
    fmt.Printf("Hash: %s\n", hash(DOCB))
}

The output of this program (at least in the golang docker container) is:

{ "foo": 1.23e1, "bar": { "baz": true, "abc": 12 } }
Hash: d50756fbb830f8335187a3f427603944c566772365d8d8e6f6760cd2868c8a73
{ "bar": { "abc": 12, "baz": true }, "foo": 12.3 }
Hash: d50756fbb830f8335187a3f427603944c566772365d8d8e6f6760cd2868c8a73

The nice thing about this approach is that, for the cost of some performance, you get insulated from whatever dumb junk you did while marshalling your JSON in the first place (so, unlike other suggestions, you don't have to think about what you might be doing with custom Marshallers and whatnot). This is especially a big deal when you forget that this was an issue at all in version 3.8 of your code a year from now, implement something that messes with the marshal order, and start breaking things.

And, of course, you could always add the hash to the resulting struct and marshal again with the extra item in the map. Obviously you want to optimize a bit for performance if you're worried about it at all and properly handle errors, but this is a good prototype anyway :-)

Oh, and if you're super-worried about edge cases biting you, you could also use canonical JSON to marshal, since it's specifically designed for this type of use (though, honestly, I couldn't come up with an example in my testing where c-json worked but go's default json didn't).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.