33

Is there a way, in golang, to see if I can differentiate between a json field being set to null vs a json field not being there when unmarshalled into a struct? Because both set the value in the struct to be nil, but I need to know if the field was there to begin with and to see if someone set it to null.

{
  "somefield1":"somevalue1",
  "somefield2":null
}

VS

{
  "somefield1":"somevalue1",
}

Both jsons will be nil when unmarshalled into a struct. Any useful resources will be very appreciated!

6 Answers 6

15

Use json.RawMessage to "delay" the unmarshaling process to determine the raw byte before deciding to do something:

var data = []byte(`{
        "somefield1":"somevalue1",
        "somefield2": null
}`)

type Data struct {
    SomeField1 string          
    SomeField2 json.RawMessage
}

func main() {
    d := &Data{}

    _ = json.Unmarshal(data, &d)

    fmt.Println(d.SomeField1)

    if len(d.SomeField2) > 0 {
        if string(d.SomeField2) == "null" {
            fmt.Println("somefield2 is there but null")
        } else {
            fmt.Println("somefield2 is there and not null")
            // Do something with the data
        }
    } else {
        fmt.Println("somefield2 doesn't exist")
    }
}

See the playground https://play.golang.org/p/Wganpf4sbO

3
  • 1
    Sorry, I am a newbie to Go, and I am not sure but wouldn't this say it is null if say SomeField2 was "hello"? Apr 13, 2016 at 14:45
  • 1
    @AlexanderTrakhimenok thanks for the elaboration edited as commented. Apr 13, 2016 at 15:07
  • @CetinBasoz yes I've added the conditions to the code to cover that. Apr 13, 2016 at 15:09
15

If you're on Go 1.18+ you can use a simple generic struct to know when a JSON value is undefined or null:

type Optional[T any] struct {
    Defined bool
    Value   *T
}

// UnmarshalJSON is implemented by deferring to the wrapped type (T).
// It will be called only if the value is defined in the JSON payload.
func (o *Optional[T]) UnmarshalJSON(data []byte) error {
    o.Defined = true
    return json.Unmarshal(data, &o.Value)
}

That's all, you can then just use this type in your structs:

type Payload struct {
    Field1 Optional[string] `json:"field1"`
    Field2 Optional[bool]   `json:"field2"`
    Field3 Optional[int32]  `json:"field3"`
}

And you'll be able to use the Defined field to know if the field was null or undefined.

Check this playground link for a full example: https://go.dev/play/p/JZfZyVVUABz

Original answer (pre-generics)

Another way to do this, with a custom type:

// OptionalString is a struct that represents a JSON string that can be
// undefined (Defined == false), null (Value == nil && Defined == true) or 
// defined with a string value
type OptionalString struct {
    Defined bool
    Value   *string
}

// UnmarshalJSON implements the json.Unmarshaler interface.
// When called, it means that the value is defined in the JSON payload.
func (os *OptionalString) UnmarshalJSON(data []byte) error {
    // UnmarshalJSON is called only if the key is present
    os.Defined = true
    return json.Unmarshal(data, &os.Value)
}


// Payload represents the JSON payload that you want to represent.
type Payload struct {
    SomeField1 string         `json:"somefield1"`
    SomeField2 OptionalString `json:"somefield2"`
}

You can then just use the regular json.Unmarshal function to get your values, for example:

    var p Payload
    _ = json.Unmarshal([]byte(`{
            "somefield1":"somevalue1",
            "somefield2":null
        }`), &p)
    fmt.Printf("Should be defined == true and value == nil: \n%+v\n\n", p)


    p = Payload{}
    _ = json.Unmarshal([]byte(`{"somefield1":"somevalue1"}`), &p)
    fmt.Printf("Should be defined == false \n%+v\n\n", p)
    
    p = Payload{}
    _ = json.Unmarshal([]byte(`{
            "somefield1":"somevalue1",
            "somefield2":"somevalue2"
        }`), &p)
    fmt.Printf("Parsed should be defined == true and value != nil \n%+v\n", p)
    if p.SomeField2.Value != nil {
      fmt.Printf("SomeField2's value is %s", *p.SomeField2.Value)
    }

Should give you this output:

Should be defined == true and value == nil: 
{SomeField1:somevalue1 SomeField2:{Defined:true Value:<nil>}}

Should be defined == false 
{SomeField1:somevalue1 SomeField2:{Defined:false Value:<nil>}}

Parsed should be defined == true and value != nil 
{SomeField1:somevalue1 SomeField2:{Defined:true Value:0xc000010370}}
SomeField2's value is somevalue2

Link to the playground with the full example: https://play.golang.org/p/AUDwPKHBs62

Do note that you will need one struct for each type you want to wrap, so, if you need an optional number you'll need to create an OptionalFloat64 (all JSON numbers can be 64 bit floats) struct with a similar implementation. If/when generics land in Go, this could be simplified to a single generic struct.

7

Good question.

I believe you can use https://golang.org/pkg/encoding/json/#RawMessage as:

type MyMessage struct {
  somefield1 string
  somefield2 json.RawMessage
}

So after unmarshalling you should have []byte("null") in case of null and nil if missing.

Here is a playground code: https://play.golang.org/p/UW8L68K068

5

If you unmarshall the object into a map[string]interface{} then you can just check if a field is there

type unMarshalledObject map[string]interface{}
json.Unmarshal(input, unMarshalledObject)
_, ok := unMarshalledObject["somefield2"]

Go Playground

2
  • 3
    The use of map[string]interface{}, though interesting, in this case is an overkill and overly complicate. Apr 13, 2016 at 15:11
  • Agree with @pie-o-pah, this would hide type of somefield1 but it's very interesting to know it's working at all. Apr 13, 2016 at 15:15
0

If struct field is a pointer, JSON decoder will allocate new variable if the field is present or leave it nil if not. So I suggest to use pointers.

type Data struct {
    StrField *string
    IntField *int
}
...
if data.StrField != nil {
    handle(*data.StrField)
}
1
  • 1
    The value of the variable will be also nil if the value was "null" in the JSON.
    – Fenistil
    Jul 30, 2021 at 20:34
0

By using github.com/golang/protobuf/ptypes/struct and jsonpb github.com/golang/protobuf/jsonpb, you can do like this:

func TestFunTest(t *testing.T) {
    p := &pb.KnownTypes{}
    e := UnmarshalString(`{"val":null}`, p)
    fmt.Println(e, p)
    p = &pb.KnownTypes{}
    e = UnmarshalString(`{"val":1}`, p)
    fmt.Println(e, p)
    p = &pb.KnownTypes{}
    e = UnmarshalString(`{"val":"string"}`, p)
    fmt.Println(e, p)
    p = &pb.KnownTypes{}
    e = UnmarshalString(`{}`, p)
    fmt.Println(e, p)
}

Output:

[ `go test -test.run="^TestFunTest$"` | done: 1.275431416s ]
    <nil> val:<null_value:NULL_VALUE > 
    <nil> val:<number_value:1 > 
    <nil> val:<string_value:"string" > 
    <nil> 
    PASS

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.