I have a Go project where I'm using Zap structured logging to log the contents of structs. That's how I initialise the logger:
zapLog, err := zap.NewProductionConfig().Build()
if err != nil {
panic(err)
}
Initially I started with my own structs with json tags and it all worked perfectly:
zapLog.Info("Event persisted", zap.Any("event", &event))
Result:
{"level":"info","ts":1626448680.69099,"caller":"persisters/log.go:56",
"msg":"Event persisted","event":{"sourceType":4, "sourceId":"some-source-id",
"type":"updated", "value":"{...}", "context":{"foo":"bar"}}}
I now switched to protobuf and I'm struggling to achieve the same result. Initially I just got the "reflected map" version, when using zap.Any():
zapLog.Info("Event persisted", zap.Any("event", &event))
{"level":"info","ts":1626448680.69099,"caller":"persisters/log.go:56",
"msg":"Event persisted","event":"sourceType:TYPE_X sourceId:\"some-source-id\",
type:\"updated\" value:{...}, context:<key: foo, value:bar>}
I tried marshalling the object with the jsonpb marshaller, which generated the correct output on itself, however, when I use it in zap.String()
, the string is escaped, so I get an extra set of '\' in front of each quotation mark. Since there's processing of the logs at a later point, this causes problems there and hence I want to avoid it:
m := jsonpb.Marshaler{}
var buf bytes.Buffer
if err := m.Marshal(&buf, msg); err != nil {
// handle error
}
zapLog.Info("Event persisted", zap.ByteString("event", buf.Bytes()))
Result:
{"level":"info","ts":1626448680.69099,"caller":"persisters/log.go:56",
"msg":"Event persisted","event":"{\"sourceType\":\"TYPE_X\", \"sourceId\":\"some-source-id\",
\"type\":\"updated\", \"value\":\"{...}\", \"context\":{\"foo\":"bar\"}}"}
I then tried using zap.Reflect()
instead of zap.Any()
which was the closest thing I could get to what I need, except that enums are rendered as their numerical values (the initial solution did not have enums, so that didn't work in the pre-protobuf solution either):
zapLog.Info("Event persisted", zap.Reflect("event", &event))
Result:
{"level":"info","ts":1626448680.69099,"caller":"persisters/log.go:56",
"msg":"Event persisted","event":{"sourceType":4, "sourceId":"some-source-id",
"type":"updated", "value":"{...}", "context":{"foo":"bar"}}}
The only option I see so far is to write my own MarshalLogObject()
function:
type ZapEvent struct {
event *Event
}
func (z *ZapEvent) MarshalLogObject(encoder zapcore.ObjectEncoder) error {
encoder.AddString("sourceType", z.event.SourceType.String()
// implement encoder for each attribute
}
func processEvent(e Event) {
...
zapLog.Info("Event persisted", zap.Object("event", &ZapEvent{event: &e}))
}
But since it's a complex struct, I would rather use a less error prone and maintenance heavy solution. Ideally, I would tell zap to use the jsonpb marshaller somehow, but I don't know if that's possible.