153

I am trying to create a generic method in Go that will fill a struct using data from a map[string]interface{}. For example, the method signature and usage might look like:

func FillStruct(data map[string]interface{}, result interface{}) {
    ...
}

type MyStruct struct {
    Name string
    Age  int64
}

myData := make(map[string]interface{})
myData["Name"] = "Tony"
myData["Age"]  = 23

result := &MyStruct{}
FillStruct(myData, result)

// result now has Name set to "Tony" and Age set to 23

I know this can be done using JSON as an intermediary; is there another more efficient way of doing this?

5
  • 1
    Using JSON as an intermediary will use reflection anyway.. assuming you're going to be using the stdlib encoding/json package to do that intermediate step.. Can you give an example map and example struct that this method could be used on? Commented Nov 4, 2014 at 21:09
  • Yea, that is the reason I am trying to avoid JSON. Seems like there hopefully is a more efficient method that I don't know about.
    – tgrosinger
    Commented Nov 4, 2014 at 22:01
  • Can you give an example use case? As in - show some pseudocode that demonstrates what this method will do? Commented Nov 4, 2014 at 22:02
  • Mmm... there might be a way with the unsafe package .. but I dare not try it. Other than that .. Reflection is required, as you need to be able to query the metadata associated with a type in order to place data into its properties. It would be fairly straight forward to wrap this in json.Marshal + json.Decode calls.. but that's double the reflection. Commented Nov 4, 2014 at 22:19
  • I have removed my comment about reflection. I am more interested in just doing this as efficiently as possible. If that means using reflection that is okay.
    – tgrosinger
    Commented Nov 4, 2014 at 22:21

9 Answers 9

182

The simplest way would be to use https://github.com/mitchellh/mapstructure

import "github.com/mitchellh/mapstructure"

mapstructure.Decode(myData, &result)

If you want to do it yourself, you could do something like this:

http://play.golang.org/p/tN8mxT_V9h

func SetField(obj interface{}, name string, value interface{}) error {
    structValue := reflect.ValueOf(obj).Elem()
    structFieldValue := structValue.FieldByName(name)

    if !structFieldValue.IsValid() {
        return fmt.Errorf("No such field: %s in obj", name)
    }

    if !structFieldValue.CanSet() {
        return fmt.Errorf("Cannot set %s field value", name)
    }

    structFieldType := structFieldValue.Type()
    val := reflect.ValueOf(value)
    if structFieldType != val.Type() {
        return errors.New("Provided value type didn't match obj field type")
    }

    structFieldValue.Set(val)
    return nil
}

type MyStruct struct {
    Name string
    Age  int64
}

func (s *MyStruct) FillStruct(m map[string]interface{}) error {
    for k, v := range m {
        err := SetField(s, k, v)
        if err != nil {
            return err
        }
    }
    return nil
}

func main() {
    myData := make(map[string]interface{})
    myData["Name"] = "Tony"
    myData["Age"] = int64(23)

    result := &MyStruct{}
    err := result.FillStruct(myData)
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println(result)
}
7
  • 4
    Thank you. I am using a slightly modified version. play.golang.org/p/_JuMm6HMnU
    – tgrosinger
    Commented Nov 4, 2014 at 22:58
  • I want the FillStruct behavior on all my various structs and not have to define func (s MyStr...) FillStruct ... for every one. Is it possible to define FillStruct for a base struct then have all my other structs 'inherit' that behavior? In the paradigm above it's not possible since only the base struct ... in this case "MyStruct" will actually have it's fields iterated
    – Michael M
    Commented Aug 12, 2015 at 23:31
  • I mean you could have it work for any struct with something like this: play.golang.org/p/0weG38IUA9
    – dave
    Commented Aug 12, 2015 at 23:39
  • Is it possible to implement tags in Mystruct?
    – vicTROLLA
    Commented Mar 18, 2016 at 19:48
  • 1
    @abhishek there is certainly a performance penalty you'll pay for first marshaling to text and then unmarshaling. That approach is also certainly simpler. It's a tradeoff, and generally I'd go with the simpler solution. I answered with this solution because the question stated "I know this can be done using JSON as an intermediary; is there another more efficient way of doing this?". This solution will be more efficient, the JSON solution will generally be easier to implement and reason about.
    – dave
    Commented Jan 19, 2020 at 5:28
93

Hashicorp's https://github.com/mitchellh/mapstructure library does this out of the box:

import "github.com/mitchellh/mapstructure"

mapstructure.Decode(myData, &result)

The second result parameter has to be an address of the struct.

5
84
  • the simplest way to do that is using encoding/json package

just for example:

package main
import (
    "fmt"
    "encoding/json"
)

type MyAddress struct {
    House string
    School string
}
type Student struct {
    Id int64
    Name string
    Scores float32
    Address MyAddress
    Labels []string
}

func Test() {

    dict := make(map[string]interface{})
    dict["id"] = 201902181425       // int
    dict["name"] = "jackytse"       // string
    dict["scores"] = 123.456        // float
    dict["address"] = map[string]string{"house":"my house", "school":"my school"}   // map
    dict["labels"] = []string{"aries", "warmhearted", "frank"}      // slice

    jsonbody, err := json.Marshal(dict)
    if err != nil {
        // do error check
        fmt.Println(err)
        return
    }

    student := Student{}
    if err := json.Unmarshal(jsonbody, &student); err != nil {
        // do error check
        fmt.Println(err)
        return
    }

    fmt.Printf("%#v\n", student)
}

func main() {
    Test()
}
6
  • 1
    Thanks @jackytse. This is actually the best way to do that !! map structure doesn’t often work with nested map within an interface. So it’s better consider a map string interface and handle it as a json. Commented Aug 13, 2019 at 8:35
  • 1
    Go playground link for the above snippet: play.golang.org/p/JaKxETAbsnT
    – Junaid
    Commented Jun 6, 2020 at 12:49
  • This is the best way, why is this not the accepted answer
    – karthikcru
    Commented Sep 3, 2021 at 19:45
  • 2
    @karthikcru because this is exactly what OP said they want to avoid doing, see "I know this can be done using JSON as an intermediary; is there another more efficient way of doing this?"
    – Petr
    Commented Dec 23, 2021 at 9:10
  • This should be marked as the answer, best solution!!! Thanks! Commented Apr 14, 2022 at 17:24
17

There are two steps:

  1. Convert interface to JSON Byte
  2. Convert JSON Byte to struct

Below is an example:

dbByte, _ := json.Marshal(dbContent)
_ = json.Unmarshal(dbByte, &MyStruct)
16

You can do it ... it may get a bit ugly and you'll be faced with some trial and error in terms of mapping types .. but heres the basic gist of it:

func FillStruct(data map[string]interface{}, result interface{}) {
    t := reflect.ValueOf(result).Elem()
    for k, v := range data {
        val := t.FieldByName(k)
        val.Set(reflect.ValueOf(v))
    }
}

Working sample: http://play.golang.org/p/PYHz63sbvL

2
  • 1
    This appears to panic on zero values: reflect: call of reflect.Value.Set on zero Value Commented Apr 23, 2017 at 4:17
  • @JamesTaylor Yes. My answer assumes you know exactly what fields you're mapping. If you're after a similar answer with more error handling (including the error you're experiencing), I would suggest Daves answer instead. Commented Apr 23, 2017 at 5:00
8

You can roundtrip it through JSON:

package main

import (
   "bytes"
   "encoding/json"
)

func transcode(in, out interface{}) {
   buf := new(bytes.Buffer)
   json.NewEncoder(buf).Encode(in)
   json.NewDecoder(buf).Decode(out)
}

Example:

package main
import "fmt"

type myStruct struct {
   Name string
   Age  int64
}

func main() {
   myData := map[string]interface{}{
      "Name": "Tony",
      "Age": 23,
   }
   var result myStruct
   transcode(myData, &result)
   fmt.Printf("%+v\n", result) // {Name:Tony Age:23}
}
1

I adapt dave's answer, and add a recursive feature. I'm still working on a more user friendly version. For example, a number string in the map should be able to be converted to int in the struct.

package main

import (
    "fmt"
    "reflect"
)

func SetField(obj interface{}, name string, value interface{}) error {

    structValue := reflect.ValueOf(obj).Elem()
    fieldVal := structValue.FieldByName(name)

    if !fieldVal.IsValid() {
        return fmt.Errorf("No such field: %s in obj", name)
    }

    if !fieldVal.CanSet() {
        return fmt.Errorf("Cannot set %s field value", name)
    }

    val := reflect.ValueOf(value)

    if fieldVal.Type() != val.Type() {

        if m,ok := value.(map[string]interface{}); ok {

            // if field value is struct
            if fieldVal.Kind() == reflect.Struct {
                return FillStruct(m, fieldVal.Addr().Interface())
            }

            // if field value is a pointer to struct
            if fieldVal.Kind()==reflect.Ptr && fieldVal.Type().Elem().Kind() == reflect.Struct {
                if fieldVal.IsNil() {
                    fieldVal.Set(reflect.New(fieldVal.Type().Elem()))
                }
                // fmt.Printf("recursive: %v %v\n", m,fieldVal.Interface())
                return FillStruct(m, fieldVal.Interface())
            }

        }

        return fmt.Errorf("Provided value type didn't match obj field type")
    }

    fieldVal.Set(val)
    return nil

}

func FillStruct(m map[string]interface{}, s interface{}) error {
    for k, v := range m {
        err := SetField(s, k, v)
        if err != nil {
            return err
        }
    }
    return nil
}

type OtherStruct struct {
    Name string
    Age  int64
}


type MyStruct struct {
    Name string
    Age  int64
    OtherStruct *OtherStruct
}



func main() {
    myData := make(map[string]interface{})
    myData["Name"]        = "Tony"
    myData["Age"]         = int64(23)
    OtherStruct := make(map[string]interface{})
    myData["OtherStruct"] = OtherStruct
    OtherStruct["Name"]   = "roxma"
    OtherStruct["Age"]    = int64(23)

    result := &MyStruct{}
    err := FillStruct(myData,result)
    fmt.Println(err)
    fmt.Printf("%v %v\n",result,result.OtherStruct)
}
1

Here function to convert map to struct by tag. If tag not exist it will find by fieldByName.

Thanks to https://gist.github.com/lelandbatey/a5c957b537bed39d1d6fb202c3b8de06

type MyStruct struct {
    Name string `json:"name"`
    ID   int    `json:"id"`
}

myStruct := &MyStruct{}

for k, v := range mapToConvert {
        err := MapToStruct(myStruct, k, v)
        if err != nil {
            fmt.Println(err)
        }
    }
func MapToStruct(s interface{}, k string, v interface{}) error {
    var jname string
    structValue := reflect.ValueOf(s).Elem()
    fieldByTagName := func(t reflect.StructTag) (string, error) {
        if jt, ok := t.Lookup("keyname"); ok {
            return strings.Split(jt, ",")[0], nil
        }
        return "", fmt.Errorf("tag provided %s does not define a json tag", k)
    }
    fieldNames := map[string]int{}
    for i := 0; i < structValue.NumField(); i++ {
        typeField := structValue.Type().Field(i)
        tag := typeField.Tag
        if string(tag) == "" {
            jname = toMapCase(typeField.Name)
        } else {
            jname, _ = fieldByTagName(tag)
        }
        fieldNames[jname] = i
    }

    fieldNum, ok := fieldNames[k]
    if !ok {
        return fmt.Errorf("field %s does not exist within the provided item", k)
    }
    fieldVal := structValue.Field(fieldNum)
    fieldVal.Set(reflect.ValueOf(v))

    return nil
}

func toMapCase(s string) (str string) {
    runes := []rune(s)
    for j := 0; j < len(runes); j++ {
        if unicode.IsUpper(runes[j]) == true {
            if j == 0 {
                str += strings.ToLower(string(runes[j]))
            } else {
                str += "_" + strings.ToLower(string(runes[j]))
            }
        } else {
            str += strings.ToLower(string(runes[j]))
        }
    }
    return str
}
0

Simple way just marshal it json string and then unmarshat it to struct

here is the link

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.