I have an instance of a struct that I defined and I would like to convert it to an array of bytes. I tried []byte(my_struct), but that did not work. Also, I was pointed to the binary package, but I am not sure which function I should use and how I should use it. An example would be greatly appreciated.
10 Answers
One possible solution is the "encoding/gob"
standard package. The gob package creates an encoder/decoder that can encode any struct into an array of bytes and then decode that array back into a struct. There's a great post, here.
As others have pointed out, it's necessary to use a package like this because structs, by their nature, have unknown sizes and cannot be converted into arrays of bytes.
I've included some code and a play.
package main
import (
"bytes"
"encoding/gob"
"fmt"
"log"
)
type P struct {
X, Y, Z int
Name string
}
type Q struct {
X, Y *int32
Name string
}
func main() {
// Initialize the encoder and decoder. Normally enc and dec would be
// bound to network connections and the encoder and decoder would
// run in different processes.
var network bytes.Buffer // Stand-in for a network connection
enc := gob.NewEncoder(&network) // Will write to network.
dec := gob.NewDecoder(&network) // Will read from network.
// Encode (send) the value.
err := enc.Encode(P{3, 4, 5, "Pythagoras"})
if err != nil {
log.Fatal("encode error:", err)
}
// HERE ARE YOUR BYTES!!!!
fmt.Println(network.Bytes())
// Decode (receive) the value.
var q Q
err = dec.Decode(&q)
if err != nil {
log.Fatal("decode error:", err)
}
fmt.Printf("%q: {%d,%d}\n", q.Name, *q.X, *q.Y)
}
-
1perfect - great package - ty didn't know about gob– user1971598Sep 20, 2020 at 1:19
-
1This might be a dumb question but does gob also serialize unexported fields?– slimDec 17, 2020 at 15:09
-
3@slim not at all! I'm actually not 100% sure but I would expect that it wouldn't. Same as encoding/json not serializing unexported fields.– AdamDec 18, 2020 at 22:59
I assume you want something like the way C handles this. There is no built in way to do that. You will have to define your own serialization and deserialization to and from bytes for your struct. The binary package will help you encode the fields in your struct to bytes that you can add to the byte array but you will be responsible for specifying the lengths and offsets in the byte array that will hold the fields from your struct.
Your other options are to use one of the encoding packages: http://golang.org/pkg/encoding/ such as gob or json.
EDIT:
Since you want this for making a hash as you say in your comment the easisest thing to do is use []byte(fmt.Sprintf("%v", struct))
like so: http://play.golang.org/p/yY8mSdZ_kf
-
Thank you for your quick answer. The reason I'm trying to do this is to be able to get a hash (I'm trying to use SHA-256, but it could be another) of my struct. Do you know of an easier way of doing this?– abw333May 2, 2013 at 4:56
-
1
-
Thanks again. I tried doing that, but I got the following error message: "multiple-value fmt.Printf() in single-value context". Do you know why that is happening?– abw333May 2, 2013 at 5:18
-
@abw333, it is because usage of
Printf
seems to be a typo. You should useSprintf
, as is demonstrated by the code behind the link in Jeremy's answer. May 2, 2013 at 7:19 -
I know this thread is old, but none of the answers were accepted, and there's a pretty simple way to do this.
https://play.golang.org/p/TedsY455EBD
important code from playground
import (
"bytes"
"fmt"
"encoding/json"
)
type MyStruct struct {
Name string `json:"name"`
}
testStruct := MyStruct{"hello world"}
reqBodyBytes := new(bytes.Buffer)
json.NewEncoder(reqBodyBytes).Encode(testStruct)
reqBodyBytes.Bytes() // this is the []byte
-
9This will convert the struct to a bytes array in "JSON" format. I'm assuming the question is about converting the struct to a bytes array in "Binary" format (For lower memory/disk consumption).– TomerApr 21, 2018 at 17:39
-
Indeed it will return an array of [123 34 110 97 109 101 34 58 34 104 101 108 108 111 32 119 111 114 108 100 34 125 10] or {"name":"hello world"}LF Jan 30, 2019 at 22:00
Just use json marshal, this is a very simple way.
newFsConfig := dao.ConfigEntity{EnterpriseId:"testing"}
newFsConfigBytes, _ := json.Marshal(newFsConfig)
-
4Exactly, I only scrolled down looking if someone had given this answer. The rest of the answers are way too over convoluted for such a simple problem. Good job! Apr 12, 2020 at 21:27
-
1
Serialization is likely proper answer.
But if you consent to unsafety and actually need to read struct as bytes, then relying on byte array memory representation might be a bit better than relying on byte slice internal structure.
type Struct struct {
Src int32
Dst int32
SrcPort uint16
DstPort uint16
}
const sz = int(unsafe.SizeOf(Struct{}))
var asByteSlice []byte = (*(*[sz]byte)(unsafe.Pointer(&struct_value)))[:]
Works and provides read-write view into struct, zero-copy. Two "unsafe" should hint enough that it may break badly.
You should use a bytes buffer instead of a string, the other suggested methods create a SHA1 of variable length, the SHA1 standard length must be 20 bytes (160 bits)
package main
import (
"crypto/sha1"
"fmt"
"encoding/binary"
"bytes"
)
type myStruct struct {
ID string
Data string
}
func main() {
var bin_buf bytes.Buffer
x := myStruct{"1", "Hello"}
binary.Write(&bin_buf, binary.BigEndian, x)
fmt.Printf("% x", sha1.Sum(bin_buf.Bytes()))
}
Try it yourself: http://play.golang.org/p/8YuM6VIlLV
It's a really easy method and it works great.
-
2
-
Thank you Der, I'll edit this. Was my first comment. Jan 22, 2015 at 18:13
-
8Doesn't look like this works because
myStruct
isn't fixed size. play.golang.org/p/IGA_lgRVNX Sep 21, 2016 at 21:00 -
4Did it work? Check the err returned from binary.Write reveals "binary.Write: invalid type main.myStruct"....– nothizeApr 18, 2017 at 5:55
-
2Proof that this does not work. Would've been a great thing though.. play.golang.org/p/gpBUXMnaOUJ Feb 2, 2018 at 12:14
package main
import (
"crypto/sha1"
"fmt"
"encoding/binary"
"bytes"
)
type myStruct struct {
ID [10]byte
Data [10]byte
}
func main() {
var bin_buf bytes.Buffer
x := myStruct{"1", "Hello"}
binary.Write(&bin_buf, binary.BigEndian, x)
fmt.Printf("% x", sha1.Sum(bin_buf.Bytes()))
}
binary.Write takes a struct which has fixed length memory allocated datatype.
-
1This is probably the best answer to the question in the title. Nice to learn that, thanks! Sep 29, 2020 at 15:44
json.Marshal is the best option to convert a struct to []byte, see example below:
package main
import (
"encoding/json"
"fmt"
)
type ExampleConvertToByteArray struct {
Name string
SurName string
}
func main() {
example := ExampleConvertToByteArray{
Name: "James",
SurName: "Camara",
}
var exampleBytes []byte
var err error
exampleBytes, err := json.Marshal(example)
if err != nil {
print(err)
return
}
fmt.Println(string(exampleBytes))
}
Go playground -> https://play.golang.org/p/mnB9Cxy-2H3
Take a look at https://blog.golang.org/go-slices-usage-and-internals Specifically slice internals. The idea is to mimic slice's internal structure and point to our struct instead of a byte sequence:
package main
import (
"fmt"
"unsafe"
)
// our structure
type A struct {
Src int32
Dst int32
SrcPort uint16
DstPort uint16
}
// that is how we mimic a slice
type ByteSliceA struct {
Addr *A
Len int
Cap int
}
func main() {
// structure with some data
a := A{0x04030201,0x08070605,0x0A09, 0x0C0B}
// create a slice structure
sb := &ByteSliceA{&a, 12, 12} // struct is 12 bytes long, e.g. unsafe.Sizeof(a) is 12
// take a pointer of our slice mimicking struct and cast *[]byte on it:
var byteSlice []byte = *(*[]byte)(unsafe.Pointer(sb))
fmt.Printf("%v\n", byteSlice)
}
Output:
[1 2 3 4 5 6 7 8 9 10 11 12]
-
@chewbapoclypse this only works with structs with no internal pointers. Tt won't work with strings, since
string
has an internal pointer to an underlying sequence of elements. You have to keep your string data in arrays for this to work.– KaxJun 22, 2019 at 4:01