23

I'm trying to map an array to a file via Mmap, the array could be any type, like float64. In C, I find this one. After reading some texts, I wrote this sample. I don't know if it is correct, and it is not writing the values to the file. If I increase the size of array a lot, e.g from 1000 to 10000, it crashes. If someone know how to do that in the correctly way, please, tell me.

Thanks!

1 Answer 1

31

For example, revising your sample program,

package main

import (
    "fmt"
    "os"
    "syscall"
    "unsafe"
)

func main() {
    const n = 1e3
    t := int(unsafe.Sizeof(0)) * n

    map_file, err := os.Create("/tmp/test.dat")
    if err != nil {
        fmt.Println(err)
        os.Exit(1)
    }
    _, err = map_file.Seek(int64(t-1), 0)
    if err != nil {
        fmt.Println(err)
        os.Exit(1)
    }
    _, err = map_file.Write([]byte(" "))
    if err != nil {
        fmt.Println(err)
        os.Exit(1)
    }

    mmap, err := syscall.Mmap(int(map_file.Fd()), 0, int(t), syscall.PROT_READ|syscall.PROT_WRITE, syscall.MAP_SHARED)
    if err != nil {
        fmt.Println(err)
        os.Exit(1)
    }
    map_array := (*[n]int)(unsafe.Pointer(&mmap[0]))

    for i := 0; i < n; i++ {
        map_array[i] = i * i
    }

    fmt.Println(*map_array)

    err = syscall.Munmap(mmap)
    if err != nil {
        fmt.Println(err)
        os.Exit(1)
    }
    err = map_file.Close()
    if err != nil {
        fmt.Println(err)
        os.Exit(1)
    }
}
5
  • 1
    Thanks, PeterSO, it's working. Do you know if it's safe to use mmap in Go?
    – tfmoraes
    Commented Feb 9, 2012 at 11:05
  • 5
    What do you mean by "safe"? It recasts pointers and requires therefore the "unsafe" package. If you don't handle the pointers correctly, it might crash (and GAE doesn't allow any packages which uses the unsafe package in the first place). But if you handle them correctly, you can write fast and efficient programs. The codesearch app by Russ Cox (one of the Go authors) also uses mmap a lot, so you might want to take a look at that for inspiration.
    – tux21b
    Commented Feb 9, 2012 at 11:19
  • 1
    @tux21b: I wonder if one can use e.g. the math package in Go GAE? See link
    – zzzz
    Commented Feb 9, 2012 at 16:58
  • I think so, but they might be either shipping a custom math package or they allow the usage of unsafe in the approved standard library. That's definitely a good point and something which should be tried someday :)
    – tux21b
    Commented Feb 10, 2012 at 0:08
  • 3
    Note that this is unix only. Go in Windows has a different api for memory mapped files.
    – kristianp
    Commented Jan 12, 2013 at 5:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.