15

I'm at an initial stage of creating a microservice application in Go, but due to the way that the import paths and directories are handled I'm not quite sure what's best way to structure the project files.

Normally, the project would look something like this in Java:

|-- gateway_microservice
   |-- src
   |-- docker
|-- config_microservice
   |-- src
   |-- docker
|-- recommendation_microservice
   |-- src
   |-- docker
|-- users_microservice
   |-- src
   |-- docker

Now if I do it the same way in Go, the import paths become somewhat cumbersome:

import (
       "fmt" 
       "github.com/user/myproject/gateway_microservice/src/package1"
       "github.com/user/myproject/gateway_microservice/src/package2"
)

Additionally, I hear that the idiomatic way is to put all main.go files in a separate cmd directory, which adds to the confusion. Would it look something like this:

|-- cmd
   |-- gateway_microservice
      |-- main.go
   |-- config_microservice
      |-- main.go
   |-- recommendation_microservice
      |-- main.go
   |-- users_microservice
      |-- main.go
|-- gateway_microservice
   |-- src
   |-- docker
|-- config_microservice
   |-- src
   |-- docker
|-- recommendation_microservice
   |-- src
   |-- docker
|-- users_microservice
   |-- src
   |-- docker

What is the 'correct' or idiomatic way of structuring a project like this in Go?

5 Answers 5

18

I'm structuring it like this; mono-repo per. project approach. Taking into account that these services are closely related:

github.com/user/some_project/
├── pkg/ (common own-created packages for all services)
|   ├── errors/
|   ├── log/
|   ├── metrics/
|   ├── sd/
|   |   ├── consul/
|   |   └── kubernetes/
|   └── tracing/
├── services/
|   ├── account/
|   |   ├── pb/
|   |   |   ├── account.proto
|   |   |   └── account.pb.go
|   |   ├── handler.go
|   |   ├── main.go
|   |   ├── main_test.go
|   |   ├── Dockerfile
|   |   └── README.md
|   ├── auth/
|   ├── frontend/
|   └── user/
├── vendor/ (common vendor-packages for all services)
├── docker-compose.yml
├── go.mod
├── go.sum
├── Makefile
└── README.md

Alternative 2:

github.com/user/some_project/
├── pkg/
├── service.account/
|   ├─ cmd/
|   |  └─ main.go
|   ├─ pb/
|   ├─ Dockerfile
|   ├─ go.mod
|   └─ go.sum
├── service.auth/
├── service.frontend/
├── service.user/
├── docker-compose.yml
├── go.mod (used primarly for packages in the /pkg dir.)
├── go.sum
├── Makefile
└── README.md

With the introduction of go-modules, I'm leaning more to the second alternative.

At some later time, when you start on your second macro/micro/nano-services project, many of the these packages in the /pkg folder would be required there too. What to do? Copy/paste? No! Instead, extract these packages from the project, i.e. log, metric and make your own kit.

Remember that if you use some kind of CI/CD (you really should), you have the option to write a script placed in the project root that will only detect the changes you make in the repository, thus only the affected services will be built and delivered. There are several examples out there how to do this.

1
  • this doesnt explain where the configs go for each service? Commented Oct 20, 2022 at 23:28
13

The other answer here advocates putting each microservice into its own repository. There may be valid reasons for splitting things up that way, but there may be equally valid reasons from wanting to keep everything in one repository as well (it really depends on your project / circumstances)

If you want all the code in one repository, you can- you just need to follow Go's package rules. (this is a good read: https://golang.org/doc/code.html#Workspaces)

If you have a mix of commands and libraries, the directory structure you proposed in your question comes close, but you probably don't need the src directories in there. Here's an example of how a directory structure within a repo with libraries and commands might look:

lib1/
-- some.go
-- source.go
lib2/
-- more.go
-- source.go
cmd/
-- microservice1/
   -- main.go
-- microservice2/
   -- anothermain.go

To use this repository, you would clone it inside a Go workspace on your system (see the link I shared above). Assuming your repository lives in github.com/mybiz/project, and your GOPATH was ~/go, the workspace would look as follows:

~/go/src/github.com/mybiz/
  -- project/
     <clone repo in here>

The file cmd/microservice1/main.go would include the library lib1 via a path it expects it in relative to $GOPATH/src as follows:

import "github.com/mybiz/project/lib1"

Now, your code has access to the exported symbols in that package using the package name declared in the files under lib1... usually just:

package lib1

In cmd/microservice1/main.go, with the import above, you could use lib1 symbols as follows:

lib1.CallMe()

I hope that helps clear up how Go's directory structure works.

1
  • Thanks, I'm not able to separate the repository into multiple separate repositories so I was thinking along the same lines. I came across a nice article which clarified some things: peter.bourgon.org/go-in-production but your answer also made things clearer.
    – Rtsne42
    Commented Aug 20, 2017 at 1:17
10

Thanks to @karl-andresen. I was doing research on the same topic and came up with the below structure hope this helps someone

github.com/username/container/
├── pkg/ ('username' created packages - common for all services & reusable in other projects)
|   ├── errors/
|   ├── log/
|   ├── metrics/
|   ├── infra/     (sub category in packages)
|   |   ├── consul/
|   |   └── kubernetes/
|   └── tracing/
├── services/ (where all microservices will be imported as submodules - may or may not be reused)
|   ├── account/
|   |   ├── handler.go
|   |   ├── handler_test.go (unit testing, note filename with '_test')
|   |   ├── main.go
|   |   ├── main_test.go    (another unit testing)
|   |   ├── account.cfg     (configuration file for account microservice)
|   |   ├── submodule/      (sub directory)
|   |   |   ├── submodule.go
|   |   |   └── submodule_test.go   (submodule unit test)
|   |   ├── Dockerfile
|   |   └── README.md
|   ├── auth/
|   ├── booking/
|   └── user/
├── api/ (OpenAPI/Swagger specs, JSON schema files, protocol definition files.)
|   ├── proto/  (protocol buffer files)
|   |   ├── v1/
|   |   |   ├── account.proto
|   |   |   ├── account.pb.go
|   |   |   ├── booking.proto
|   |   |   └── booking.pb.go
|   |   └── v2/
|   └── rest/   (json files)
|       ├── v1/
|       |   ├── booking.json
|       |   └── account.json
|       └── v2/
├── configs/ (project config settings, default configs, file templates)
├── scripts/ (Scripts to perform various build, install, analysis, etc operations.)
├── build/ (Packaging and Continuous Integration.)
├── test / (system and module level tests)
├── docs/ (project documents folder)
├── examples/ (project examples for service interactions)
├── third_party/ (all open source, third party codes, where applicable fork and add as submodule)
├── githooks/ (project git hooks)
├── assets/ (common assests for all services)
├── Makefile
├── README.md
└── docker-compose.yml
1
  • Does this mean that account.pb.go and booking.pb.go belongs to package v1? What if you have like 50+ services with own messages and rpc methods? Isn't that hard to work with in IDE? Just trying to research/learn this thing. I kinda like this Go Standard Project Layout. As well, i probably would move *.pb.go into /pkg/api/v1 or mby even /internal/api/v1. Not sure.
    – Dzintars
    Commented Jun 13, 2019 at 14:50
4

How do I organize my projects?

|-- github.com/avelino/service1
   |-- Dockerfile
   |-- main.go
|-- github.com/avelino/service2
   |-- Dockerfile
   |-- main.go
|-- github.com/avelino/service3
   |-- Dockerfile
   |-- main.go

Packages

All Go code is organized into packages. A package in Go is simply a directory/folder with one or more .go files inside of it. Go packages provide isolation and organization of code similar to how directories/folders organize files on a computer.

All Go code lives in a package and a package is the entry point to access Go code. Understanding and establishing good practices around packages is important to write effective Go code.

3
  • 2
    Thanks for the response, so do you keep each service in a separate repository?
    – Rtsne42
    Commented Aug 19, 2017 at 21:50
  • 1
    Yep, isolating each service in your repository
    – avelino
    Commented Aug 19, 2017 at 23:04
  • 2
    yup, this is how microservice should be structured. Because each service should be independent to eachother which mean they should be separated their repo. Commented Mar 7, 2018 at 4:14
-2

Each microservice should be an independent service and communicate with each other through network either using RESTful, RPC or messaging. Follow 12 Factor of App is a virtue when designing a microservice to make sure it easy to ship. Check typical-go-server for an example of how to make microservice friendly project.

Not the answer you're looking for? Browse other questions tagged or ask your own question.