31

I have a go grpc service. I'm developing on a mac, sierra. When running a grpc client against the service locally, all is well, but when running same client against same service in the docker container I get this error:

transport: http2Client.notifyError got notified that the client transport was broken EOF.
FATA[0000] rpc error: code = Internal desc = transport is closing

this is my docker file:

FROM golang:1.7.5

RUN mkdir -p /go/src/github.com/foo/bar
WORKDIR /go/src/github.com/foo/bar

COPY . /go/src/github.com/foo/bar
# ONBUILD RUN go-wrapper download
RUN go install

ENTRYPOINT /go/bin/bar

EXPOSE 51672

my command to build the image:

docker build -t bar .

my command to launch the docker container:

docker run -p 51672:51672 --name bar-container bar

Other info:

  • client program runs fine from within the docker container
  • connecting to a regular rest endpoint works fine (http2, grpc related?)
  • running the lsof command in OS X yields these results

    $lsof -i | grep 51672
    com.docke   984 oldDave   21u  IPv4 0x72779547e3a32c89      0t0  TCP *:51672 (LISTEN)
    com.docke   984 oldDave   22u  IPv6 0x72779547cc0fd161      0t0  TCP localhost:51672 (LISTEN)
    
  • here's a snippet of my server code:

    server := &Server{}
    endpoint := "localhost:51672"
    lis, err := net.Listen("tcp", endpoint)
    if err != nil {
        log.Fatalf("failed to listen: %v", err)
    }
    
    s := grpc.NewServer(grpc.Creds(creds))
    
    pb.RegisterExpServiceServer(s, server)
    
    // Register reflection service on gRPC server.
    reflection.Register(s)
    
    log.Info("Starting Exp server: ", endpoint)
    
    if err := s.Serve(lis); err != nil {
        log.Fatalf("failed to serve: %v", err)
    }
    

3 Answers 3

57

When you specify a hostname or IP address​ to listen on (in this case localhost which resolves to 127.0.0.1), then your server will only listen on that IP address.

Listening on localhost isn't a problem when you are outside of a Docker container. If your server only listens on 127.0.0.1:51672, then your client can easily connect to it since the connection is also made from 127.0.0.1.

When you run your server inside a Docker container, it'll only listen on 127.0.0.1:51672 as before. The 127.0.0.1 is a local loopback address and it not accessible outside the container.

When you fire up the docker container with "-p 51672:51672", it'll forward traffic heading to 127.0.0.1:51672 to the container's IP address, which in my case is 172.17.0.2.

The container gets an IP addresses within the docker0 network interface (which you can see with the "ip addr ls" command)

So, when your traffic gets forwarded to the container on 172.17.0.2:51672, there's nothing listening there and the connection attempt fails.

The fix:

The problem is with the listen endpoint:

endpoint := "localhost:51672"

To fix your problem, change it to

endpoint := ":51672"

That'll make your server listen on all it container's IP addresses.

Additional info:

When you expose ports in a Docker container, Docker will create iptables rules to do the actual forwarding. See this. You can view these rules with:

iptables -n -L 
iptables -t nat -n -L
5
  • @olafure setting the endpoint to just a ":[port]" doesn't work on the nodejs implementation of grpc. I just get this error {"created":"@1548557943.020372671","description":"getaddrinfo failed","file":"../deps/grpc/src/core/lib/iomgr/tcp_uv.cc","file_line":73,"grpc_status":14,"os_error":"unknown node or service"} Jan 27, 2019 at 2:59
  • 2
    Thanks for the answer. Additionally, if you're running containers via Docker Compose, you can use that container name specified in the yaml file as the host name and it'll automatically map to the correct IP address.
    – zMan
    Mar 4, 2019 at 23:42
  • 2
    Thank you so much for this answer. I had been scratching my head on this same issue for hours.
    – liam923
    Jul 19, 2020 at 3:55
  • 2
    Changing localhost to 0.0.0.0 worked for me, which I believe is analogous to omitting the host name. Jun 4, 2021 at 0:45
  • Why is it that you can't hit the IP address directly after publishing the port? I understand publishing means exposing it to the outer world aka beyond the container itself but unsure why you can't hit 172.17.0.2:<port#> after publishing but can access it through localhost
    – Paul
    Sep 8 at 1:37
0

If you use docker container to run grpc service,you should make sure grpc service and grpc client on ths same bridge

0

if you are using docker container, u can define network by IP and give this IP to your IP address(like: 178.20.0.5).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.