Using gRPC with Golang, Python, and Ruby

30 minute read     Updated:

Alex Couture-Beil %
Alex Couture-Beil

I was surprised to learn that Google protocol buffers (protobufs), were first introduced nearly two decades ago. They were used internally at google as early as 2001 and were open sourced 2008.

Following this success, in 2016 Google released gRPC. gRPC offered a way to define remote procedure calls using protobufs for serialization. Due to protobuf’s binary serialization format, it offered a significant speed up compared to using JSON over HTTP. The use of proto files for a precise definition of a service’s API. This was a big innovation.

gRPC is a great solution for communicating between internal microservices. There’s plenty of articles and documentation that covers protobufs and gRPC, but when I am considering a new technology, I learn best by seeing a working example. In this blog post I’m going build an example using Go, Python and Ruby.

First Step: Implementing a gRPC Client using Go

Let’s write an in-memory key/value micro-service in Go, and some clients in both Python and Ruby.

Our server will allow users to set and get data from a key/value store.

hand drawn cartoon for a person talking to a computer server

Interacting with the server

First let’s design our API in a proto file:

    syntax = "proto3";
    package simplekeyvalue;
    option go_package = "/kvapi";
    // The key/value API contains two procedures for storing
    // and retrieving data
    service KeyValue {
      rpc Set (SetRequest) returns (SetReply) {}
      rpc Get (GetRequest) returns (GetReply) {}
    // SetRequest contains a key and value to set
    message SetRequest {
      string key = 1;
      string value = 2;
    // SetReply contains nothing
    message SetReply {
    // GetRequest contains a key and value to set
    message GetRequest {
      string key = 1;
    // GetReply contains the value
    message GetReply {
      string value = 1;

Next we need to compile this proto file into Go code. On a Mac one might be tempted to run brew install protobuf, or if you’re on Linux you might want to see if apt-get install protoc will magically work, but rather than do that, we will use earthly to containerize these tools. This will allow you to share this code with other developers, and ensure everyone can compile proto files across multiple platforms using the same version to eliminate compatibility issues.

Here’s what an Earthfile would look like for installing Google protobufs inside an Ubuntu image, and generating the protobuf code using the protoc-gen-go-grpc tool:

    FROM ubuntu:20.10
    WORKDIR /defs
    RUN apt-get update && apt-get install -y wget unzip
    # setup protoc
    RUN wget -O
    RUN unzip -d /usr/local/
      RUN apt-get install -y golang git
      ENV GO111MODULE=on
      ENV PATH=$PATH:/root/go/bin
      RUN go get \

      COPY api.proto /defs
      RUN mkdir /defs/go-api
      RUN protoc --proto_path=/defs --go_out=/defs/go-api \
          --go-grpc_out=/defs/go-api /defs/api.proto
      SAVE ARTIFACT ./go-api/kvapi AS LOCAL kvapi

This will then produce two go files under the kvapi directory: api.pb.go and api\_grpc.pb.go which contains the auto generated protobuf and grpc code respectively.

At this point, assuming that earth is already installed, give it a try for yourself with code from our example repository:

    git clone
    cd example-grpc-key-value-store/proto
    earth +proto-go

The next step is to write the server code that will implement the set and get methods:

    package main
    import (
    const (
     port = ":50051"
    var errKeyNotFound = fmt.Errorf("key not found")
    // server is used to implement kvapi.KeyValueServer
    type server struct {
     data map[string]string
    // Set stores a given value under a given key
    func (s *server) Set(ctx context.Context, in *kvapi.SetRequest) (*kvapi.SetReply, error) {
     key := in.GetKey()
     value := in.GetValue()
     log.Printf("serving set request for key %q and value %q", key, value)
[key] = value
     reply := &kvapi.SetReply{}
     return reply, nil
    // Get returns a value associated with a key to the client
    func (s *server) Get(ctx context.Context, in *kvapi.GetRequest) (*kvapi.GetReply, error) {
     key := in.GetKey()
     log.Printf("serving get request for key %q", key)
     value, ok :=[key]
     if !ok {
      return nil, errKeyNotFound
     reply := &kvapi.GetReply{
      Value: value,
     return reply, nil
    func main() {
     lis, err := net.Listen("tcp", port)
     if err != nil {
      log.Fatalf("failed to listen: %v", err)
     log.Printf("Listening on %s", port)
     serverInstance := server{
      data: make(map[string]string),
     s := grpc.NewServer()
     kvapi.RegisterKeyValueServer(s, &serverInstance)
     if err := s.Serve(lis); err != nil {
      log.Fatalf("failed to serve: %v", err)

Next we will compile the go code and save it as a docker image with the following Earthfile:

    FROM golang:1.13-alpine3.11
    WORKDIR /kvserver
        COPY go.mod go.sum ./
        RUN go mod download
        COPY ../proto+proto-go/kvapi kvapi
        COPY --dir cmd ./
        RUN go build -o kvserver cmd/server/main.go
        SAVE ARTIFACT kvserver
        FROM alpine:latest
        COPY +kvserver/kvserver /kvserver
        ENTRYPOINT /kvserver
        SAVE IMAGE as kvserver:latest

You can give it a try on your own by using our example code in our GitHub repository, just run:

    git clone
    cd example-grpc-key-value-store/go-server
    earth +kvserver-docker

Then start up the server in Docker, by running:

    docker run --rm --network=host kvserver:latest

Next step: Implementing a gRPC Client Using Python

Now that we’ve built and launched our Go-based key-value-store server, we’ll cover how to talk to it using a Python client. Remember that initial Earthfile that generated the Go code? We’ll extend it to pip install grpc tooling, and generate Python code:

      RUN apt-get install -y python3 python3-pip
      RUN pip3 install grpcio grpcio-tools
      COPY api.proto /defs
      RUN mkdir /defs/py-api
      RUN python3 -m grpc_tools.protoc -I /defs --python_out=/defs/py-api \
          --grpc_python_out=/defs/py-api /defs/api.proto
      SAVE ARTIFACT ./py-api /py-pb AS LOCAL py-pb

Then we’ll create a client that reads command line arguments, and if the argument contains an equals sign, it will store the value in the server, and otherwise it will retrieve the value from the server:

    import sys
    import grpc
    import api_pb2
    import api_pb2_grpc
    addr = ''
    if len(sys.argv) < 2:
        print('program requires arguments in the form key, or key=value')
    channel = grpc.insecure_channel(addr)
    stub = api_pb2_grpc.KeyValueStub(channel)
    for arg in sys.argv[1:]:
        if '=' in arg:
            # send a value to the server
            key, value = arg.split('=')
                set_request = api_pb2.SetRequest(key=key, value=value)
                set_response = stub.Set(set_request)
            except grpc.RpcError as e:
                print(f'failed to send key to server: {e.details}')
                print(f'sent "{key}" to server')
            # get a value from the server
            key = arg
                get_request = api_pb2.GetRequest(key=key)
                get_response = stub.Get(get_request)
            except grpc.RpcError as e:
                print(f'failed to get key from server: {e.details}')
                value = get_response.value
                print(f'server returned value "{value}" for key "{key}"')

We then store this python code, along with the generated gRPC protobuf code with the following Earthfile:

    FROM python:3
    RUN pip install grpcio protobuf pycodestyle
    WORKDIR /kvclient
        COPY .
        COPY ../proto+proto-go/go-pb kvapi
        FROM +code
        RUN pycodestyle
        FROM +code
        SAVE IMAGE as python-kvclient:latest
        BUILD +lint
        BUILD +kvclient-docker

You can give it a try for yourself with the example code:

    git clone
    cd example-grpc-key-value-store/python-client
    earth +kvclient-docker

Then you can run it and set the weather to sunny with:

    docker run --rm --network=host python-kvclient:latest python3 \
    /kvclient/ weather=sunny

And if all went well, you should see some output on both the client and server consoles:

    # client output
    sent "weather" to server
    # server output
    2020/11/12 23:15:18 Listening on :50051
    2020/11/12 23:15:34 serving set request for key "weather" and value "sunny"

Final Step: Implementing a gRPC Client Using Ruby

We’ve come a long ways with our Go and Python gRPC examples, but what if you also wanted to include a Ruby gRPC client implementation too? Well let’s extend our proto Earthfile to generate Ruby protobufs too:

      RUN apt-get install -y ruby
      RUN gem install grpc grpc-tools
      COPY api.proto /defs
      RUN mkdir /defs/rb-api
      RUN grpc_tools_ruby_protoc -I /defs --ruby_out=/defs/rb-api \
      --grpc_out=/defs/rb-api /defs/api.proto
      SAVE ARTIFACT ./rb-api /rb-pb AS LOCAL rb-pb

We can then use this generated Ruby gRPC code with a simple ruby client example that performs a get request for keys listed as command line arguments:

    $LOAD_PATH.unshift '.'
    require 'grpc'
    require 'api_pb'
    require 'api_services_pb'
    stub =
      '', :this_channel_is_insecure
 do |arg|
      request = arg)
      response = stub.get(request)
      puts response.value
    git clone
    cd example-grpc-key-value-store/ruby-client
    earth +kvclient-docker

Then you can try querying the server to see what the weather was set to:

    docker run --rm --network=host ruby-kvclient:latest \
    ruby /kvclient/client.rb weather
And if all went well, it’ll tell you that it’s sunny outside.
drawing of the sunn

It’s Sunny Outside

So there we go. You can find the code for the server, the two clients, and a bonus integration test in GitHub.

Alex Couture-Beil %
Alex enjoys writing code, growing vegetables, and the great outdoors.



Get notified about new articles!
We won't send you spam. Unsubscribe at any time.