Common Code Be Damned — How we reduced dependencies between services

Just saying… This whole initiative is a sequence of conversations, POCs, and lots of teamwork.

In today’s world, it’s very common that a startup reaches a crossroad in which they need to decide if they choose to keep the monolithic service that got them so far, or if they should change it up. In Augury we reached this point in time and decided on moving to a more distributed system. To do so we are using guidelines derived from DDD in order to create our new microservice architecture. Today we already have a couple of dozens microservices.

Let’s start with stating the obvious — working with microservices has two sides:

• Decoupling is awesome, velocity increased and autonomy is sky high
• It’s too complicated, I need to start separating all of my common code too and keeping backwards compatibility is a b*$%h

I started working in Augury a year ago and the way we handled microservices and common code was pretty simple. We have one common repository and all the services are dependent on it. In that common code repository we have tooling clients (mongo, redis, nsq etc.), internal clients which are our in-house services and also just generic code that is used between different services.

So let’s break it down, why single common code repo?

Pros:

• Simple, you know where you consume from
• Only one repository to update yourself against
• If you make a change, it’s only needed in one place

Cons:

• If you break something, you broke it for everyone
• If you added a dependency, you added it to everyone
• Some code if never changed, but its version it’s getting bumped
• Having a distributed system that is still dependent is just a big ball of mud.
• It is hard to keep track of all the changes that has been made
• Can’t create versioning per package
• A lot of developers working on the same repo, can create conflicts, lots of rebases and more

As we move through this blog post I will share diagrams which represent our issues. Our shared common code package is called go-clients and we have multiple services in the system.

At the beginning we had 2 big monoliths:

And as we started using the microservices pattern more and more we started having:

What’s the first step?

Take out the api (http) package out of the common code. Meaning the common methods which perform the POST/GET/etc. Until this point in time, every time someone wanted to implement an internal client, they needed to implement the methods by themselves. Which caused code duplications and even worse, we had different implementations that did the same thing. So this was the first issue we tackled.

Now what?

Now we can start separating the internal apis themselves into different repositories.

Let’s think about it -

Pros:

• Each repo will be handled as a separate package with its own versioning
• Depending on a client package will not require depending on the entire go clients package.
• Each repo can depend on the new shiny api package
• Each repo ownership can be easily assigned
• New capabilities on the http api layer will be implemented in a single place

Cons:

• Handling a lot of repositories
• New capabilities will require propagation to multiple repositories

We decided on separating to different sdks. Each microservice has its own “go-sdk-<service-name>” to go along with it.

But oh no, the clients themselves is still coupled to the common code (dam dam dam)

At some point we figured out that we left something out. In our common code we have a configuration object which holds all of the dependencies a service might need. From this configuration object we afterwards init each api’s requests context or nsq events contexts to avoid creating a client per request/event.

Because we include go-clients as a whole into our services, we actually include into our service the dependency into all the tools and clients that are defined in this object.

So actually we ended up with:

Never give up

Once we figured out what the issue was, we started looking into a solution for the configuration object. But trouble never travels alone. On top of having a configuration object, we also create static common functions for the enrichment of the context of each api request or nsq event with our dependencies. You see, in Go it’s common to keep values in the context of requests or events and use them as a sort of dependency injection. This creates coupling to the services clients because they appear in the functions signatures and the fact that Go is statically typed.

To be able to remove the dependency of the go-sdks, we need to make the configuration object specific per service and define methods in the service instead of using the ones from go-clients. This way, each service will only need to include into its dependencies the sdks it is really dependent on.

Once again, let’s break it down -

Pros:

• Without this, we can’t reach our decoupling

Cons:

• Lots of code duplication between the microservices for defining the static functions that load the variables onto the context

I think that the fact that without it we can continue is a clear winner but let’s brainstorm for a moment. Code duplication is bad, but… sometimes it’s ok in the naming of decoupling. We are not duplicating logic, we are only duplicating simple set get methods. Also, Go being a baby programming language, it’s just reaching its high school years and features like generics that might would have enabled us to create this functions in a very generic way, are not supported in the current go version we are using (generics in Go was introduced in Go 1.17 which is still in the making)

One last hiccup

I was pretty pleased with myself writing the POC for this solution. Started copy pasting the methods for updating the contexts, created a super lean configuration struct in my microservices and… boom! We are using the static functions in our Gin middlewares. Gin is a commonly used web framework in Go and you have all the standard functionalities including the option to add middlewares on your routes. Our middlewares, like all of our common code, is in go-clients. The middlewares uses the static functions and this is “no bueno” because in the services we load the context from the local configuration object.

We decided to define an interface per middleware that will implement all of the usages that are currently the static methods. In addition we created a new middleware called v2 that consumes the interface. In the shared common code we also implemented the interface with the calls to the static functions.

This way we are backwards compatible and finally have a ready solution!!!

So what did we have -

• Created a “go-api” package that give http services to all sdks
• Decouple microservices from a central configuration struct
• Decouple microservice from static functions that are defined in common code
• Enable backwards compatible solution for our http gin middlewares
• Create “go-sdk-<microservice>” for each microservice

There are always improvements that could be made, but this is one step into a more decoupled world.