Why I use Go

I take the opportunity of the publication of this little article. One more as there are hundreds of them to share a little of my experience with this language.

Analyzing my repos, I can state that I use Go professionally for about 3 years.

It was at LibMed, a startup that developed a platform for managing paramedical replacements.

Like any many platforms, LibMed is made up of a multitude of micro-services that collaborate with each other to deliver the functionalities offered to users (web front end, data graph management, generation of employment contracts, electronic signature, intelligent notification system, etc.).

The advantage of micro-service architectures is, among other things, that they allow freedom of choice of weapons (the development language) for each service as long as it supports the API protocol chosen for communication between services (in this case in gRPC).

Many micro-services at LibMed have been developed in the language probably the most distant from the Go philosophy : Scala.

This choice was made for the power and expressiveness that a functional language like Scala can bring.

When a service has to implement numerous and complex business rules, the expressiveness of a language is an important aspect, Scala is very good in this domain, as would be Kotlin (non-existent at the time of the creation of LibMed) or even Rust to a lesser extent.

But Scala has two problems that it inherits from hyperexpressive languages:

• The power he derives from his expressiveness is paid for at the price of a code that quickly becomes incomprehensible to those who do not spend eight hours a day in front of it. This raises two problems in software engineering: the cost of a new team member and the cost of maintenance. In addition to a very long learning curve (several years to really master the nuances of the language, which makes candidates rather rare on the market), newcomers will have to invest several weeks before they can fully master the basic code. The same goes for the unfortunate developer who will have to take over code written several months or years ago by someone else (or by himself).
• Its compilation time is very long (despite the progress made over the versions), it may seem like anecdotal at the time when any laptop has 4 or even 8 cores and 16GB of RAM, or even at a time where the build was delegated to a CI-CD chain provided for this purpose, but it is in fact a source of frustration, loss of time and therefore productivity for the developer (because few developers are coding for 8 hours without ever testing on their workstation).

That said, I remain convinced that hyperexpressive languages such as Scala, Kotlin or even Rust (which has other advantages) remain an excellent choice when it comes to implementing business rules, which are themselves complex.

But, because there is a “but”, a platform is not only made up of components that have to manage complex business rules (in fact they are rather rare).

In the vast majority of cases, the problems a developer has to solve are simply complicated, not complex.

The properties that we will look for in the tool (the language) to solve them are therefore different.

Hyperexpressiveness (which is expensive, I remind you) is no longer essential, on the opposite of:

• maximum productivity in the development cycle: therefore a fast compilation and a well supplied standard library.
• a minimum cost of entry (cognitive investment) for the new developer: therefore a simple, readable and straightforward language
• minimum maintenance cost: we take the previous properties and add language stability over the releases as well as full backward compatibility.

As these components (or microservices) represent the vast majority of the platform, the overall performance of the platform and its operating cost therefore depend above all on them.

We then add the following properties to our wish list :

• a high level of (run-time) performance,
• minimum resource consumption (CPU, memory) (this is what will determine the amount of your Cloud provider invoice at the end of the month),
• a fast start (this has become a necessity with the auto-scalling functions of infrastructure orchestrators, Kubernetes for example,
• a packaging / delivery system that simplifies deployment and updates as much as possible (we will avoid systems relying on a multitude of libraries that must be individually maintained up to date or systems requiring an interpreter (Python, JS, Perl, …) or a virtual machine (JVM, Erlang / Elixir VM, …) for the same reasons.

Three years ago, while analyzing this list, one candidate emerged: Go.

I routinely (at least once a year) go through the exercise of re-chalenging this choice in accordance with the evolutions of other languages or newcomers (I keep an eye on V for example).

Until today, Go systematically wins the exercise, compared to this list and for this type of use, and here is why.

Productivity

I must have written over the last three years close to a hundred programs in Go, from the small CLI tool to the deferred notification system implementing CQRS , event sourcing and FSM through very production-oriented applications (medical imaging, GIS) or more common for public.

I’m just not sure I could have done the same with another language.

It is of course possible to write a program very quickly in Java, Kotlin, Perl or Ruby, but would it be of “production” quality?

I mean by that:

• Modular enough to be able to evolve quickly
• Robust enough (with the error management that this implies) to be able to run for months or even years without human intervention in production.
• Powerful enough to never represent a bottleneck in the whole system
• Safe enough (mermory safety) not to present a significant risk

From my modest experience: no.

Go is a recent language (it has just turned 11 years old), it is not necessarily a quality, but it has an advantage: its standard library meets today’s needs.

• Need an HTTP server? Boom! in 10 lines of code you have a multithreaded (rather multi-Go-routines, but let’s simplify) ultra robust and with the same level of performance as an Apache or an NGinx.
• Need to parse or write JSON? Boom, it’s natively managed in the data structures with a couple of annotations.
• Need to interact with a database? Boom, this is also in the standard library
• Need to zip / unzip files? same
• Need to send / parser emails? guess what…

If you want to do Corba or SOAP, you will certainly have to go through external libraries, but to implement a REST or gRPC service, you need almost nothing but the standard library… and very few lines of code.

What about compilation speed?

One of my longest projects to build is a storage provider for Kubernetes (to be simple, which is a kind of driver that Kubernetes uses to provide storage space).

My code is not very long, but during the build, Go also has to compile the Kubernetes dependencies (there is no library system precompiled in Go, I’ll come back to that), that is to say a significant part of it (about 3.3 million lines of code according to scc).

Here is what it looks like on my modest MacBook 13":

• First compilation (no cache, you have to build everything): go build . 86.94s user 15.95s system 542% cpu 18.978 total
• Second build (it simply checks the cache for accuracy and compiles what has changed): go build . 1.25s user 1.09s system 524% cpu 0.446 total

Go compiled about 3.3 million lines in less than 19 seconds on a single laptop.

Hard to beat. The daily impact for the developer is significant.

Admission fee

Go is a deliberately simple language.

It has been a top priority for its creators since the first day of its conception (which, according to the legend, took place while waiting for a C++ build at Google).

Let’s summarize :

• It is not an object-language (even if it allows to implement “methods” on data structures): no classes, no superclasses, no polymorphism, no inheritance, no “protected”, no abstract types or anything else that makes the OOP quickly complex, not to say unusable.
• It is not a functional language (even if it knows how to handle lambda, higher order functions and composition): no immutability, no prohibition of side effects, no functor, no monad or other vulgarities that scare away most newcomers.

It carries the minimum (and not much more):

• 7 data types and nothing else : Integers (Signed and UnSigned), Floats, Complex Numbers, Byte, Rune, String, Booleans.
• Functions and “methods” (functions applicable to a data structure)
• Interfaces to bring some modularity
• Packages to organize your code
• And an elegant and incredibly efficient management of concurrent programming based on the CSP concept .

Well, that’s about it.

The complete language specification is available here.

A normal person (with 90–100 billion neurons) can learn the language in a day or two and start being productive in a week.

This deliberate simplification has its detractors: the error handling is repetitive (there is no notion of exception, you have to check the return of each function call), there are no generics (but it is a work in progress), it lacks expressiveness, etc…

But it also has huge advantages:

• As I said, a newcomer to a team is very quickly operational, even if he didn’t know the language when he joined.
• The code is explicit and crystal clear: open the source code of any project written in Go (even the big one : Kubernetes) and you will immediately (or almost) understand how it works.
• Error handling may be repetitive, but it is so explicit that it leaves no room for forgetting and there is no break in the execution flow (as can be the case with exceptions).
• As the language is simple, the compilation is fast (There’s no magic bullet…)

Maintenance cost

An explicit and easily readable code is a more easily maintainable code (even if it requires double the number of lines of an ultra-expressive / compressed and therefore unreadable code).

A developer spends on average 90% of his working time reading code and only 10% writing it.
Ease of reading should therefore be a priority, and if he has to type one more line when he writes it, it wins (for his employer/customer too).

The other relatively surprising feature of Go, considering its young age, is its stability.

Go is already ultra-present in Cloud computing infrastructures, for example :

• Docker
• Kubernetes
• Traefik
• Containerd
• CoreDNS
• Harbor
• Jaeger
• Prometheus
• Helm
• etcd
• CRI-O
• NATS
• Drone.io
• …

Services that need to run 24/7/365 without interruption and with a minimum of human maintenance.

A combination of factors explains this phenomenon:

• Its memory safety by design which eliminates a very large source of potential bugs.
• Its explicit management of errors that leaves no room for “we assume it should work”.
• A Go binary is autonomous, it does not depend on any external library (not even the libc), on any JVM or other VM: the risk of incompatibility between several products is therefore nil and the risk of error during the update process is reduced to its strict minimum (one file is replaced by another).
• The overall stability of the language: in 11 years of evolution, the novelties (at the level of the language itself) are almost non-existent. You only have to read the release notes to be convinced of this. Some people find it a bit annoying (I am one of them), but you have to admit that reusing code that was written 11 years ago is not a big deal.

Performance

This is THE subject that often obsesses programming enthusiasts, often forgetting that the final performance is not so much determined by the language and compiler as by the algorithmic talent of the developer.

Simply speaking, Go is fast, the right term would be: fast enough.

Faster than Java, a little less than Rust, a little less than C++ and much faster than Python.

And the best part is that it achieves this level of performance easily, without requiring years of experience to master its subtleties, without having to rely on the black magic of 20 lines of compilation options or the fine tuning of a garbage collector.

The ease induced by the language to do concurrent programming naturally encourages the developer to actually use all the cores available on his CPU(s), which is much less frequent than one might think.

Resource consumption

Go is rather frugal, especially with memory.

It had sometimes to rewrite in Go components initially developed in Java, Scala or even PHP, the benefit in terms of memory consumption is obvious from a factor of 10 often observed to a factor of 100 in some extreme cases.

When we know that memory is the most determining dimension (because it is the most expensive) in the billing formula of Cloud providers, we wonder why so few companies pay attention to it.

Finally, and even if it is less significant, a binary (or let’s say a container) in Go will also easily weigh 20 to 100 times less than its Java, Kotlin or Scala equivalent.

Since Go does not need any library or VM to run, the build script of a docker image is reduced to its simplest expression:

FROM scratch 
COPY hello / 
CMD ["/hello"]

scratch being an empty image, hello being the binary from your compilation.

No need to worry about the choice of a base Linux distribution, a JVM distribution, its version, the frequency at which it will be necessary to update all this in order to fix new vulnerabilities…

Quick start?

Micro-service oriented architectures encourage the division of a business problem into functional sub-domains.

For example, an online store can be divided into modules such as :

• A module for registering new accounts
• An identity management module
• A stock management module
• A data presentation module (for the front)
• A module for calculating the amount of the basket (managing discounts and other conditional promotions)
• A payment management module
• A module for managing shipments / deliveries
• A returns management module
• A module for managing communication (targeted marketing) to customers

Each module can be called upon with different intensity over time, some of them could even be launched event-driven (for example the calculation of the logistic modalities inherent to the preparation of an order).

If we consider that the company wants to control its IT costs as tightly as possible, it is in its advantage that this software architecture behaves in an elastic manner, i.e. the number of instances of a given module increases during peak periods of use and rapidly decreases (potentially down to 0) when there is little to no traffic.

The true “Pay as you Grow” is at this price.

This implies that in case of a rapid increase of traffic on the online store, starting a new instance of a module can take no more than 2 or 3 seconds.

The JVM on which languages such as Scala, Java or Kotlin are based was not designed for this.

It was born at a time when large monolithic applications were designed that could take several minutes to start, as they were almost never stopped. And for good reason : they were run on machines that the company had purchased and for which they financed the entire depreciation, whether they were used at 5 or 100% of their capacity.

The paradigm of Cloud computing and orchestrators such as Kubernetes, effectively capable of efficiently controlling the elasticity of a program according to a measurable demand, has changed everything.

Initiatives have emerged to try to correct this inadequacy with current needs, such as the excellent Quarkus, but the efforts to be deployed to adapt legacy applications are such that the question of their complete redevelopment in a technology natively armed for these new constraints is sometimes relevant.

The vast majority of Go applications start up in less than a second, even those that embed a Web server and/or need to connect to a database.

Packaging / deployment

As we have seen, Go produces standalone binaries, dependencies are linked statically.

Indeed, and although it is allowed by the compiler almost nobody produces precompiled libraries.

When you have a dependency on an external library, you just have to indicate the URL of the repository of the latter, for example :

import ( 
   "github.com/sirupsen/logrus" 
)

The dependency management system, go modules, will then retrieve the latest release (you can still choose a specific version) of the latter, download its sources which will then be compiled with yours during the build.

In the example, this is a Git repo, but Go supports several types:

Bazaar .bzr 
Fossil .fossil 
Git .git 
Mercurial .hg 
Subversion .svn

This simple yet elegant system has two positive consequences:

• No more need to deploy or use an artifact repository such as Maven, Nuget, CPAN, npm, RubyGems, etc… to publish a library (a package in Go), you just have to commit, push and version it. It’ s hard to make it simpler.
• No need to build your library, it will be done when you use it. And that’s fortunate, because Go being multi-platforms and multi-architectures, it would have to be compiled and stored in all possible combinations, it would be tedious and expensive.

You might argue that this will be paid when building the final binary (since it has to compile everything) but remember the performance of the compiler (3 million lines in less than 19s in my previous example).

In the vast majority of the time, there is only one file to deploy in production or to inject in a Docker image.

However, it may happen that you need to add resources to it, for Web applications in particular: images, CSS styles, JS scripts, …

There are three ways to handle this :

1. Your target format is a Docker image and you can simply copy your resources into it.
2. Your target format is a single binary: you can inject your resources into it, either with libraries such as packr or omeid but also soon simply with the standard library.
3. Your target format is a binary and its resources: well you deliver your binary and its resources (in a zip or in any other format).

Personally, I most often deploy as a Docker image (the targets being most often K8s clusters).

With a minimum of discipline in the versioning strategy (but here again, the language help you), managing deployments and upgrades is child’s play.

This dependency and target format management system is so simple and effective that one may wonder why other languages have not adopted it (anyone who has had to do an npm build recently will understand).

Conclusion

Go is not an academic language, it was not designed to make progress in the science of languages or to explore new paradigms.

Go was designed by Engineers for Engineers, it is a tool, not a subject of study.

You can feel it when you use it, it is pragmatic, efficient, simple, sometimes basic, but it can quickly and reliably solve 99% of my daily problems and that’s all I ask of it.

It doesn’t claim to be a universal language, it is for example very little used in mobile development or for desktop UIs (to which I am relatively unexposed with my customers).

It has been considered for a long time as “the language of the Cloud”, of infrastructure, but it has been open to many other types of applications over the last 11 years.

Whether you’re a developer, tech lead, CTO, or in a more generic way a decision maker, it’s a technology that deserves your attention because it can make your life easier, make you more productive, and lead to savings and efficiencies for your business and your customers.

That’s why I use Go.

Originally published at https://www.solutions.im on November 14, 2020.