Writing maintainable code is definitely hard. It’s even hard to define what “maintainable” means. After watching an enlightening talk by the author of Clojure lang, I’d say that maintainable code should primarily be simple.
Simple is often erroneously mistaken for easy. “Easy” means “to be at hand”, “to be approachable”. “Simple” is the opposite of “complex” which means “being intertwined”, “being tied together”. Simple != easy. – Rich Hickey, Simple Made Easy, 20 October 2011.
Reality is that we prefer easy “at hand” solutions that look innocent but lead to a poor maintenance experience. Let’s explore how simple code differs from easy in practice using Hystrix library as an example.
When we make changes that span across multiple services in a distributed system we face series of challenges. Risk of breaking the whole system is very high.
This is very unfortunate, but we need to adapt to these obstacles and develop better tools and practices to overcome these problems. GitHub developed a small library for fearless refactoring called scientist. The idea is to make data-driven decisions on refactored code rollout instead of pushing it into production and hoping that nothing will crash (while keeping in mind a rollback). I’d like to extend this idea to cover not only refactoring but also the system level changes.
Services communication is a fundamental part of any distributed system design. There are multiple ways you can make services communicate, and all of them are either synchronous or asynchronous. The choice of which one to use is essential. It’s not just about choosing sync HTTP client for the ease of development and then switching to async one to make service perform better. This choice defines system’s reliability. Let’s design the same system using both approaches to services communication and compare them.
Event Sourcing became very popular with the recent rise of reactive systems. However, the idea is not new and has a very concise definition polished with the years of research:
Although event sourcing is not directly related to eventual consistency (apart from the word “event” in the name), the common misconception is that you get eventual consistency for free while building event sourced system. In reality, the only kind of consistency you get for free is inconsistency. Let’s explore how not to get into this trap and build an event sourced system providing strong consistency guarantees.
Data Science is a trendy topic nowadays. We analyze customers behavior when they are browsing an e-commerce website to improve user experience, apply fraud detection mechanisms to prevent malicious transactions and use image recognition to know what’s a hot dog and what is not. At the same time, there are not too many data-powered instruments that would help us, software engineers, improve our experience of writing high-quality code. Git, in its turn, is not only a must-have tool in our day-to-day job but also a source of valuable data hiding the insights about both source code and people who develop it.
In the perfect world, services in distributed systems hide their data behind an interface. Although the shared database is a well-known services integration anti-pattern, it’s still one of the most used ones due to its reduced initial complexity and promising consistency guarantees.
Extracting services in a system with an overused shared database integration style becomes a nightmare. Even if you find a sub-domain to isolate, you still need to decouple its data from the rest of the system.
My point is that we can gradually solve this problem avoiding massive rewrites. But we need better tools to do this.