“Simplicity is a prerequisite for reliability” – Edsger W. Dijkstra
Managing and measuring complexity is one of the most important and most difficult, open-ended problems in software development. In this article we will describe the principles and guidelines that Lemma teams follow to manage complexity. In a future article we’ll focus more on what complexity measurements are available and what insights we can learn from them.
Simple != Easy
Something simple may not necessarily be easy, and something easy may not necessarily be simple.
Something easy is subjectively familiar, or within one’s current understanding. It’s quick to pick up because it doesn’t require one to learn a new paradigm. Something hard for someone may be easy for someone else. “Easy” is a subjective experience.
The roots of “simple” are “sim” and “plex”, and means “one twist”. The opposite, which would be complex, is “multiple twists” or “braided together”. Something simple does not complect (conflate, confuse) different concerns — it’s consistent and reduced to its most basic elements. Simplicity is an objective trait.
A deeper analysis on this topic can be found in the truly wonderful talk, Simple Made Easy by Richard Hickey, the creator of Clojure. Studying this talk is a requirement of our onboarding process for any new engineer that joins Lemma.
It’s very common to mistake “simple” with “easy” and the reason this differentiation is so important is because managing complexity is a primary software design consideration that we can only address and discuss if we are equipped with the right vocabulary and mental models.
For example, many people would say that Ruby on Rails is a simple framework, but we would argue that Ruby on Rails has historically been optimized for pragmatism and ease of use, *at the expense* of both operational and tremendous underlying complexity, or what some call “magic”. In many cases, the trade-offs can be reasonable, making it a viable framework to use. But it’s important to be aware of the existence of those trade-offs, and if one wanted to optimize for simplicity, there are many other frameworks or micro-frameworks that could be better candidates.
Less Is More - Dependencies
We recommend limiting the number and size of dependencies or external libraries to optimize for the control and adaptability of the codebase. Adding more dependencies comes with detrimental side effects, such as:
To give just one example of the risks associated with indiscriminate use of dependencies, consider the case of left-pad, the famous 11-LOC NPM library that was used by most NodeJs projects in existence inadvertently, which then was un-published by its author, leaving most NodeJs projects in a broken build state.
You may think this example is unique and in practice this isn’t that big of a concern, but actually most libraries out there at one point or another have had one, many, or in some cases myriads of bugs.
For example, let’s take one of the most popular Ruby Gems ever made, Devise; and just look at the list of over 100 closed bugs tracked in GitHub. You will find all kinds of bugs, from security vulnerabilities to performance issues to broken configuration options to… you name it. Any one of those bugs could have affected your application in unpredictable and dangerous ways, without you knowing. And this is just a single (extremely popular and well-maintained) Ruby Gem.
Think about projects that have hundreds or thousands of dependencies. What bugs, inefficiencies or other problems may lurk in the web of dependencies? At what point can you no longer realistically keep track of all of it?
There is no silver bullet solution to this problem, so it’s imperative to be aware of this consideration, to understand the complexity trade-offs that come with any new dependency added to the codebase and to build the right team collaboration and peer review processes which ensure that enough attention is given to the topic of complexity and reliability throughout the Software Development Life Cycle in general.
Less Is More - Less Code
Much of the same cost that comes from external dependencies, also comes from the code we write ourselves. As we will discuss in future articles, the sheer amount of code that we write is in and of itself a measurement of complexity.
“Less Code” is also about focusing on the essential complexity of the problem we’re trying to solve and expressing solutions that are succinct and unambiguous.
One of our engineering mentors, Lucas Tolchinsky, has given an excellent talk on this topic, which we highly recommend watching. You can also find the slides for that talk, here.
Last, but not least, we recommend reading the following article titled “Human Error”, by Michel Martens. This article explores in greater detail the importance of building solid mental models and how complexity hinders our ability to do so.
Simplicity Driven Development
These are some of the opinions, techniques and guidelines that we use to prioritize simplicity in our projects.
Focus on the essential domain complexity of the problem you need to solve. Avoid unnecessary features, use cases or other distractions.
- Err on the side of YAGNI.
- Don’t write placeholder-type code. If you need mocked functionality for any purpose (like a functionality demo or prototype) then be explicit about it and do it in a separate branch or encapsulate it as much as possible.
- Any extra feature the team writes today may have different requirements when it’s actually needed and deferring its implementation also means that the team will know more about the domain and users at that point in time.
- Breaking the YAGNI principle also means breaking the prioritization planning set by the team.
Prioritize the team’s ability to read and understand code, more than the convenience or optimizations for writing code.
- Prioritize designs that are explicit rather than implicit. It is better to invest in writing a solution that’s slightly longer but more obvious, than a solution that takes less space but requires more effort to mentally parse and understand.
- There are fewer worthy situations to use metaprogramming than you may think. If you find yourself considering the use of metaprogramming, first assume you are wrong and challenge your instinct.
Optimize software design and implementation for maintenance efficiency.
- Every single line of code has to be maintained for the lifetime of a project. It is more important to reduce the effort of maintenance than to reduce the initial effort of implementation.
- The ease of maintenance of a piece of software is usually proportional to the simplicity of its individual components.
- Assume that all code you write will need to change or be removed over time and design your solutions accordingly.
Avoid premature optimizations. An optimization is premature when the project could succeed without it.
- Premature optimizations typically result in more code and higher complexity. The potential benefits of premature optimizations may never be realized.
- On the other hand, this is not a license to write unnecessarily slow or fragile code. Collaborate with your team to define acceptance criteria for quality (including of course performance and reliability) so the team as a whole can make that judgment call.
Ensure disciplined and laser-focused code contributions.
- Don’t push commits that do multiple things. Don’t throw in small fixes in the same Pull Request where you are implementing a new feature.
- Ensure that all changes to the codebase are small enough that they are always as easy as possible to test and QA. Smaller, but more frequent contributions typically also result in less bugs being produced in the first place as the team will deal with smaller change sets and fewer blindspots.
Invest in frugal, high quality comments in your code.
- Code comments should explain why the code is doing something, not what it is doing. The “what” should be self-explanatory from the code itself.
Measure code coverage.
- Are 100% of users using 100% of the codebase? What’s the risk of accruing technical debt or legacy code for something that’s not impactful to the overall project?
Follow industry standard principles, such as the Law of Demeter, SOLID, KISS, YAGNI and others.
- These will give you excellent tools and heuristics to reign in complexity and design elegant solutions.
- Use these to help you, but avoid being dogmatic or losing sight of your goals when applying them.
- The same applies to design patterns for OOP or FP.
Encapsulate unavoidable complexity.
- Use adapters, wrappers or other techniques to encapsulate complex components in the codebase, so that the rest of the codebase only knows about simple interfaces.
- Consider as an example how something as immensely complex as the Go programming language hides its inner complexity.
