Thesis: “when building large scale distributed systems, high performance functional programming languages provide the quickest turnaround from idea to concept; however such advantage disappears as we move from concept to production, and the overall time from idea to production across all programming languages is of the same order of magnitude”.
I posted this diagram, without justification, yesterday evening, in an attempt to gauge the reactions of the community, in twitter. Thank you to all of you that commented. With the experiment done, let me now provide my thesis and hopefully address most of the feedback so far. I intend to make this post fluid and keep updating as the conversation evolves.
Now, let me qualify the thesis. First, this is a thesis applicable to multiple domains, whether that’s a low-level network switch appliance, a http proxy gateway or a web application. As a consequence, this is not a framework comparison, it’s not about Rails vs Django vs Play vs Express vs …. This is not an argument about dynamic vs statically typed programming languages, although that definitely plays a partial role in the thesis.
Mine is a thesis about the complete team productivity by programming language and across the full product life-cycle from inception to retirement. The thesis is purely based on my observations throughout the years. There is not data to back it up, except anecdotal evidence. In summary, mine is a qualitative statement, not a quantitative one.
If the thesis were true, this would mean that teams using high-performance and functional programming languages can iterate more quickly from idea to concept. Such quick turnaround allows a constant validation of ideas in the code. Because it is quickly possible to see the idea running, we can afford to have more ideas. The quicker we are able to iterate between idea and concept, the more we are innovating.
Such quick turnaround has a real trade-off and a false trade-off.
High-performance functional programming languages trade strictness for innovation potential. Without the additional investment in checks and tests, these systems are not predictable. The time we gained upfront, we pay later for. I’d argue such trade-off is good, and allows quick iteration to find the right idea.
The second trade-off is about tech debt. The time to production is identical across languages, but because the developer in high-performance functional programming languages had to compensate for the lack of compiler infrastructure and runtime guardrails by adding completeness of tests and checks, the system is easier to maintain and has overall a better architecture. This is almost counter intuitive and I think most people don’t think this way because they move too quickly from concept to production, and don’t implement the necessary strictness. There is a tendency in the developer to move quickly from concept to production, and therefore accruing overall technical debt.