On Modern Web Applications

Introduction

We all know that fashion waves come and go in software development, just like everywhere else in life. When in the middle of such a wave, it is extremely difficult to raise any concerns about the actual main stream, canonical method or truth. Currently it is agile and functional programming. We have a tunnel vision, where we keep repeating '4 legs good, 2 legs bad'. This article attempts to go against the accepted and unchallenged, and look at a few pretty big cracks in the system, trying to come up with some recommendations as well.

The problem

It has been a while now that I noticed a few things I did not like about how we write software. I started to collect these points, and was thinking about why we do things the way we do them, and how these individual aspects might correlate, catalysing each other. It is best if we start with the end product of it all, the software that we write.

If you look at a typical web application made in the past few years, you will notice the following few attributes:

it is using FP as programming paradigm

it is using a framework like Redux for application logic

it has no noticeable software design

it has tons of unit tests

it has a layered organisation of files (services, actions, etc.)

When you try to apply even the tiniest change to an application written like this, you immediately notice the following engineering principles all being violated:

Open-Closed Principle

Single Responsibility Principle

Liskov Substitution Principle

Interface Segregation Principle

Dependency Inversion Principle

First of all, any functionality change needs to involve all the layers, actions, reducers, components, meaning lots of changes. Since there is a tendency to introduce dependencies between pieces of state in the reducer(s) these changes are all potentially risky - hence you need to put a load of unit tests in place to make sure things still work. Things are flat and open, there is no resistance in this software, people can do pretty much anything, software contracts, etc. are not expressed by any means. And they even think that this is a good thing...

There is no noticeable design, that is, no entities, processes, interactions, the software is an unintelligible assembly of framework specific technicalities like actions and reducer functions that operate on atomic bits like a Boolean variable that has a particular meaning. Looking at this from ten thousand feet it looks as though we are again at the elementary level of assembly programming - our software is close to the metal (close to the framework and the DOM) and far away from the reality it is modelling. Yes, it seems history does repeat itself.

This type of software will obviously be very costly to maintain. As I said before, simple functional changes will need to be woven through the fabric of the application (lots of changes, lots of unit tests) but larger changes, like supporting a different financial product with a slightly different set of attributes/behaviours will be painful due the complete lack of abstraction and tight coupling everywhere. I like to call this type of software shrink wrap software. It is tailor made to the exact set of requirements known at the time of writing the software, with absolutely zero flexibility to withstand or help with change.

Under the hood

So what is driving all this? We all know that back in the 90s people started to feel that Waterfall doesn't really cut the mustard as it was unable to keep up with the fast changing business reality. By the way, this was the era of Object Oriented programming and software design (Design Patterns by GoF, etc. - apparently people had time for all that! :)

Coincidentally, application development took another turn at the time, more like around the early 2000s. Web applications started to replace the heavy duty GUIs and their object oriented designs that were developed with costly waterfall methodology. The fast paced business world found its ideal match, functional programming and agile.

The focus of agile is short term, the actual iteration, there is not much room for anything else. It is but natural that developers picked up a tool which is more proper for this type work, functional programming. Functional languages are good at small scale work, as in writing lots of small functions that can be combined to carry out more complicated work. Due to their dynamic nature they are also good for quick prototyping (funnily enough most prototypes in an Agile project end up being used as the real thing - clearly showing some confusion in the minds).

Functional programming, however, inherently is not so good at expressing larger scale designs. If you try to google for functional design patters you will find none. Everything is a function, end of story. You can play with scopes etc, but it is not idiomatic functional programming any more. Functional programming is wonderful, and it is very efficient for an array of things. It is a mistake however to try to use it for everything. It leads to the messy, unmaintainable codebases we call agile software.

A Way out

So far I tried to show where I see the problems with modern software development. It is ephemeral, short sighted, lacks design and uses a tool that is inherently incapable of producing structured, reusable and maintainable software. So what shall we do, shall we go back to waterfall and Object Oriented languages?

Well, that would hardly work, there was a reason for leaving all that behind.

There are a few things we need to take into account when trying to come up with a better way of crafting software.

1) Businesses change rapidly, so only tools/metholodigies that are able to keep up will be viable
2) Agile is unlikely to go away

Since coming up with an application design for each new application is not really viable due to the points above, we need a better framework that allows us to craft software, component by component that blends into the super-structure of the application, which is easy to maintain over time. This is the exact opposite of things like Redux, where things melt away, as you keep adding more and more to the application. They dissolve into atomic pieces of the state and fragmented logic, which are very difficult to reason about. I think this is a key problem with Redux. It forces a brittle, fragmented architecture on your application - and there is a massive price to pay for this down the line.

So the way forward can be a plugin-like architecture, where you can develop pieces of functionality (yes, using FP if you like!) where these plugins provide clear integration points and APIs, and it is easy to see how they are assembled together to provide a piece of functionality. These plugins can then be easily reused in other application, since plugins are self-contained and encapsulated units of markup, state and async resources. Contrast this with the fragmented nature of modern web apps, with the entangled web of action creators, reducers, services, utilities, and UI components - where it is impossible to isolate anything for reuse.

This plugin architecture will also help with testing, by allowing easy replacement of the dependencies these plugins declare.

It is still a question, how much of this is possible to implement in TypeScript, which is tied to JavaScript which has some limitations around implementing these kind of 'meta' designs, which are based on indirection, inversion of control, decoupling and lazy/conditional loading.

Conclusion

It is long overdue to come up with a better architecture and organisation for web applications. Finally we have a simple and elegant UI component framework, Svelte, but we are still suffering from an even bigger problem, not being able to model business problems in a clean, reusable and maintainable way.

My proposal is to combine the good from both OO and FP to come up with a framework that allows rapid application development and yet it does not compromise on the maintainability of the resulting code.

Sounds like a daring proposition?