Cutting the
Gordian Knott

In his recent edition of First View, Faster Horses, First Derivative CEO, David Collins, discussed the area of system design.

In this edition I elaborate on this subject. What follows is intentionally opinionated. If your perspective differs, even better – I’d welcome the discussion.

If we start with what we are trying to achieve, I would argue that the purpose of software engineering is to craft elegant, high-performance solutions that are intuitive, adaptable, and resilient – designed to operate seamlessly in real time while remaining easy to modify, scale, and maintain.

The problem, however, is that often the way we go about designing solutions does not make the best use of the tools now at our disposal. We are using new tools to create old solutions. We often end up replicating legacy thinking and processes on new technology rather than rethinking the problem itself.

Typical anti-patterns include:

• Asking people what they want, only to have them describe what they already have. This just results in a duplication of the legacy system, albeit on new technology
• Using automation to replicate old-fashioned, sequential human processes. In this instance, the result may be faster, but all the inefficiencies caused by organisational silos and process hand-offs will be retained. Anyone familiar with Conway’s Law will recognise this (and if you’re not familiar with it, it is a concept that should be on every architect’s radar)
• Where people expect strategic automation of a function but end up with point automation of human tasks

Creating solutions where data flows in real time from one decision maker to another generating immediately actionable insights to create competitive advantage will involve getting a number of ingredients right:

Whilst all are important, there is typically one driving force behind the design of a new solution. Usually, it is easy to tell what route has been taken: user-centric, process-centric, service-oriented or data-centric.

If we are to create solutions where data flows in real time, operational processes are automated, and human intervention is minimised, then a data-centric approach is needed.

There have been times in the last few decades where data was seen as a goal in itself, particularly in the early days of Big Data. That just resulted in billions being spent on data lakes, generally with questionable return on investment. So that mistake needs to be avoided by hardwiring investment in data with use cases which enhance user experience and create competitive advantage.

Whereas in the past, the data architecture has often been an afterthought, in a world powered by AI, it is the foundation upon which everything else will be built.

As legend would have it, in 300BC, traditional methods had failed to untangle the intricate knot tied by King Gordius of Phrygia. Alexander the Great succeeded by taking a different approach. Solving modern challenges requires more than untangling legacy systems – it calls for decisive data-driven reinvention.