Part One: Enterprise Chess (Part Two will appear Monday)
By Gerben Wierda
In January 2015, my book Chess and the Art of Enterprise Architecture was published. I had started writing it in 2010, mostly because I was deeply dissatisfied with the ineffective nature of EA and the obviously impractical ideas of orthodox EA that I thought were the cause of that ineffectiveness. But then, in 2011, I was offered a Lead Architect position and a chance to set up an EA practice. So, the book project got shelved — incidentally, the second one that got shelved — and I started setting up a practice based on my own ideas. I had to build a team, convince people to work with my ideas, and get the whole thing off the tarmac, all of this in a high-strung very complex environment. A perfect testing ground, as it turned out.
Between 2011 and 2014 we were busy trying and developing my kind of ‘enterprise architecture’ and it was hard for many reasons. I learned a lot about what did work, but also a lot about what did not, and what unexpected hurdles there were. After I moved to another job, I restarted my book project and the result was published in 2015 and it is enjoying a still steadily growing readership.
In the meantime, I have of course encountered more challenges, thought of solutions, experienced success and hurdles — all ‘on the shop floor’ —, and I have kept on writing, mostly on my blog. In these two articles, I’ll give an overview of the main points of the original book and follow up with how these ideas have progressed since it was published.
Why change the EA practice?
Well, that one was quite obvious for me. There was this complete disconnect between what was being sold as ‘best practice’ and how executable and effective these ‘best practices’ were in the real world. They were seldom a practice — nobody was executing them as they were supposed to be executed — let alone ‘best’: after decades of the ‘orthodox’ philosophy, the landscapes were still a mess, projects were still late, expensive and incomplete, or failed altogether. In other words: the EA profession was telling the world they could solve the issues, but their recipes did not make a lot of difference. The general reaction of the EA community itself was that there was nothing wrong with their medicine, it would be fine if it was just properly used. I had two reasons to doubt that belief. One was that more than thirty years of trying was more than enough to conclude that nobody had succeeded in making them work and the other was that I could see fundamental flaws in the reasoning that lay behind the orthodoxy. The problem was: the orthodoxy sounded very logical, it was a story that was told in very convincing ways.
Why use chess as an analogy?
People often assume I must be a chess buff. And while I do like the game and did play a bit of chess when I was in my teens, I am not at all a very good chess player. But chess presented me with a good analogy to explain some of the shortcomings of what I consider ‘orthodox enterprise architecture’. By the way, I prefer to use the term ‘digital architecture’ instead of ‘enterprise architecture’. The term ‘enterprise’ has become a tail that wags the dog (see below). The only reason we are doing EA, I think, is because IT has enabled us to create a level of landcape complexity that merits specific attention.
Having said that: many people understand the basics of chess, and chess thus turned out to be a very usable analogy to present the following main insights:
- Using a ‘future state’ to work towards has little practical effect. Stay in the present.
- Architecture principles have little effect and are often even toxic
- Details count
- Architects need true insight in real technology
These four are in direct opposition of what orthodox digital architecture has been preaching for decades.
The attractiveness of a future state is obvious. Since the start of the IT-revolution, organisations have experienced that they are in a constant situation that the IT they have at some point doesn’t really fit the IT they want or need. In the previous century, this problem has been christened ‘Business-IT Alignment’, and it has been a core goal for the architecture profession to improve this alignment. The orthodox way to make this happen is to design a ‘future state’ for the landscape and try to work towards that goal. The problem is: the business goals are far more volatile than architecture can be, nor can the far-away goal give enough to influence decisions in the here and now. The same is true in chess: the grand master doesn’t work backwards from a desired end state at all. Instead, the grand master constantly works in the here and now: finding the best next move that creates opportunities and/or improves robustness. Counter to what people think, chess grand masters do not think that many moves ahead. Only near the end when the goal is in sight does that change.
So, in the ‘enterprise chess’ approach, architecture is mostly seen as a result all the changes we constantly make, not a specific target to work towards. Every change in the Business-IT landscape is then like a move in chess, it changes the landscape. The future state will not define our moves, but our moves will define the future state.
In EA orthodoxy, the common future state goal is also seen as contributing to the coherence of design choices. As this is clearly not enough, architecture principles have been espoused as a way to make future design choices ‘good’. Architecture principles, are like simple tactical ‘gameplay rules’ in chess. My favourite chess gameplay rule example is that one can give each piece in chess a weight (queen is 9, castle is 5, bishop and knight is 3 and pawn is 1) and that if you exchange material then you should take the exchange when you win more points than you lose. That is the chess equivalence of an architecture principle: it defines ‘the choice you should make’.
Following such a rule blindly is a sure way to lose a game of chess, though. Simply focusing on getting more material, will not make you win, you might not even end up with more material. But what is true is that winners do in general end up with more pieces than losers. Such a tactical gameplay rule of chess are thus descriptive and not prescriptive. Architecture principles generally are ‘desired outcomes’ (check your own if you have them) but that doesn’t mean following them will get you that outcome. The same is true in EA.
In reality, the situation is even worse. An ‘architecture principle’ comes with the ‘comply or explain’ mechanism. And this, in effect, means: if you follow the rule, that is by definition good. In the real world, this results in that — if only because even if architects are involved in decision making, they’re not all masters — architecture principles are too often followed when following them is really a bad idea. I have seen a 150 million euro project fail completely mostly because of a single, very popular, architecture principle that was followed when it should not have.
The idea that a set of design rules could get you a well-designed landscape is simply to good to be true. Besides: the whole idea of comply or explain is deeply contradictory: how can something by definition be good and at the same time have that ‘or’? ‘Comply or explain’ is a toxic oversimplification.
How often have I heard orthodox architects state that details are not important? But ignoring details is something that blows up in our faces time and again. Organisations do not stumble over mountains, they stumble over molehills, and keeping your eyes focused on that far away mountain is bound to trip you up. It is likewise in chess: sometimes the position or a move of a pawn can win or lose you the game. At the same time, it is impossible to take all — often yet unknown — details into account. And thus, the question becomes: how do we recognise the relevant details?
Which brings us to a key question: what makes a good architect? The orthodoxy pays very little attention to that question, focused as it is on frameworks and rules, models and designs. But frameworks do not create architectures, people — architects, designers — do. If we look at chess, then something interesting can be seen. Our best chess players do not think further ahead than good amateurs. Research into the way chess players think has unearthed that the difference between the two is — surprisingly — not better or deeper calculation. It is better recognition. From that same research: on average every position in a chess game has about only three potential good moves. The difference between good amateurs and the best players is that the good amateurs spot on average about two out of three good potential moves (and some bad ones as potentially good), whereas the professionals spot all three good ones. In digital architecture, the same is true: good architects see all the possible good solutions, and also — like good chess players — recognise bad ones.
‘Legacy’ is a derogatory term these days, but that is because we almost never have a good legacy… Basically, that tells us how poor our architecture decisions in the past have been. Our goal for architecture should be that we can often say: “Man, am I glad we decided to do X a few years back!”. If we would become good at architecture, the word ‘legacy’ would not be seen as universally negative.
Playing ‘Enterprise Chess’
Compared to digital architecture, chess is simple. Really, really simple. In chess, the goal is simple, we have only six types of pieces, a very clear landscape where everything is always known, and two players who politely take turns in making a move. An enterprise, though, has thousands of overlapping ‘squares’, hundreds of types of ‘pieces’ and many, many players — of varying skill — who make moves concurrently all over the place. Add to that that the goals are often vague, contradictory, and volatile. Enterprise architecture is the chess game from hell and if these simple approaches fail already in chess, there is no chance they will work in EA.
But, not all hope is lost.
First, while the EA problem is infinitely more complex than a game of chess, we’re not a lone player: we have an army of colleagues. As it is impossible to know everything about everything and because recognising the important details before they become show stoppers, enterprise architecture should thus be a deeply collaborative effort, and not a top-down ivory tower one. That means that a lot depends on making sure the right people discuss the design choices at the right time as efficiently and effectively as possible. So, if you create something like an ‘architecture board’ for instance, make sure the people in that board are able to understand technical discussions so that they as a group can form a ‘virtual architect’ for the complex landscape.
A part of collaboration is setting up checks and balances. There are two types of checks and balances: one is to make sure architecture goals are part of the total mix of goals when the organisation prioritises work to be done. One of the hurdles for good architecture is that there is no pressure from outside of the organisation to have a good architecture. Neither customers, nor regulators care, it thus requires quite a strong conviction of management to spend much on it (and given that they have in the past and not much came from it, they are not very willing to do so these days, and if they are, they generally are happy to try the same failed architecture approaches again as these are seductive).
The second checks and balances is to make sure that design choices undergo rigorous checks by designers from different domains.
Such processes may seem a hopeless effort. How can you get everybody from north to south and east to west to agree? But there is a decision making mechanism that can help. I only later learned from someone who had read my book that what I had been doing was called ‘consent-based decision making’. That kind of decision making asks at the end of a discussion a simple question. Not “do you agree with the choice?” But: “Can you live with the choice?” This tends to help focus on the essential choices.
Is there no place for thinking about the medium to long term future then? Yes there is. But not as simple as it has been so alluringly up-front, top-down presented by the orthodoxy. For instance, from an architecture perspective, the key uncertainties and scenarios are much more important than that future state.
It dawned on me — when later we had to confront the need for Agile and DevOps — that the ‘enterprise chess’ approach could be called ‘agile enterprise architecture’ and that it is based on a view where architecture is more an emergent aspect than something that is top-down/up-front designed. The whole enterprise chess philosophy was created and first executed in a classic project-based up-front design world, and funny enough, that classic project-based approach made a more flexible approach to architecture possible, mostly because design decisions are so handliy packaged in… designs.
Enterprise Architecture started out as the enterprise-wide expansion of IT architecture. It really became a pressing need when systems more and more got connected and dependent on each other. The existing attention to ‘software architecture’ clearly wasn’t enough. Incidentally, this is why the “International Association of Software Architects” — IASA — is not about software architecture anymore and the acronym has become a proper name. So, the name ‘enterprise architecture’ was coined. But in a twist where the tail started wagging the dog, the professionals in the field took that term ‘enterprise’ and started to claim that EA ‘was not about IT’ or even that they should be ‘architecting enterprises’. That was another way in which orthodox EA made itself ineffective. After all, there already was a party responsible for ‘enterprise design’ — general management — and EA had little of true value to add to what they did. EA in effect told general management that they were going to do general management’s job.
In the end there has been only one reason why we started with EA: the IT-revolution was having side effects that required managing: the constant lack of alignment and the too few benefits and too high costs of IT-heavy change initiatives. Looking at EA as that ‘chess game from hell,’ helps managing that Business-IT complexity.
Part Two can be viewed at https://www.architectureandgovernance.com/applications-technology/beyond-chess-and-the-art-of-enterprise-architecture-part-two/
Gerben Wierda is author, occasional speaker, and a lead architect at a large financial institution. He has written this in a personal capacity. His blog is at R&A IT Strategy & Architecture.