By Christian Siegers, of KPMG Netherlands
In a previous article, The Resilience Paradox, I argued that AI is forcing organizations to rethink resilience. As enterprises become increasingly dependent on AI platforms, data ecosystems and interconnected technology services, ensuring systems remain operational is becoming a growing architectural challenge.
However, operational continuity is only part of the story.
A system can be resilient, available and technically healthy while still producing outcomes that are inconsistent, undesirable or outside accepted policy boundaries. In other words, a system can continue to function exactly as designed while the organization gradually loses confidence in how that system behaves.
This distinction becomes increasingly important as organizations introduce AI agents, autonomous workflows and AI-driven decision-making into business processes. For decades, enterprise governance was built around implementation. Architecture reviews, security assessments and compliance processes were effective because most important decisions were made during design and implementation.
AI changes that assumption.
Increasingly, decisions are being made during execution. Outcomes are influenced by models, context, retrieved knowledge, interactions and orchestration logic that continue to evolve long after a solution has been deployed. As a result, organizations are discovering that governing implementation does not necessarily mean governing behaviour.
This creates an uncomfortable reality. In many organizations, the default response to AI is the creation of additional governance structures. Yet more governance activity does not necessarily create more control. Organizations can become increasingly effective at reviewing designs and approving solutions while remaining largely blind to behaviour in production.
The challenge is not that governance has become less important. If anything, it has become more important than ever. The challenge is that governance remains concentrated around the point where decisions used to be made, while AI is steadily moving decision-making into runtime.
This may be one of the most significant governance challenges enterprise architects face today.
Governance was built for a more predictable world
Enterprise governance evolved around a relatively simple idea: if an organization could control change, it could largely control outcomes. Governance processes therefore focused on the introduction of change. Architecture reviews assessed designs before implementation. Security teams validated controls before release. Risk and compliance functions evaluated solutions before they entered production.
For traditional enterprise systems, this approach proved highly effective. The behaviour of a system was largely determined by code, configuration and predefined business rules. Once deployed, applications generally behaved as designed until another change was introduced. Architects could review a solution, understand its intended behaviour and make reasonable assumptions about how it would operate in production.
AI fundamentally weakens that relationship.
An AI-enabled solution may use approved platforms, approved data sources and approved integration patterns. It may pass every architecture review and satisfy every governance requirement. Yet its behaviour can still vary depending on context, retrieved knowledge, model behaviour, available tools and interactions that occur during execution. The architecture may remain unchanged while outcomes continue to evolve.
This creates a governance challenge that many organizations are only beginning to recognize. Traditional governance was designed to evaluate decisions made during design and implementation. Increasingly, however, many of the decisions that matter most are being made after deployment.
Traditional governance was effective because most important decisions were made during design and implementation. AI changes that assumption. Increasingly, decisions are made during execution. Governance must inevitably follow.
The illusion of control
Many organizations assume that governance and control are closely related. In practice, they are not the same thing.
Governance creates confidence that processes have been followed. Control provides confidence that outcomes remain acceptable.
For years, the distinction was relatively easy to ignore because most enterprise systems behaved predictably. If governance processes were followed, organizations could usually assume that outcomes would remain broadly aligned with intentions.
AI exposes the gap between governance and control.
An organization may have extensive governance structures in place. Architecture review boards may approve every solution. Risk teams may validate every control framework. Compliance functions may confirm policy alignment. Security teams may verify every safeguard. Yet despite all of this activity, the organization may still struggle to explain why a system behaved in a particular way, whether that behaviour remains acceptable, or how it might evolve over time.
The uncomfortable reality is that many governance activities provide assurance without necessarily providing control. Historically, governance and control were closely aligned because behaviour was relatively predictable. AI increasingly separates those concepts. Systems may be implemented correctly, comply with standards and pass every governance review while their behaviour continues to evolve during operation.
Assurance remains important, but assurance alone does not tell an organization how a system is behaving in production. It does not explain how decisions are being influenced by context, retrieved information or interactions between multiple AI components. Most importantly, it does not automatically provide the ability to intervene when behaviour moves outside acceptable boundaries.
Increasingly, organizations need both governance and control.
This is why the governance discussion is shifting away from implementation. The question is no longer whether a solution is compliant before deployment. The more important question is whether an organization remains in control after deployment.
Runtime governance as an architectural capability
The response to this challenge is often framed as a tooling discussion. Organizations invest in governance platforms, monitoring solutions, policy engines and compliance technologies. While these technologies can provide value, they do not address the underlying issue on their own.
The real challenge is architectural.
For many years, governance existed around technology rather than within it. Architects reviewed designs. Security validated controls. Risk assessed compliance. Governance surrounded the technology lifecycle but was rarely designed as an operational capability embedded within the architecture itself.
As systems become increasingly autonomous, this approach becomes less effective. Governance remains concentrated around deployment while critical decisions increasingly occur during execution.
Organizations now require capabilities that allow them to continuously understand what systems are doing, determine whether behaviour remains within acceptable boundaries and intervene when necessary. This requires observability, policy enforcement, auditability, human oversight and intervention mechanisms that operate alongside the systems themselves.
This is where runtime governance emerges as an architectural capability.
Rather than treating governance as a series of checkpoints, organizations need governance capabilities that operate continuously. The objective is no longer limited to deciding whether a solution should be deployed. The objective is maintaining confidence that the solution continues to behave appropriately after deployment.
For enterprise architects, this represents a significant shift in responsibility. Architecture can no longer focus solely on how systems are built. It must increasingly address how systems remain governable throughout their operational lifecycle. The ability to observe, constrain, explain and influence behaviour is becoming as important as scalability, security and resilience.
This shift also exposes what may become a significant blind spot in enterprise AI architecture.
Most AI reference architectures spend considerable time describing data platforms, models, orchestration frameworks and agents. Far fewer explain how organizations maintain control once these capabilities begin operating autonomously across business processes.
Organizations have faced similar challenges before. Identity management evolved from an application concern into a shared enterprise capability. Security controls followed a similar path. Integration evolved from project-specific implementations into enterprise services. In each case, organizations eventually recognized that certain capabilities were too important to be implemented differently by every project.
Governance is now approaching a similar point.
As AI adoption expands, organizations can no longer rely on individual solutions to define their own monitoring approaches, escalation mechanisms and control structures. Doing so inevitably creates inconsistency, duplication and governance debt.
Instead, governance increasingly needs to become a reusable enterprise capability.
This is where the idea of a governance control plane becomes valuable. A governance control plane is not a single product or technology platform. It represents a set of shared capabilities for policy enforcement, behavioural monitoring, auditability, escalation and intervention. Its purpose is to provide a consistent mechanism through which organizations can maintain oversight and influence across increasingly autonomous systems.
The organizations that scale AI successfully will not necessarily be those with the most governance processes. They will be those that can operationalize governance as a reusable enterprise capability.
Conclusion
For decades, enterprise governance worked because the most important decisions were made during design and implementation. Governance naturally concentrated itself around those activities, and for largely deterministic systems, this proved highly effective.
AI challenges that assumption.
As organizations introduce increasingly autonomous systems, the relationship between design and behaviour becomes less direct. Solutions are no longer shaped solely by code and configuration, but by models, context, knowledge, interactions and decisions that emerge during execution.
This is not a governance failure.
It is a governance mismatch.
Many governance mechanisms were designed to govern implementation. Increasingly, organizations need mechanisms capable of governing behaviour.
This distinction matters because governance and control are not the same thing. An organization can have extensive governance structures, well-defined policies and rigorous approval processes while still struggling to understand or influence how systems behave in production. One of the more uncomfortable realities of enterprise AI is that organizations can have more governance than ever before while simultaneously having less control over system behaviour.
The rise of runtime governance is therefore not about creating more governance. Most organizations already have no shortage of governance processes, committees and controls. The challenge is ensuring that governance remains effective in an environment where decisions are increasingly made after deployment rather than before it.
For enterprise architects, the objective is no longer simply to design systems that are secure, scalable and compliant at the point they go live. The objective is to design systems that remain observable, controllable and governable throughout their operational lifecycle.
In The Resilience Paradox, I argued that organizations must rethink how they keep increasingly complex AI-enabled systems operational. Runtime governance addresses a different question: not whether systems continue to operate, but whether organizations remain in control as they do.
Because governance, much like resilience, must ultimately follow the realities of how systems behave rather than how they were designed.
And increasingly, those realities emerge at runtime.