By Dr. Nikita Golovko, AI Architect, Siemens

Your AI system may already have an architecture document. It lists the data sources. It describes models. It defines the KPIs the system is meant to improve. Someone spent an effort writing it.

Now let us shift to a harder test. Let us suppose that you are building an architecture of loans giving system with an AI risk scoring under the hood. A regulator reviews a loan decision that your system made six months ago. They ask three direct questions. Why was this applicant declined at that threshold. When did the model’s behavior change in a meaningful way. Who approved that change, and what evidence supported the decision.

If your documentation cannot answer those questions with verifiable evidence, you do not have an accountable AI system. You have a system that is well documented. That is not the same thing. In practice, accountability starts where description ends and traceable evidence begins.

The Governance Illusion

Today, many organizations use documentation as their first response to AI governance. They create model cards, system cards, and new AI sections in architecture documents. This work matters. It gives teams a shared language. It makes data ownership clearer. It brings design assumptions into the open instead of leaving them hidden.

Still, documentation has a clear limit. It describes what the system is meant to do and what the team believes is true about it. It does not prove that these things are true in daily operation. It also does not prove that they stay true after the system changes in production.

This is not a weakness in documentation itself. The real issue begins when teams expect the document to govern the system on its own. A document defines the intent. It sets the rules and expectations. Real governance starts after that. It depends on the mechanisms that test, enforce, monitor, and record whether the running system still matches what the document says it should be.

Four Gaps That Documentation Cannot Close

Let us consider four governance problems that keep appearing in production AI systems, and why documentation on its own does not solve them.

 

Data lineage drift.

An architecture document can tell you where the training data came from and which transformations were applied before the model went live. That helps at the start. The problem begins after deployment. Data sources do not stay fixed. A sensor may be replaced. A new upstream system may change field values. A preprocessing step may be updated during a later retraining cycle. Once that happens, the original document no longer describes the data the model is using today.

This is why data lineage drift is hard to control with documentation alone. A manually maintained document gives you a snapshot of the design. It does not warn you when the character of the data changes in production. After the first retraining cycle, the document often starts to fall behind. A few weeks later, it may still look complete, but it describes the system as it once was, not the system that is running now.

Model drift without detection.

An architecture document often states when a model needs retraining. That is not the same as knowing when retraining is truly needed. Writing down a threshold does not make the system watch for it. If the document says the model must be retrained when accuracy drops, confidence shifts, or input data changes, the production system still needs active monitoring to detect those signals. Without that monitoring, the threshold remains a rule on paper. The requirement exists, but the mechanism that detects the breach and triggers a response is still missing.

Governance authority gaps.

An architecture document may state who must approve model promotion before the model reaches production. That helps define responsibility. It does not ensure the approval will happen.

In real projects, teams work under delivery pressure. Deadlines are tight. Incidents appear. Business expectations push the release forward. In that situation, any approval step that depends only on people remembering to follow the written process is easy to skip. The pipeline runs, the model goes live, and the required sign-off never happens.

This is the core problem. Documented responsibility is not the same as enforced responsibility. If accountability depends only on manual compliance, it will often break at the exact moment when the decision carries the highest risk.

Missing audit trails.

Architecture documents often say which decision logs an AI system should produce. That is useful as a design statement. It does not create the logs.

The logs only exist when the team builds and runs the infrastructure that captures them, stores them, and keeps them available over time. Until then, the document is only a description of intended behavior. It is not the audit trail itself.

This difference matters most during a regulatory review. In that moment, the document is not evidence. The real evidence is the record the system produced in operation, with the right detail, the right retention, and the right integrity. If that record does not exist, the architecture document cannot fill the gap.

The Architect’s Accountability Responsibility

When teams see this as a documentation problem, they usually respond in the same way. They write more documents, add more templates, and define more process steps. That gives the appearance of control, but it does not solve the real issue. The real issue is enforcement.

Once you frame it correctly, the architectural response changes.

The system needs a trusted place that always shows which model is running in production. That place should be the model registry. It should not depend on someone updating a field by hand after a release. If the current version matters, and in production it always does, the system itself must record it and expose it as the source of truth.

The same applies to approvals. A written policy that says a model must be approved before release is not enough. Under the time-pressure, teams skip steps. The pipeline still runs, and the model still goes live. A stronger design puts the approval into the release process itself. The deployment pipeline should stop unless the required approval artifact is present. In this setup, approval is no longer a reminder in a document. It becomes a real release condition.

Decision logs work the same way. Many architecture documents say the system should record important decisions, but that does not create the record. The system must produce that log as part of normal operation. It must store it in the right place, keep it for the required time, and make it available when someone needs evidence later. If logging is treated as something to add later, it will often be incomplete when it matters most.

This also changes how governance authority works. A document may name a model owner, but a title alone does not create authority. Real authority exists only when the organization gives that person the power to stop a release, demand review, or block a change that does not meet the rules. Without that power, the role is symbolic. The name exists in the documentation, but the control does not exist in practice.

This is the central point. Good AI governance needs two things working together. It needs technical mechanisms that enforce important rules inside the system. It also needs an organizational structure that gives the right people the authority to use those mechanisms. When one of these parts is missing, governance stays weak. When both are present, architecture starts to support real accountability instead of only describing it.

Four Fixes: Closing Gaps in Practice

Your organization does not need perfect governance before making progress. The better path is to close the biggest gaps one by one. Each gap has a practical architectural fix. Your team adds these fixes step by step, without throwing away the systems already in place.

First fix: Replace manual lineage with automatic provenance.

Data lineage should not live in a spreadsheet or in a document someone updates after a retraining cycle. That approach falls behind almost at once. The system itself should record where the data came from, when the pipeline used it, and which transformations shaped it along the way.

This is where lineage tooling matters. Tools such as OpenLineage, dbt, and DataHub capture lineage directly from the data pipeline. They record facts as the system runs. That changes the role of architecture documentation. The document no longer tries to list every lineage detail by hand. Instead, it defines the policy around lineage. Which fields count as sensitive. Which transformations need review. Which upstream changes should trigger a retraining decision. The system records the facts. The architecture records the rules. Together, they give your team lineage that stays useful and current.

Second fix: Detect drift automatically and connect detection to action.

Many teams document model drift well. Far fewer build a system that responds when drift appears. A threshold written in a document does nothing on its own. Your system needs active monitoring that watches for drift and triggers a defined response.

The important step comes first. Your team must decide what counts as meaningful drift for this specific system. In a credit scoring system, that might mean a Population Stability Index limit or a clear shift in the distribution of predicted scores across customer groups. Once your team defines those limits, the monitoring layer should enforce them. A small shift might raise an alert. A larger shift might escalate to review. A severe shift might stop further inference until the team checks what is happening.

The key idea is simple. Drift thresholds should live in a governed artifact, such as an Architecture Decision Record or a versioned configuration file. They should not live only in one engineer’s head or in one dashboard setting. When the team changes a threshold, there should be a record of why that change happened.

Third fix: Make governance authority part of the structure, not part of the process description.

One of the most common failures in AI governance happens when approval exists in documents but not in the release path. The document says a model owner must approve production release. Then deadline pressure rises, the pipeline runs, and the model goes live anyway.

A stronger design solves this in two places. First, in the technical flow. The CI/CD pipeline should require a machine-readable approval artifact before promotion to production. This might be a signed review record, a completed checklist with a named approver, or a ticket in the required resolved state. The pipeline should not ask whether the team followed the process. The pipeline should check whether the required evidence exists.

Second, in the organizational setup. The person named as model owner must have real authority to stop the release. A title without decision power is not governance. It is only responsibility without control. Real accountability starts when the named owner has the right to block a release that does not meet the agreed conditions.

Fourth fix: Treat decision logs as a core system output.

Any AI system that affects people should produce an auditable record of its decisions as part of normal operation. This record should not depend on later reconstruction from application logs. The inference path itself should write the decision, the active model version, the feature values used, and the threshold in force at that moment into an immutable store.

This matters because audits ask precise questions. Why did the system make this decision at that time. Which model made it. Which threshold applied. Which inputs shaped the result. If your system does not produce those answers directly, your team ends up rebuilding the story after the fact, often from partial evidence.

That is why the decision log schema deserves the same discipline as an API contract. Your team should version it, document it, and protect backward compatibility. A schema change in the decision log is not a small operational detail. It is an architectural change, because it affects the system’s ability to explain itself later.

The main lesson across all four fixes is clear. Good governance does not start with more documents. Good governance starts when your architecture records facts automatically, enforces important decisions structurally, and keeps evidence ready before anyone asks for it.

Where to start

Most organizations do not need to fix all four governance gaps at once. A better starting point is to look at the current system honestly and ask a few direct questions. Is data lineage tracked automatically or does someone update it by hand. Are drift thresholds defined in a governed way, or are they based on informal team habits. Is model promotion approval enforced by the release pipeline, or does the team rely on people to remember the process. Do decision logs exist as built-in system outputs, or does the team have to reconstruct them later when someone asks for evidence.

These answers usually make the first priority clear. One gap will stand out as the biggest risk, whether that risk is regulatory, operational, or reputational. That gap should become the first architectural investment.

This also matters because the fixes strengthen each other. Automated lineage makes drift detection more trustworthy. Governed drift thresholds make promotion gates easier to justify. Strong decision logs make the whole governance setup easier to verify.

Organizations that treat AI governance as a documentation task will keep producing more detailed documents about systems that still cannot explain themselves under scrutiny. Organizations that treat AI governance as an architectural discipline will build systems that can account for their own behavior. In practice, that is what accountable AI requires.

Nikita Golovko is a software and solution architect specializing in AI architectures for real industrial systems. With a PhD in machine learning focused on industrial control and optimization—an early form of what is now called Physical AI—he combines deep academic grounding with hands-on experience building AI solutions for metallurgical equipment and other production environments.