One of the peculiar things about computer programs is that they can keep doing their jobs long after everyone has forgotten why they do them that way. Usually this is considered a success. The checks clear, the employees get paid, and nobody has to think very hard about what is happening inside the computer. This arrangement can last for decades, which is impressive until somebody has to change something.
COBOL is an obvious example. The language was developed for business computing in 1959, and it is still used in systems that move money, maintain records, and keep large organizations functioning. The usual story is about old programmers retiring and taking the last knowledge of COBOL with them. We imagine a room full of mainframes, a fellow who has been there since the Carter administration, and a telephone number somebody had better not lose.
The skills problem is real, but I think that story misses the more important one. COBOL is a programming language. We can teach people to read it, build tools to analyze it, and translate it into other languages. Finding someone who knows how the program works is a different problem from finding someone who knows why it works that way.
The Rule Nobody Remembers
Consider an entirely hypothetical payroll system. It calculates pay, overtime, leave, and deductions. Somewhere in the program is a special rule for a small group of employees. If an employee meets certain conditions, their overtime is calculated differently for one pay period near the end of the year. Nobody in payroll knows why. The exception has been there for forty years, and changing it would be a nuisance, so everybody leaves it alone.
Perhaps a labor agreement established the exception, but nobody removed it when the agreement expired. Perhaps it accommodated an older accounting system. Perhaps a court ordered a particular treatment for a particular class of employees. It might even have been a bug that produced an incorrect result in 1987, after which someone introduced a compensating calculation elsewhere. Each explanation is consistent with the code. Only some justify keeping the rule.
A competent programmer can trace the conditions and explain the calculation. That is useful, but it does not establish which explanation is correct. The programmer has reconstructed what the software does, not what the institution intended. And until we know what the institution intended, we cannot decide whether preserving the behavior is a requirement or a mistake.
This distinction becomes important during modernization. A typical effort documents the current behavior and builds tests so a replacement will produce the same results. Those tests are essential. Nobody wants to discover after a payroll conversion that everyone was underpaid because a rounding rule changed. But there is a catch: for the cases it covers, a regression test tells us whether the new system behaves like the old one. It does not tell us whether the old behavior is correct.
Suppose the original rule was a workaround for a reporting limitation that disappeared twenty years ago. A perfectly faithful replacement preserves that workaround. We might spend a great deal of money moving the calculation from COBOL to Java without improving the underlying process. We have a newer computer program that makes precisely the same old decision for precisely the same forgotten reason.
Software Is Not Institutional Memory
This is not a hypothetical concern about the age of the language. In July 2025, the Government Accountability Office identified eleven federal computer systems most in need of modernization. Eight used older programming languages, including COBOL and assembly language. Eight lacked fully documented modernization plans. These systems perform functions the government cannot simply stop doing, which makes replacing them risky.
The problem also exists outside the code. In August 2026, GAO’s own inspector general reported that GAO had paid about a million dollars in 2021 for an IT modernization plan, then changed to an ongoing modernization strategy less than eighteen months after the plan was finalized. The decision was not documented, and subsequent officials were not fully aware of the reasoning behind it. There is no need to blame COBOL for that one. The institution forgot why it changed its plan before the modernization effort was finished.
The program and the plan have the same weakness. Organizations make decisions, and those decisions eventually become procedures. Some are written into software. Others become policies, forms, checklists, budget arrangements, or practices that everybody follows because everybody else follows them. The procedure survives its author. Without the reasons, the next person inherits a rule they cannot evaluate.
That creates a peculiar kind of risk. Systems that fail obviously receive attention. Systems that run correctly, or at least run consistently, can carry an obsolete decision for years without anyone noticing. An exception may affect only a few cases, so its outputs do not look unusual. Users adjust to it, auditors learn to expect it, and a mistake can become normal behavior. Longevity makes the rule familiar, but familiarity is not evidence that the rule remains appropriate.
What AI Can Recover
Artificial intelligence seems particularly well suited to this problem. Large language models can read unfamiliar code, describe complicated branches, identify repeated calculations, and help construct tests. IBM’s discussion of COBOL modernization includes code understanding, refactoring, translation, and the use of AI to help document existing applications. These are useful capabilities, especially when the application is large and the original developers are no longer around.
But we need to distinguish two kinds of knowledge. The software contains evidence of how a calculation was implemented. It may also contain comments, identifiers, and related documents that help explain the reasoning. A model can examine those materials and offer a plausible account. It cannot recover an undocumented meeting or an assumption the original programmer never wrote down. It might guess correctly. The guess is still a guess.
Worse, a convincing explanation may be harder to challenge than an obvious lack of information. If an AI says a mysterious payroll exception reflects a statutory requirement, the development team might preserve it out of caution. If it says the same exception is unnecessary, they might remove it. Both decisions require evidence from the institution, not merely a persuasive account of the code. This is a question of authority as much as software. The model can identify a decision, but explaining programs does not give it authority to make policy.
So modernization has to include more than code translation. We need to identify the rules the system implements, decide which remain valid, and record the reasoning for those we keep. We should preserve the source of a requirement, whether that is a statute, an agreement, a policy decision, or a documented operational constraint. Where the original reasoning is gone, we should record that uncertainty instead of quietly inventing a history for it. Policy owners need to decide what the replacement should do before the programmers reproduce what the old program happens to do.
None of this is especially glamorous. It involves interviews, old documents, test cases, and people from different offices sitting in the same room long enough to agree on what a rule means. The code may accurately record the implementation without recording the policy behind it.
The last person who knows COBOL can be replaced by another programmer. The last person who knows why the rule exists is a more serious problem. If that person is still around, we should probably talk to them before we buy the new computer.