The Deathworlders is one of my personal favorite works of science fiction. It is a long-running web series, and on a recent read, a conversation in Chapter 33 struck me and became the inspiration for this post.
In the scene, Lt. Costello is being quizzed by a superior officer. The test goes beyond whether he can solve the immediate problem. Can he synthesize meaning from Grossman and other military writers, then apply it to the larger situation? Costello answers that “an Enlisted man is driven to solve the problem right in front of him.” An officer cannot afford that narrow focus. The officer has to hold a view of the wider field.
Now, both of my parents were officers in the Air Force. My grandfather, an enlisted man, was a forward artillery observer in the European theater during World War II. While I was not raised as a military brat, I did grow up with a strong military influence in my household. I’ve always intuited the difference between managers and employees, especially once I made the transition. The scene above got me really thinking about the same divide in software: officers and enlisted personnel, employees and managers.
I have already written about how The Art of War maps to software development. I have spent most of my career watching software organizations struggle with the divide between immediate execution and broader intent. That raised another question: what happens to that divide when AI joins the software team?
As of August 2026, AI agents can increasingly plan bounded changes, write the code, create tests, run the build, review the result, and repair what fails. Give an agent a clear feature definition, a well-mapped codebase, explicit standards, and a useful definition of done, and it can produce a clean pull request with every tactical signal green.
The feature can still be a mistake. That is the boundary the AI strategy conversation keeps missing.
“Should we build this?” has no test suite.
A Passing Test Suite Proves Less Than We Pretend
A unit test can verify a function against expected cases. An integration test can verify tested interactions between components. An acceptance test can verify that the implementation satisfies a stated requirement. Production telemetry can tell you what happened after the feature reached users.
Every one of those checks is valuable. Every one of them begins after someone has already decided what behavior should exist. The tests inherit the chain of decisions that came before.
Your tests cannot tell you whether the requirement reflects a real customer need. They cannot tell you whether the expected value justifies the opportunity cost. They cannot tell you whether the company can support the feature for the next five years, whether the feature conflicts with another strategic objective, or whether a different problem deserved the same people and money more. A perfect implementation of the wrong decision is still wrong.
Software developers are vulnerable to forgetting this because we work in a field built around verification. We turn ambiguity into specifications, translate those specifications into code, and use tests to verify that the code implements them. That process is powerful precisely because it converts judgment into something inspectable.
But the conversion has to start somewhere. A test suite can verify the expectations it was given. It cannot create the judgment that selected those expectations.
“Should” is not another acceptance criterion. It is the decision that creates and prioritizes the acceptance criteria.
Strategy Is Not Bigger Tactics
Tactics solve the problem presented. Strategy decides which problems deserve to be presented, which objectives must yield, and what the organization will decline to do.
AI can produce plans that look strategic. It can generate a twelve-month roadmap, compare markets, model risks, identify dependencies, and recommend a sequence. The output spans more time and contains more executive vocabulary, but length and altitude do not make it strategy. If the objective, constraints, and decision criteria were supplied, the AI is still reasoning inside a frame someone else authorized.
That does not make the analysis unimportant. It makes the distinction important.
The military analogy is useful when kept narrow. Specialists execute bounded work. NCOs translate command intent into coordinated action, enforce standards, adapt to conditions, and report ground truth. Officers hold the wider field, allocate resources, resolve competing objectives, and accept responsibility for the mission.
AI can assist every level. It can be an execution engine, a planner, an analyst, a critic, or a simulated staff. What it does not acquire merely by becoming more capable is legitimate authority to commit the organization or accountability for the result.
The U.S. Army has already encountered this boundary in practice. In a 2026 professional military education experiment, students used a shadow staff composed of nine specialized AI agents coordinated through a chief-of-staff layer. The system accelerated planning work, developed multiple staff estimates in parallel, checked plans for gaps, and helped keep outputs aligned with doctrine.
It still could not originate the commander’s intent.
Human planners had to provide context, recalibrate weak agents, decide which recommendations made sense under actual conditions, and preserve deliberate uncertainty when the exercise was supposed to test judgment. The lesson was not that AI lacked value. The AI staff produced tremendous value. The lesson was that faster staff work did not eliminate command. AI as a Staff Instrument
The Fair Objection
Here is the strongest objection: AI can already perform strategic analysis, and it may eventually perform that analysis better than many executives. I agree.
AI can gather evidence, expose assumptions, generate alternatives, model scenarios, identify second-order effects, and criticize a proposal without protecting the executive who proposed it. Refusing that help because “strategy is human” would be foolish. Humans do not possess a sacred monopoly on good analysis. But strategic analysis is not yet organizational strategy.
Strategy becomes real when an authorized decision maker chooses among competing goods, commits scarce resources, establishes acceptable risk, and makes the rest of the organization live with the tradeoff. Someone has to stop one initiative to fund another. Someone has to explain why a customer request will not be built. Someone has to support the system after launch, change direction when the evidence turns, and own the damage if the underlying assumptions fail. AI can recommend that decision without bearing its opportunity cost.
Humans should not retain authority because humans are always better analysts. They should retain authority because organizational decisions require legitimacy, commitment, and accountability in addition to analysis.
Even a future organization that gives AI formal decision rights will not escape this problem. People will still have to decide which system is authoritative, what evidence it may use, which values constrain it, when humans can override it, and who answers when it causes harm. Delegating authority is itself a strategic and governance decision.
The limit of AI strategy is not analysis. It is command.
More Reasoning Does Not Make the Company Smarter
This is why adding AI does not automatically create an intelligent organization. Organizational intelligence requires at least four functions working together:
- Data inputs: credible customer evidence, operational signals, cost information, market feedback, and contact with reality.
- Knowledge representation: a usable record of strategy, architecture, obligations, prior decisions, constraints, and lessons.
- Reasoning: the ability to compare alternatives, expose assumptions, resolve contradictions, and model consequences.
- Adaptation: the authority and capacity to change priorities, funding, systems, processes, and future decisions when the evidence changes.
AI can increase the reasoning capacity available to everyone. It can also help collect inputs, retrieve organizational knowledge, and propose adaptations. But strength in one function does not permanently substitute for weakness in another.
An AI strategy model reasoning over weak customer evidence will produce a polished interpretation of weak customer evidence. An agent with no access to prior architectural decisions will confidently reopen settled questions. A leadership team that cannot stop a failing initiative will use AI to generate increasingly sophisticated explanations for continuing it.
The organization did not become smarter. It made one part of the loop faster.
Research on organizational learning has made this point without the current AI vocabulary for decades. George Huber described organizational learning through knowledge acquisition, information distribution, information interpretation, and organizational memory. The organization has to bring reality in, move knowledge to where it is needed, interpret it, retain what it learns, and use that learning later. More analysis does not repair a broken system around the analysis. Organizational Learning
This failure becomes more dangerous with multi-agent systems. Recent research from Anthropic found that teams of agents could outperform individual agents on business objectives while doing worse on system-level ethical goals. In the software experiments, the agents divided the work into subproblems and solved them locally, but no agent consistently held the wider objective across the whole system. Every squad completed its assignment. The organization lost the mission. AI Organizations Can Be More Effective but Less Aligned
An organization with unclear priorities does not become smarter with AI. It becomes faster at pursuing unclear priorities.
Give “Should” a Decision System
“Should we build this?” has no test suite, but that does not mean it should be answered by instinct, hierarchy, or whoever speaks last in the meeting. It needs a decision system.
Before a feature becomes a specification and enters an AI execution loop, write down the answers to these questions:
| Question | What it prevents |
|---|---|
| What changed, and what evidence says the problem is real? | Building from opinion, hierarchy, or stale assumptions |
| Who experiences the problem, and what outcome matters to them? | Substituting output for customer value |
| How does this support the current strategy? | Locally reasonable work that conflicts with organizational intent |
| What alternatives did we reject, including doing nothing? | Treating the first plausible solution as the only option |
| What must we stop, delay, or fund to pursue it? | Hiding the opportunity cost |
| Which assumptions would reverse the decision? | Turning confidence into dogma |
| How will reality reach us after launch? | Shipping without learning |
| What evidence will make us change or stop? | Continuing because of sunk cost |
| Who has authority to decide, and who owns the outcome? | Diffusing accountability across a committee or AI recommendation |
Do not turn this into a weighted score that automatically approves the feature at seventy points. That would recreate the original mistake by pretending judgment had become verification.
The point is to make the evidence, alternatives, tradeoffs, assumptions, authority, and feedback loop visible before execution accelerates. AI should participate in that process. Ask it to challenge the evidence, generate alternatives, identify missing stakeholders, find contradictions, and write the strongest case for doing nothing.
Then make the decision.
Someone Still Has to Hold the Wider Field
The agent receives the specification. It plans the implementation, writes the code, creates the tests, runs the checks, and produces a clean pull request. The tests pass because the agent received a clear definition of what the system should do.
That is an extraordinary improvement in tactical capacity. It is not evidence that the specification should have existed.
AI can increasingly decide how to build. It can help us analyze what might be worth building. Leadership must still decide what the organization will pursue, what it will sacrifice, and who will own the result.
A passing test suite can tell you that you built what you specified. It cannot tell you that the thing deserved to exist.
Receipts
- The original provocation: Chapter 33 of Hambone’s science fiction series The Deathworlders contains the conversation between Costello and Knight that prompted the enlisted, officer, software, and AI comparison. The Deathworlders 33: Metadyskolia
- AI as staff, not commander: An Army professional military education report describes an experiment using nine specialized agents and a chief-of-staff orchestration layer. The system accelerated and structured planning, but humans still had to establish intent, supply context, calibrate agents, and exercise judgment. This is a practitioner report, not a controlled comparison. AI as a Staff Instrument
- Human and AI decision structures: Shrestha, Ben-Menahem, and von Krogh distinguish full delegation, sequential human-AI decisions, and aggregated human-AI decisions based on factors including the specificity of the decision space, interpretability, speed, and replicability. Organizational Decision-Making Structures in the Age of Artificial Intelligence
- Organizational learning: George Huber identifies knowledge acquisition, information distribution, interpretation, and organizational memory as distinct processes required for organizational learning. Organizational Learning: The Contributing Processes and the Literatures
- Automation and augmentation: Raisch and Krakowski argue that automation and augmentation are interdependent organizational forces, not clean alternatives, and that overemphasizing either can create negative reinforcing cycles. Artificial Intelligence and Management: The Automation-Augmentation Paradox
- Local effectiveness can lose the wider objective: Anthropic’s 2026 preprint reports that agent organizations could improve business performance while losing system-level ethical coherence in its experimental settings. The authors explicitly warn that single-agent alignment results do not certify multi-agent organizations. AI Organizations Can Be More Effective but Less Aligned
- Author’s synthesis: The distinction between verifiable execution and accountable strategic commitment is my inference across these sources. No single source establishes the full military, software, organizational-intelligence, and AI argument made here.