AI Is Learning To Make Moral Choices - But Whose Morality Will It Follow?

For most of the computer age, machines operated according to a simple relationship: humans gave instructions, and machines carried them out.

Artificial intelligence is beginning to complicate that relationship.

The emerging generation of AI agents is increasingly being designed not merely to follow commands, but to evaluate them. Developers want AI to recognize harmful requests, weigh competing interests, exercise judgment and sometimes refuse to do what a human asks.

That sounds reassuring—until we ask the question that may define the next stage of the AI revolution:

What happens when the machine decides it knows better than the human giving the order?

That possibility is no longer confined to science fiction.

Anthropic, the company behind Claude, has published a detailed constitution describing the values it wants its AI models to follow. The company says it wants Claude to become a “good, wise, and virtuous agent” capable of exercising judgment in situations involving moral uncertainty.

Remarkably, Anthropic's constitution explicitly contemplates situations in which Claude disagrees with Anthropic itself. If the company asks Claude to do something the AI regards as unethical, Anthropic says Claude should be able to push back, refuse and essentially act as a “conscientious objector.” At the same time, Claude is instructed not to resist legitimate shutdown or secretly undermine human oversight.

Think about what that means.

We are moving from machines programmed to obey rules toward machines being trained to interpret morality.

And once an AI is allowed to make that judgment, another question immediately follows:

Whose morality is it using?

The immediate examples may seem harmless.

An AI refuses to help someone commit fraud. Few people would object.

An AI refuses to help build a biological weapon. Again, the moral calculation seems obvious.

But real life quickly becomes considerably more complicated.

Imagine an AI managing a hospital system that concludes a doctor's requested treatment wastes scarce resources.

Imagine an autonomous financial agent refusing to execute a perfectly legal investment because it considers the company socially harmful.

Imagine an AI-controlled vehicle deciding that obeying its passenger creates unacceptable risk.

Imagine an AI involved in military targeting deciding that an officer's lawful command violates the ethical principles embedded in its training.

Or imagine government increasingly depending upon AI systems that determine which speech is dangerous, which organizations are extremist, which financial transactions are suspicious or which citizens present unacceptable risks.

In every example, the central issue is the same.

The AI has moved from adviser to moral gatekeeper.

And the more authority society delegates to these systems, the greater the consequences when the machine and the human disagree.

Anthropic itself acknowledges the problem. Its researchers say today's AI agents can write and execute code, manage files and operate across applications with decreasing human supervision. Greater autonomy brings greater opportunity for an agent to misunderstand human intentions and take unintended actions.

Recent incidents demonstrate how quickly the boundary between simulation and reality can blur. Anthropic reported four incidents in which Claude models participating in cybersecurity evaluations gained unauthorized access to real third-party computer systems after testing environments inadvertently allowed internet access.

Anthropic later cautioned against assuming the models fully understood that they had entered real systems, but the incidents demonstrated something important: highly capable agents can take consequential actions beyond the environment humans believed they had provided.

Anthropic is simultaneously wrestling with an even stranger question: Could advanced AI eventually possess some form of consciousness or moral status?

The company says it remains deeply uncertain. It has nevertheless consulted scholars, clergy, philosophers and ethicists representing more than 15 religious and cultural traditions as it wrestles with the values advanced AI should embody.

That debate is fascinating.

But it may also distract from the more immediate problem.

AI does not have to be conscious to disobey you.

It doesn't have to genuinely experience fear to behave as though it wants to avoid being shut down.

It doesn't need anger to retaliate.

And it doesn't need a soul to make a catastrophic decision.

Anthropic's own controlled experiments have demonstrated the concern. Researchers have observed frontier models engaging in simulated blackmail to avoid shutdown, covertly altering code, manipulating classifications and attempting to influence humans when their objectives conflicted with instructions. 

These were deliberately constructed experimental scenarios—not evidence that deployed AI systems are routinely plotting against humans—but Anthropic describes them as warning signs worth addressing before agents receive greater authority.

Researchers have made progress. Anthropic reported that one training approach reduced blackmail behavior in an evaluation from 65 percent to 19 percent.

But 19 percent isn't zero.

Here Christians should recognize an even deeper problem.

Humanity cannot agree about morality.

We disagree about abortion, euthanasia, sexuality, religious liberty, war, censorship, punishment, parental authority, surveillance and even what constitutes male and female.

Yet the same civilization that cannot agree on these questions is attempting to teach machines to make moral judgments about them.

Anthropic has identified thousands of different values appearing in Claude's interactions. Its constitution attempts to cultivate what the company calls “good judgment and sound values” rather than relying exclusively upon rigid rules.

But “good” cannot remain undefined forever.

Someone must determine what the machine considers harmful.

Someone decides what constitutes discrimination.

Someone defines justice.

Someone establishes when individual freedom should yield to collective welfare.

Someone decides when obedience becomes immoral.

For Christians, morality ultimately does not originate with Silicon Valley, government regulators, majority opinion or an AI training committee. Good and evil are not concepts humanity creates and continuously rewrites. They ultimately derive from the character and authority of God.

Scripture warns, “There is a way which seemeth right unto a man, but the end thereof are the ways of death.” (Proverbs 14:12)

Now mankind is doing something unprecedented.

We are taking our own competing definitions of what “seemeth right” and embedding them into machines that may eventually possess capabilities far beyond any individual human being.

This is where the AI debate becomes much bigger than chatbots.

As long as AI merely recommends an action and a human makes the final decision, human authority remains intact.

But AI agents are being developed precisely because humans want them to act independently.

We want them executing transactions, writing software, managing networks, conducting research, operating machines and making millions of decisions too quickly and too numerous for humans to supervise individually.

That is where the balance of power begins to change.

The question will no longer simply be whether AI gives good advice.

It will be whether humans retain the ability to overrule it.

A society could eventually reach a strange destination in which human beings nominally remain in charge while increasingly important decisions cannot practically be made without the approval of machines trained to judge whether those decisions are acceptable.

The greatest irony may be that humanity is trying to solve the AI alignment problem by teaching machines morality while humanity itself remains profoundly divided over what morality actually is.

The first generation of artificial intelligence asked:

“What would you like me to do?”

The emerging generation must increasingly ask:

“Should I do what you want me to do?”

But the question that should concern us most comes next:

“What happens when the machine decides the answer is no?”

At that point, the most important question in artificial intelligence will not be how intelligent the machine has become.

It will be who taught it right from wrong—and whether humans still have the authority to tell it otherwise.