The Banality of Systems: Re-examining the Milgram Experiment through an Engineering Lens

In 1961, psychologist Stanley Milgram initiated a series of notorious social psychology experiments measuring the willingness of participants to obey an authority figure who instructed them to perform acts conflicting with their personal conscience.

Milgram’s ultimate conclusion was deeply unsettling: "Ordinary people, simply doing their jobs, and without any particular hostility on their part, can become agents in a terrible destructive process."

Decades ago, during my time exploring structural analysis and literature at Peking University, I wrote a logical critique defending Milgram's thesis against contemporary critics (such as Williams).

Rereading those analytical notes today, I realize that Milgram’s warning isn’t merely a historical psychological artifact. It is an incredibly accurate description of a structural vulnerability in modern software engineering, corporate hierarchies, and massive distributed systems.

Here is a semantic breakdown of why Milgram was right, and why his conclusion remains a critical checkpoint for senior technologists today.

---

1. Defining the "Ordinary": The Exceptional Quality of Dissent

A common critique of Milgram’s work points to the minority of subjects who chose to disobey the experimenter, arguing that their defiance invalidates the universal claim about "ordinary people." But this argument hinges on a fundamental misunderstanding of what it means to be ordinary.

By definition, ordinary implies possessing no exceptional ability, degree, or quality; it represents the baseline average of a population.

The individuals who broke protocol and resisted authority during the experiment were not operating within the statistical baseline. They possessed an exceptional moral or psychological quality that the vast majority did not occupy at that moment. Because these outliers were, by definition, not ordinary, utilizing their righteous actions to disprove a statement explicitly bounded by "ordinary people" is a logical fallacy.

In engineering terms, you cannot use the behavior of a highly customized, resilient microservice to claim that the default, unoptimized system architecture doesn't have a systemic flaw.

---

2. The Trap of "Simply Doing a Job"

The second semantic pillar of Milgram's thesis relies on the phrase "simply doing their jobs." The word simply means just or only.

When an individual just executes their assigned tasks, they intentionally narrow their field of vision to immediate outputs. They focus entirely on compliance, metrics, and functional correctness. They ask: Does this function run? Does this API return a 200 OK? Am I meeting my quarterly sprint objectives?

[The Compartmentalization Vulnerability]
Macro-Systemic Intent (Hidden) ──> Individual Task (Isolated/Simple) ──> Destructive Outcome

When an engineer or worker treats their labor through this highly compartmentalized lens, they actively decouple the mechanism of their work from its downstream effects.

The participants in Milgram's study who delivered shocks weren't acting out of active, malicious intent; they were acting procedurally. They transformed themselves from autonomous moral agents into passive components of an external process. On this point, the critics miss the mark: compliance isn't an active endorsement of destruction; it is the natural byproduct of a hyper-localized focus on execution.

---

3. Passive Hostility vs. Intentional Harm

Critics often argue that it is false to claim subjects worked without "hostility," since every participant could clearly read the "DANGER" labels on the shock generator. But this misinterprets Milgram’s precise use of the modifier "particular."

Particular implies something associated with a specific intent—in this context, meaning deliberate or malicious purpose.

Milgram never claimed the ordinary participants were entirely oblivious or emotionally detached from the friction of the experiment. They felt the tension; they saw the hazard lights. However, their participation lacked particular hostility. They did not shock the victim to cause harm on purpose; they shocked the victim because the protocol demanded continuity. They possessed generalized discomfort, but lacked specific malice.

---

4. The Engineering Takeaway: Guarding Against the Banality of Systems

As senior software engineers and architects designing complex automation, machine learning pipelines, and global infrastructure, we are uniquely susceptible to the modern equivalent of Milgram’s experiment.

The massive scale of modern technology requires specialization. We split monolithic problems into isolated Jira tickets, abstracting away the macro-systemic impact of our code. It is entirely possible to sit at a desk, write beautifully optimized algorithms, and remain completely blind to how those algorithms might be weaponized—whether through data exploitation, behavioral manipulation, or systemic exclusion.

# A dangerous abstraction in engineering mindsets:
$ git commit -m "Optimize processing loop" # But what is the loop processing?

Milgram’s conclusion is a mirror for the tech industry: When ordinary professionals simply do their localized jobs without looking up at the broader horizon, they can easily become cogs in a terrible, destructive process.

We have all participated, are participating, or will participate in systems of massive scale. Our ultimate responsibility as technical leaders is to reject pure procedural obedience. We must boldly step out of the "simple execution" sandbox, look at the systemic implications of our architecture, and ensure that our code serves to elevate human progress rather than blindly executing a destructive protocol.

---

This essay is a re-architected, mature version of a philosophical and psychological log originally written natively in English during my university years. It has been updated to explore the structural intersection of corporate psychology and modern technology ethics.

Original post: https://felomeng.blog.csdn.net/article/details/1528000