A Central Fallacy in Robot Ethics

Funny white robot stay with pen and documentFifteen years ago a team of archaeologists uncovered a strange set of artefacts while on a routine dig in a remote area of China. Their find consisted of thousands of incredibly thin “metallic” tablets, each inscribed with strange markings, all of them stored within a larger “metallic” case. One of the tablets, which the archaeologists dubbed Rosetta II included ancient Greek alongside the strange markings. Carbon dating suggested that each of the tablets was produced during a radically different period in history, with the earliest tablet and storage container suggesting a date on the order of billions of years ago. Naturally the carbon dating was deemed enigmatically inconclusive.

Until recently, that is, when researchers finally uncovered some of the meaning behind the tablets.

Rosetta II, they have told reporters, became the focus of intense research after the find, the hope being that the Greek markings would be the key to translating the unfamiliar markings inscribed on all the tablets. After nearly ten years of study, researchers are now confident that they have succeeded in deciphering the text. Rosetta II was indeed the key to translation.

Incredibly, the tablets appear to belong to an extra-terrestrial researcher of sorts. Our earthly researchers are describing the tablets as field notes, similar to those that were scribbled down by Darwin on his trips to the Galapagos Islands so many years ago. These alien field notes appear to describe life on earth from its earliest days. Indeed, our earthly researchers are convinced that some of the earliest tablets refer to single-celled life in the primordial soup, others more recent refer to the dinosaurs, while the most recent notes refer to humans living roughly in the middle ages. Though the identity of the alien researcher remains a mystery, its notes suggest that it has been visiting our planet off and on for a very long time.

One of the most troubling aspects of the translations is the way in which our alien visitor appears to describe life on Earth. It seems that, owing to the nature of the first life it observed—single-celled organisms—our visiting researcher concluded that those organisms were mere machines. Its field notes contain lengthy descriptions of DNA and biology, and though our earthly researchers are still trying to piece together the full meaning of the text, it seems that the alien researcher was able to deduce various formulae and models describing precisely how life on Earth works. To put it in more familiar terms, our alien visitor appears to have reverse engineered the “program” that is organic life on Earth. It seems that because of this fundamental understanding of the mechanics of life, our alien observer thought of us as mere self-replicating machines, as programs running via “wetware”, in other words our alien observer was convinced that we were fundamentally different that it. In several notes our alien observer mentions the “delusion” (this is our best translation) that earthly machines suffer from, namely that we “act as if non-automata”, “express beliefs, desires, and understanding, though none are possible due to [our] crude physicality”. In short, our alien visitor denied the possibility of our inner mental lives precisely because of the simplicity of our constituent building blocks.

Earthly researchers urgently contacted media outlets today to reveal their find, indicating that they had finished translating the final tablet. In the press conference they cobbled together they presented reporters with images of the tablet along with bubble quotes pointing to specific passages of interest. Those passages refer to dates in the not-too-distant future. Though our earthly researchers did not provide the specific dates to reporters, they assure us they are known. The alien observer, it seems, refers also to a “harvest” event, though the specific meaning is unclear.

Based on translations from other tablets, earthly researchers are convinced that the alien plans to return, most likely with others of its kind, for the “harvest” event. They are also convinced that it is our apparent lack of consciousness that the alien observer has used as the primary justification behind initiating the “harvest” event. There is clear evidence the aliens have a well-defined ethics surrounding “harvesting”.

With all this in mind researchers are asking the media to appeal to human kind to come up with a way to convince the aliens, upon their imminent return, that we are not mere machines, that we are conscious and have inner mental lives worthy of preservation according to alien ethics which, coincidentally, seem almost identical to ours. The task is urgent. Our continued existence depends on it.


This thought experiment is meant to describe a fallacy that is commonly encountered when discussing technology. The fallacy is committed whenever someone points to a piece of technology that most likely has no consciousness, say a mobile phone or other computer, and concludes that any future technology derived from similar constituent building blocks is therefore not capable of consciousness.

In ordinary conversational contexts committing this fallacy would seem entirely unproblematic. Indeed, it would hardly qualify as a fallacy. After all, we are surrounded by technology that is relatively mechanistic compared to the complex physical systems we call life. Even our most sophisticated machines today seem to deserve being labeled mere machines.

In philosophy, robot ethics in particular, the question whether or not technology based on the crude building blocks we have assembled today will ever be capable of consciousness is wide open. It is, in fact, one of the central questions of the field. To commit the fallacy is to pretend there is an answer to that question, where no such answer actually exists. Putting it differently, when discussing future robots the fallacy amounts to a form of question begging.

John Searle’s widely cited thought experiment, the Chinese Room, is often used to support the conclusion that robots based on today’s technological building blocks cannot have consciousness. Less often discussed is precisely what Searle’s argument does, and does not, do. Searle makes two central claims in his argument. First, he claims that the Chinese Room example demonstrates that mere symbol manipulation is not enough to give rise to human like understanding (consciousness). Since computer programs are symbol manipulation systems, he concludes that artificial intelligence (AI) based on computer programs cannot be conscious. Searle’s second claim is that only machines with very special characteristics can give rise to human like consciousness, namely machines with the same causal powers that give rise to it in human brains. What are those causal powers? Searle cannot answer that question. To date, no one can.

What Searle provides is an argument why computer programs might not give rise to consciousness. He absolutely does not provide a proof to that effect, unless one is willing to redefine ‘proof’. He also points to an unexplained characteristic of the human brain in an attempt to explain consciousness. However, since he cannot demonstrate that today’s technology does not possess the causal powers required for consciousness, he leaves open the possibility that future technology based on today’s constituent building blocks might possess those causal powers. He definitely leaves open the possibility that some future technology could possess the same causal powers of the human brain. Consciousness for Searle is not magic; we are physical systems, a trait we share with machines.

Thus, Searle leaves wide open the possibility that machines could one day be conscious. In Searle’s own words, “only a machine could think, and indeed only very special kinds of machines, namely brains and machines that had the same causal powers as brains”.[1]

Like our alien researcher we look to technology with an understanding of how it came to be, how it works, and what it is made of down to the tiniest bits and pieces. We can manufacture it, reverse engineer it and explain it away as mere machinery. Like our alien researcher we are also prone to drawing unsupported conclusions based on those abilities.

But we have good reason to avoid fallacies of this sort in robot ethics. The question of machine consciousness is open, and it should remain so in our technological speculations. Where consciousness is concerned the moral stakes are high. Today’s mobile phones and driverless cars might be to tomorrow’s technology what single-celled organisms of the past are to humans. So long as we give up on magic, degrees of complexity seem in this world to make a difference in the properties we must consider belonging to a thing. We are complex collections of single celled organisms, yet we claim to be conscious. If we follow in the alien’s footsteps and, based on our understanding of the basic building blocks falsely conclude that some future machine cannot be conscious, we run the risk of paving the way for a “harvest event” of our own making where none is justified.

How could we go about convincing the aliens that we are indeed conscious, thus avoiding a “harvest” event? That is a difficult question, one that deserves careful attention. One thing is certain, their bias would complicate things immeasurably.

[1] Searle, J. (1980). “Minds, Brains, and Programs.” Behavioural and Brain Sciences 3. Original emphasis.


2 thoughts on “A Central Fallacy in Robot Ethics

  1. Good job Jason, but I assume this is a start of a much longer discourse? As per my FB post “Interesting view, I cannot help with the direction you are going we’ll end up in the morals versus ethics argument. So let me jump there and postulate whether it is actually our moral compass that would preclude us from ever accepting an ethical view of a robot race. Damn, I dropped the race card… (will continue on your blog)” if I read your preamble and apply the ending, is what you are asking really whether we can appeal to the morals of the alien; e.g. convince them that the ‘harvest’ is wrong and rather than a full cull based on ‘data/observations’ that they now have an ethical reason to change the rule(s) (either for the criteria of the harvest or for the specific harvest impacting mankind) you have spun in the opening.

    To further dance on my soap box (and I am admittedly far from an expert) can we assume that for mankind to properly advance a discussion on robot ethics that first from a moral standpoint we need to (re)ground our basic principals on what an autonomous life form means in terms of a ‘life’ means? Are we asking that to have a new acceptance that life isn’t necessarily defined as cleanly as we know it and in some cases in then raptures of our God complex can actually create life from sticks and stones! Even if we intended something far inferior?

    The challenge with all of this mental, moral, ethical change IHMO is the requirement of a societal shift in thinking, education and acceptance that would then have to mirror robot evolution. Alas you and I can probably agree this will lag at best. So if it lags the evolution of a robotic mind/persona will robots been seen as sub-anthropomorphic and their conscience just a gift of our hand not an imbedded truism/gift. As such it is ours to own, judge and do with as we please outside of the morals or ethics we’d practice on another ‘living’ thing?

    Just food for thought, more where that came from that we can share when you return from the conference.

    1. Joel, you’ve captured the main questions surrounding the thought experiment nicely! I do intend the thought experiment to raise questions about how we would hope to be able to persuade the aliens. I also hope to use it to get people to think about how we might find ourselves in the same epistemological position re robots sometime in the (admittedly distant) future.

Leave a comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s