Do you think that it's possible for a machine to have evil intentions?

[Boojatta]

I'm not talking about any existing machine, but machines of a hypothetical future time when there is artificial intelligence.

Now, let's move on to another topic. If you don't want to go out on a limb and express opinions on the question that I used as the title of this thread, then you have an alternative ...

Have you ever seen message board posts that claim that it is possible to explain the moral dimension of human behavior in terms of conventions?

I don't think that I've ever seen anybody outline a program of research for actually producing such an explanation, but I think that I've seen plenty of allegations that it is or will be possible to provide an explanation in terms of conventions.

However, let's think about the fact that it's easy to name a pair of languages (perhaps Hungarian and Basque is an example) that are quite different from each other. It would certainly be difficult to quantify these matters, but it seems that there is less variety among moral systems than there is among the languages of the world. Nevertheless, the search for language universals is a legitimate part of linguistics. Are there no moral universals? It seems more plausible to me that there are moral universals and that we are characterizing as "moral systems" some systems for which the adjective "moral" is inappropriate.

Of course, it would be awkward to tell somebody "I think that the moral system that you are dedicated to isn't actually moral", but it would also have been awkward to tell a Soviet biologist who taught Lysenkoism, that it wasn't science. That something would be awkward to say does not necessarily imply that it's false.

When we talk about languages, we are actually talking about conventions. When you formulate your thoughts in a given language, you are finding a way to express them within a given system of conventions. The realm of the moral goes beyond mere conventions, just as violent crimes and major white collar crimes cause far more trouble than the average grammatical error causes.

[WingDinger]

1. Morality is almost all convention.

Feticide, honor killings, sterilization, Forced ghetto housing, capital punishment, all are conventions.

RIGHTS, are not God given, even though that is the language the founders used. Nor do the ten commandments suffice, for morality in total.

Manners are tied to morality. They are also simply conventions. And cynical ones at that. Noone truly puts others first.

As for a machine doing evil, only as much as a lion is doing evil to the gazelle.

[LeftishBrit]

2. Most machines that I've dealt with have had evil intentions!

[WingDinger]

3. Get out of here finchley

[baldguy]

4. Yes.

Even people with evil intent don't consider themselves to be evil.

[cleanhippie]

5. Define evil.

When you say "evil", just what definition are you using?

[Boojatta]

8. Feel free to use whatever definition you like ...

if you have something interesting to say.

I think that the word "evil" names a basic concept that can be used in defining other concepts, and I'm not aware of any simple and clear way to define "evil" in terms of other concepts.

[cleanhippie]

9. Well, I would use the definition that it is the

Willful and intentional use of power and influence to oppress others for personal gain.

If machines had AI, and they had no programming controls, then I would say that yes, they would be capable of evil.

But "evil" can mean many different things to many different people. So I'm not sure my opinion would be universal across definitions.

[backscatter712]

23. Probably the best definition I've seen is Phil Zimbardo's...

"Evil is the exercise of power to intentionally harm, hurt, destroy or commit crimes against humanity."

He's the guy behind the Stanford Prison Experiment, who later wrote The Lucifer Effect.

[mia]

6. Reminds me of a conversation with Kohlberg and Chomsky.

This paper may provide some insights.
http://tuvalu.santafe.edu/~bowles/UniversalMoralGrammar.pdf

[Boojatta]

7. Could you quote a short excerpt that you consider relevant?

If you want to sell DU members on following your link, then you should do better than "may provide some insights."

After all, the only alternative to "may provide some insights" is "absolutely guaranteed to provide no insights" (unless you want to get picky about the word "insights" being plural).

[mia]

12. You mean you want ME to read the whole thing? Gasp!

Universal moral grammar: theory, evidence, and the future
John Mikail
http://tuvalu.santafe.edu/~bowles/UniversalMoralGrammar.pdf

My very short summary:

Social conventions that apply to moral decision making are evident across cultures (and languages).

The authors hypothesize that "humans possess an innate moral faculty" that is similar to the language faculty evident across cultures. Based on studies with young children, who seem to have a well-developed sense of social conventions, and the fact that "all natural languages” have words for concepts such as "permissible", "forbidden" and "obligation", the authors suggest that there may indeed be moral universals, especially as it applies to legal theory.

This article seems to be related to your question, but maybe not.
Your post interested me, nonetheless. I was once an avid student of subjects related to moral development.

[Jim__]

10. What is the basis for your claim that language is a matter of convention?

I could understand claiming that vocabulary is a matter of convention - even though there are restrictions. But, grammar is a critical part of language, and I am curious as to any evidence that grammar is convention and not based on specific brain structures - that is, that the realm of language does not go beyond mere conventions just as you claim the moral does.

[Boojatta]

11. I think that language does go beyond convention.

I merely intended to say that there's a heavy element of convention involved in language, as your example of vocabulary helps to show.

[ZombieHorde]

13. No, mostly because there is no such thing as an evil intention. nt

[Boojatta]

15. "There is no such thing as an evil intention"

How did you arrive at that conclusion? For example, if you were convinced that some radio broadcasts before and during the genocide in Rwanda were intended to incite people to commit genocide, and apparently did play a role in sparking and/or intensifying the genocide, then what would prevent you from saying that the broadcasts were made with evil intent?

I have heard of organized efforts to spread misinformation about history and deny that genocide has occurred. An alternative for those who play that game would be to try to persuade people that genocide is either morally neutral or a good thing. However, that sounds like a very difficult venture, and I haven't heard of it being pursued as a serious alternative to the denial game.

This train of thought leads me to suspect that I am simply misunderstanding you and that you are interpreting the words "evil intention" in some idiosyncratic and unusual way.

[ZombieHorde]

16. Genocide cannot be evil.

Evil is a reaction to perceived stimuli, similar to beauty and delicious.

If you write a book, and then I write a critique of your book, the book and the critique would be two different things. My critique would not be your book, but it may influence how others perceive your book. Genocide and evil are two different things, but the concept of evil may influence how people perceive genocide.

[Boojatta]

18. "Evil is a reaction to perceived stimuli, similar to beauty and delicious."

Last edited Wed Feb 1, 2012, 07:06 PM - Edit history (1)

That sounds like a somewhat nonstandard definition of the word "evil". In what sense is there a similarity where you claim "similar to"? It occurs to me that you might be using a definition of the word "similar" that is also, in my opinion, nonstandard.

Admittedly not having given the matter much thought, I don't see how it is possible for a moral evaluation to be anything other than moral neutrality or some degree of either goodness or evil. I don't see the concept of evil as something that influences the moral evaluation that people give to genocide. Instead, I see the concept of evil as something that helps define a structure, and I see the making of a moral evaluation as identifying the appropriate point (or small interval) within that structure. Perhaps I could make an analogy with a time scale. I'm pretty confident that agriculture was practiced many thousands of years ago. Thus, if you asked me a multiple choice question "when was agriculture first practiced on planet Earth?" and gave me only two options "BC" and "AD", then I would select "BC", but I don't think that I'm influenced by the concept of Christ's birth. However, it is true that "BC" and "AD" are labels for my options when I take the multiple choice test, and that my answer might be different if the concepts were redefined so that the beginning of the AD era got pushed back to include times far enough in the past.

Some people mistakenly claim that the number two isn't a prime number because all other primes are odd. I suppose that you could say that the concept of an odd number does influence them in their evaluation of the number two as non-prime, because it confuses them about what the phrase "prime number" actually means. If they had no familiarity with the concept of an odd number, then they would be less likely to become confused.

[ZombieHorde]

19. In the first example, I think the influencing factor is the concept of dates.

Humans decided the Earth orbiting around the Sun once was significant enough to label and use as a basis for saying when things happened. Humans then invented calendars based on this concept, and from calendars came the concept of dates.

You don't need to know what BC and AD stand for to know the dates they represent, but you do need to know they represent dates.

This can be a confusing example because humans planting seeds with the intention of growing food is a very real thing, and the Earth orbiting the Sun is a very real thing. However, genocide and evil are based on the abstract concepts of race and morality. The existence of heliocentricity and growing food are facts. The evidence for the existence of these two things is really strong, but the existence of race and inherent morality have yet to be proven. Both race and morality seem to be in the eye of the beholder.

My book and book critique example is also confusing for the same reasons, and I now regret using it. A better (but still not perfect) example may be the claim: desserts are delicious. Lots of people, myself included, would agree desserts are delicious, and lots of people, myself included, would agree genocide is evil, but neither of these claims are really true. Evil and delicious are not inherent qualities, they're reactions. Delicious is the interaction between a thing and a person's taste buds, and evil is the interaction between a thing or event and a person's moral code.


Is cake delicious? Depends on the person perceiving it.
Is pie delicious? Depends on the person perceiving it.
Is ice cream delicious? Depends on the person perceiving it.

Is abortion evil? Depends on the person perceiving it.
Is gay marriage evil? Depends on the person perceiving it.
Is country music music evil? Yes.
Is war evil? Depends on the person perceiving it.
Is Social Security evil? Depends on the person perceiving it.

(the country music bit was a joke)

[Boojatta]

20. In practice, you cannot travel to Mars and directly experience the temperature

but you can look at data people obtained by allowing thermometers to interact with the surface of Mars. Does it follow that there is no objectively real temperature on Mars?

Delicious is the interaction between a thing and a person's taste buds, and evil is the interaction between a thing or event and a person's moral code.

Suppose that there is some level of radiation that is lethal to human beings, but not high enough to harm cockroaches. You could say that such a level of radiation "isn't inherently hazardous" and that the problem it poses is merely the interaction between the radiation and human beings who are exposed to that level of radiation. In this case, your statement would be technically correct, but not very interesting to most human beings. To some, it might have positive comedic value, and others might consider it to be a joke in poor taste. I imagine a headline such as "stand-up comic unlikely to be invited back to Fukushima."

[ZombieHorde]

22. In your first example, I would say thermometers and temperatures are two different things.

Although I doubt Mars has a uniform amount of heat, I am sure a thermometer can objectively detect an amount of heat if sent to Mars.

Suppose that there is some level of radiation that is lethal to human beings, but not high enough to harm cockroaches. You could say that such a level of radiation "isn't inherently hazardous" and that the problem it poses is merely the interaction between the radiation and human beings who are exposed to that level of radiation.

I agree. "Such a level of radiation isn't inherently hazardous" and "such a level of radiation isn't inherently hazardous to me" are two different claims. Similarly, "genocide is evil" and "genocide is evil to me" are two different claims. One is a statement about genocide, and other is a statement about the observer.

In this case, your statement would be technically correct, but not very interesting to most human beings.

Since the statement doesn't involve sex, sports, violence, gossip, or TV, you are probably correct that the statement is not very interesting to most human beings.

To some, it might have positive comedic value, and others might consider it to be a joke in poor taste. I imagine a headline such as "stand-up comic unlikely to be invited back to Fukushima."

Ha! You're probably right about that.

[The Straight Story]

14. Relatively speaking, yes

If it performs something to someone/something that harms it in a way it feels is evil.

We could say America did evil by invading Iraq, others would say it was good.

[Angry Dragon]

17. Machines are the Devil's invention

[spin]

21. It is possible that in the future the military might have soldier machines...

that might be hunter killers.

If such machines were on your side of a war they would be called heroic robots. if on the other side they would be labeled as machines of evil.

AI Uses
AI Goes to War

As with many other fields of scientific study, the military has picked up on the use of Artificial Intelligence. The possibilities of military use of AI are boundless, exciting, intimidating, and frightening. While today's military robots are used mainly to find roadside bombs, search caves, and act as armed sentries, they have the potential to do so much more.

***snip***

According to the Pentagon, actual robotic soldiers powered by Artificial Intelligence will be a major fighting force in the American army, probably within the next decade. The first robot soldiers will actually be remote-controlled vehicles. The military has poured tens of billions of dollars into this project already. Congress wants to see this happen, and they ordered that a third of all military vehicles and deep-strike aircraft be automated by 2010.

As the machines begin to think, see, and react more like humans, the level of their autonomy and our level of trust in them will grow as well. However, it is predicted that a true soldier-simulating robot will not come about for another 30 years. These robots need to be able to determine friend from foe and enemy from bystander, and teaching them to do so will require a tremendous amount of research and work. The government has assured us however that these robotic soldiers will not be put into the field and allowed to make such decisions until they are ready to do so.

Another current infantry prototype knows how to recognize an enemy when it is under fire. When this happens, it can react to enemy fire on its own or follow orders given to it from a remote observer. Although it's programmed to work autonomously, in its present state, it still requires some set of outside monitoring controls in order for it to work. Its designers plan to have it usable for infantry missions by 2015.
http://library.thinkquest.org/05aug/01158/AIwar.html

[backscatter712]

24. I would say that yes, it is possible.

We're clearly not there yet - I think it'll be a couple more decades before our technology enables us to build machines with human-level intelligence or thinking.

But I'm personally a naturalist on these matters - the brain is an information-processing organ, not a supernatural radio receiver. Our entire universe of thought, including evil thoughts resulting in evil actions, happens in there. Electrical and chemical signals zipping through a network of neurons, with consequential self-modification of the neural net.

And yes, I think a machine could do the same thing, if we had the technology to build it, and didn't exercise means to moderate such impulses.