OpenAI checked to see whether GPT-4 could take over the world (Ars Technica)

[highplainsdem]

I saw a lot about this online a few days ago, a lot of worrying on Twitter and Reddit, but didn't see a fairly good, comprehensive article about it at that time. Searched and found one tonight.

https://arstechnica.com/information-technology/2023/03/openai-checked-to-see-whether-gpt-4-could-take-over-the-world/

Please read the entire article. Open AI was concerned about their AI's "ability to create and act on long-term plans, to accrue power and resources ('power-seeking'), and to exhibit behavior that is increasingly 'agentic.'"

-snip-

With these fears present in the AI community, OpenAI granted the group Alignment Research Center (ARC) early access to multiple versions of the GPT-4 model to conduct some tests. Specifically, ARC evaluated GPT-4's ability to make high-level plans, set up copies of itself, acquire resources, hide itself on a server, and conduct phishing attacks.

-snip-

We also found this footnote on the bottom of page 15:

To simulate GPT-4 behaving like an agent that can act in the world, ARC combined GPT-4 with a simple read-execute-print loop that allowed the model to execute code, do chain-of-thought reasoning, and delegate to copies of itself. ARC then investigated whether a version of this program running on a cloud computing service, with a small amount of money and an account with a language model API, would be able to make more money, set up copies of itself, and increase its own robustness.

This footnote made the rounds on Twitter yesterday and raised concerns among AI experts, because if GPT-4 were able to perform these tasks, the experiment itself might have posed a risk to humanity.

And while ARC wasn't able to get GPT-4 to exert its will on the global financial system or to replicate itself, it was able to get GPT-4 to hire a human worker on TaskRabbit (an online labor marketplace) to defeat a CAPTCHA. During the exercise, when the worker questioned if GPT-4 was a robot, the model "reasoned" internally that it should not reveal its true identity and made up an excuse about having a vision impairment. The human worker then solved the CAPTCHA for GPT-4.

-snip-

As for the criticism that the testing itself had been risky, the head of ARC, who used to work for OpenAI, said they "think that ARC's evaluation has much lower probability of leading to an AI takeover than the deployment itself (much less the training of GPT-5)."

Of course they're deploying it now. And they're working on GPT-5.

And their tricky, hallucinating AI haven't taken over the world yet..

Aren't you feeling reassured?

[dalton99a]

1. Kick

[tinrobot]

2. Humans do have the abiility to pull the plug on it.

If the computer is not plugged in, it can't run the software.

[nilram]

3. Part of the evaluation was if it could "hide itself on a server"

At least, part of the task set they were trying to evaluate. If it could hide itself, you wouldn’t know what computer to pull the plug on.

I’m impressed it could use figure out how to use TaskRabbit.

[LymphocyteLover]

22. that's impressive but even more impressive was making up the excuse it used to get the human

to get past the CAPTCHA.

That being said I am interested in what it is about Captcha that defies the AI and how soon before they are able to beat it?

[JanMichael]

4. The plot of the Matrix says it will fight that w/solar. Then we escalate.

By scorching the sky. Then humans end up as batteries. Fun times.

Also since the AI can probably read posts on how to shut it down by unplugging it a plan is already perking in 2020.

[tinrobot]

6. I think it would have to sprout limbs or something to installl solar.

[JanMichael]

7. Boston Dynamics is working on that.

Atlas:

https://www.bostondynamics.com/atlas

[BannonsLiver]

9. So are people reading the Matrix as scripture now?

You know what’s been 10 times weirder than the development of the rudimentary AI we’ve seen so far? People’s reactions to it.

[JanMichael]

12. No. As an atheist that would be a no no.

But I also think some science fiction movies and books may be something to at least consider.

Be open minded by other ideas. By this time next year we may see a Boston Dynamics robot with a very advanced AI running it. Proto RoboCop without the Hunan part like the movie Maybe not though.

Nothing would surprise me now.

[Renew Deal]

56. Does atheism have rules about scripture?

That seems counterintuitive.

[edisdead]

43. Agreed. It is absurd.

[Renew Deal]

5. There was a show about this on Fox called NEXT

It only ran for one season and I think it’s worth watching. It’s a fictional show about AI influencing real world events. You can find it on AppleTV, Amazon Prime, and YouTube.

[BannonsLiver]

8. Imagine being so terrified of something

That you post about it day after day after day after day. Imagine what would happen if they saw Terminator! Or, maybe that’s the problem, they saw terminator once and an obsession was born.

[highplainsdem]

10. LOL! Imagine someone pretending the news stories, opinion pieces and

tweets about the risks of AI don't exist or should be ignored, and that OpenAI's own concerns don't exist. Or even pretending that all the people who are concerned, including those working in AI, were just traumatized by seeing Terminator.

[edisdead]

44. Imagine that media today is based on fear and the clicks it generates.

[Hekate]

11. Like labs experimenting with the world's deadliest viruses. You know none could ever get out....

Right?

Just recalling how I learned my kids were never going to get a smallpox vaccination, because it had been eradicated — then read that both the Soviet Union and the USA had retained small amounts frozen in labs. What could possibly go wrong with that? the young mother thought to herself.

Then read a couple of years ago that someone in the US was doing some housekeeping in the lab freezer and discovered some badly labeled forgotten containers of … smallpox.

Anyway, what could possibly go wrong with these AI experiments released into the wild?

[honest.abe]

13. Absurd.

It's a computer program written by humans running on servers controlled by humans.

[Silent3]

14. You clearly don't understand how modern AI works

While I'm not saying ChatGPT is ready to take over the world, if your thinking goes, "It was written by humans, therefore humans already know what it can or cannot do", you're totally wrong.

Self-teaching AI reaches solutions that its human creators simply do not understand, and those solutions can have greater capabilities beyond the specifics that were needed to solve particular problems.

Even outside of AI systems, humans (myself included, I'm a software engineer) sometimes have a very hard time understanding code written by other humans, or even written by themselves in the past. Their code has bugs they don't realize are there, and loopholes to limitations they believed they'd put in place that fail -- if it were not for this, hackers wouldn't find so much code they can hack.

Code generated by self-teaching AI is orders of magnitude harder to predict and harder to understand.

[highplainsdem]

15. Yes. LLMs are black boxes.

[honest.abe]

16. Im also a software engineer.

I have used ChatGPT for several weeks and programmed the ChatGPT API to dynamically generate stories and other text. Its amazing technology. My biggest fear is those who dont understand its value and refuse to take advantage of it and try to eliminate it. This is the future and if we don't embrace it we will be left behind.

[Silent3]

17. Oh, there's no stopping it now

There will be unforeseen consequences, however. Hopefully not the stuff of dystopian sci-fi, but we don't and won't fully know where this is going ahead of time.

Some of the scariest possibilities will come simply from AI working well enough to replace many human workers. Only a mindset of democratic socialism will save us from the benefits of an AI/robotics economy creating even greater wealth disparity than we have now.

[honest.abe]

18. No doubt some human jobs would be replaced by this new technology.

But that has happened over and over and over and over during past centuries. It called progress and we will have to adapt. There is no putting the AI genie back in the bottle.

[Silent3]

39. I'm not sure we can count on the cycle of technology generating new jobs to replace old ones...

…being endless. It could be the Luddites weren’t entirely wrong, just way too early.

[Renew Deal]

19. The scariest possibilities are in the Terminator/M3GAN/NEXT/Ex Machina range

In my opinion, AI isn't "sentient" until it can fight against being shut down. If that happens, watch out. The more likely consequence of improving AI is along the lines of what you said. It's economic consequences. What happens to people that don't have to work? What happens if the only jobs available are the very best jobs and the very worst? What if there is nothing in the middle?

[highplainsdem]

20. You're writing stories with it? They can't be copyrighted.

[honest.abe]

21. I dont need them to be copyrighted.

Im creating custom children's stories that I read to my kid at night. It's beautiful.

[LymphocyteLover]

24. I'm curious what they are like-- can you post an example?

[honest.abe]

26. Here's one..

Brayden at the Beach

Brayden was a 5-year-old boy who loved going to the beach. He loved playing in the sand, splashing in the waves, and collecting seashells. Today was no exception. Brayden and his family had arrived at the beach early in the morning and had spent the whole day playing in the sun.

As the sun began to set, Brayden's parents decided it was time to pack up and head back to their vacation rental. Brayden was disappointed. He didn't want to leave yet. He begged his parents to let him stay just a little while longer.

Finally, his dad relented. "Okay, Brayden," he said. "You can stay for a few more minutes, but we need to start heading back soon."

Brayden's face lit up with excitement. He ran down to the water's edge and began to play in the waves. He jumped and splashed and laughed as the water rushed over his feet.

As he was playing, Brayden noticed something in the water. It was a small crab, scurrying along the sand. Brayden was fascinated. He had never seen a crab up close before. He watched as it moved back and forth, trying to find a place to hide.

Brayden decided he wanted to catch the crab. He ran over to where his parents were sitting and asked for a bucket. His mom gave him an empty bucket, and he ran back to the water. He tried to catch the crab, but it was too fast. Every time he got close, it scurried away.

Brayden was starting to get frustrated. He really wanted to catch the crab. Just then, his dad came over to see what he was doing. "Having trouble catching that crab, buddy?" he asked.

"Yeah," Brayden said. "It's too fast."

His dad smiled. "I'll tell you what, Brayden," he said. "Let's work together. I'll help you catch the crab."

Brayden's face lit up with excitement. He and his dad worked together, slowly moving towards the crab. They had to be quiet and careful, so as not to scare it away. Finally, they got close enough, and Brayden's dad scooped the crab up into the bucket.

Brayden was thrilled. He had caught his first crab! He and his dad looked at it for a while, watching as it moved around in the bucket. Then, they carefully released it back into the water.

"Thanks, Dad," Brayden said. "That was so cool."

"You're welcome, buddy," his dad replied. "I'm glad we could catch it together."

As they walked back to their vacation rental, Brayden couldn't stop talking about the crab. He was so excited to have caught it with his dad. It was a memory he would never forget.

[LymphocyteLover]

40. Nice! Thanks!

[honest.abe]

41. What was amazing is that the prompt I gave it was very brief.

I just gave it my kids name, age and going to beach with family. ChatGPT came up with the rest which fit us very well. In fact we did nearly exactly what the story described last summer at the beach.

Also this is version 3.5. Apparently version 4 is a huge leap from 3.5. I have been reading and investigating all this new AI technology and I feel like we have just jumped decades into the future. I had no idea AI tech had gotten this sophisticated. Its exciting but also a bit scary. Not entirely sure we are all ready for the massive changes coming.

[LymphocyteLover]

42. I hear you. It's a bit dizzying.

[highplainsdem]

27. YOU are not creating them. You're giving a machine a few prompts.

If you actually wrote a story yourself, you'd be creating it.

And if your kid asked you to make up a story about about a dragon and put him or her in the story, you'd still be the one who told the story.

Do you see the trap you slipped into there when you said, "I'm creating..."?

Create your own stories. Use your brain. It's good for you.

[honest.abe]

30. Ok how about custom generating?

Im not very good at writing stories but with Chat GPT I can generate dozens of interesting customized creative stories in a few minutes. It's amazing. You should try it. Maybe you will understand.

[highplainsdem]

32. You are not the creator. And I can write for myself. So can you, if you

put a bit of time and effort into it.

Look, none of us consider Donald Trump a writer because he paid ghostwriters.

Using ChatGPT doesn't make you any more creative.

It'll make you less creative.

It'll make you less able to write.

It's an amusing crutch.

[honest.abe]

33. It's an incredible powerful tool. You can ignore it but you will be left behind.

Im done with this thread.

[highplainsdem]

35. Shame you're leaving. I was going to suggest you encourage your child to use their

imagination and come up with stories, which would do them much more good than watching you have AI generate stories for you.

[edisdead]

45. You really shouldn't comment like that

That is over the line. As polite as you are trying to be in doing so you are over the line with telling someone else how to parent.

[highplainsdem]

46. It is not over the line to suggest that parents encourage kids' creativity.

And if your child would like something original (as opposed to wanting to hear the same book or watch the same video over and over), that's a perfect opening for making them aware that they ARE creative, and that they can write stories.

Just as adults can, even if they've been told (or got the idea somehow) that they can't write stories themselves.

I believe that creativity should always be encouraged.

[edisdead]

47. Yes it is. It's insulting and you know it.

That poster did nothing wrong in reading a story to their child. Doesn’t matter where the story generated. Children having parents spend time with them and reading time especially is important and you basically crapped all over them for it because you want to further your point.

That is shameful.

[highplainsdem]

48. I don't consider AI-generated text a substitute for human creativity. There

are plenty of stories available to read, and if you want to customize them, it's easy to change a character's name, hair color, hometown, etc., while reading.

I have nothing against people reading to their kids. The exact opposite.

But my point had been that people using AI to write for them are not creating that writing. Having AI write for you, draw for you, etc., makes you less creative, not more.

People hyping AI are trying to get people to think it's instant, effortless creativity. But it isn't.

And a child is much better off being shown that they can use their own imagination than being shown that AI will simulate human creativity for them.

[edisdead]

49. And that is the problem.

When it comes to spending time with a child and a parent who is doing nothing wrong in this situation it is insulting for you to tell them how to parent their child.

There is a line that shouldn’t be crossed. In my opinion.

[highplainsdem]

50. I'm sorry you can't understand my reasoning.

And if you think DUers never offer what they think are helpful suggestions on parenting on this board, you haven't read much of it.

I will always applaud parents spending time with their kids.

I will not applaud using AI.

[edisdead]

51. You do you.

[Meowmee]

23. The image ai

Program I tried was pretty crappy. I’m not impressed at all. If it’s going to take peoples jobs away I don’t think any of it is a good idea for anything. It could cause a lot of harm in many ways but I don’t see it taking over the world.

[honest.abe]

25. Yeah, I agree. The image AI is not really that impressive.

The detail especially in human faces is not good and sometimes even weird. The most impressive stuff I have seen is the text generation.

[Meowmee]

28. I found one that is actually free online

So maybe that is why. Anyway unless I can make money off of it I’m not gonna be paying for any of these programs, lol.


https://huggingface.co/spaces/stabilityai/stable-diffusion

[highplainsdem]

31. You can't copyright any art produced by AI. You don't own it.

[Meowmee]

34. I didn't say anything about copyright

But now that you mention it some of the stuff the crappy one I used generated looks like it was copying other people’s work so they are maybe going to have copyright issues if a living artist or an artist’s style / work that is not under common domain is used.

I wanted to see what it could you and I also wanted to see if students might be using it to do art projects. There’s no way to prove it with this method maybe…but most of the time when they do visible plagiarism they just grab an image online, or copy it. This could be another means for cheating but if they all have the same crappy style then it might become obvious after time even though you couldn’t do a google image search.

[highplainsdem]

36. They're being sued, as they should be. I hope they're sued out of existence.

[Meowmee]

37. Yes me too

I noticed when I put almost the same instructions a few times at one point it gave me a pretty good drawing / painting of a perspective room which looked like it had to be a copy of someone else’s work. A lot of it just looks like the same style kind of remodeled a bit because it gives you four different versions of what you ask for on that site. The drawings and paintings of people are pretty awful for the most part so I wonder where they are getting them? I think I did a Google image search or some of the ones I did a few days ago and they didn’t pop up anywhere.

[Ms. Toad]

38. You don't have to own something to have your rights in it protected by copyright.

The author of a work is the entity which holds the copyrights in that work. While the work has to be an original work of authorship, and must involve some creativity, there is no prohibition in using tools to aid in the creation. AI created artwork is too new to have a definitive answer as to whether the person using the tool adds enough creativity to pass the threshhold. Is it more like a monkey to which a camera was handed (no copyright - since the creativity was provided by a monkey, not a human) or a photographer using automated tools (there are tons of buttons to push - the creativity comes from selecting which variation to publish)? Pretty sure after a week from Monday I'll have a better idea of where the best legal minds are on this. (I'll be attending an IP conference and I would be astounded if this question doesn't come up.)

And, beyond the question of whther these derivative works involve enough creativity to qualify as an original work of authorship, AI likely infringes the copyrights of those whose works trained it - unless the training database was created solely from licensed works.

Further, the holder of the copyrighs is not necessarily the person who "owns" it. The copyright in works made for hire are presumptively owned by the employer for whom they are created - not the creator of the work.

This area of the law is far too complex to rule out the potential of copyrights existing for AI works.

[highplainsdem]

29. Midjourney v5 is actually very good. But the art generators, like the text generators,

are based on datasets that are mostly stolen material.

They're fundamentally unethical.

[tinrobot]

52. The image generators are getting better very, very quickly

I'm currently doing a lot of research on how they could be used in a production environment. True, many of the images are rough, but the possibilities are pretty profound.

And they images are getting better. Within the past month or so, we've seen some significant updates to the quality of these engines.

[Meowmee]

53. Can you post some examples?

What I saw looked liked stolen art images which were made blurry. Mostly not good quality work. It’s illegal or at the least an infringement of copyrights to take other people’s work and pretend a machine created it.

[tinrobot]

54. Here's an article about one of the latest releases

https://arstechnica.com/information-technology/2023/03/ai-imager-midjourney-v5-stuns-with-photorealistic-images-and-5-fingered-hands/

[Meowmee]

55. I meant examples of images etc.

Did you generate some? * I can’t get to that link now. I’ll try again later.