University at Buffalo recommends ways for teachers to catch ChatGPT cheating - all time-consuming

[highplainsdem]

And while it's good that teachers may be able to catch the cheating, ChatGPT is creating serious enough problems in schools that IMO OpenAI should never have released it - especially giving it a widespread free release designed to maximize hype and get people, including students who want to cheat, hooked on the app.

https://www.buffalo.edu/ubnow/stories/2023/03/chatgpt.html

Faculty are encouraged to talk with their students early in the semester about whether ChatGPT is acceptable to use in the course, said Jeffrey Kohler, associate director of teaching transformation in the Office of Curriculum, Assessment and Teaching Transformation.

“Make sure that you are specifying your policies about AI use in your course, whether you want students to be using AI or not,” Kohler said. “Make sure that they understand not only the parameters that you set, but also the ramifications for use if you are asking them to refrain from doing so.”

-snip-

Faculty who suspect students are using ChatGPT dishonestly may want to use detection tools, which are proliferating along with use of the application, Ahuna said. Kohler suggested faculty collect in-class writing samples so they have something to compare. Ahuna recommended closely fact-checking papers.

“We know ChatGPT is making a lot of errors, so as the expert in your field, hopefully that’s something you can catch quickly and that might raise some suspicion,” Ahuna said.

Of course the detection tools that exist so far don't work.

And think of the time required, with a large class, to compare in-class writing samples to essays written as assignments.

ChatGPT is notorious for making up citations, whether inventing nonexistent sources or linking to sources that might look plausible at first glance but don't contain the information cited. But it takes a lot of time to check those, too, especially if the reference is to a web page with a lot of text to skim.

I have to applaud this university for at least offering some tips.

But the villain here is OpenAI, as much as the students using it to cheat.

I don't for one second believe the people at OpenAI weren't perfectly aware of how much trouble their app would cause.

They released it anyway because they were greedy, they were desperate to be first with this tech, and they didn't care about the harm it would do.

[dsc]

1. I wonder how much the answers vary

say as a writing prompt I give a class, you are now President describe your first day. If in a class of 25, 5 use the AI tool, how different would those 5 answers look from each other?

[Renew Deal]

12. It's fairly easy to test this.

ChatGPT allows you to create "threads." The threads don't include history of your other conversations. Create 5 different threads and ask the question in the same way in each thread. That should answer your question. It's possible that answers vary to a larger degree as time goes by and the system learns more or advances technology. This should be a good test over a short period of time.

[bucolic_frolic]

2. So when I write something now, could i be falsely accused of AI?

I mean, generic writing is generic writing

[highplainsdem]

4. It's a risk. Detection tools give false positives as well as false

negatives.

You can expect to see a lot of ChatGPT-written fake letters to the editor and internet misinformation, too.

Anyone at OpenAI could have seen the havoc they could wreak tossing this shiny new toy out there.

[dalton99a]

3. AI companies expect teachers to do extra work to detect the AI bullshit

because their AI detection tools are practically useless

[usonian]

5. Do a web search on detect ai papers.

Too many results. Hope that one of them helps.

I use DuckDuckGo for search. Google and others spy too much and retain your search terms forever, whether you log into them or not.

[highplainsdem]

8. I've already posted other threads about this. As for DuckDuckGo, until

last August it was limited in the Microsoft tracking it could block because of its deal with MS to use Bing for its search results.

https://techcrunch.com/2022/08/05/duckduckgo-microsoft-tracking-scripts/

[usonian]

9. I noticed that DDG results are just like Bing.

At least they claim to erase search terms daily.

I can't follow the AI detection in detail. Swamped. And not using AI to summarize!

Thx

[sanatanadharma]

6. Like 25 general answers and five rehashes of previous Presidents' first days

The five AIbot answers would look like plagiarism, with too many mirror sentences.
Unless one student asked for a steam-punk answer or a Chaucer-ian reply.

How many AI exist? Are they several AI or several instances of one AI?
Would AI identify as singular, plural, automaton or consciousness?
Could AI explain the nature and origin of 'consciousness' if it so-claimed to be?
Can we?

If we or AI claim that AI is (has) consciousness, then clearly the body-brain is not a necessity for conscious existence.

[JCMach1]

7. Writing assignments can no longer be busy work, or students will use busywork tools

ChatGPT is the equivalent of a calculator.

Just stop giving students the writing assignment equivalent of simple computation.

[Cuthbert Allgood]

10. This. This right here.

I teach HS English. I also teach a public speaking class that students can take for credit through the local university.

Stop giving generic prompts. And when you write a prompt, go to the AI and see what they have to say about it.

It's not that hard, really.

[JCMach1]

17. And we teachers are partially to blame. My son had a typical busywork assignmet

That required writing just basic, basic bs (think copy paste/ he's 12). As a writing professor, I showed him how to use chatGPT to generate text and then make it his own and fact check. ChatGPT will also generate a list of sources it used for you as well.

For the rulesy folks on DU, it was 1AM already with a 6AM wakeup call. This teacher literally had no sense of how long this particular project would take. Total nightmare.

So, like math teachers when calculators came out, other disciplines will need to adjust around the new tools available.

[highplainsdem]

21. You're a teacher, yet you showed your son how to cheat on an assignment?

I'm assuming your son wasn't told to use ChatGPT or you'd have mentioned that.

It doesn't matter that you're a teacher and felt his assignment would have had him writing "just basic, basic bs" as you called it. It was not your call. He isn't your student.

You taught him to cheat and let him know you're okay with cheating. Hell of a lesson, JCMach1.

For the rulesy folks on DU, it was 1AM already with a 6AM wakeup call. This teacher literally had no sense of how long this particular project would take. Total nightmare.

It wasn't 1AM when he was given the assignment. If he didn't start on it till then, that was his fault - or yours for not teaching him to be more responsible. By the age of 12, he really should have known better.

And if that particular project was going to require much time, it was NOT "basic BS." It was fairly important.

But your son hadn't started on it in time - which again shows you hadn't taught him to be responsible - and you didn't want him to have to deal with the consequences of that. So you showed him how to cheat.

I've known a lot of teachers (friends after I was out of school; I'm not counting the ones who were my teachers). I don't think any of them would have been okay with what you did.

And they wouldn't have given a fuck if you called them "rulesy" for minding that you taught a child to cheat.

[JCMach1]

28. Are calculators cheating? Card catalogs? They are tools

So no, verifying it, documenting it,.and putting it in your own words is definitely not cheating.

It's in fact EXACTLY what the task was at hand.

I strongly resent your insulting characterization and will be glad to block further nonsense from you.

[highplainsdem]

29. Unless the assignment was to have the students use ChatGPT that

way, it WAS cheating.

And there are plenty of teachers and students who consider it cheating.

Which you're aware of, or you would not have made that crack about "rulesy folks" or tried to run down both your son's teacher and the assignment.

The fact you feel this way certainly explains why you're so eager to defend this sort of use of ChatGPT.

I don't know if you'll see this reply, since your message suggests you're blocking me. So I'm not going to waste my time trying to respond to your nasty, name-calling email, which I saw before I saw your reply here.

But I will link to a couple of earlier threads that are relevant:

https://www.democraticunderground.com/100217617842

https://www.democraticunderground.com/100217693512

[highplainsdem]

18. No, it isn't at all the equivalent of a calculator.

Writing is basic communication. It requires reasoning as well as an ability to put words together.

[Renew Deal]

11. Have you tried ChatGPT?

Have you signed up for an account on OpenAI’s website and tried ChatGPT?

[highplainsdem]

22. Not interested. I don't need a machine to write for me. I also

respect good writing and accuracy, neither of which ChatGPT is capable of.

[Renew Deal]

23. So you basically have no idea how it works or what it does

Other than what you have read. I encourage you to try it. You don't have to like it, but I think you'd have a better understanding of what you're posting about.

https://openai.com/blog/chatgpt

[highplainsdem]

26. Renew Deal, my personal experience with it, even if I thought it was

amusing, would be completely irrelevant compared to the problems it's creating.

The companies behind it KNOW it isn't reliable. They admit that. They warn people not to rely on it. They know it gets things wrong. They know it hallucinates, that it invents sources, that it will argue relentlessly that it's right even when it's wrong.

Even when it's arguing with a computer expert it insists is dead and invents imaginary obituaries for. See https://www.theregister.com/2023/03/02/chatgpt_considered_harmful/ . That's a British tech news site, and the headline is "Why ChatGPT should be considered a malevolent AI – and be destroyed."

It's a bullshitter, as others have pointed out.

Sometimes the bullshit is amusing. , even if people can see it's bullshit. Sometimes people can't see it's bullshit, and then it can become really harmful.

And its flaws, its unreliability, are what make all the comparisons to calculators so ridiculous.

Calculators would never have been widely used if they had often provided incorrect answers and required fact-checking.

ChatGPT - like Meta's Galactica and Google's Bard - is badly flawed and should never have been released. It was released because companies are desperate to be first no matter how flawed it is.

[SheltieLover]

30. I agree, it should be destroyed! And banned!

But we know that won't happen.

[Takket]

13. AI is an emerging technology with many potential benefits

This is obviously one of the drawbacks. Like any technology it will grow and change over time but I’m glad to see it emerging because it has many applications to improve our lives.

I don’t see any reason to hold it back just because people might use it to cheat. People have been plagiarizing for as long as there have been books. Detection software needs to improve and if people are still cheating, well, they’ll find out later when they get fired because they don’t actually know the material that the cheating wasn’t actually worth it.

[lindysalsagal]

14. Wow. I think if I were a university prof I'd revert to in-class paper tests and essays

and not allow any devices in the room. If personal knowledge and memory are the point.

Otherwise, it doesn't matter how well you write term papers and research documents anymore: you can insert your data into the system and let it write it for you. Just like cursive, these skills are no longer important.

I guess you'd have to provide a paper to the class and get them to find the mistakes and logical/research errors.

[Renew Deal]

15. The question about the usefulness of various skills is actually very important.

When do we cross the line from possessing skills to perform AI assisted writing being more important than manual writing and research?

[lindysalsagal]

16. Correct. We all took typing in middle school in the 70's. THAT's gone. Paper filing?

Secretaries taking short hand? So many tasks are automated now. Even 50 years ago, did the scientists really write the papers, or did their phd candidates write them? Is it more important to have a scientist who can write up the work, or one that's purely analytical?

Personally, I'd prefer a surgeon who cuts well, and pilot who lands well, and a nutritionist who listens well.

Leave the writing to the writers and/or computers.

[hunter]

27. Ultimately the circle will be complete and nobody will be reading this AI crap but crappy AIs.

Meanwhile actual humans will be off somewhere else doing their usual human things.

I imagine students cheating by using AIs to write their papers, and teachers cheating by using AIs to read and grade these papers...

[SheltieLover]

31. Yup. Remember those little blue books hor handwritten tests?

[Silent3]

19. If OpenAI waited longer to release ChatGPT, it would probably be better and even harder to detect...

...when it became available.

I also can't imagine any feasible arrangement that could have been made by OpenAI and thousands of different academic institutions ahead of public release. How would that even work?

[FakeNoose]

20. Is "OpenAI" the same thing as "Open Source"?

Highplainsdem, I know you are following this story on the ChatGPT software very closely. Much closer than I am, and probably most of us. I'm not familiar with the term OpenAI, but I'm wondering if it's equivalent to Open Source?

If it is the same or equivalent, then it means the barn door is already open and the horses have fled the scene. Open Source programming is released by the original coders for free, and anyone can use it as is, improve it, update it or otherwise make revisions on their own version of the code. It means there are no copyright laws to prevent this from happening, and the original coder(s) are not legally responsible for whatever changes might be made by others.

If that's the case, then it would be bad news for teachers and test graders everywhere.

[highplainsdem]

24. No. See this Vice article: "OpenAI Is Now Everything It Promised Not to Be: Corporate, Closed-Source

and For-Profit":

https://www.vice.com/en/article/5d3naz/openai-is-now-everything-it-promised-not-to-be-corporate-closed-source-and-for-profit

Now, eight years later, we are faced with a company that is neither transparent nor driven by positive human impact, but instead, as many critics including co-founder Musk have argued, is powered by speed and profit. And this company is unleashing technology that, while flawed, is still poised to increase some elements of workplace automation at the expense of human employees. Google, for example, has highlighted the efficiency gains from AI that autocompletes code, as it lays off thousands of workers.

When OpenAI first began, it was envisioned as doing basic AI research in an open way, with undetermined ends. Co-founder Greg Bockman told The New Yorker, “Our goal right now…is to do the best thing there is to do. It’s a little vague.” This resulted in a shift in direction in 2018 when the company looked to capital resources for some direction. “Our primary fiduciary duty is to humanity. We anticipate needing to marshal substantial resources to fulfill our mission,” the company wrote in an updated charter in 2018.

By March 2019, OpenAI shed its non-profit status and set up a “capped profit” sector, in which the company could now receive investments and would provide investors with profit capped at 100 times their investment. The company’s decision was likely a result of its desire to compete with Big Tech rivals like Google and ended up receiving a $1 billion investment shortly after from Microsoft. In the blog post announcing the formation of a for-profit company, OpenAI continued to use the same language we see today, declaring its mission to “ensure that artificial general intelligence (AGI) benefits all of humanity.” As Motherboard wrote when the news was first announced, it’s incredibly difficult to believe that venture capitalists can save humanity when their main goal is profit.

-snip-

Emily M. Bender, a professor of linguistics at the University of Washington and the co-author of that paper, tweeted: “They don't want to address actual problems in the actual world (which would require ceding power). They want to believe themselves gods who can not only create a ‘superintelligence’ but have the beneficence to do so in a way that is ‘aligned’ with humanity.”

[Straw Man]

25. I'm a writing teacher ...

... and I would use only in-class real-time assignments, except for the fact that, due to COVID, I now teach only remotely. As a semi-retired adjunct instructor, I don't get paid enough to do the detective work required to ferret out papers that were produced by AI. In my opinion, it's now up to the universities to invest in the development of detection software. That's the least they can do: Justify the whopping fees they charge by proving that their degrees are worth more than the paper they're printed on.