Since ChatGPT came in late November 2022, there's been a heap of chatter in academia about how to go about marking essays. I was talking to a fellow tutor today who has just marked about 150 essays in the sciences department and they reckon about 75% were written using AI. Or that AI wrote these essays. Pick your weapons here.
I can just imagine those dispiriting hours, trawling over essays that have been written by a bot and trying to work out which students have written their own work and whatnot. The standard plagiarism software that crawls the web doesn't work here, so it's up to the tutor to work out whether or not someone has plugged in some commands to a chat bot to create the perfect essay.
So far, with such small classes, I haven't had to encounter it so far. Plus working in the Humanities areas of creative writing and the classics of English literature, the emphasis for essays is on individual takes rather than regurgitating facts. There has always been a bit of essay sharing stuff going on but that stuff is easily picked up.
My feeling is that a lot of students are relying on their final marks, as opposed to actually learning anything. High marks lead to scholarships, placements and doctorate awards. These awards are worth $$$K. So regularly handing in a perfect essay using a chat bot probably works for them. That's my devil's advocate argument. If you are paying a shitload for a degree, then why not monetise the said degree?
Personally, I see AI like ChatGPT as being useful for grunt work, and quite often women's work. That grant application, that letter to a politician, that job requiring vacuuming the lint out of corners. The demeaning or tedious work could be done by a bot. Toilets are a pretty good example. I'm all for bots cleaning our toilets. In education it gets spicier.
In intellectual property rights, it gets spicier still. I can ask ChatGPT to write a story based on Sarah Toa's A WineDark Sea and it will crawl the web, find my content and produce a story. No IPR are attributed to me but the 'writer' can add commands like 'make it funnier' or 'make it more heartfelt' and it will proceed to churn out a blog post pretty much the same tone as me writing.
The local newspaper has been using AI for a while now. Like I said, women's work. They've been struggling for funding and/or advertisers and this is the easy way for the editor to go. It's ... yes it's awful ... but it's here.
It would appear to me that originality could be dead.
ReplyDeleteRespectfully Rachel, I disagree. I think originality and critical thinking will be countering AI
ReplyDeleteThat's a relief.
ReplyDeleteHaving heard the results comparing A.I. with human content, I think I can say that - as far as the Arts goes - it is not very good - yet, anyway. The music is awful, for instance. There should be a command of 'make it better'. Then we should begin worrying.
ReplyDeleteThe other day we tried instructing a chatbot to write a short piece based on Virginia Woolf's stream of consciousness, Chernobyl and trees. Pretty lame.
DeleteWhen evaluating a student's paper, if ChatGPT has been used to facilitate/write the paper, how do you determine a student's proficiency and skill? This also raises the question of, what constitutes the student's original thought? Regarding newspaper articles, I would greatly appreciate a ChatGPT article that presents multiple sides of any topic. Bringing back investigative journalism would be beneficial. I recently heard about NewsBots presenting news. They have not reached the US as yet.
ReplyDeleteTo be bleak, Susan. We'll have a generation of environmental scientists going out into the field who have learned bugger all except how to instruct a chatbot. Personally, I think AI will be amazing ... but terrible for education.
DeleteI had one essay come in yesterday that I think was AI. It was an English Lit class. Tom, there is a command of make it better. A lot of essay-sharing stuff going on but that is now 'old school'! It's making it hard work for us tutors to work out what is and what isn't.
ReplyDeleteI helped my grand daughter to write a thesis for her training in paediatric nursing. She had read all the books and I just helped with structure and grammar of what she wanted to say, because she is dyslexic. It was a nightmare because she was late with her submission and the plagiarism software had become super sensitive, having already trawled through everyone else's. I was constantly having to find words to substitute for perfectly good ones which could not be used without setting off alarm bells. Very frustrating. So there is a 'make it better' command?!
DeletePlagiarism software is still pretty good but chatbots get through scot-free. This is the conundrum - it's easier to use AI to write your essay than it is to steal someone else's work - and in the end, AI trawling the web for answers is actually using someone else's work to write a so-called original essay. Fun times in the academy!
DeleteI believe that AI builds itself and the knowledge it holds and delivers gets better everytime.
ReplyDeleteI enjoy that too Rachel. Even as we as tutors and teachers are trying to work out which essay questions get past AI, we are in the process of training it to respond to those questions.
ReplyDeleteFirst time here. Try zerogpt. Students submit essays digitally, I upload to zerogpt, and it catches almost all AI. It gives a probability score. When I question the student, they almost always fess up. I teach a science at uni in Wisconsin USA.
ReplyDeleteOh wow - thank you Celie! I'll give it a try :)
ReplyDeleteI'm suspicious of some supposed AI detectors. For example, ZeroGPT is not the same as ZeroGPT.cc — the former is under development by OpenAI (the developer of ChatGPT) and is not yet publicly available; the latter is from a company calling itself ZeroGPT, but good luck finding out anything more about that company.
ReplyDeleteDo you have institutional access to Turnitin? If so, check the AI estimate in the box at the bottom of the right-hand column in the Feedback Studio. If the estimate looks suspiciously high, click it and you'll be shown the assignment with suspicious parts flagged. That's a good starting point for further investigation (students do not see this; it's accessible for teachers only). Crucially, Turnitin is well aware of the problem of false positives (falsely identifying human-written text as AI-generated), so it's conservative in flagging AI-generated material (rightly so, IMO). Other supposed AI detectors seem less scrupulous about falsely accusing students of using an AI to write parts of their assignment.
Our university is struggling to deal with this problem. Investigating a possible case of AI cheating is hugely time-consuming. I'd better stop now — I have marking to do 🙂
Oh Pete, thanks for your comment. Yes I have access to turnitin and am finding Zero a bit problematic too. Turnitin have been putting up some pretty strange red flags re white text to conceal word count and I feel that Zero is possibly disingenuous in their findings. Plus I fear that we are training Zero's AI as well. It's such a minefield! And so much more work for us markers.
Delete