ChatGPT detection tool thinks Macbeth was generated by AI. What happens now? – AI News Update

ChatGPT yesterday released a new classification tool to detect AI-generated text that, within hours, was found to be flawed at best. It turns out that when it comes to detecting generative AI – whether it’s text or images – there may not be a silver bullet.

Sebastian Raschkaan artificial intelligence and machine learning researcher who is a senior AI educator at Lightning AI, began testing the OpenAI Text Classifier to ChatGPT with text excerpts from a book he published in 2015. Three different passages received varying results – the tool reported that it was “unclear” whether the book’s preface was written by HAVE; but the foreword was “maybe AI” and a paragraph in the first chapter was “probably” AI.

Even more concerning was how the tool ranked the first page of Shakespeare’s Macbeth:

“The classifier considers the text to be likely generated by the AI.”

When asked if he was surprised by the results, Raschka replied “Yes and no – they don’t share the paper so I can’t say 100% how it works, but from the short description they have on the website, it looks like they’re training a classifier to predict if something is human-generated or AI-generated.The problem, he explained, is that there are false negatives and false positives depending on the dataset the tool was trained on.

With Macbeth, for example, Raschka said he believed the tool was not trained on Old English. “It’s not normal spoken English, it’s almost like a foreign language.”

Open AI admits the classifierwhich is a GPT model that is refined via supervised learning to perform binary classification, with a training dataset of human-written and AI-written passages of text, is only accurate to about 26 %.

However, he says the tool can still be useful in tandem with other methods. In an email, the company said “The classifier aims to help mitigate false claims that AI-generated text was written by a human. However, it still has a number of limitations – it should therefore be used in addition to other methods of determining the source of the text rather than being the main decision-making tool.

The company added on its website that it was making the classifier publicly available “for feedback on the usefulness of flawed tools like this,” adding that it would continue to work on detecting text generated by AI and “hope to share improved methods in the future.”

OpenAI is far from alone in trying to tackle the wild west of generative AI sensing. There is a wave of other tools trying to meet the challenge.

GPTZero, for example, provides a score that must then be interpreted by the user. In a blog post, Raschka explained, “GPTZero does not recommend whether the text was AI-generated or not. Instead, it only returns the perplexity score for a relative comparison between texts. This is good because it forces users to critically compare similar texts instead of blindly trusting a predicted label.

Detect GPT, Raschka explained, “disrupts” the text: that is, he explained, if the probability of the new text is significantly lower than that of the original, it is generated by the AI. Otherwise, if it’s roughly the same, it’s man-generated. The problem, he added, is that the method involves the use of a specific LLM model, which “may not be representative of the AI ​​model to generate the text in question.”

Watermarking is another approach, he added – the idea of ​​reducing the odds of certain words so they are less likely to be used by LLMs, using a “to avoid list”. However, Raschka explained, this requires an LLM that has been modified with this avoidance list. If the list to avoid is known, he said, the AI-generated text can be modified.

What does this mean for AI generative sensing?

Raschka said it’s unclear how this will all play out and whether generative AI detection tools will make headway in overcoming the challenge of discerning between human-created content and computer-generated text. AI. Will the internet itself become unusable, flooded with untrustworthy generated content?

“What that means to me, or how I think about the way forward, is that the internet was where you searched for content and you mostly trusted what you found,” he said. declared. Going forward, it will be more about being selective and finding credible websites.

Whatever the future, Pandora’s box is already open when it comes to generative AI, he pointed out, adding that he currently finds ChatGPT useful as a “sophisticated grammar checker” to aid writing. .

“I don’t think we can go back,” he said. “Everyone is going to use these systems and I think it’s fine if we use them responsibly – I don’t think there’s going to be any way to avoid the use of these models.”

For now, generative AI detection tools are “definitely not good enough” to be used for important decisions, he said, which includes efforts to use them in grading college essays. students – in response to fear of cheating and plagiarism.

“Models like this can cause real-world damage as educators adopt it for grading,” Raschka tweeted yesterday. “So let’s add some transparency on false positives and false negatives.”