With increasing concerns over ChatGPT and other AI writing technologies, several developers have created solutions to help users detect AI content and maintain academic and work integrity.
One of the recently emerged tools is a Chrome extension called DetectGPT. This is separate from another AI content detector of the same name made by Stanford researchers. Thomas Petracco, a developer currently living in Taiwan, created this particular extension.
According to Petracco, DetectGPT trained on OpenAI’s GPT-2 dataset. But even so, he claims the tool works well with GPT-3 and ChatGPT content.
Being a Chrome extension, DetectGPT does not require additional software installation and is incredibly easy to use. Simply open the extension in your browser, paste the text you want to check for AI content, or allow the tool to read text on your current web page.
Read on to see what we thought about DetectGPT and how it fares against other AI content detection tools.
Time to test whether DetectGPT’s Chrome Extension works as intended.
For this experiment, we used seven pieces of content generated by Jasper.Ai’s GPT-3 writing tool. Jasper.AI is one of the most popular AI content generators today, with over 3000+ positive customer reviews.
For comparison, we also checked Originality.AI’s results on the same writing samples. Originality.AI is both a plagiarism checker and an AI content detector. Aside from employing AI algorithms to check whether a text is made by AI, the tool also runs the text against millions of publicly available documents and pages online.
Here are the results:
Before we go through the results, we should mention that the DetectGPT Chrome Extension has a 250-word limit. So we could only test the samples’ first four or five paragraphs instead of the entire document. To keep it fair, we tested the same amount of words per sample with Originality.ai.
Originality.AI is the clear winner of the two as it shows higher percentages in most samples. The only one that DetectGPT scored better at is sample #6. We can attribute this improved accuracy to the fact that Originality.ai is trained on several language models, namely GPT-3, GPT-3.5, and ChatGPT.
Looking at DetectGPT’s numbers, you can tell that the tool is currently inconsistent – especially on samples generated by a GPT-3 app. This is why the DetectGPT Chrome extension should not be the sole basis of assessment for AI plagiarism.
It should be noted that the DetectGPT tool did not work correctly during our initial testing. For every web page we tried, the tool showed the result:
“Page content: There is not enough text on this page to analyze.”
It also did not respond to any clicks of the “Refresh” button or “Check Content” button when you manually submitted text.
We tried troubleshooting it by reinstalling, updating through developer mode, and toggling the different options available, but the extension doesn’t seem to read any data we input. If you check the reviews on Google Chrome and several comments on Product Hunt, you’ll see that other users have experienced the same problem on initial installations.
A few hours after we sent a report to Google Chrome about our concern, the tool finally worked as intended.
The DetectGPT Chrome Extension may need several updates before it can be on par with more powerful AI content detectors like Originality.AI. Since many modern AI writing tools use GPT-3 and ChatGPT as their language models, the DetectGPT Chrome Extension may be unable to detect all of the content generated by these AI tools.
There should also be clarity on who exactly the tool is for so that users know what kind of content to test. For example, Originality.AI intends to catch AI-generated content marketing pieces. A few others are generally geared toward detecting AI plagiarism in student work.
However, even with its current inconsistent level of performance, DetectGPT can still provide users with a quick and easy way to check for AI-generated content in the text they are creating or consuming.
Where it does shine is in its convenience. Some competitors require you to visit their websites to access their tools. Others even need you to sign up for an account. On the other hand, DetectGPT is free and can be easily accessed on the top-right panel of your browser.
Branding-wise, DetectGPT could encounter issues with another tool of the same name. A quick Google search reveals that the top pages mainly refer to the Stanford-developed tool. The DetectGPT Chrome extension may be able to change that with a few tweaks and performance upgrades.
The DetectGPT Chrome Extension can sometimes be inconsistent, especially when detecting GPT-3 or ChatGPT content. However, this is an incredibly easy tool to use and provides users with a quick way to check for AI-generated content. Furthermore, DetectGPT does not require additional software and can be accessed directly from your browser.
So if you need a straightforward content checker, DetectGPT might be able to do the job. But it’s important to remember that DetectGPT is just one of many tools available. If you’re looking for a reliable and trustworthy solution, check out other resources too.
