top 50 comments

sorted by: hot top controversial new old
[–] 248 points 5 months ago (22 children)

tools like these are used to reject CVs and grade school papers btw

no matter how much ai is trash do NOT use ai checkers, they do not work

  • source
  • hideshow 22 child comments
  • [–] 68 points 5 months ago* (last edited 5 months ago) (6 children)

    Yep, they’re all trash and should not be relied upon.

    I got anywhere from 35% to 70% AI generated results on a book I wrote in 2019, before AI was even released.

    eta: it’s not about plagiarism, either. I also ran my novel through plagiarism checkers, since it’s easy to accidentally write passages similar to existing work. 0% on those, but high numbers in the AI checkers.

  • source
  • parent
  • hideshow 6 child comments
  • [–] 18 points 5 months ago

    I had to write a short story for English literature class in 2006 and I still have the file. Apparently over half of that is AI generated, which is pretty impressive on my part I must say.

  • source
  • parent
  • [–] 37 points 5 months ago (3 children)

    I witnessed an interaction where a grad school professor used AI detector and threatened to fail a student for submitting "AI generated" paper. It was so stupid, even after showing them how if you just add a few spelling mistakes the detection says human written, or even putting their own email in AI detector to show an example. It's like the saying "little knowledge is dangerous"

  • source
  • parent
  • hideshow 3 child comments
  • [–] 7 points 5 months ago

    When I was at university I was pretty belligerent and if a professor tried that on me I'd have reported them for academic misconduct. They should be grading in the damn papers themselves, if they're not going to do that then what is the point in them?

  • source
  • parent
  • [–] 5 points 5 months ago

    Yeah these are the kind of awful situations that will probably happen way more often as people turn to AI detectors to "find out" if someone is using AI not realizing that they aren't completely accurate, or even remotely accurate.

  • source
  • parent
  • [–] 15 points 5 months ago

    Yeah, LLM-based checkers will still have LLM-based problems, most notably being incapable of true analysis, which is the whole point of an AI checker. It's just the same text predictor shit.

    Oh and also there's an arms race where generative AI has the advantage because eventually it will be capable of generating things entirely indistinguishable from what a human would make (though it will still be susceptible to the hallucinations and errors it's already famous for).

  • source
  • parent
  • [–] 5 points 5 months ago

    That sucks. I had a hunch that my above-average level in french, my native language, (not just according to me, but also... almost all of my French teachers throughout my entire education) might be tripping these...

  • source
  • parent
  • load more comments (7 replies)
    [–] 126 points 5 months ago

    Still no statement from Mary. Sounds like she is guilty and doesn’t know how to respond.

  • source
  • [–] 58 points 5 months ago

    Her defense was that it wasn't an "artificial" intelligence: "It's alive. It's alive!"

  • source
  • [–] 52 points 5 months ago

    So the AI thinks this human-made text is actually AI-made and offers an AI tool that'll turn this human-made text into an AI-made text that'll appear more human than the human-made text? I wonder how it'd rewrite this paragraph.

    Sometimes it feels like the formal texts I write (like anything I write in the context of a job application) sound a bit like AI, but I just try to immitate the dumb way HR people write their job postings.

  • source
  • [–] 43 points 5 months ago (1 child)

    She used the inhouse LLM abby normal.

  • source
  • hideshow 1 child comment
  • [–] 37 points 5 months ago* (last edited 5 months ago) (4 children)

    I think this is the most convincing proof that time travel is possible I've seen so far.

  • source
  • hideshow 4 child comments
  • [–] [S] 28 points 5 months ago (3 children)

    Occam would probably punch you for ignoring his razor so thoroughly 😄

  • source
  • parent
  • hideshow 3 child comments
  • [–] 28 points 5 months ago (1 child)

    LLMs are predictive models. They scraped as much text as possible to create a model that predicts the next word accurately. To generate text, the LLM assembles a sequence of likely next words.

    That exact same sort of model can be turned around and asked, how closely did the actual next word match the predicted one? Good test for training the LLM. A better model will make more accurate predictions.

    AI checkers are usually doing that test. Does the real text match what the AI predicted? It sounds like a test of the text, but it really isn't. In this case, yes. Of course an AI trained on Mary Shelly's Frankenstein can accurately predict the next word of Mary Shelly's Frankenstein. It has the whole book memorized, if it were accurate to anthropomorphize computer code.

    So the "checker" calls it AI generated. These checkers don't work.

  • source
  • hideshow 1 child comment
  • [–] 20 points 5 months ago (8 children)

    It is just comparing against well-known public texts available to AI crawlers.

  • source
  • hideshow 8 child comments
  • load more comments (8 replies)
    [–] 15 points 5 months ago* (8 children)

    Perhaps that so many people have quoted that chapter in college and high school papers, book review and film reviews, and cultural criticism that maybe there is a weird "shoot the moon" situation where a "works of origin" begin to look like a "works of derivation" in LLMs.

  • source
  • hideshow 8 child comments
  • [–] 16 points 5 months ago (5 children)

    Frankenstein is out of copyright.

    I would be unsurprised if you couldn't tease out the entire book. I wonder if Mary Shelly was a fan of dashes.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 8 points 5 months ago (4 children)

    Being out of copyright is kinda irrelevant. There are lawsuits right now, because the AI firms apparently fed the AI's tons of copyrighted books.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 5 points 5 months ago* (3 children)

    It is and it isn't. Those lawsuits mean they at least try to stop it from producing copyrighted work. They won't make Simpsons characters or produce anything from the house of mouse without major cajoling or some trickery in the prompt.

    For the text from Frankenstein they are not even going to try.

    Incidentally after writing this content I tried to get chatgpt to reproduce the first paragraph of chapter 3. It refused and offered a summary. I "reminded" it that the book is in the public domain and then it reproduced it without issue.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 3 points 5 months ago (1 child)

    I tried to get chatgpt to reproduce the first paragraph of chapter 3. It refused and offered a summary. I “reminded” it that the book is in the public domain and then it reproduced it without issue.

    I bet you could do exactly the same thing for a book that's still copyrighted.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 3 points 5 months ago

    They still obviously trained it on the copyrighted text. Which I think is what some claim is illegal without payment?

    Mind you, I don't think copyright should cover that, for text at least. It is not in society's interest.

  • source
  • parent
  • [–] 10 points 5 months ago

    Yeah, or perhaps there is no need to make up excuses for the Copyright Infringement, world bruning, infinite lying machine lying about what text is real vs generated by it. LLMs lie, LLM based LLM detectors lie about lies.

  • source
  • parent
  • [–] 3 points 5 months ago

    The problem is it's not plagiarism detector (it would also be a pretty bad one since it can't detect quotes) it's an AI detector.

    It's saying that a direct quote is AI, Which obviously isn't true, it's a quote, which is a different thing.

    If 10% of my thesis is quoting other works that's not the same as my thesis being 10% AI generated. The distinction needs to be made.

  • source
  • parent
  • [–] 11 points 5 months ago (1 child)

    People somehow fail to realize that the AI checker is also AI which is basically just an electric dumbass

  • source
  • hideshow 1 child comment
  • [–] 4 points 5 months ago

    If Mary was smart she would have used the Free AI Text Humanizer.

  • source
  • [–] 2 points 5 months ago

    I mean, scanning a well known piece of literature which likely exists online in its entirety is an obvious way to generate a false positive. This doesn't mean that these tools can't be useful. You need to understand how to use them. When it's appropriate, what context, how much trust to attribute to it, and what other tools you might use along side it. At the end of the day, it's up to the human to be informed, critical, and to form reasonable suspicions or judgements. This isn't novel to AI use, but most tools.

  • source
  • load more comments
    view more: next ›