you are viewing a single comment's thread
view the rest of the comments
[–] 4 points 3 days ago* (last edited 3 days ago) (1 child)

Skimming the linked nature article... is my understanding correct that you can basically disable the watermarking by turning temperature down to 0?

For example, if the LLM distribution is very low entropy, meaning it almost always returns the exact same response to the given prompt, then Tournament sampling cannot choose tokens that score more highly under the g functions.

Other highlights from the linked paper... to get a true positive rate of 90% with a false positive rate of 1% you need 400 tokens (which should be a few paragraphs worth of text)? (If I'm reading figure 3 right?) That actually isn't that much, relative to the lengths of essays people write for high school and college classes. ...well actually... 1% false positive doesn't sound too bad, but if you have thousands of freshmen students all taking classes involving writing essays and checking for watermarks becomes the norm, that is dozens and dozens of false positive, which means lots of false accusations, and as we've seen from how teachers and institutions have tried utilizing the existing "AI detection" tools that are much much less reliable... I'm getting angry just thinking about it.

Edit: on turning temperature down, it should be noted Anthropic and OpenAI have been increasingly denying the end user internals of their models, such as summarizing or even outright hiding the thinking traces, and not allowing them access to temperature settings either.

  • source
  • hideshow 2 child comments
  • [–] [S] 4 points 3 days ago*

    yeah. The word-choice watermarking is an interesting trick for a paper, needs quite a chunk of text, will have a ghastly rate of both false positives and false negatives in real world fraud detection, and is evidently good enough for Gemini not to drink too much its own piss but not much else.

  • source
  • parent