That was a good read.

Corey doc wrote:

It's not "unethical" to scrape the web in order to create and analyze data-sets. That's just "a search engine"

Equivocating what LLMs do and what goes into LLM web scraping with "a search engine" is messed up. His article that he links about scraping is mostly about how badly copyright works and how analysing trade-secret-walled data can be beneficial both to consumers and science but occasionally bad for citizen privacy, which you'll recognize as mostly irrelevant to the concerns people tend to have against LLM training data providers ddosing the fuck out of everything, and all the rest of the stuff tante does a good job of explaining.

Corey also provides this anecdote:

As a group of human-rights defending forensic statisticians, HRDAG has always relied on cutting edge mathematics in its analysis. With its Colombia project, HRDAG used a large language model to assign probabilities for responsibility for each killing documented in the databases it analyzed.

That is, HRDAG was able to rigorously and legibly say, “This killing has an X% probability of having been carried out by a right-wing militia, a Y% probability of having been carried out by the FARC, and a Z% probability of being unrelated to the civil war.”

The use of large language models — produced from vast corpuses of scraped data — to produce accurate, thorough and comprehensible accounts of the hidden crimes that accompany war and conflict is still in its infancy. But already, these techniques are changing the way we hold criminals to account and bring justice to their victims.

Scraping to make large language models is good, actually.

what the actual shit

edit: I mean, he tried transformer powered voice-to-text and liked it, and now he's all in on the LLMs are a rigorous and accurate tool actually bandwagon?

Also the web scraping article is from 2023 but CD linked it in the recent pluralistic post so I assume his views haven't changed.