654
PSA Huggingface is scraping GitHub (thelemmy.club)
submitted 1 month ago by to c/programmer_humor@programming.dev
126 comments fedilink hide all child comments
 

https://huggingface.co/spaces/HuggingFaceCode/in-the-stack

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 1 month ago* (3 children)

Yeah, they aren't supposed to scrape that stuff. That's kind of where I'm wondering if they limited their collections.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 3 points 1 month ago

    Well, they included some MPL 2.0 repos of mine at least, but skipped others that use GPL3. Yet another that is a mirror of some otherwise lost firmware files for early 2000s wifi cards (and definitely isn't free software) is also included.

    So possibly they filter out GPL2/3 specifically, rather than only include known permissive licenses. Which is a pretty bad way of doing it.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 2 points 1 month ago (1 child)

    Okay, I see you, I think I may have misunderstood how you were phrasing it. Thinking you were saying they were scanning all of GitHub because it's all permissive.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 3 points 1 month ago* (last edited 1 month ago)

    We'll they said they only scanned stuff that would have allowed them implicitly, now did they really respect that? Their ''stack'' is public, so people can review it. It's opensource, people can audit it at least.

    The big giants can lift whatever they want from Github and we wouldn't have the means to prove it. I'm sure Microsoft is using private repos as they like. It's not a coincidence that Copilot was one of the earlier coding LLMs.

  • permalink
  • fedilink
  • source
  • parent