submitted 1 day ago* (last edited 1 day ago) by to c/opensource@lemmy.ml
 

About a decade ago, I tried to index the web via the Dewey Decimal System. I had a site laid out similar to Google, where you could browse sites continuously starting from a given call, but the DDS is proprietary, and those people hate anyone who uses their IP without a license. You can Google everyone they've shutdown -- places that weren't even libraries -- for using anything similar to the DDS. I reached out to the group that manages the DDS, and was taken offline before my project even started.

With all the corporate BS lately, and people looking for alternative options, I thought I'd take a search engine old school, and we'd index like Usenet. Except with an XXX.XXX.XXX format.

Problem is, now I've got to find categories, sub categories, and local categories. Categories for animals, chat rooms, cars, construction, wrestling, etc.

I need a brilliantly created categorical system with a TLD.MLD.LLD (top level subject domain, mid level domain, and local level domain) format.

I was going to let site owners Disney Land it, by letting them create an index.json file in the root of their site, choosing for themselves where they wanted to appear in the index. Problem is, as trends change and systems become more exact, site owners would have to update their index. Which could become cumbersome.

Further, if we divided into academic disciplines, there would be things like gaming and model construction which wouldn't fit into those.

It would help if there were an already established categorization system, so that site owners didn't have choice anxiety. And then, as the site grew, site owners could recategorize themselves with user feedback.

Could you guys suggest a few categorization systems (can't be DDS), either of your own making or other sources?

I'm already looking at Usenet: https://www.livinginternet.com/u/uw_hier.htm

Edit: Current UX: https://imgur.com/a/sURm4s1

  • Haven't chosen a name yet.
  • Adult is probably going to be dropped, and when the site owner chooses a category, they can put it in the ADT TLD.
you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 10 hours ago

Bless your heart OP, but attempting to hierarchically taxonomise all of the internet is just futile.

And not just even talking about the infrastructure required, but the fact that pinning ideas into a taxonomy or ontology frames them in image of the taxonomising taxonomer, and not in the one querying it.

It is also one of the reasons that the ideas of Web 2.0 never truly caught on, and why the population of json users dwarfs that of RDF users.

Perhaps instead of a hierarchical categorisation you could use vector embeddings with a similarity measure. That frees you from the burden of taxonomising and perhaps leaves you more energy to refine your indexed corpus for relevance and utility.

Best of luck!

  • source