submitted 1 day ago* (last edited 1 day ago) by to c/opensource@lemmy.ml
 

About a decade ago, I tried to index the web via the Dewey Decimal System. I had a site laid out similar to Google, where you could browse sites continuously starting from a given call, but the DDS is proprietary, and those people hate anyone who uses their IP without a license. You can Google everyone they've shutdown -- places that weren't even libraries -- for using anything similar to the DDS. I reached out to the group that manages the DDS, and was taken offline before my project even started.

With all the corporate BS lately, and people looking for alternative options, I thought I'd take a search engine old school, and we'd index like Usenet. Except with an XXX.XXX.XXX format.

Problem is, now I've got to find categories, sub categories, and local categories. Categories for animals, chat rooms, cars, construction, wrestling, etc.

I need a brilliantly created categorical system with a TLD.MLD.LLD (top level subject domain, mid level domain, and local level domain) format.

I was going to let site owners Disney Land it, by letting them create an index.json file in the root of their site, choosing for themselves where they wanted to appear in the index. Problem is, as trends change and systems become more exact, site owners would have to update their index. Which could become cumbersome.

Further, if we divided into academic disciplines, there would be things like gaming and model construction which wouldn't fit into those.

It would help if there were an already established categorization system, so that site owners didn't have choice anxiety. And then, as the site grew, site owners could recategorize themselves with user feedback.

Could you guys suggest a few categorization systems (can't be DDS), either of your own making or other sources?

I'm already looking at Usenet: https://www.livinginternet.com/u/uw_hier.htm

Edit: Current UX: https://imgur.com/a/sURm4s1

  • Haven't chosen a name yet.
  • Adult is probably going to be dropped, and when the site owner chooses a category, they can put it in the ADT TLD.
you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 21 hours ago*

You can't use the DDC because classification systems require a custodian who is in charge of maintaining the system as the dataset changes.

For the same reason you can't just slap any UPC code on an item and sell it. There are standards for a reason.

You would need to invent your own classification system, and a mechanism that would not only be able to automate the process but also process the dataset quickly enough to keep up with change rate of the internet.

The infrastructure for a project this size would demand a buisness model to finance its upkeep. Its impractical in the same way trying to run your own search engine index isn't really practical, which is why metasearch engines like SearX are a thing.

Maybe start with something more realistic and achievable like indexing the Lemmy Fediverse.

  • source