• 0 Posts
  • 6 Comments
Joined 14 days ago
cake
Cake day: February 27th, 2025

help-circle
  • Maybe there should be known public crawlers hosted by several people that would set up some kind of shared but separated indices, and people could self host their own search page and set it to subscribe to these indices and filter for sites they care about. The index hosted by one person must be public and easily recreated elsewhere, so that if they can’t host it anymore for whatever reason, others can fill the gap.

    Or! Each server would kind of be in some federation, and we all have our own index, some overlapping. The overlapping simply becomes a kind of redundancy.

    Sorry if that makes zero sense or is a bad idea. Just tossing it out cause I thought it might be somewhat viable after some (or much) tweaking. Been somewhat interested in information retrieval lately and this is making my little brain kind of excited.