As the saying goes, it happened before and will happen again : it now the turn of DuckDuckGo to enshitify my searches by bubblification and ignoring quotes.
Also, not interested in IA chatbot, I prefer multiple sources and to cross-check the information myself, thank you.
What alternatives are you using / recommending ?
I once tried to index the web with local Mistral 7b. Here’s what I did:
1. Copy/pasted an English dictionary and removed the definitions, so I just had an Array of about 500k words.
2. Wrote a simple loop to iterate over the words, with the instruction to Mistral to try to guess 10 URLs related to that word. I would typically get 2-3 sites per word.
3. When visiting a site, also check that site for links to other URLs, and summarize it with hash tags, visiting the new sites as well and recursively doing that.
4. Keep a tally of how many sites are being linked to for the page’s “rank”.
In the end you should have a massive list of objects with valid URLs ranked by how popular they are (to each other, by link), and a set of searchable hash tags for each.
I ran this for days and I never got past B, and almost every site was adult content. Most of the sites came from other sites. But it does work! Janky page crawling and ranking.
Anyway more people should take on web search, I agree. LLMs offer shortcuts.
p.s ecosia does have their annoying obligatory ai chat bot that attempts to answer things which i could do without but i just mostly ignore it
If you just want a free search engine, Brave is probably the best index. Bonus points to Marginalia for finding the obscure things Brave doesn't surface.
But if you want to go down the rabbit hole, it's easy to build your own metasearch engine and search several engines at once - an AI model can make one for you in a single night with just a small PHP script, if you have your own website / webost. In my case, the combination of Brave, Mojeek & Marginalia covers all my search needs.
Deeper down the rabbit hole, you can build your own search index with SQLite, as an additional index for your metasearch. There are projects like Hister that will help make your own search index:
Feels like Google Search from 2008.
other things that came up in a search not mentioned in post yet (that I have heard of before):
mojeek or yacy - are these any good?
list of searx instances (I've used some of these a while ago but not recently so idk of their quality currently): https://searx.space/
This could easily yield very interesting (i.e. "not just another bing proxy") results once it starts gaining traction.
I use DuckDuckGo at the moment, only because Startpage started making themselves impossible to use when they started aggressively blocking access via VPNs. As other people have already said,see `noai.duckduckgo.com` and `lite.duckduckgo.com` which might address your specific concerns.
If you are a Mullvad user, there is also Mullvad Leta[2], which has been security audited[3].
I might shift more towards Ecosia/Qwant.
[1] https://blog.ecosia.org/eusp-germany-rollout/ [2] https://mullvad.net/en/blog/introducing-mullvad-leta-a-searc... [3] https://mullvad.net/en/blog/2023/5/16/security-audit-of-our-...
As for Kagi - they've earned a life-long ban from me for political reasons.
It will definitively be a problem in the long term for monetization of the authors. But for me as a user, it's the best.