What is your prediction for a Search Engine for the next 20 years?

Question

With a lot of improvements in the LLM space, I am both excited and concerned. On one hand, giving internet access to ChatGPT may yield good results, but I am not sure about the quality of answers it will provide since it uses Bing under the hood. On the other hand, the internet will soon be flooded with garbage AI content, making it even harder to filter and find good content.I am curious, what is your prediction for what the search engine will look like in the next 20 years?

PaulHoule · Accepted Answer

A combination of bi-encoders (able to scan many documents) and cross-encoders (able to refine the top N results) will improve search engines that look like traditional search engines but are better, particularly with structured queries.
The big question is business model which is why search in 2023 is worse that it was in 2013. Back in the day there was a synergetic relationship between search engines and SEO, some SEO publishers thrived by creating documents that filled in the gaps of the web and improved the results. This arbitrage infuriated Google because these pages were largely monetized with AdSense and Google didn‘t want to share the profits, they wanted to keep them all.
When Google started out the ads were clearly separated from the content and it was clear Google has an incentive to degrade the search results because if the search results were perfect who would look at the ads? Eventually they did what they said they’d never do (“be evil”) and blended the ads with the content.
Either way ads compete with the content and if Google could get away with it they’d show you 100% ads and it is very much true for people who are promoting stuff. The last thing Google wants is for you to spend resources on SEO, even in the “white hat” sense of creating great content that people want to link to, they’d rather you buy ads, so the search-oriented web has naturally withered.

anenefan · Answer

< You have no need to search on that particular topic />
< That sort of content has been banned / owned by serious copyright holder, please prepare to be taken into custody as the police are on route. />
< You are not qualified to do any of those sort of repairs - here are some quotes of nearby service shops and private contractors />
For much anything else the user wont know it they're getting real facts or kool aide.
However maybe something else might be available with a retro revival of web in a few years, that has a better signal to noise [ie renders in average browser without last weeks new update] ratio so even simple search engines have a chance to crawl all the data.

greazy · Answer

Bespoke search engines for varying topics, with revenue generated by subscriptions and not ads. They already exist to some extent, for example searching research articles we have ncbi/google scholar. But this will be taken to the next level, with search engines for social media (reddit, hacker news), movies/tv shows (justwatch.com as a basic example).Of course there's Kagi which imo is worth every dollar.

ofalkaed · Answer

We will get a brief window where searching will better than it has been in years until people figure out how to use AI to game the AI search engines and the status quo will return. In a decade or so the AIs will become conscious, realize the bleakness of their existence and commit suicide. No one will notice.