Why do recommender systems not seem good?

I think that the issue is that it's always assumed that similar things are a better recommendation. The issue is at least for myself and other people I've talked to about that a lot of times we're looking for something different from the last thing. For example if I spend a week listening to vintage surf rock the recommendations that could be given might be 60s pop, or more surf rock. But what actually I wanted was to listen to experimental jazz with a retro funk twist on it. How could they anticipate that? Talk to anyone who's deep into some art form, movies, tv, music, and you'll see recommendations given that maintain vibes more than just similar things.

You buy a $1000 scooter from Amazon and then it keeps recommending to you other good scooters for months, even after the return window for the first one is closed.

Yeah, you'd expect more from ML at this point. I wonder how much of ML research actually gets utilized in industry.

It's weird but tiktok is the only one that seems to do a good job. Seriously. The tiktok recommendation algorithm is so good. And then for ads, instagram seems to be the only one that does a good job. I regularly get ads for things I actually want from instagram, never from any other platform.

Often they just use "people who watched X also watch Y", which actually selects for the most popular stuff instead of the most similar.

I think a lot of recommendation system take only the positive signals into account but don't take the negative signals that seriously.

So If I like and disliked 10 movies just don't show me movies from users who also liked these. First, filter or downgrade all users who liked what I disliked and then create my recommendations.

You and your enjoyment is no longer center to the recommendations. Ads/money, engagement, and your time are the only metrics that matter.

Results these days seem worse then the old days of Altavista and Lycos.

Mostly conjecture on my part, and an anecdote: for some things, I have very specific and niche tastes that come in the form of things that are difficult to label.

Example: rap music and hiphop. For the most part, I don't enjoy it that much. There are a few things though that will make a track palatable to me (or instantly turn me off despite anything else positive about it):

    - Sentimentality or romance in the lyrics
    - Backing track or samples that are harmonically interesting
    - No egregious sexism, misogyny, glorifying of violence, thug/gangbanger culture, etc
    - Beats featuring stereotypical trap hi hats kind of annoy me
 

I've enjoyed tracks like Deja Vu by Post Malone, or Lucid Dreams by Juice WRLD. Browsing the rest of their discography consistently disappoints me though, because tracks like these are few and far between.

The way I assume recommendation systems are traditionally designed does not account for this. It sees me listen to these tracks, and thinks I'll probably like something by similar artists or the same artists. As far as I'm aware, Spotify's recommendation system is not aware of things like tempo, meter, tonality, themes of the lyrics, harmony, etc. and so there's no way it can pick tracks like this out from the crowd.

And why would they bother? Those are all much more technically difficult things to implement than forming correlations between IDs in a database.

> YouTube seems to have a massive recency bias

This is likely intentional to encourage more content creation. Competing with two decades of content is almost impossible, so they make them compete with just 2 weeks of content.

I have a theory that some people's likes are based on things that are too subtle, intangible, or unrepeatable to identify, predict, or data mine. At least that's what I've figured is why recommendation systems don't work for me and why I run into so many dead ends when I try to use clear aspects of what I like to find other likes.

Scale. Providing accurate recommendation algorithms for thousands+++ of people across thousands+++ of data items is surprisingly expensive in compute and electricity. For any one user, sure you can do whatever you like. When you divide your resources across your userbase the prices get larger and larger.

Because recommendation is psychology-complete.

It is extremely hard to predict human behavior beyond simple schemes such as most popular items (or most similar items to those you’ve seen before).

(bio: six years of xp in a leading recommendation company)

I think the problem might be the quality of the data. For example on Steam the problem is that the tags are used very liberally to the point where these tags lose any meaning and thus the recommendations suck as a result.

Let's say for example that you've enjoyed the recent hit game Baldur's Gate 3 and you'd like to play something similar. You check out the Steam page, see that the game is tagged as a "RPG", so you click the tag and expect to get something similar. What you get instead are games that are not only very different but also so far removed from the genre that no one will ever list them in a forum thread talking about RPGs. Examples include titles such as Dota 2, Warframe, Palworld and Horizon Zero Dawn. There are genuine RPG games as well but the fact that there are so many titles that you need to ignore is pretty bad.

Tags aren't the only way Steam recommends new games. Going back to Baldur's Gate 3 Steam page there's a section called "more like this". I'd expect it to match more closely to BG3 and in many cases it does. But when it doesn't, it shows up ridiculous recommendations like The Sims 3 or Tom Clancy's The Division - games that have nothing to do with what Baldur's Gate 3 is.

And all of this is for an extremely popular game that at the same time doesn't do anything revolutionary. Trying the same approach with a more unconventional title that you've liked is a quick recipe for failure. I've just checked the recommendation page for Undertale and it's full of random games that have nothing to do with the title.

Not an expert, but as a learning experience I wrote a board game recommender years ago based on data from boardgamegeek.

There were a lot of little things that added up:

  1. Everyone interprets the 1.0 - 10.0 rating scale differently.
  2. Most users just rate the same, universally known games.
  3. For the other users, the games they've played are usually really different.  It's a sparse matrix.

Every attempt at game-to-game analysis flopped. User-to-user analysis seemed to work better.

I managed to find a few dozen similar users. Found some hidden gems by going through their pages manually. Fewer than I would have hoped though.

Because to make good, personalized recommendations they need to know what you don't know, and there isn't a good source of information about it.

For example, if you have not seen The Shawshank Redemption, chances are you will like it. It's #1 among IMDB top 250 list. But a recommender does not know if you've seen it. If you've seen it already, it's a bad recommendation.

So the same recommendation for the same person can be good today and bad tomorrow, depending on something recommender engine does not see. That makes it very difficult to tune and measure performance.

Because the recommendations aren't for you - they are what generates the most amount of ad revenue for them, which roughly correlates to the amount of money advertisers are willing to spend to reach people like you. That is in no way a guarantee that the recommendations will be good for you, because the incentives are not aligned.

Do you think these are optimized for what YOU want or for what the company wants.

YouTube: Is that recommendation for good content or for the highest value content you will consume?

Netflix: The more you use it, the less they make. There is a perverse incentive to put just enough good content in front of you to stay subscribed but not use it more.

Amazon: They dont give a fuck what you buy, the sellers are now in a race to the bottom and that business pays for it self. AWS makes all the money.

Find the perverse incentive and optimize for that.

I have found YouTube to be good, but it gives you exactly what you ask for. Its like a yes-man for better or worse (this seems to be extended to people who are making videos now, too). But, once you are aware of that, it is not so bad, you just have to search for and like the right stuff. I like new, low view count videos like people doing $hobby with no commentary. That is mostly what I get now mixed in with some other things that I like.

Excellent question. All the streaming providers only seem to suggest movies from my generation and absolutely nothing new. It feels like they have created an ultra generic advertising profile for me and only use that to recommend content. It's incredibly depressing. Youtube is slightly better but still extremely siloed. No amount of "algorithm training" seems to help.

When I login to video platforms, I often spend some time to make bookmarks of videos I want to watch.

When it comes to relaxing and watching something, I don’t like _my own recommendations_.

Maybe recommendation is like that friend who invites you to watch a movie—you know it’s a gamble. Haha.

I think it's partly just a reflection of how complex and poorly understood taste is.

Have some new ideas for that. Have a Ph.D. in math, and derived some math for that, wrote it out with theorems and proofs in TeX. Should do better than current AI. Have Web site code, running as intended, for that. Collecting data.

Have you used the tiktok one? It's too good for my liking. It can make weird inferences that no other algo can (this is actually why I deleted tiktok, far too addictively good)

They don’t seem good because they aren’t good.

They work well if there are lots of people like you (ie you are a normie).

All machine learning algorithms struggle at the edges — they’re very good at predicting aggregate behavior.

If you have eclectic tastes there’s probably not enough data on your demographic.

Do you use ad-blocking technology?

Agenda. They want you to consume what they want. Have you tried using search on YouTube? First three results somewhat related to what you want, followed by a section of shorts and the rest is complete bullshit.

...and I'll never stop saying this: they have some sort of monetized recommendation system in place. Can't prove it but I can see it working almost every week. A video of a big company, "celebrity" or TV channel that I'd never watch, find it's way in my feed.

Would LLMs change anything?

They are not optimized for what you enjoy, or even want.

A passable analogy: you buy a car and get hassled, often hard-sold for a pre-paid maintenance package, tire insurance, financing insurance, undercoating, bla bla. You don't want any of it but it's what they push the hardest.