Implementing a recommendation system for a video website

Not for 'how-to' coding questions but PHP theory instead, this forum is here for those of us who wish to learn about design aspects of programming with PHP.

Moderator: General Moderators

Post Reply
brownca
Forum Newbie
Posts: 4
Joined: Wed Feb 02, 2011 11:11 am

Implementing a recommendation system for a video website

Post by brownca »

Hi,
I hope this is the right forum to post this in.

I'm making a website (mysql) that allows a user to view a video, and then recommends the next video to be viewed based on many variables (already stored in various tables). My initial thought of how to do this would be to make a 'score' for each video - the higher the score, the more relevant to the user, and thus the top 5 (say) recommended videos can be shown. However, this would mean storing a score for every video and for every user, as well as calculating new scores for every video every time a list of recommendations is requested. This sounds to me like it might be a bit of a problem, but as a newbie to php/mysql I'd really appreciate the input of those more experienced than myself. Is there a better way of doing it that I haven't thought of? Would this idea be feasible with heavy traffic?

If I haven't explained this very well please let me know and I'll try and go into a bit more detail. Thanks.
josh
DevNet Master
Posts: 4872
Joined: Wed Feb 11, 2004 3:23 pm
Location: Palm beach, Florida

Re: Implementing a recommendation system for a video website

Post by josh »

Bayesian or k nearest neighbor the most # of records you'll need is # of videos ^ 2. You don't need a row for each "hit", just a total count for each possible "path" thru the site (and there's likely existing implementations that have been enhanced to use way less storage.)

All you need to know is that after viewing video A that the user is x% likely to goto video C, etc.

More advanced 'related videos' engine would do image recognition and find visually similar things, and also weight in a high relevance to the 'SEO' of the video (title, description, tags). Heck you could do it without tracking views, just use mysql full text search to find videos that have tags that match the tags of the current video, for example. That already returns a 'score' you can go off. You could then use a function of mysql's score with the # of views of the video, and you're done.
brownca
Forum Newbie
Posts: 4
Joined: Wed Feb 02, 2011 11:11 am

Re: Implementing a recommendation system for a video website

Post by brownca »

I see what you're saying, but I want to go a bit more in-depth with the recommendation. For example, I want to give more weight to the videos watched next by 'similar' users, or videos from a playlist that has been subscribed to; and less weight to a video watched recently by the user (they don't want to keep being recommended the same thing), etc. This is mostly all data that's being stored in tables anyway. The difficult part is getting all these different ideas into one score. I mean, calculating the score is not too much of an issue, my main focus is on where I should store it and when I should calculate it. Intuitively I feel I shouldn't be calculating and storing thousands of 'scores' in a database every time a user requests a suggestion, so that's probably the wrong way to go about it. Or is it?
josh
DevNet Master
Posts: 4872
Joined: Wed Feb 11, 2004 3:23 pm
Location: Palm beach, Florida

Re: Implementing a recommendation system for a video website

Post by josh »

brownca wrote:I see what you're saying, but I want to go a bit more in-depth with the recommendation. For example, I want to give more weight to the videos watched next by 'similar' users, or videos from a playlist that has been subscribed to;
The former is already possible with the correlation algorithms I suggested. K-nearest neighbor would literally find the users with the "closets" viewing history to the current user. It would then look at the top videos they viewed. Weighting in additional factors like subscription count is a combination of elementary multiplication, and trial & error.
Intuitively I feel I shouldn't be calculating and storing thousands of 'scores' in a database every time a user requests a suggestion, so that's probably the wrong way to go about it. Or is it?
You'd calculate user's viewing histories. You'd store that. A cron script would run whenever economically feasible. This scheduled task would update the recommendations.

There's a distinction between recommendation videos & related videos.

Recommendation - you need a way to collect feedback, don't go off views, instead have thumbs up & thumbs down like youtube. A view could indicate the user liked or disliked. You have to ask. Just like TIVO.
Related - you should not take the current user into account. Better suited to Bayesian as you're simply interested in "users who watched A are 10% likely to view B next"
brownca
Forum Newbie
Posts: 4
Joined: Wed Feb 02, 2011 11:11 am

Re: Implementing a recommendation system for a video website

Post by brownca »

OK, thanks very much for the advice. I'll read into it :)
Post Reply