1132 lines
40 KiB
Plaintext
1132 lines
40 KiB
Plaintext
|
{
|
|||
|
|
"cells": [
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "literary-toyota",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"# Content-based recommenders\n",
|
|||
|
|
"\n",
|
|||
|
|
"Content-based recommenders in their recommendations rely purely on the features of items. Conceptually it can be expressed as a model of the form (personalized):\n",
|
|||
|
|
"<center>\n",
|
|||
|
|
"$$\n",
|
|||
|
|
" score \\sim (user, item\\_feature_1, item\\_feature_2, ..., item\\_feature_n)\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"</center>\n",
|
|||
|
|
"or (not personalized)\n",
|
|||
|
|
"<center>\n",
|
|||
|
|
"$$\n",
|
|||
|
|
" score \\sim (item\\_feature_1, item\\_feature_2, ..., item\\_feature_n)\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"</center>\n",
|
|||
|
|
"\n",
|
|||
|
|
" + Content-based recommenders do not suffer from the cold-start problem for new items.\n",
|
|||
|
|
" - They do not use information about complex patterns of user-item interactions - what other similar users have already discovered and liked."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "exciting-specific",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"%matplotlib inline\n",
|
|||
|
|
"%load_ext autoreload\n",
|
|||
|
|
"%autoreload 2\n",
|
|||
|
|
"\n",
|
|||
|
|
"import numpy as np\n",
|
|||
|
|
"import pandas as pd\n",
|
|||
|
|
"import matplotlib.pyplot as plt\n",
|
|||
|
|
"import seaborn as sns\n",
|
|||
|
|
"from IPython.display import Markdown, display, HTML\n",
|
|||
|
|
"from collections import defaultdict\n",
|
|||
|
|
"from sklearn.model_selection import KFold\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Fix the dying kernel problem (only a problem in some installations - you can remove it, if it works without it)\n",
|
|||
|
|
"import os\n",
|
|||
|
|
"os.environ['KMP_DUPLICATE_LIB_OK'] = 'True'"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "administrative-charleston",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"# Load the data"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "architectural-andrews",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"ml_ratings_df = pd.read_csv(os.path.join(\"data\", \"movielens_small\", \"ratings.csv\")).rename(columns={'userId': 'user_id', 'movieId': 'item_id'})\n",
|
|||
|
|
"ml_movies_df = pd.read_csv(os.path.join(\"data\", \"movielens_small\", \"movies.csv\")).rename(columns={'movieId': 'item_id'})\n",
|
|||
|
|
"ml_df = pd.merge(ml_ratings_df, ml_movies_df, on='item_id')\n",
|
|||
|
|
"ml_df.head(10)\n",
|
|||
|
|
"\n",
|
|||
|
|
"display(HTML(ml_movies_df.head(10).to_html()))\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Filter the data to reduce the number of movies\n",
|
|||
|
|
"rng = np.random.RandomState(seed=6789)\n",
|
|||
|
|
"left_ids = rng.choice(ml_movies_df['item_id'], size=100, replace=False)\n",
|
|||
|
|
"\n",
|
|||
|
|
"ml_ratings_df = ml_ratings_df.loc[ml_ratings_df['item_id'].isin(left_ids)]\n",
|
|||
|
|
"ml_movies_df = ml_movies_df.loc[ml_movies_df['item_id'].isin(left_ids)]\n",
|
|||
|
|
"ml_df = ml_df.loc[ml_df['item_id'].isin(left_ids)]\n",
|
|||
|
|
"\n",
|
|||
|
|
"print(\"Number of left interactions: {}\".format(len(ml_ratings_df)))"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "effective-renaissance",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"# Recommender class\n",
|
|||
|
|
"\n",
|
|||
|
|
"Remark: Docstrings written in reStructuredText (reST) used by Sphinx to automatically generate code documentation. It is also used by default by PyCharm (type triple quotes after defining a class or a method and hit enter)."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "cooperative-synthesis",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"class Recommender(object):\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" Base recommender class.\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" \n",
|
|||
|
|
" def __init__(self):\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" Initialize base recommender params and variables.\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" pass\n",
|
|||
|
|
" \n",
|
|||
|
|
" def fit(self, interactions_df, users_df, items_df):\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" Training of the recommender.\n",
|
|||
|
|
" \n",
|
|||
|
|
" :param pd.DataFrame interactions_df: DataFrame with recorded interactions between users and items \n",
|
|||
|
|
" defined by user_id, item_id and features of the interaction.\n",
|
|||
|
|
" :param pd.DataFrame users_df: DataFrame with users and their features defined by user_id and the user feature columns.\n",
|
|||
|
|
" :param pd.DataFrame items_df: DataFrame with items and their features defined by item_id and the item feature columns.\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" pass\n",
|
|||
|
|
" \n",
|
|||
|
|
" def recommend(self, users_df, items_df, n_recommendations=1):\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" Serving of recommendations. Scores items in items_df for each user in users_df and returns \n",
|
|||
|
|
" top n_recommendations for each user.\n",
|
|||
|
|
" \n",
|
|||
|
|
" :param pd.DataFrame users_df: DataFrame with users and their features for which recommendations should be generated.\n",
|
|||
|
|
" :param pd.DataFrame items_df: DataFrame with items and their features which should be scored.\n",
|
|||
|
|
" :param int n_recommendations: Number of recommendations to be returned for each user.\n",
|
|||
|
|
" :return: DataFrame with user_id, item_id and score as columns returning n_recommendations top recommendations \n",
|
|||
|
|
" for each user.\n",
|
|||
|
|
" :rtype: pd.DataFrame\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" \n",
|
|||
|
|
" recommendations = pd.DataFrame(columns=['user_id', 'item_id', 'score'])\n",
|
|||
|
|
" \n",
|
|||
|
|
" for ix, user in users_df.iterrows():\n",
|
|||
|
|
" user_recommendations = pd.DataFrame({'user_id': user['user_id'],\n",
|
|||
|
|
" 'item_id': [-1] * n_recommendations,\n",
|
|||
|
|
" 'score': [3.0] * n_recommendations})\n",
|
|||
|
|
"\n",
|
|||
|
|
" recommendations = pd.concat([recommendations, user_recommendations])\n",
|
|||
|
|
"\n",
|
|||
|
|
" return recommendations"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "cleared-warehouse",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"# Evaluation measures"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "overall-perspective",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"## Explicit feedback - ratings"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "tamil-anderson",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"### RMSE - Root Mean Squared Error\n",
|
|||
|
|
"\n",
|
|||
|
|
"<center>\n",
|
|||
|
|
"$$\n",
|
|||
|
|
" RMSE = \\sqrt{\\frac{\\sum_{i}^N (\\hat{r}_i - r_i)^2}{N}}\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"</center>\n",
|
|||
|
|
"\n",
|
|||
|
|
"where $\\hat{r}_i$ are the predicted ratings and $r_i$ are the real ratings and $N$ is the number of items in the test set.\n",
|
|||
|
|
"\n",
|
|||
|
|
" + Very well-behaved analytically and therefore extensively used to train models, especially neural networks.\n",
|
|||
|
|
" - The scale of errors dependent on data which reduced comparability between different datasets."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "entitled-snake",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"def rmse(r_pred, r_real):\n",
|
|||
|
|
" return np.sqrt(np.sum(np.power(r_pred - r_real, 2)) / len(r_pred))\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Test\n",
|
|||
|
|
"\n",
|
|||
|
|
"print(\"RMSE = {:.2f}\".format(rmse(np.array([2.1, 1.2, 3.8, 4.2, 3.6]), np.array([3, 2, 4, 5, 1]))))"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "unknown-arrival",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"### MRE - Mean Relative Error\n",
|
|||
|
|
"\n",
|
|||
|
|
"<center>\n",
|
|||
|
|
"$$\n",
|
|||
|
|
" MRE = \\frac{1}{N} \\sum_{i}^N \\frac{|\\hat{r}_i - r_i|}{|r_i|}\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"</center>\n",
|
|||
|
|
"\n",
|
|||
|
|
"where $\\hat{r}_i$ are the predicted ratings and $r_i$ are the real ratings and $N$ is the number of items in the test set.\n",
|
|||
|
|
"\n",
|
|||
|
|
" + Easily interpretable (average percentage error) and with a meaning understandable for business.\n",
|
|||
|
|
" - Blows up when there are values close to zero among the predicted values."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "dying-course",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"def mre(r_pred, r_real):\n",
|
|||
|
|
" return 1 / len(r_pred) * np.sum(np.abs(r_pred - r_real) / np.abs(r_real))\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Test\n",
|
|||
|
|
"\n",
|
|||
|
|
"print(\"MRE = {:.4f}\".format(mre(np.array([2.1, 1.2, 3.8, 4.2, 3.6]), np.array([3, 2, 4, 5, 1]))))"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "imported-contribution",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"### TRE - Total Relative Error\n",
|
|||
|
|
"\n",
|
|||
|
|
"<center>\n",
|
|||
|
|
"$$\n",
|
|||
|
|
" TRE = \\frac{\\sum_{i}^N |\\hat{r}_i - r_i|}{\\sum_{i}^N |r_i|}\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"</center>\n",
|
|||
|
|
"\n",
|
|||
|
|
"where $\\hat{r}_i$ are the predicted ratings and $r_i$ are the real ratings and $N$ is the number of items in the test set.\n",
|
|||
|
|
"\n",
|
|||
|
|
" + Easily interpretable (total percentage error) and with a meaning understandable for business.\n",
|
|||
|
|
" + Reliable even for very small predicted values.\n",
|
|||
|
|
" - Does not distinguish between a case when one prediction is very bad and other are very good and a case when all predictions are mediocre."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "premium-trouble",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"def tre(r_pred, r_real):\n",
|
|||
|
|
" return np.sum(np.abs(r_pred - r_real)) / np.sum(np.abs(r_real))\n",
|
|||
|
|
"\n",
|
|||
|
|
"# Test\n",
|
|||
|
|
"\n",
|
|||
|
|
"print(\"TRE = {:.4f}\".format(tre(np.array([2.1, 1.2, 3.8, 4.2, 3.6]), np.array([3, 2, 4, 5, 1]))))"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "quantitative-navigation",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"## Implicit feedback - binary indicators of interactions"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "obvious-egypt",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"### HR@n - Hit Ratio \n",
|
|||
|
|
"How many hits did we score in the first n recommendations.\n",
|
|||
|
|
"<br/>\n",
|
|||
|
|
"<br/>\n",
|
|||
|
|
"<center>\n",
|
|||
|
|
"$$\n",
|
|||
|
|
" \\text{HR@}n = \\frac{\\sum_{u} \\sum_{i \\in I_u} r_{u, i} \\cdot 1_{\\hat{D}_n(u)}(i)}{M}\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"</center>\n",
|
|||
|
|
"\n",
|
|||
|
|
"where:\n",
|
|||
|
|
" * $r_{u, i}$ is $1$ if there was an interaction between user $u$ and item $i$ in the test set and $0$ otherwise, \n",
|
|||
|
|
" * $\\hat{D}_n$ is the set of the first $n$ recommendations for user $u$, \n",
|
|||
|
|
" * $1_{\\hat{D}_n}(i)$ is $1$ if and only if $i \\in \\hat{D}_n$, otherwise it's equal to $0$,\n",
|
|||
|
|
" * $M$ is the number of users.\n",
|
|||
|
|
"\n",
|
|||
|
|
"\n",
|
|||
|
|
" + Easily interpretable.\n",
|
|||
|
|
" - Does not take the rank of each recommendation into account."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "found-amazon",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"def hr(recommendations, real_interactions, n=1):\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" Assumes recommendations are ordered by user_id and then by score.\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" # Transform real_interactions to a dict for a large speed-up\n",
|
|||
|
|
" rui = defaultdict(lambda: 0)\n",
|
|||
|
|
" \n",
|
|||
|
|
" for idx, row in real_interactions.iterrows():\n",
|
|||
|
|
" rui[(row['user_id'], row['item_id'])] = 1\n",
|
|||
|
|
" \n",
|
|||
|
|
" hr = 0.0\n",
|
|||
|
|
" \n",
|
|||
|
|
" previous_user_id = -1\n",
|
|||
|
|
" rank = 0\n",
|
|||
|
|
" for idx, row in recommendations.iterrows():\n",
|
|||
|
|
" if previous_user_id == row['user_id']:\n",
|
|||
|
|
" rank += 1\n",
|
|||
|
|
" else:\n",
|
|||
|
|
" rank = 1\n",
|
|||
|
|
" \n",
|
|||
|
|
" if rank <= n:\n",
|
|||
|
|
" hr += rui[(row['user_id'], row['item_id'])]\n",
|
|||
|
|
" \n",
|
|||
|
|
" previous_user_id = row['user_id']\n",
|
|||
|
|
" \n",
|
|||
|
|
" hr /= len(recommendations['user_id'].unique())\n",
|
|||
|
|
" \n",
|
|||
|
|
" return hr\n",
|
|||
|
|
"\n",
|
|||
|
|
" \n",
|
|||
|
|
"recommendations = pd.DataFrame(\n",
|
|||
|
|
" [\n",
|
|||
|
|
" [1, 13, 0.9],\n",
|
|||
|
|
" [1, 45, 0.8],\n",
|
|||
|
|
" [1, 22, 0.71],\n",
|
|||
|
|
" [1, 77, 0.55],\n",
|
|||
|
|
" [1, 9, 0.52],\n",
|
|||
|
|
" [2, 11, 0.85],\n",
|
|||
|
|
" [2, 13, 0.69],\n",
|
|||
|
|
" [2, 25, 0.64],\n",
|
|||
|
|
" [2, 6, 0.60],\n",
|
|||
|
|
" [2, 77, 0.53]\n",
|
|||
|
|
" \n",
|
|||
|
|
" ], columns=['user_id', 'item_id', 'score'])\n",
|
|||
|
|
"\n",
|
|||
|
|
"display(HTML(recommendations.to_html()))\n",
|
|||
|
|
"\n",
|
|||
|
|
"real_interactions = pd.DataFrame(\n",
|
|||
|
|
" [\n",
|
|||
|
|
" [1, 45],\n",
|
|||
|
|
" [1, 22],\n",
|
|||
|
|
" [1, 77],\n",
|
|||
|
|
" [2, 13],\n",
|
|||
|
|
" [2, 77]\n",
|
|||
|
|
" \n",
|
|||
|
|
" ], columns=['user_id', 'item_id'])\n",
|
|||
|
|
"\n",
|
|||
|
|
"display(HTML(real_interactions.to_html()))\n",
|
|||
|
|
" \n",
|
|||
|
|
"print(\"HR@3 = {:.4f}\".format(hr(recommendations, real_interactions, n=3)))"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "behind-munich",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"### NDCG@n - Normalized Discounted Cumulative Gain\n",
|
|||
|
|
"\n",
|
|||
|
|
"How many hits did we score in the first n recommendations discounted by the position of each recommendation.\n",
|
|||
|
|
"<br/>\n",
|
|||
|
|
"<br/>\n",
|
|||
|
|
"<center>\n",
|
|||
|
|
"$$\n",
|
|||
|
|
" \\text{NDCG@}n = \\frac{\\sum_{u} \\sum_{i \\in I_u} \\frac{r_{u, i}}{log\\left(1 + v_{\\hat{D}_n(u)}(i)\\right)}}{M}\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"</center>\n",
|
|||
|
|
"\n",
|
|||
|
|
"where:\n",
|
|||
|
|
" * $r_{u, i}$ is $1$ if there was an interaction between user $u$ and item $i$ in the test set and $0$ otherwise, \n",
|
|||
|
|
" * $\\hat{D}_n(u)$ is the set of the first $n$ recommendations for user $u$, \n",
|
|||
|
|
" * $v_{\\hat{D}_n(u)}(i)$ is the position of item $i$ in recommendations $\\hat{D}_n$,\n",
|
|||
|
|
" * $M$ is the number of users.\n",
|
|||
|
|
"\n",
|
|||
|
|
"\n",
|
|||
|
|
" - Takes the rank of each recommendation into account."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "floral-anatomy",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"def ndcg(recommendations, real_interactions, n=1):\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" Assumes recommendations are ordered by user_id and then by score.\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" # Transform real_interactions to a dict for a large speed-up\n",
|
|||
|
|
" rui = defaultdict(lambda: 0)\n",
|
|||
|
|
" \n",
|
|||
|
|
" for idx, row in real_interactions.iterrows():\n",
|
|||
|
|
" rui[(row['user_id'], row['item_id'])] = 1\n",
|
|||
|
|
" \n",
|
|||
|
|
" ndcg = 0.0\n",
|
|||
|
|
" \n",
|
|||
|
|
" previous_user_id = -1\n",
|
|||
|
|
" rank = 0\n",
|
|||
|
|
" for idx, row in recommendations.iterrows():\n",
|
|||
|
|
" if previous_user_id == row['user_id']:\n",
|
|||
|
|
" rank += 1\n",
|
|||
|
|
" else:\n",
|
|||
|
|
" rank = 1\n",
|
|||
|
|
" \n",
|
|||
|
|
" if rank <= n:\n",
|
|||
|
|
" ndcg += rui[(row['user_id'], row['item_id'])] / np.log2(1 + rank)\n",
|
|||
|
|
" \n",
|
|||
|
|
" previous_user_id = row['user_id']\n",
|
|||
|
|
" \n",
|
|||
|
|
" ndcg /= len(recommendations['user_id'].unique())\n",
|
|||
|
|
" \n",
|
|||
|
|
" return ndcg\n",
|
|||
|
|
"\n",
|
|||
|
|
" \n",
|
|||
|
|
"recommendations = pd.DataFrame(\n",
|
|||
|
|
" [\n",
|
|||
|
|
" [1, 13, 0.9],\n",
|
|||
|
|
" [1, 45, 0.8],\n",
|
|||
|
|
" [1, 22, 0.71],\n",
|
|||
|
|
" [1, 77, 0.55],\n",
|
|||
|
|
" [1, 9, 0.52],\n",
|
|||
|
|
" [2, 11, 0.85],\n",
|
|||
|
|
" [2, 13, 0.69],\n",
|
|||
|
|
" [2, 25, 0.64],\n",
|
|||
|
|
" [2, 6, 0.60],\n",
|
|||
|
|
" [2, 77, 0.53]\n",
|
|||
|
|
" \n",
|
|||
|
|
" ], columns=['user_id', 'item_id', 'score'])\n",
|
|||
|
|
"\n",
|
|||
|
|
"display(HTML(recommendations.to_html()))\n",
|
|||
|
|
"\n",
|
|||
|
|
"real_interactions = pd.DataFrame(\n",
|
|||
|
|
" [\n",
|
|||
|
|
" [1, 45],\n",
|
|||
|
|
" [1, 22],\n",
|
|||
|
|
" [1, 77],\n",
|
|||
|
|
" [2, 13],\n",
|
|||
|
|
" [2, 77]\n",
|
|||
|
|
" \n",
|
|||
|
|
" ], columns=['user_id', 'item_id'])\n",
|
|||
|
|
"\n",
|
|||
|
|
"display(HTML(real_interactions.to_html()))\n",
|
|||
|
|
" \n",
|
|||
|
|
"print(\"NDCG@3 = {:.4f}\".format(ndcg(recommendations, real_interactions, n=3)))"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "appointed-baltimore",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"# Testing routines (offline)"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "bizarre-elevation",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"## Train and test set split"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "fatty-blackjack",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"### Explicit feedback"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "negative-cigarette",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"def evaluate_train_test_split_explicit(recommender, interactions_df, items_df, seed=6789):\n",
|
|||
|
|
" rng = np.random.RandomState(seed=seed)\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Split the dataset into train and test\n",
|
|||
|
|
" \n",
|
|||
|
|
" shuffle = np.arange(len(interactions_df))\n",
|
|||
|
|
" rng.shuffle(shuffle)\n",
|
|||
|
|
" shuffle = list(shuffle)\n",
|
|||
|
|
"\n",
|
|||
|
|
" train_test_split = 0.8\n",
|
|||
|
|
" split_index = int(len(interactions_df) * train_test_split)\n",
|
|||
|
|
"\n",
|
|||
|
|
" interactions_df_train = interactions_df.iloc[shuffle[:split_index]]\n",
|
|||
|
|
" interactions_df_test = interactions_df.iloc[shuffle[split_index:]]\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Train the recommender\n",
|
|||
|
|
" \n",
|
|||
|
|
" recommender.fit(interactions_df_train, None, items_df)\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Gather predictions\n",
|
|||
|
|
" \n",
|
|||
|
|
" r_pred = []\n",
|
|||
|
|
" \n",
|
|||
|
|
" for idx, row in interactions_df_test.iterrows():\n",
|
|||
|
|
" users_df = pd.DataFrame([row['user_id']], columns=['user_id'])\n",
|
|||
|
|
" eval_items_df = pd.DataFrame([row['item_id']], columns=['item_id'])\n",
|
|||
|
|
" eval_items_df = pd.merge(eval_items_df, items_df, on='item_id')\n",
|
|||
|
|
" recommendations = recommender.recommend(users_df, eval_items_df, n_recommendations=1)\n",
|
|||
|
|
" \n",
|
|||
|
|
" r_pred.append(recommendations.iloc[0]['score'])\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Gather real ratings\n",
|
|||
|
|
" \n",
|
|||
|
|
" r_real = np.array(interactions_df_test['rating'].tolist())\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Return evaluation metrics\n",
|
|||
|
|
" \n",
|
|||
|
|
" return rmse(r_pred, r_real), mre(r_pred, r_real), tre(r_pred, r_real)\n",
|
|||
|
|
"\n",
|
|||
|
|
"recommender = Recommender()\n",
|
|||
|
|
"\n",
|
|||
|
|
"results = [['BaseRecommender'] + list(evaluate_train_test_split_explicit(\n",
|
|||
|
|
" recommender, ml_ratings_df.loc[:, ['user_id', 'item_id', 'rating']], ml_movies_df))]\n",
|
|||
|
|
"\n",
|
|||
|
|
"results = pd.DataFrame(results, \n",
|
|||
|
|
" columns=['Recommender', 'RMSE', 'MRE', 'TRE'])\n",
|
|||
|
|
"\n",
|
|||
|
|
"display(HTML(results.to_html()))"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "naval-croatia",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"### Implicit feedback"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "separated-enclosure",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"**Task 1.** Implement the following method for train-test split evaluation for implicit feedback."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "considerable-sunrise",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"def evaluate_train_test_split_implicit(recommender, interactions_df, items_df, seed=6789):\n",
|
|||
|
|
" # Write your code here\n",
|
|||
|
|
" pass"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "muslim-tunisia",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"## Leave-one-out, leave-k-out, cross-validation"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "adjusted-spirit",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"### Explicit feedback"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "alpine-luxembourg",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"**Task 2.** Implement the following method for leave-one-out evaluation for explicit feedback."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "flexible-runner",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"def evaluate_leave_one_out_explicit(recommender, interactions_df, items_df, max_evals=100, seed=6789):\n",
|
|||
|
|
" # Write your code here\n",
|
|||
|
|
" pass"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "engaged-lloyd",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"### Implicit feedback"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "surrounded-newton",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"def evaluate_leave_one_out_implicit(recommender, interactions_df, items_df, max_evals=10, seed=6789):\n",
|
|||
|
|
" rng = np.random.RandomState(seed=seed)\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Prepare splits of the datasets\n",
|
|||
|
|
" kf = KFold(n_splits=len(interactions_df), random_state=rng, shuffle=True)\n",
|
|||
|
|
" \n",
|
|||
|
|
" hr_1 = []\n",
|
|||
|
|
" hr_3 = []\n",
|
|||
|
|
" hr_5 = []\n",
|
|||
|
|
" hr_10 = []\n",
|
|||
|
|
" ndcg_1 = []\n",
|
|||
|
|
" ndcg_3 = []\n",
|
|||
|
|
" ndcg_5 = []\n",
|
|||
|
|
" ndcg_10 = []\n",
|
|||
|
|
" \n",
|
|||
|
|
" # For each split of the dataset train the recommender, generate recommendations and evaluate\n",
|
|||
|
|
" \n",
|
|||
|
|
" n_eval = 1\n",
|
|||
|
|
" for train_index, test_index in kf.split(interactions_df.index):\n",
|
|||
|
|
" interactions_df_train = interactions_df.loc[interactions_df.index[train_index]]\n",
|
|||
|
|
" interactions_df_test = interactions_df.loc[interactions_df.index[test_index]]\n",
|
|||
|
|
" \n",
|
|||
|
|
" recommender.fit(interactions_df_train, None, items_df)\n",
|
|||
|
|
" recommendations = recommender.recommend(interactions_df_test.loc[:, ['user_id']], items_df, n_recommendations=10)\n",
|
|||
|
|
" \n",
|
|||
|
|
" hr_1.append(hr(recommendations, interactions_df_test, n=1))\n",
|
|||
|
|
" hr_3.append(hr(recommendations, interactions_df_test, n=3))\n",
|
|||
|
|
" hr_5.append(hr(recommendations, interactions_df_test, n=5))\n",
|
|||
|
|
" hr_10.append(hr(recommendations, interactions_df_test, n=10))\n",
|
|||
|
|
" ndcg_1.append(ndcg(recommendations, interactions_df_test, n=1))\n",
|
|||
|
|
" ndcg_3.append(ndcg(recommendations, interactions_df_test, n=3))\n",
|
|||
|
|
" ndcg_5.append(ndcg(recommendations, interactions_df_test, n=5))\n",
|
|||
|
|
" ndcg_10.append(ndcg(recommendations, interactions_df_test, n=10))\n",
|
|||
|
|
" \n",
|
|||
|
|
" if n_eval == max_evals:\n",
|
|||
|
|
" break\n",
|
|||
|
|
" n_eval += 1\n",
|
|||
|
|
" \n",
|
|||
|
|
" hr_1 = np.mean(hr_1)\n",
|
|||
|
|
" hr_3 = np.mean(hr_3)\n",
|
|||
|
|
" hr_5 = np.mean(hr_5)\n",
|
|||
|
|
" hr_10 = np.mean(hr_10)\n",
|
|||
|
|
" ndcg_1 = np.mean(ndcg_1)\n",
|
|||
|
|
" ndcg_3 = np.mean(ndcg_3)\n",
|
|||
|
|
" ndcg_5 = np.mean(ndcg_5)\n",
|
|||
|
|
" ndcg_10 = np.mean(ndcg_10)\n",
|
|||
|
|
" \n",
|
|||
|
|
" return hr_1, hr_3, hr_5, hr_10, ndcg_1, ndcg_3, ndcg_5, ndcg_10\n",
|
|||
|
|
"\n",
|
|||
|
|
"recommender = Recommender()\n",
|
|||
|
|
"\n",
|
|||
|
|
"results = [['BaseRecommender'] + list(evaluate_leave_one_out_implicit(\n",
|
|||
|
|
" recommender, ml_ratings_df.loc[:, ['user_id', 'item_id']], ml_movies_df))]\n",
|
|||
|
|
"\n",
|
|||
|
|
"results = pd.DataFrame(results, \n",
|
|||
|
|
" columns=['Recommender', 'HR@1', 'HR@3', 'HR@5', 'HR@10', 'NDCG@1', 'NDCG@3', 'NDCG@5', 'NDCG@10'])\n",
|
|||
|
|
"\n",
|
|||
|
|
"display(HTML(results.to_html()))"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "optional-chain",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"# Linear Regression Recommender\n",
|
|||
|
|
"\n",
|
|||
|
|
"For every movie we transform its genres into one-hot encoded features and then fit a linear regression model to those features and actual ratings."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "sonic-horror",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"from sklearn.linear_model import LinearRegression\n",
|
|||
|
|
"from sklearn.preprocessing import MultiLabelBinarizer\n",
|
|||
|
|
"\n",
|
|||
|
|
"class LinearRegressionRecommender(object):\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" Base recommender class.\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" \n",
|
|||
|
|
" def __init__(self):\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" Initialize base recommender params and variables.\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" self.model = None\n",
|
|||
|
|
" self.mlb = None\n",
|
|||
|
|
" \n",
|
|||
|
|
" def fit(self, interactions_df, users_df, items_df):\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" Training of the recommender.\n",
|
|||
|
|
" \n",
|
|||
|
|
" :param pd.DataFrame interactions_df: DataFrame with recorded interactions between users and items \n",
|
|||
|
|
" defined by user_id, item_id and features of the interaction.\n",
|
|||
|
|
" :param pd.DataFrame users_df: DataFrame with users and their features defined by user_id and the user feature columns.\n",
|
|||
|
|
" :param pd.DataFrame items_df: DataFrame with items and their features defined by item_id and the item feature columns.\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" \n",
|
|||
|
|
" interactions_df = pd.merge(interactions_df, items_df, on='item_id')\n",
|
|||
|
|
" interactions_df.loc[:, 'genres'] = interactions_df['genres'].str.replace(\"-\", \"_\", regex=False)\n",
|
|||
|
|
" interactions_df.loc[:, 'genres'] = interactions_df['genres'].str.replace(\" \", \"_\", regex=False)\n",
|
|||
|
|
" interactions_df.loc[:, 'genres'] = interactions_df['genres'].str.lower()\n",
|
|||
|
|
" interactions_df.loc[:, 'genres'] = interactions_df['genres'].str.split(\"|\")\n",
|
|||
|
|
" \n",
|
|||
|
|
" self.mlb = MultiLabelBinarizer()\n",
|
|||
|
|
" interactions_df = interactions_df.join(\n",
|
|||
|
|
" pd.DataFrame(self.mlb.fit_transform(interactions_df.pop('genres')),\n",
|
|||
|
|
" columns=self.mlb.classes_,\n",
|
|||
|
|
" index=interactions_df.index))\n",
|
|||
|
|
" \n",
|
|||
|
|
"# print(interactions_df.head())\n",
|
|||
|
|
" \n",
|
|||
|
|
" x = interactions_df.loc[:, self.mlb.classes_].values\n",
|
|||
|
|
" y = interactions_df['rating'].values\n",
|
|||
|
|
" \n",
|
|||
|
|
" self.model = LinearRegression().fit(x, y)\n",
|
|||
|
|
" \n",
|
|||
|
|
" def recommend(self, users_df, items_df, n_recommendations=1):\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" Serving of recommendations. Scores items in items_df for each user in users_df and returns \n",
|
|||
|
|
" top n_recommendations for each user.\n",
|
|||
|
|
" \n",
|
|||
|
|
" :param pd.DataFrame users_df: DataFrame with users and their features for which recommendations should be generated.\n",
|
|||
|
|
" :param pd.DataFrame items_df: DataFrame with items and their features which should be scored.\n",
|
|||
|
|
" :param int n_recommendations: Number of recommendations to be returned for each user.\n",
|
|||
|
|
" :return: DataFrame with user_id, item_id and score as columns returning n_recommendations top recommendations \n",
|
|||
|
|
" for each user.\n",
|
|||
|
|
" :rtype: pd.DataFrame\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Transform the item to be scored into proper features\n",
|
|||
|
|
" \n",
|
|||
|
|
" items_df = items_df.copy()\n",
|
|||
|
|
" items_df.loc[:, 'genres'] = items_df['genres'].str.replace(\"-\", \"_\", regex=False)\n",
|
|||
|
|
" items_df.loc[:, 'genres'] = items_df['genres'].str.replace(\" \", \"_\", regex=False)\n",
|
|||
|
|
" items_df.loc[:, 'genres'] = items_df['genres'].str.lower()\n",
|
|||
|
|
" items_df.loc[:, 'genres'] = items_df['genres'].str.split(\"|\")\n",
|
|||
|
|
" \n",
|
|||
|
|
" items_df = items_df.join(\n",
|
|||
|
|
" pd.DataFrame(self.mlb.transform(items_df.pop('genres')),\n",
|
|||
|
|
" columns=self.mlb.classes_,\n",
|
|||
|
|
" index=items_df.index))\n",
|
|||
|
|
" \n",
|
|||
|
|
"# print(items_df)\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Score the item\n",
|
|||
|
|
" \n",
|
|||
|
|
" recommendations = pd.DataFrame(columns=['user_id', 'item_id', 'score'])\n",
|
|||
|
|
" \n",
|
|||
|
|
" for ix, user in users_df.iterrows():\n",
|
|||
|
|
" score = self.model.predict(items_df.loc[:, self.mlb.classes_].values)[0]\n",
|
|||
|
|
" \n",
|
|||
|
|
" user_recommendations = pd.DataFrame({'user_id': [user['user_id']],\n",
|
|||
|
|
" 'item_id': items_df.iloc[0]['item_id'],\n",
|
|||
|
|
" 'score': score})\n",
|
|||
|
|
"\n",
|
|||
|
|
" recommendations = pd.concat([recommendations, user_recommendations])\n",
|
|||
|
|
"\n",
|
|||
|
|
" return recommendations"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "colored-favorite",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Quick test of the recommender\n",
|
|||
|
|
"\n",
|
|||
|
|
"lr_recommender = LinearRegressionRecommender()\n",
|
|||
|
|
"lr_recommender.fit(ml_ratings_df, None, ml_movies_df)\n",
|
|||
|
|
"recommendations = lr_recommender.recommend(pd.DataFrame([[1], [2]], columns=['user_id']), ml_movies_df, 1)\n",
|
|||
|
|
"\n",
|
|||
|
|
"recommendations = pd.merge(recommendations, ml_movies_df, on='item_id')\n",
|
|||
|
|
"display(HTML(recommendations.to_html()))"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "national-sight",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"lr_recommender = LinearRegressionRecommender()\n",
|
|||
|
|
"\n",
|
|||
|
|
"results = [['LinearRegressionRecommender'] + list(evaluate_train_test_split_explicit(\n",
|
|||
|
|
" lr_recommender, ml_ratings_df.loc[:, ['user_id', 'item_id', 'rating']], ml_movies_df, seed=6789))]\n",
|
|||
|
|
"\n",
|
|||
|
|
"results = pd.DataFrame(results, \n",
|
|||
|
|
" columns=['Recommender', 'RMSE', 'MRE', 'TRE'])\n",
|
|||
|
|
"\n",
|
|||
|
|
"display(HTML(results.to_html()))"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "static-mozambique",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"# TF-IDF Recommender\n",
|
|||
|
|
"TF-IDF stands for term frequency–inverse document frequency. Typically Tf-IDF method is used to assign keywords (words describing the gist of a document) to documents in a corpus of documents.\n",
|
|||
|
|
"\n",
|
|||
|
|
"In our case we will treat users as documents and genres as words.\n",
|
|||
|
|
"\n",
|
|||
|
|
"Term-frequency is given by the following formula:\n",
|
|||
|
|
"<center>\n",
|
|||
|
|
"$$\n",
|
|||
|
|
" \\text{tf}(g, u) = f_{g, u}\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"</center>\n",
|
|||
|
|
"where $f_{g, i}$ is the number of times genre $g$ appear for movies watched by user $u$.\n",
|
|||
|
|
"\n",
|
|||
|
|
"Inverse document frequency is defined as follows:\n",
|
|||
|
|
"<center>\n",
|
|||
|
|
"$$\n",
|
|||
|
|
" \\text{idf}(g) = \\log \\frac{N}{n_g}\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"</center>\n",
|
|||
|
|
"where $N$ is the number of users and $n_g$ is the number of users with $g$ in their genres list.\n",
|
|||
|
|
"\n",
|
|||
|
|
"Finally, tf-idf is defined as follows:\n",
|
|||
|
|
"<center>\n",
|
|||
|
|
"$$\n",
|
|||
|
|
" \\text{tfidf}(g, u) = \\text{tf}(g, u) \\cdot \\text{idf}(g)\n",
|
|||
|
|
"$$\n",
|
|||
|
|
"</center>\n",
|
|||
|
|
"\n",
|
|||
|
|
"In our case we will measure how often a given genre appears for movies watched by a given user vs how often it appears for all users. To obtain a movie score we will take the average of its genres' scores for this user."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "infrared-southwest",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"from sklearn.feature_extraction.text import TfidfVectorizer\n",
|
|||
|
|
"\n",
|
|||
|
|
"class TFIDFRecommender(object):\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" Recommender based on the TF-IDF method.\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" \n",
|
|||
|
|
" def __init__(self):\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" Initialize base recommender params and variables.\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" self.tfidf_scores = None\n",
|
|||
|
|
" \n",
|
|||
|
|
" def fit(self, interactions_df, users_df, items_df):\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" Training of the recommender.\n",
|
|||
|
|
" \n",
|
|||
|
|
" :param pd.DataFrame interactions_df: DataFrame with recorded interactions between users and items \n",
|
|||
|
|
" defined by user_id, item_id and features of the interaction.\n",
|
|||
|
|
" :param pd.DataFrame users_df: DataFrame with users and their features defined by user_id and the user feature columns.\n",
|
|||
|
|
" :param pd.DataFrame items_df: DataFrame with items and their features defined by item_id and the item feature columns.\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" \n",
|
|||
|
|
" self.tfidf_scores = defaultdict(lambda: 0.0)\n",
|
|||
|
|
"\n",
|
|||
|
|
" # Prepare the corpus for tfidf calculation\n",
|
|||
|
|
" \n",
|
|||
|
|
" interactions_df = pd.merge(interactions_df, items_df, on='item_id')\n",
|
|||
|
|
" user_genres = interactions_df.loc[:, ['user_id', 'genres']]\n",
|
|||
|
|
" user_genres.loc[:, 'genres'] = user_genres['genres'].str.replace(\"-\", \"_\", regex=False)\n",
|
|||
|
|
" user_genres.loc[:, 'genres'] = user_genres['genres'].str.replace(\" \", \"_\", regex=False)\n",
|
|||
|
|
" user_genres = user_genres.groupby('user_id').aggregate(lambda x: \"|\".join(x))\n",
|
|||
|
|
" user_genres.loc[:, 'genres'] = user_genres['genres'].str.replace(\"|\", \" \", regex=False)\n",
|
|||
|
|
"# print(user_genres)\n",
|
|||
|
|
" user_ids = user_genres.index.tolist()\n",
|
|||
|
|
" genres_corpus = user_genres['genres'].tolist()\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Calculate tf-idf scores\n",
|
|||
|
|
" \n",
|
|||
|
|
" vectorizer = TfidfVectorizer()\n",
|
|||
|
|
" tfidf_scores = vectorizer.fit_transform(genres_corpus)\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Transform results into a dict {(user_id, genre): score}\n",
|
|||
|
|
" \n",
|
|||
|
|
" for u in range(tfidf_scores.shape[0]):\n",
|
|||
|
|
" for g in range(tfidf_scores.shape[1]):\n",
|
|||
|
|
" self.tfidf_scores[(user_ids[u], vectorizer.get_feature_names()[g])] = tfidf_scores[u, g]\n",
|
|||
|
|
" \n",
|
|||
|
|
"# print(self.tfidf_scores)\n",
|
|||
|
|
" \n",
|
|||
|
|
" def recommend(self, users_df, items_df, n_recommendations=1):\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" Serving of recommendations. Scores items in items_df for each user in users_df and returns \n",
|
|||
|
|
" top n_recommendations for each user.\n",
|
|||
|
|
" \n",
|
|||
|
|
" :param pd.DataFrame users_df: DataFrame with users and their features for which recommendations should be generated.\n",
|
|||
|
|
" :param pd.DataFrame items_df: DataFrame with items and their features which should be scored.\n",
|
|||
|
|
" :param int n_recommendations: Number of recommendations to be returned for each user.\n",
|
|||
|
|
" :return: DataFrame with user_id, item_id and score as columns returning n_recommendations top recommendations \n",
|
|||
|
|
" for each user.\n",
|
|||
|
|
" :rtype: pd.DataFrame\n",
|
|||
|
|
" \"\"\"\n",
|
|||
|
|
" \n",
|
|||
|
|
" recommendations = pd.DataFrame(columns=['user_id', 'item_id', 'score'])\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Transform genres to a unified form used by the vectorizer\n",
|
|||
|
|
" \n",
|
|||
|
|
" items_df = items_df.copy()\n",
|
|||
|
|
" items_df.loc[:, 'genres'] = items_df['genres'].str.replace(\"-\", \"_\", regex=False)\n",
|
|||
|
|
" items_df.loc[:, 'genres'] = items_df['genres'].str.replace(\" \", \"_\", regex=False)\n",
|
|||
|
|
" items_df.loc[:, 'genres'] = items_df['genres'].str.lower()\n",
|
|||
|
|
" items_df.loc[:, 'genres'] = items_df['genres'].str.split(\"|\")\n",
|
|||
|
|
" \n",
|
|||
|
|
" # Score items \n",
|
|||
|
|
" \n",
|
|||
|
|
" for uix, user in users_df.iterrows():\n",
|
|||
|
|
" items = []\n",
|
|||
|
|
" for iix, item in items_df.iterrows():\n",
|
|||
|
|
" score = 0.0\n",
|
|||
|
|
" for genre in item['genres']:\n",
|
|||
|
|
" score += self.tfidf_scores[(user['user_id'], genre)]\n",
|
|||
|
|
" score /= len(item['genres'])\n",
|
|||
|
|
" items.append((item['item_id'], score))\n",
|
|||
|
|
" \n",
|
|||
|
|
" items = sorted(items, key=lambda x: x[1], reverse=True)\n",
|
|||
|
|
" user_recommendations = pd.DataFrame({'user_id': user['user_id'],\n",
|
|||
|
|
" 'item_id': [item[0] for item in items][:n_recommendations],\n",
|
|||
|
|
" 'score': [item[1] for item in items][:n_recommendations]})\n",
|
|||
|
|
"\n",
|
|||
|
|
" recommendations = pd.concat([recommendations, user_recommendations])\n",
|
|||
|
|
"\n",
|
|||
|
|
" return recommendations"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "oriented-service",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Quick test of the recommender\n",
|
|||
|
|
"\n",
|
|||
|
|
"tfidf_recommender = TFIDFRecommender()\n",
|
|||
|
|
"tfidf_recommender.fit(ml_ratings_df, None, ml_movies_df)\n",
|
|||
|
|
"recommendations = tfidf_recommender.recommend(pd.DataFrame([[1], [2]], columns=['user_id']), ml_movies_df, 3)\n",
|
|||
|
|
"\n",
|
|||
|
|
"recommendations = pd.merge(recommendations, ml_movies_df, on='item_id')\n",
|
|||
|
|
"display(HTML(recommendations.to_html()))"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "qualified-westminster",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"tfidf_recommender = TFIDFRecommender()\n",
|
|||
|
|
"\n",
|
|||
|
|
"results = [['TFIDFRecommender'] + list(evaluate_leave_one_out_implicit(\n",
|
|||
|
|
" tfidf_recommender, ml_ratings_df.loc[:, ['user_id', 'item_id']], ml_movies_df, max_evals=300, seed=6789))]\n",
|
|||
|
|
"\n",
|
|||
|
|
"results = pd.DataFrame(results, \n",
|
|||
|
|
" columns=['Recommender', 'HR@1', 'HR@3', 'HR@5', 'HR@10', 'NDCG@1', 'NDCG@3', 'NDCG@5', 'NDCG@10'])\n",
|
|||
|
|
"\n",
|
|||
|
|
"display(HTML(results.to_html()))"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "beautiful-snapshot",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"# Tasks"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "growing-maria",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"**Task 3.** Implement the MostPopularRecommender (check the slides for class 1), evaluate it with leave-one-out procedure for implicit feedback, print HR@1, HR@3, HR@5, HR@10, NDCG@1, NDCG@3, NDCG@5, NDCG@10."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "strategic-commons",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Write your code here"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "black-schedule",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"**Task 4.** Implement the HighestRatedRecommender (check the slides for class 1), evaluate it with leave-one-out procedure for implicit feedback, print HR@1, HR@3, HR@5, HR@10, NDCG@1, NDCG@3, NDCG@5, NDCG@10."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "likely-vatican",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Write your code here"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "handy-palmer",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"**Task 5.** Implement the RandomRecommender (check the slides for class 1), evaluate it with leave-one-out procedure for implicit feedback, print HR@1, HR@3, HR@5, HR@10, NDCG@1, NDCG@3, NDCG@5, NDCG@10."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "minute-cliff",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Write your code here"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "animal-heart",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"**Task 6.** Gather the results for TFIDFRecommender, MostPopularRecommender, HighestRatedRecommender, RandomRecommender in one DataFrame and print it."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "optical-creator",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Write your code here"
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "markdown",
|
|||
|
|
"id": "visible-burlington",
|
|||
|
|
"metadata": {},
|
|||
|
|
"source": [
|
|||
|
|
"**Task 7\\*.** Implement an SVRRecommender - one-hot encode genres and fit an SVR model to \n",
|
|||
|
|
"\n",
|
|||
|
|
"(genre_1, genre_2, ..., genre_N) -> rating\n",
|
|||
|
|
"\n",
|
|||
|
|
"Tune params of the SVR model to obtain as good results as you can. \n",
|
|||
|
|
"\n",
|
|||
|
|
"To do tuning properly (although in practive people are often happy with leave-one-out and do not bother with dividing the set into training, validation and test sets):\n",
|
|||
|
|
" - divide the set into training, validation and test sets (randomly divide the dataset in proportions 60%-20%-20%),\n",
|
|||
|
|
" - train the model with different sets of tunable parameters on the training set, \n",
|
|||
|
|
" - choose the best tunable params based on results on the validation set, \n",
|
|||
|
|
" - provide the final evaluation metrics on the test set for the best model obtained during tuning.\n",
|
|||
|
|
"\n",
|
|||
|
|
"Recommended method of tuning: use hyperopt. Install the package using the following command: `pip install hyperopt`\n",
|
|||
|
|
" \n",
|
|||
|
|
"Print the RMSE and MAE on the test set generated with numpy with seed 6789."
|
|||
|
|
]
|
|||
|
|
},
|
|||
|
|
{
|
|||
|
|
"cell_type": "code",
|
|||
|
|
"execution_count": null,
|
|||
|
|
"id": "promotional-gregory",
|
|||
|
|
"metadata": {},
|
|||
|
|
"outputs": [],
|
|||
|
|
"source": [
|
|||
|
|
"# Write your code here"
|
|||
|
|
]
|
|||
|
|
}
|
|||
|
|
],
|
|||
|
|
"metadata": {
|
|||
|
|
"kernelspec": {
|
|||
|
|
"display_name": "Python 3",
|
|||
|
|
"language": "python",
|
|||
|
|
"name": "python3"
|
|||
|
|
},
|
|||
|
|
"language_info": {
|
|||
|
|
"codemirror_mode": {
|
|||
|
|
"name": "ipython",
|
|||
|
|
"version": 3
|
|||
|
|
},
|
|||
|
|
"file_extension": ".py",
|
|||
|
|
"mimetype": "text/x-python",
|
|||
|
|
"name": "python",
|
|||
|
|
"nbconvert_exporter": "python",
|
|||
|
|
"pygments_lexer": "ipython3",
|
|||
|
|
"version": "3.8.8"
|
|||
|
|
}
|
|||
|
|
},
|
|||
|
|
"nbformat": 4,
|
|||
|
|
"nbformat_minor": 5
|
|||
|
|
}
|