kubapok/aitech-eks-pub-22
4
5
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

3

Browse Source
This commit is contained in:
Jakub Pokrywka 2022-03-22 12:58:16 +01:00
parent 58d54c38ca
commit 8c34124a9b
1 changed files with 19 additions and 14 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

33
cw/03a_tfidf.ipynb
View File

@ -437,6 +437,20 @@
"### IDF" "### IDF"
] ]
}, },
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Wersja bez żadnej normalizacji\n",
"\n",
"\n",
"$idf_i = \\Large\\frac{|D|}{|\\{d : t_i \\in d \\}|}$\n",
"\n",
"\n",
"$|D|$ - ilość dokumentów w korpusie\n",
"$|\\{d : t_i \\in d \\}|$ - ilość dokumentów w korpusie, gdzie dany term występuje chociaż jeden raz"
]
},
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 22, "execution_count": 22,
@ -470,7 +484,7 @@
"outputs": [], "outputs": [],
"source": [ "source": [
"for i in range(len(documents_vectorized)):\n", "for i in range(len(documents_vectorized)):\n",
" documents_vectorized[i] = documents_vectorized[i]# * idf" " documents_vectorized[i] = documents_vectorized[i] * idf"
] ]
}, },
{ {
@ -961,7 +975,9 @@
{ {
"cell_type": "code", "cell_type": "code",
"execution_count": 36, "execution_count": 36,
"metadata": {}, "metadata": {
"scrolled": true
},
"outputs": [ "outputs": [
{ {
"data": { "data": {
@ -1065,18 +1081,7 @@
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": []
"### ZADANIE 4 NAPISAĆ IDF w celu zmiany wag z TF na TF- IDF \n",
"\n",
"Proszę użyć wersję bez żadnej normalizacji\n",
"\n",
"\n",
"$idf_i = \\Large\\frac{|D|}{|\\{d : t_i \\in d \\}|}$\n",
"\n",
"\n",
"$|D|$ - ilość dokumentów w korpusie\n",
"$|\\{d : t_i \\in d \\}|$ - ilość dokumentów w korpusie, gdzie dany term występuje chociaż jeden raz"
]
} }
], ],
"metadata": { "metadata": {