add metadata to cw
This commit is contained in:
Jakub Pokrywka
parent
1836dc18c1
commit
ad34aaeae0
@ -2,60 +2,76 @@
|
|||||||
"cells": [
|
"cells": [
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {
|
||||||
|
"collapsed": false
|
||||||
|
},
|
||||||
"source": [
|
"source": [
|
||||||
"# Informacje ogólne"
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 0. <i>Informacje na temat przedmiotu</i> [\u0107wiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"## Kontakt z prowadzącym\n",
|
"# Informacje og\u00f3lne"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"## Kontakt z prowadz\u0105cym\n",
|
||||||
"\n",
|
"\n",
|
||||||
"prowadzący: mgr inż. Jakub Pokrywka\n",
|
"prowadz\u0105cy: mgr in\u017c. Jakub Pokrywka\n",
|
||||||
"\n",
|
"\n",
|
||||||
"Najlepiej kontaktowąć się ze mną przez MS TEAMS na grupie kanału (ogólne sprawy) lub w prywatnych wiadomościach. Odpisuję co 2-3 dni. Można też umówić się na zdzwonko w godzinach dyżuru (wt 12.00-13.00) lub umówić się w innym terminie.\n",
|
"Najlepiej kontaktow\u0105\u0107 si\u0119 ze mn\u0105 przez MS TEAMS na grupie kana\u0142u (og\u00f3lne sprawy) lub w prywatnych wiadomo\u015bciach. Odpisuj\u0119 co 2-3 dni. Mo\u017cna te\u017c um\u00f3wi\u0107 si\u0119 na zdzwonko w godzinach dy\u017curu (wt 12.00-13.00) lub um\u00f3wi\u0107 si\u0119 w innym terminie.\n",
|
||||||
"\n",
|
"\n",
|
||||||
"\n",
|
"\n",
|
||||||
"## Literatura\n",
|
"## Literatura\n",
|
||||||
"Polecana literatura do przedmiotu:\n",
|
"Polecana literatura do przedmiotu:\n",
|
||||||
"\n",
|
"\n",
|
||||||
"\n",
|
"\n",
|
||||||
"- https://www.manning.com/books/relevant-search#toc (darmowa) Polecam chociaż przejrzeć.\n",
|
"- https://www.manning.com/books/relevant-search#toc (darmowa) Polecam chocia\u017c przejrze\u0107.\n",
|
||||||
"- Marie-Francine Moens. 2006. Information Extraction: Algorithms and Prospects in a Retrieval Context. Springer. (polecam mniej, jest trochę nieaktualna)\n",
|
"- Marie-Francine Moens. 2006. Information Extraction: Algorithms and Prospects in a Retrieval Context. Springer. (polecam mniej, jest troch\u0119 nieaktualna)\n",
|
||||||
"- Alex Graves. 2012. Supervised sequence labelling. Studies in Computational Intelligence, vol 385. Springer. Berlin, Heidelberg. \n",
|
"- Alex Graves. 2012. Supervised sequence labelling. Studies in Computational Intelligence, vol 385. Springer. Berlin, Heidelberg. \n",
|
||||||
"\n",
|
"\n",
|
||||||
"- Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. North American Association for Computational Linguistics (NAACL). \n",
|
"- Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. North American Association for Computational Linguistics (NAACL). \n",
|
||||||
"\n",
|
"\n",
|
||||||
"- Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. Journal of Machine Learning Research vol 21, number 140, pages 1-67. \n",
|
"- Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. Journal of Machine Learning Research vol 21, number 140, pages 1-67. \n",
|
||||||
"\n",
|
"\n",
|
||||||
"- Flip Graliński, Tomasz Stanisławek, Anna Wróblewska, Dawid Lipiński, Agnieszka Kaliska, Paulina Rosalska, Bartosz Topolski, Przemysław Biecek. 2020. Kleister: A novel task for information extraction involving long documents with complex layout. URL https://arxiv.org/abs/2003.02356 \n",
|
"- Flip Grali\u0144ski, Tomasz Stanis\u0142awek, Anna Wr\u00f3blewska, Dawid Lipi\u0144ski, Agnieszka Kaliska, Paulina Rosalska, Bartosz Topolski, Przemys\u0142aw Biecek. 2020. Kleister: A novel task for information extraction involving long documents with complex layout. URL https://arxiv.org/abs/2003.02356 \n",
|
||||||
"\n",
|
"\n",
|
||||||
"- Łukasz Garncarek, Rafał Powalski, Tomasz Stanisławek, Bartosz Topolski, Piotr Halama, Filip Graliński. 2020. LAMBERT: Layout-Aware (Language) Modeling using BERT. URL https://arxiv.org/pdf/2002.08087 \n",
|
"- \u0141ukasz Garncarek, Rafa\u0142 Powalski, Tomasz Stanis\u0142awek, Bartosz Topolski, Piotr Halama, Filip Grali\u0144ski. 2020. LAMBERT: Layout-Aware (Language) Modeling using BERT. URL https://arxiv.org/pdf/2002.08087 \n",
|
||||||
"\n",
|
"\n",
|
||||||
"## Zaliczenie\n",
|
"## Zaliczenie\n",
|
||||||
"\n",
|
"\n",
|
||||||
"\n",
|
"\n",
|
||||||
"\n",
|
"\n",
|
||||||
"Do zdobycia będzie conajmniej 600 punktów.\n",
|
"Do zdobycia b\u0119dzie conajmniej 600 punkt\u00f3w.\n",
|
||||||
"\n",
|
"\n",
|
||||||
"Ocena:\n",
|
"Ocena:\n",
|
||||||
"\n",
|
"\n",
|
||||||
"- -299 — 2\n",
|
"- -299 \u2014 2\n",
|
||||||
"\n",
|
"\n",
|
||||||
"- 300-349 — 3\n",
|
"- 300-349 \u2014 3\n",
|
||||||
"\n",
|
"\n",
|
||||||
"- 350-399 — 3+\n",
|
"- 350-399 \u2014 3+\n",
|
||||||
"\n",
|
"\n",
|
||||||
"- 400-449 — 4\n",
|
"- 400-449 \u2014 4\n",
|
||||||
"\n",
|
"\n",
|
||||||
"- 450—499 — 4+\n",
|
"- 450\u2014499 \u2014 4+\n",
|
||||||
"\n",
|
"\n",
|
||||||
"- 500- — 5\n",
|
"- 500- \u2014 5\n",
|
||||||
"\n",
|
"\n",
|
||||||
"\n",
|
"\n",
|
||||||
"**Żeby zaliczyć przedmiot należy pojawiać się na laboratoriach. Maksymalna liczba nieobecności to 3. Obecność będę sprawdzał poprzez panel MS TEAMS, czyli będę sprawdzał czy ktoś jest wdzwoniony na ćwiczenia. Jeżeli kogoś nie będzie więcej niż 3 razy, to nie będzie miał zaliczonego przedmiotu** \n"
|
"**\u017beby zaliczy\u0107 przedmiot nale\u017cy pojawia\u0107 si\u0119 na laboratoriach. Maksymalna liczba nieobecno\u015bci to 3. Obecno\u015b\u0107 b\u0119d\u0119 sprawdza\u0142 poprzez panel MS TEAMS, czyli b\u0119d\u0119 sprawdza\u0142 czy kto\u015b jest wdzwoniony na \u0107wiczenia. Je\u017celi kogo\u015b nie b\u0119dzie wi\u0119cej ni\u017c 3 razy, to nie b\u0119dzie mia\u0142 zaliczonego przedmiotu** \n"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -83,7 +99,13 @@
|
|||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.8.3"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
|
"lang": "pl",
|
||||||
|
"subtitle": "0.Informacje na temat przedmiotu[\u0107wiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
@ -1,181 +0,0 @@
|
|||||||
{
|
|
||||||
"cells": [
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"Opracować w języku Haskell wyspecjalizowanego robota pobierającego dane z konkretnego serwisu.\n",
|
|
||||||
"\n",
|
|
||||||
"Punkty: 80 (domyślnie - niektóre zadanie są trudniejsze, wówczas podaję osobno liczbę punktów)\n",
|
|
||||||
"\n",
|
|
||||||
"Ogólne zasady:\n",
|
|
||||||
"\n",
|
|
||||||
"* pobieramy informacje (metadane) o plikach PDF, DjVU, JPG itp, ale nie same pliki,\n",
|
|
||||||
"* nie pobierajmy całego serwisu, tylko tyle, ile trzeba, by pobrać metadane o interesujących nas zasobach,\n",
|
|
||||||
"* interesują nas tylko teksty polskie, jeśli nie jest to trudne, należy odfiltrować publikacje obcojęzyczne,\n",
|
|
||||||
"* staramy się ustalać datę z możliwie dużą dokładnością.\n",
|
|
||||||
"\n",
|
|
||||||
"Sposób pracy:\n",
|
|
||||||
"\n",
|
|
||||||
"0. Pobrać Haskell Stack\n",
|
|
||||||
"\n",
|
|
||||||
"~~~\n",
|
|
||||||
"curl -sSL https://get.haskellstack.org/ | sh -s - -d ~/bin\n",
|
|
||||||
"~~~\n",
|
|
||||||
"\n",
|
|
||||||
"Na fizycznych komputerach wydziałowych są błędnie ustawione prawa dostępu na dyskach sieciowych, Haskell Stack musi działać na fizycznym dysku:\n",
|
|
||||||
"\n",
|
|
||||||
"~~~\n",
|
|
||||||
"rm -rf /mnt/poligon/.stack\n",
|
|
||||||
"mkdir /mnt/poligon/.stack\n",
|
|
||||||
"mv ~/.stack ~/.stack-bak # gdyby już był... proszę się nie przejmować błędem\n",
|
|
||||||
"ln -s /mnt/poligon/.stack ~/.stack\n",
|
|
||||||
"~~~\n",
|
|
||||||
"\n",
|
|
||||||
"1. Pobrać repozytorium:\n",
|
|
||||||
"\n",
|
|
||||||
"~~~\n",
|
|
||||||
"git clone https://git.wmi.amu.edu.pl/filipg/twilight-library.git\n",
|
|
||||||
"~~~\n",
|
|
||||||
"\n",
|
|
||||||
"2. Wypchnąć na początek do swojego repozytorium (trzeba sobie najpierw założyć to repozytorium na <https://git.wmi.amu.edu.pl>)\n",
|
|
||||||
"\n",
|
|
||||||
"~~~\n",
|
|
||||||
"cd twilight-library\n",
|
|
||||||
"git remote set-url origin git@git.wmi.amu.edu.pl:YOURID/twilight-library\n",
|
|
||||||
"git push origin master\n",
|
|
||||||
"git remote add mother git://gonito.net/twilight-library\n",
|
|
||||||
"~~~\n",
|
|
||||||
"\n",
|
|
||||||
"3. Zobacz, czy przykładowy robot dla strony z „Alamanachem Muszyny” działa:\n",
|
|
||||||
"\n",
|
|
||||||
"~~~\n",
|
|
||||||
"~/bin/stack install # może trwać długo za pierwszym razem\n",
|
|
||||||
"~/bin/stack exec almanachmuszyny\n",
|
|
||||||
"~~~\n",
|
|
||||||
"\n",
|
|
||||||
"\n",
|
|
||||||
"W razie problemów z instalacją:\n",
|
|
||||||
"\n",
|
|
||||||
"~~~\n",
|
|
||||||
"sudo apt install libpcre3 libpcre3-dev\n",
|
|
||||||
"~~~\n",
|
|
||||||
"\n",
|
|
||||||
"3. Opracuj swojego robota wzorując się na pliku `almanachmuszyny.hs`.\n",
|
|
||||||
" (Ale dodaj swój plik, nie zmieniaj `almanachmuszyny.hs`!)\n",
|
|
||||||
"\n",
|
|
||||||
"4. Dopisz specyfikację swojego robota do `shadow-library.cabal`.\n",
|
|
||||||
"\n",
|
|
||||||
"5. Pracuj nad swoim robotem, uruchamiaj go w następujący sposób:\n",
|
|
||||||
"\n",
|
|
||||||
"~~~\n",
|
|
||||||
"~/bin/stack install\n",
|
|
||||||
"~/bin/stack exec mojrobot\n",
|
|
||||||
"~~~\n",
|
|
||||||
"\n",
|
|
||||||
"(Tzn. nie nazywaj go „mojrobot”, tylko użyj jakieś sensownej nazwy.)\n",
|
|
||||||
"\n",
|
|
||||||
"6. Jeśli publikacja (np. pojedynczy numer gazety) składa się z wielu plików, powinien zostać wygenerowany jeden\n",
|
|
||||||
"rekord, w `finalUrl` powinny znaleźć się URL do poszczególnych stron (np. plików JPR) oddzielone ` // `.\n",
|
|
||||||
"\n",
|
|
||||||
"7. Po zakończeniu prac prześlij mejla do prowadzącego zajęcia z URL-em do swojego repozytorium.\n",
|
|
||||||
"\n",
|
|
||||||
"Lista serwisów do wyboru (na każdy serwis 1 osoba):\n",
|
|
||||||
"\n",
|
|
||||||
"1. [Teksty Drugie](http://tekstydrugie.pl)\n",
|
|
||||||
"2. [Archiwum Inspektora Pracy](https://www.pip.gov.pl/pl/inspektor-pracy/66546,archiwum-inspektora-pracy-.html)\n",
|
|
||||||
"3. [Medycyna Weterynaryjna](http://www.medycynawet.edu.pl/archives) — również historyczne zasoby od 1945 roku, **120 punktów**\n",
|
|
||||||
"4. [Polskie Towarzystwo Botaniczne](https://pbsociety.org.pl/default/dzialalnosc-wydawnicza/) — wszystkie dostępne zdigitalizowane publikacje!, **130 punktow**\n",
|
|
||||||
"5. [Wieści Pepowa](http://archiwum2019.pepowo.pl/news/c-10/gazeta) — nie pominąć strony nr 2 z wynikami, **110 punktów**\n",
|
|
||||||
"6. [Czasopismo Kosmos](http://kosmos.icm.edu.pl/)\n",
|
|
||||||
"7. [Czasopismo Wszechświat](http://www.ptpk.org/archiwum.html)\n",
|
|
||||||
"8. [Czasopisma polonijne we Francji](https://argonnaute.parisnanterre.fr/ark:/14707/a011403267917yQQFAS) — najlepiej w postaci PDF-ów, jak np. [https://argonnaute.parisnanterre.fr/medias/customer_3/periodique/immi_pol_lotmz1_pdf/BDIC_GFP_2929_1945_039.pdf](), **220 punktów**\n",
|
|
||||||
"9. [Muzeum Sztuki — czasopisma](https://zasoby.msl.org.pl/mobjects/show), **220 punktów**, publikacje, teksty, czasopisma, wycinki\n",
|
|
||||||
"10. [Wiadomości Urzędu Patentowego](https://grab.uprp.pl/sites/Wydawnictwa/WydawnictwaArchiwum/WydawnictwaArchiwum/Forms/AllItems.aspx)\n",
|
|
||||||
"11. [Czas, czasopismo polonijne](https://digitalcollections.lib.umanitoba.ca/islandora/object/uofm:2222545), **140 punktów** S.G.\n",
|
|
||||||
"12. [Stenogramy Okrągłego Stołu](http://okragly-stol.pl/stenogramy/), **110 punktów**\n",
|
|
||||||
"13. [Nasze Popowice](https://smpopowice.pl/index.php/numery-archiwalne)\n",
|
|
||||||
"14. [Czasopisma entomologiczne](http://pte.au.poznan.pl/)\n",
|
|
||||||
"15. [Wiadomości matematyczne](https://wydawnictwa.ptm.org.pl/index.php/wiadomosci-matematyczne/issue/archive?issuesPage=2), **120 punktow**\n",
|
|
||||||
"16. [Alkoholizm i Narkomania](http://www.ain.ipin.edu.pl/archiwum-starsze.html)\n",
|
|
||||||
"17. [Czasopismo Etyka](https://etyka.uw.edu.pl/tag/etyka-562018/), O.K.\n",
|
|
||||||
"18. [Skup makulatury](https://chomikuj.pl/skup.makulatury.prl), **250 punktów**\n",
|
|
||||||
"19. [Hermes](https://chomikuj.pl/hermes50-1) i https://chomikuj.pl/hermes50-2, **250 punktów**\n",
|
|
||||||
"20. [E-dziennik Województwa Mazowieckiego](https://edziennik.mazowieckie.pl/actbymonths) **150 punktów**\n",
|
|
||||||
"21. [Czasopismo Węgiel Brunatny](http://www.ppwb.org.pl/wegiel_brunatny)\n",
|
|
||||||
"22. [Gazeta GUM](https://gazeta.gumed.edu.pl/61323.html)\n",
|
|
||||||
"23. [Nowiny Andrychowskie](https://radioandrychow.pl/nowiny/)\n",
|
|
||||||
"24. [Kawęczyniak](http://bip.kaweczyn.pl/kaweczyn/pl/dla-mieszkanca/publikacje/archiwalne-numery-kaweczyniaka-rok-1995-2005/kaweczyniaki-rok-1997.html)\n",
|
|
||||||
"25. [Zbór Chrześcijański w Bielawia](http://zborbielawa.pl/archiwum/)\n",
|
|
||||||
"26. [Gazeta Rytwiańska](http://www.rytwiany.com.pl/index.php?sid=5)\n",
|
|
||||||
"27. [Nasze Popowice](https://smpopowice.pl/gazeta/2005_12_nasze-popowice-nr_01.pdf)\n",
|
|
||||||
"28. [Echo Chełmka](http://moksir.chelmek.pl/o-nas/echo-chelmka)\n",
|
|
||||||
"29. [Głos Świdnika](http://s.bibliotekaswidnik.pl/index.php/archwium/116-glos-swidnika) **100 punktów**\n",
|
|
||||||
"30. [Aneks](https://aneks.kulturaliberalna.pl/archiwum-aneksu/) **90 punktów**\n",
|
|
||||||
"31. [Teatr Lalel](http://polunima.pl/teatr-lalek)\n",
|
|
||||||
"32. [Biuletyn Bezpieczna Chemia](https://www.pipc.org.pl/publikacje/biuletyn-bezpieczna-chemia)\n",
|
|
||||||
"33. [Głos Maszynisty](https://zzm.org.pl/glos-maszynisty/)\n",
|
|
||||||
"34. [Kultura Paryska](https://www.kulturaparyska.com/pl/index), całe archiwum z książkami i innymi czasopismami, **180 punktów**\n",
|
|
||||||
"35. [Gazeta Fabryczna - Kraśnik](https://80lat.flt.krasnik.pl/index.php/gazeta-fabryczna/) **120 punktów**\n",
|
|
||||||
"36. [Artykuły o Jujutsu](http://www.kobudo.pl/artykuly_jujutsu.html)\n",
|
|
||||||
"37. [Wycinki o Taekwon-Do](https://www2.pztkd.lublin.pl/archpras.html#z1996)\n",
|
|
||||||
"38. [Materiały o kolejnictwie](https://enkol.pl/Strona_g%C5%82%C3%B3wna) **180 punktów**\n",
|
|
||||||
"39. [Centralny Instytut Ochrony Pracy](http://archiwum.ciop.pl/), znaleźć wszystkie publikacje typu <http://archiwum.ciop.pl/44938>, wymaga trochę sprytu **130 punktów**\n",
|
|
||||||
"40. [Biblioteka Sejmowa - Zasoby Cyfrowe](https://biblioteka.sejm.gov.pl/zasoby_cyfrowe/), **200 punktów**\n",
|
|
||||||
"41. [Elektronika Praktyczna](https://ep.com.pl/archiwum), te numery, które dostępne w otwarty sposób, np. rok 1993\n",
|
|
||||||
"42. [Litewska Akademia Nauk](http://www.mab.lt/), tylko materiały w jęz. polskim, takie jak np.\n",
|
|
||||||
" <https://elibrary.mab.lt/handle/1/840>, **170 punktów**\n",
|
|
||||||
"43. [Litewska Biblioteka Cyfrowa](https://www.epaveldas.lt), wyłuskać tylko materiały w jęz. polskim, **190 punktów**\n",
|
|
||||||
"44. [Czasopisma Geologiczne](https://geojournals.pgi.gov.pl), **120 punktów**\n",
|
|
||||||
"45. [Czasopisma PTTK](https://www.czasopisma.centralnabibliotekapttk.pl/index.php?i3), **120 punktów**\n",
|
|
||||||
"46. [Czasopisma Polskiego Towarzystwa Dendrologicznego](https://www.ptd.pl/?page_id=7), **100 punktów**\n",
|
|
||||||
"47. [Kilka przedwojennych książek](https://dziemiela.com/documents.htm)\n",
|
|
||||||
"48. [Historia polskiej informatyki](http://klio.spit.iq.pl/a4-wyroby-polskiej-informatyki/a4-2-sprzet/) - wyjątkowo bez datowania\n",
|
|
||||||
"49. [Zeszyty Formacyjne Katolickiego Stowarzyszenia „Civitas Christania”](http://podkarpacki.civitaschristiana.pl/formacja/zeszyty-formacyjne/), tylko niektóre pliki można zdatować\n",
|
|
||||||
"50. [Józef Piłsudski Institute of America](https://archiwa.pilsudski.org/) - **220 punktów**\n",
|
|
||||||
"51. [Prasa podziemna — Częstochowa](http://www.podziemie.com.pl), również ulotki i inne materiały skanowane - **180 punktów**\n",
|
|
||||||
"52. [Tajemnica Atari](http://krap.pl/mirrorz/atari/horror.mirage.com.pl/pixel/), plik ZIP z DjVu\n",
|
|
||||||
"\n",
|
|
||||||
"\n",
|
|
||||||
"### F.A.Q.\n",
|
|
||||||
"\n",
|
|
||||||
"**P: Nie działają strony z protokołem https, co zrobić?**\n",
|
|
||||||
"\n",
|
|
||||||
"O: Trzeba użyć modułu opartego na bibliotece curl. Paczka Ubuntu została zainstalowana na komputerach wydziałowych. Na\n",
|
|
||||||
"swoim komputerze możemy zainstalować paczkę libcurl4-openssl-dev, a\n",
|
|
||||||
"następnie można sobie ściągnąć wersję twilight-library opartą na libcurl:\n",
|
|
||||||
"\n",
|
|
||||||
" git fetch git://gonito.net/twilight-library withcurl\n",
|
|
||||||
" git merge FETCH_HEAD\n"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": null,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": []
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"metadata": {
|
|
||||||
"kernelspec": {
|
|
||||||
"display_name": "Python 3",
|
|
||||||
"language": "python",
|
|
||||||
"name": "python3"
|
|
||||||
},
|
|
||||||
"language_info": {
|
|
||||||
"codemirror_mode": {
|
|
||||||
"name": "ipython",
|
|
||||||
"version": 3
|
|
||||||
},
|
|
||||||
"file_extension": ".py",
|
|
||||||
"mimetype": "text/x-python",
|
|
||||||
"name": "python",
|
|
||||||
"nbconvert_exporter": "python",
|
|
||||||
"pygments_lexer": "ipython3",
|
|
||||||
"version": "3.9.1"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"nbformat": 4,
|
|
||||||
"nbformat_minor": 4
|
|
||||||
}
|
|
||||||
@ -1,5 +1,19 @@
|
|||||||
{
|
{
|
||||||
"cells": [
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 1. <i>Wyszukiwarki wprowadzenie</i> [ćwiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
@ -234,11 +248,14 @@
|
|||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
"kernelspec": {
|
"kernelspec": {
|
||||||
"display_name": "Python 3",
|
"display_name": "Python 3",
|
||||||
"language": "python",
|
"language": "python",
|
||||||
"name": "python3"
|
"name": "python3"
|
||||||
},
|
},
|
||||||
|
"lang": "pl",
|
||||||
"language_info": {
|
"language_info": {
|
||||||
"codemirror_mode": {
|
"codemirror_mode": {
|
||||||
"name": "ipython",
|
"name": "ipython",
|
||||||
@ -249,8 +266,11 @@
|
|||||||
"name": "python",
|
"name": "python",
|
||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.8.5"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"subtitle": "1.Wyszukiwarki wprowadzenie[ćwiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
@ -1,5 +1,19 @@
|
|||||||
{
|
{
|
||||||
"cells": [
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 2. <i>Wyszukiwarki roboty</i> [ćwiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
@ -272,11 +286,14 @@
|
|||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
"kernelspec": {
|
"kernelspec": {
|
||||||
"display_name": "Python 3",
|
"display_name": "Python 3",
|
||||||
"language": "python",
|
"language": "python",
|
||||||
"name": "python3"
|
"name": "python3"
|
||||||
},
|
},
|
||||||
|
"lang": "pl",
|
||||||
"language_info": {
|
"language_info": {
|
||||||
"codemirror_mode": {
|
"codemirror_mode": {
|
||||||
"name": "ipython",
|
"name": "ipython",
|
||||||
@ -287,8 +304,11 @@
|
|||||||
"name": "python",
|
"name": "python",
|
||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.9.1"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"subtitle": "2.Wyszukiwarki roboty[ćwiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@ -1,69 +0,0 @@
|
|||||||
{
|
|
||||||
"cells": [
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 1,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": [
|
|
||||||
"def word_to_index(word):\n",
|
|
||||||
" vec = np.zeros(len(vocabulary))\n",
|
|
||||||
" if word in vocabulary:\n",
|
|
||||||
" idx = vocabulary.index(word)\n",
|
|
||||||
" vec[idx] = 1\n",
|
|
||||||
" else:\n",
|
|
||||||
" vec[-1] = 1\n",
|
|
||||||
" return vec"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": null,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": [
|
|
||||||
"def tf(document):\n",
|
|
||||||
" document_vector = None\n",
|
|
||||||
" for word in document:\n",
|
|
||||||
" if document_vector is None:\n",
|
|
||||||
" document_vector = word_to_index(word)\n",
|
|
||||||
" else:\n",
|
|
||||||
" document_vector += word_to_index(word)\n",
|
|
||||||
" return document_vector"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": null,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": [
|
|
||||||
"def similarity(query, document):\n",
|
|
||||||
" numerator = np.sum(query * document)\n",
|
|
||||||
" denominator = np.sqrt(np.sum(query*query)) * np.sqrt(np.sum(document*document)) \n",
|
|
||||||
" return numerator / denominator"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"metadata": {
|
|
||||||
"kernelspec": {
|
|
||||||
"display_name": "Python 3",
|
|
||||||
"language": "python",
|
|
||||||
"name": "python3"
|
|
||||||
},
|
|
||||||
"language_info": {
|
|
||||||
"codemirror_mode": {
|
|
||||||
"name": "ipython",
|
|
||||||
"version": 3
|
|
||||||
},
|
|
||||||
"file_extension": ".py",
|
|
||||||
"mimetype": "text/x-python",
|
|
||||||
"name": "python",
|
|
||||||
"nbconvert_exporter": "python",
|
|
||||||
"pygments_lexer": "ipython3",
|
|
||||||
"version": "3.8.3"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"nbformat": 4,
|
|
||||||
"nbformat_minor": 4
|
|
||||||
}
|
|
||||||
@ -1,708 +0,0 @@
|
|||||||
{
|
|
||||||
"cells": [
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"# Zajecia 2\n",
|
|
||||||
"\n",
|
|
||||||
"Przydatne materiały:\n",
|
|
||||||
"\n",
|
|
||||||
"https://scikit-learn.org/0.19/datasets/twenty_newsgroups.html\n",
|
|
||||||
"\n",
|
|
||||||
"https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html\n",
|
|
||||||
"\n"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"## Importy"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 1,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": [
|
|
||||||
"import numpy as np\n",
|
|
||||||
"import sklearn.metrics\n",
|
|
||||||
"\n",
|
|
||||||
"from sklearn.datasets import fetch_20newsgroups\n",
|
|
||||||
"\n",
|
|
||||||
"from sklearn.feature_extraction.text import TfidfVectorizer"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"## Zbiór danych"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 2,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": [
|
|
||||||
"newsgroups = fetch_20newsgroups()['data']"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 3,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [
|
|
||||||
{
|
|
||||||
"data": {
|
|
||||||
"text/plain": [
|
|
||||||
"11314"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"execution_count": 3,
|
|
||||||
"metadata": {},
|
|
||||||
"output_type": "execute_result"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"source": [
|
|
||||||
"len(newsgroups)"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 4,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [
|
|
||||||
{
|
|
||||||
"name": "stdout",
|
|
||||||
"output_type": "stream",
|
|
||||||
"text": [
|
|
||||||
"From: lerxst@wam.umd.edu (where's my thing)\n",
|
|
||||||
"Subject: WHAT car is this!?\n",
|
|
||||||
"Nntp-Posting-Host: rac3.wam.umd.edu\n",
|
|
||||||
"Organization: University of Maryland, College Park\n",
|
|
||||||
"Lines: 15\n",
|
|
||||||
"\n",
|
|
||||||
" I was wondering if anyone out there could enlighten me on this car I saw\n",
|
|
||||||
"the other day. It was a 2-door sports car, looked to be from the late 60s/\n",
|
|
||||||
"early 70s. It was called a Bricklin. The doors were really small. In addition,\n",
|
|
||||||
"the front bumper was separate from the rest of the body. This is \n",
|
|
||||||
"all I know. If anyone can tellme a model name, engine specs, years\n",
|
|
||||||
"of production, where this car is made, history, or whatever info you\n",
|
|
||||||
"have on this funky looking car, please e-mail.\n",
|
|
||||||
"\n",
|
|
||||||
"Thanks,\n",
|
|
||||||
"- IL\n",
|
|
||||||
" ---- brought to you by your neighborhood Lerxst ----\n",
|
|
||||||
"\n",
|
|
||||||
"\n",
|
|
||||||
"\n",
|
|
||||||
"\n",
|
|
||||||
"\n"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"source": [
|
|
||||||
"print(newsgroups[0])"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"## Naiwne przeszukiwanie"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 5,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": [
|
|
||||||
"all_documents = list() \n",
|
|
||||||
"for document in newsgroups:\n",
|
|
||||||
" if 'car' in document:\n",
|
|
||||||
" all_documents.append(document)"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 6,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [
|
|
||||||
{
|
|
||||||
"name": "stdout",
|
|
||||||
"output_type": "stream",
|
|
||||||
"text": [
|
|
||||||
"From: lerxst@wam.umd.edu (where's my thing)\n",
|
|
||||||
"Subject: WHAT car is this!?\n",
|
|
||||||
"Nntp-Posting-Host: rac3.wam.umd.edu\n",
|
|
||||||
"Organization: University of Maryland, College Park\n",
|
|
||||||
"Lines: 15\n",
|
|
||||||
"\n",
|
|
||||||
" I was wondering if anyone out there could enlighten me on this car I saw\n",
|
|
||||||
"the other day. It was a 2-door sports car, looked to be from the late 60s/\n",
|
|
||||||
"early 70s. It was called a Bricklin. The doors were really small. In addition,\n",
|
|
||||||
"the front bumper was separate from the rest of the body. This is \n",
|
|
||||||
"all I know. If anyone can tellme a model name, engine specs, years\n",
|
|
||||||
"of production, where this car is made, history, or whatever info you\n",
|
|
||||||
"have on this funky looking car, please e-mail.\n",
|
|
||||||
"\n",
|
|
||||||
"Thanks,\n",
|
|
||||||
"- IL\n",
|
|
||||||
" ---- brought to you by your neighborhood Lerxst ----\n",
|
|
||||||
"\n",
|
|
||||||
"\n",
|
|
||||||
"\n",
|
|
||||||
"\n",
|
|
||||||
"\n"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"source": [
|
|
||||||
"print(all_documents[0])"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 7,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [
|
|
||||||
{
|
|
||||||
"name": "stdout",
|
|
||||||
"output_type": "stream",
|
|
||||||
"text": [
|
|
||||||
"From: guykuo@carson.u.washington.edu (Guy Kuo)\n",
|
|
||||||
"Subject: SI Clock Poll - Final Call\n",
|
|
||||||
"Summary: Final call for SI clock reports\n",
|
|
||||||
"Keywords: SI,acceleration,clock,upgrade\n",
|
|
||||||
"Article-I.D.: shelley.1qvfo9INNc3s\n",
|
|
||||||
"Organization: University of Washington\n",
|
|
||||||
"Lines: 11\n",
|
|
||||||
"NNTP-Posting-Host: carson.u.washington.edu\n",
|
|
||||||
"\n",
|
|
||||||
"A fair number of brave souls who upgraded their SI clock oscillator have\n",
|
|
||||||
"shared their experiences for this poll. Please send a brief message detailing\n",
|
|
||||||
"your experiences with the procedure. Top speed attained, CPU rated speed,\n",
|
|
||||||
"add on cards and adapters, heat sinks, hour of usage per day, floppy disk\n",
|
|
||||||
"functionality with 800 and 1.4 m floppies are especially requested.\n",
|
|
||||||
"\n",
|
|
||||||
"I will be summarizing in the next two days, so please add to the network\n",
|
|
||||||
"knowledge base if you have done the clock upgrade and haven't answered this\n",
|
|
||||||
"poll. Thanks.\n",
|
|
||||||
"\n",
|
|
||||||
"Guy Kuo <guykuo@u.washington.edu>\n",
|
|
||||||
"\n"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"source": [
|
|
||||||
"print(all_documents[1])"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"### jakie są problemy z takim podejściem?\n"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"## TFIDF i odległość cosinusowa- gotowe biblioteki"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 8,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": [
|
|
||||||
"vectorizer = TfidfVectorizer()\n",
|
|
||||||
"#vectorizer = TfidfVectorizer(use_idf = False, ngram_range=(1,2))"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 9,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": [
|
|
||||||
"document_vectors = vectorizer.fit_transform(newsgroups)"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 10,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [
|
|
||||||
{
|
|
||||||
"data": {
|
|
||||||
"text/plain": [
|
|
||||||
"<11314x130107 sparse matrix of type '<class 'numpy.float64'>'\n",
|
|
||||||
"\twith 1787565 stored elements in Compressed Sparse Row format>"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"execution_count": 10,
|
|
||||||
"metadata": {},
|
|
||||||
"output_type": "execute_result"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"source": [
|
|
||||||
"document_vectors"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 11,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [
|
|
||||||
{
|
|
||||||
"data": {
|
|
||||||
"text/plain": [
|
|
||||||
"<1x130107 sparse matrix of type '<class 'numpy.float64'>'\n",
|
|
||||||
"\twith 89 stored elements in Compressed Sparse Row format>"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"execution_count": 11,
|
|
||||||
"metadata": {},
|
|
||||||
"output_type": "execute_result"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"source": [
|
|
||||||
"document_vectors[0]"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 12,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [
|
|
||||||
{
|
|
||||||
"data": {
|
|
||||||
"text/plain": [
|
|
||||||
"matrix([[0., 0., 0., ..., 0., 0., 0.]])"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"execution_count": 12,
|
|
||||||
"metadata": {},
|
|
||||||
"output_type": "execute_result"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"source": [
|
|
||||||
"document_vectors[0].todense()"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 13,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [
|
|
||||||
{
|
|
||||||
"data": {
|
|
||||||
"text/plain": [
|
|
||||||
"matrix([[0., 0., 0., ..., 0., 0., 0.],\n",
|
|
||||||
" [0., 0., 0., ..., 0., 0., 0.],\n",
|
|
||||||
" [0., 0., 0., ..., 0., 0., 0.],\n",
|
|
||||||
" [0., 0., 0., ..., 0., 0., 0.]])"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"execution_count": 13,
|
|
||||||
"metadata": {},
|
|
||||||
"output_type": "execute_result"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"source": [
|
|
||||||
"document_vectors[0:4].todense()"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 14,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": [
|
|
||||||
"query_str = 'speed'\n",
|
|
||||||
"#query_str = 'speed car'\n",
|
|
||||||
"#query_str = 'spider man'"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 15,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": [
|
|
||||||
"query_vector = vectorizer.transform([query_str])"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 16,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [
|
|
||||||
{
|
|
||||||
"data": {
|
|
||||||
"text/plain": [
|
|
||||||
"<11314x130107 sparse matrix of type '<class 'numpy.float64'>'\n",
|
|
||||||
"\twith 1787565 stored elements in Compressed Sparse Row format>"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"execution_count": 16,
|
|
||||||
"metadata": {},
|
|
||||||
"output_type": "execute_result"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"source": [
|
|
||||||
"document_vectors"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 17,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [
|
|
||||||
{
|
|
||||||
"data": {
|
|
||||||
"text/plain": [
|
|
||||||
"<1x130107 sparse matrix of type '<class 'numpy.float64'>'\n",
|
|
||||||
"\twith 1 stored elements in Compressed Sparse Row format>"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"execution_count": 17,
|
|
||||||
"metadata": {},
|
|
||||||
"output_type": "execute_result"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"source": [
|
|
||||||
"query_vector"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 18,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": [
|
|
||||||
"similarities = sklearn.metrics.pairwise.cosine_similarity(query_vector,document_vectors)"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 19,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [
|
|
||||||
{
|
|
||||||
"data": {
|
|
||||||
"text/plain": [
|
|
||||||
"array([0.26949927, 0.3491801 , 0.44292083, 0.47784165])"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"execution_count": 19,
|
|
||||||
"metadata": {},
|
|
||||||
"output_type": "execute_result"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"source": [
|
|
||||||
"np.sort(similarities)[0][-4:]"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 20,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [
|
|
||||||
{
|
|
||||||
"data": {
|
|
||||||
"text/plain": [
|
|
||||||
"array([4517, 5509, 2116, 9921])"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"execution_count": 20,
|
|
||||||
"metadata": {},
|
|
||||||
"output_type": "execute_result"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"source": [
|
|
||||||
"similarities.argsort()[0][-4:]"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 21,
|
|
||||||
"metadata": {
|
|
||||||
"scrolled": false
|
|
||||||
},
|
|
||||||
"outputs": [
|
|
||||||
{
|
|
||||||
"name": "stdout",
|
|
||||||
"output_type": "stream",
|
|
||||||
"text": [
|
|
||||||
"From: ray@netcom.com (Ray Fischer)\n",
|
|
||||||
"Subject: Re: x86 ~= 680x0 ?? (How do they compare?)\n",
|
|
||||||
"Organization: Netcom. San Jose, California\n",
|
|
||||||
"Distribution: usa\n",
|
|
||||||
"Lines: 36\n",
|
|
||||||
"\n",
|
|
||||||
"dhk@ubbpc.uucp (Dave Kitabjian) writes ...\n",
|
|
||||||
">I'm sure Intel and Motorola are competing neck-and-neck for \n",
|
|
||||||
">crunch-power, but for a given clock speed, how do we rank the\n",
|
|
||||||
">following (from 1st to 6th):\n",
|
|
||||||
"> 486\t\t68040\n",
|
|
||||||
"> 386\t\t68030\n",
|
|
||||||
"> 286\t\t68020\n",
|
|
||||||
"\n",
|
|
||||||
"040 486 030 386 020 286\n",
|
|
||||||
"\n",
|
|
||||||
">While you're at it, where will the following fit into the list:\n",
|
|
||||||
"> 68060\n",
|
|
||||||
"> Pentium\n",
|
|
||||||
"> PowerPC\n",
|
|
||||||
"\n",
|
|
||||||
"060 fastest, then Pentium, with the first versions of the PowerPC\n",
|
|
||||||
"somewhere in the vicinity.\n",
|
|
||||||
"\n",
|
|
||||||
">And about clock speed: Does doubling the clock speed double the\n",
|
|
||||||
">overall processor speed? And fill in the __'s below:\n",
|
|
||||||
"> 68030 @ __ MHz = 68040 @ __ MHz\n",
|
|
||||||
"\n",
|
|
||||||
"No. Computer speed is only partly dependent of processor/clock speed.\n",
|
|
||||||
"Memory system speed play a large role as does video system speed and\n",
|
|
||||||
"I/O speed. As processor clock rates go up, the speed of the memory\n",
|
|
||||||
"system becomes the greatest factor in the overall system speed. If\n",
|
|
||||||
"you have a 50MHz processor, it can be reading another word from memory\n",
|
|
||||||
"every 20ns. Sure, you can put all 20ns memory in your computer, but\n",
|
|
||||||
"it will cost 10 times as much as the slower 80ns SIMMs.\n",
|
|
||||||
"\n",
|
|
||||||
"And roughly, the 68040 is twice as fast at a given clock\n",
|
|
||||||
"speed as is the 68030.\n",
|
|
||||||
"\n",
|
|
||||||
"-- \n",
|
|
||||||
"Ray Fischer \"Convictions are more dangerous enemies of truth\n",
|
|
||||||
"ray@netcom.com than lies.\" -- Friedrich Nietzsche\n",
|
|
||||||
"\n",
|
|
||||||
"0.4778416465020907\n",
|
|
||||||
"----------------------------------------------------------------------------------------------------\n",
|
|
||||||
"----------------------------------------------------------------------------------------------------\n",
|
|
||||||
"----------------------------------------------------------------------------------------------------\n",
|
|
||||||
"From: rvenkate@ux4.cso.uiuc.edu (Ravikuma Venkateswar)\n",
|
|
||||||
"Subject: Re: x86 ~= 680x0 ?? (How do they compare?)\n",
|
|
||||||
"Distribution: usa\n",
|
|
||||||
"Organization: University of Illinois at Urbana\n",
|
|
||||||
"Lines: 59\n",
|
|
||||||
"\n",
|
|
||||||
"ray@netcom.com (Ray Fischer) writes:\n",
|
|
||||||
"\n",
|
|
||||||
">dhk@ubbpc.uucp (Dave Kitabjian) writes ...\n",
|
|
||||||
">>I'm sure Intel and Motorola are competing neck-and-neck for \n",
|
|
||||||
">>crunch-power, but for a given clock speed, how do we rank the\n",
|
|
||||||
">>following (from 1st to 6th):\n",
|
|
||||||
">> 486\t\t68040\n",
|
|
||||||
">> 386\t\t68030\n",
|
|
||||||
">> 286\t\t68020\n",
|
|
||||||
"\n",
|
|
||||||
">040 486 030 386 020 286\n",
|
|
||||||
"\n",
|
|
||||||
"How about some numbers here? Some kind of benchmark?\n",
|
|
||||||
"If you want, let me start it - 486DX2-66 - 32 SPECint92, 16 SPECfp92 .\n",
|
|
||||||
"\n",
|
|
||||||
">>While you're at it, where will the following fit into the list:\n",
|
|
||||||
">> 68060\n",
|
|
||||||
">> Pentium\n",
|
|
||||||
">> PowerPC\n",
|
|
||||||
"\n",
|
|
||||||
">060 fastest, then Pentium, with the first versions of the PowerPC\n",
|
|
||||||
">somewhere in the vicinity.\n",
|
|
||||||
"\n",
|
|
||||||
"Numbers? Pentium @66MHz - 65 SPECint92, 57 SPECfp92 .\n",
|
|
||||||
"\t PowerPC @66MHz - 50 SPECint92, 80 SPECfp92 . (Note this is the 601)\n",
|
|
||||||
" (Alpha @150MHz - 74 SPECint92,126 SPECfp92 - just for comparison)\n",
|
|
||||||
"\n",
|
|
||||||
">>And about clock speed: Does doubling the clock speed double the\n",
|
|
||||||
">>overall processor speed? And fill in the __'s below:\n",
|
|
||||||
">> 68030 @ __ MHz = 68040 @ __ MHz\n",
|
|
||||||
"\n",
|
|
||||||
">No. Computer speed is only partly dependent of processor/clock speed.\n",
|
|
||||||
">Memory system speed play a large role as does video system speed and\n",
|
|
||||||
">I/O speed. As processor clock rates go up, the speed of the memory\n",
|
|
||||||
">system becomes the greatest factor in the overall system speed. If\n",
|
|
||||||
">you have a 50MHz processor, it can be reading another word from memory\n",
|
|
||||||
">every 20ns. Sure, you can put all 20ns memory in your computer, but\n",
|
|
||||||
">it will cost 10 times as much as the slower 80ns SIMMs.\n",
|
|
||||||
"\n",
|
|
||||||
"Not in a clock-doubled system. There isn't a doubling in performance, but\n",
|
|
||||||
"it _is_ quite significant. Maybe about a 70% increase in performance.\n",
|
|
||||||
"\n",
|
|
||||||
"Besides, for 0 wait state performance, you'd need a cache anyway. I mean,\n",
|
|
||||||
"who uses a processor that runs at the speed of 80ns SIMMs? Note that this\n",
|
|
||||||
"memory speed corresponds to a clock speed of 12.5 MHz.\n",
|
|
||||||
"\n",
|
|
||||||
">And roughly, the 68040 is twice as fast at a given clock\n",
|
|
||||||
">speed as is the 68030.\n",
|
|
||||||
"\n",
|
|
||||||
"Numbers?\n",
|
|
||||||
"\n",
|
|
||||||
">-- \n",
|
|
||||||
">Ray Fischer \"Convictions are more dangerous enemies of truth\n",
|
|
||||||
">ray@netcom.com than lies.\" -- Friedrich Nietzsche\n",
|
|
||||||
"-- \n",
|
|
||||||
"Ravikumar Venkateswar\n",
|
|
||||||
"rvenkate@uiuc.edu\n",
|
|
||||||
"\n",
|
|
||||||
"A pun is a no' blessed form of whit.\n",
|
|
||||||
"\n",
|
|
||||||
"0.44292082969477664\n",
|
|
||||||
"----------------------------------------------------------------------------------------------------\n",
|
|
||||||
"----------------------------------------------------------------------------------------------------\n",
|
|
||||||
"----------------------------------------------------------------------------------------------------\n",
|
|
||||||
"From: ray@netcom.com (Ray Fischer)\n",
|
|
||||||
"Subject: Re: x86 ~= 680x0 ?? (How do they compare?)\n",
|
|
||||||
"Organization: Netcom. San Jose, California\n",
|
|
||||||
"Distribution: usa\n",
|
|
||||||
"Lines: 30\n",
|
|
||||||
"\n",
|
|
||||||
"rvenkate@ux4.cso.uiuc.edu (Ravikuma Venkateswar) writes ...\n",
|
|
||||||
">ray@netcom.com (Ray Fischer) writes:\n",
|
|
||||||
">>040 486 030 386 020 286\n",
|
|
||||||
">\n",
|
|
||||||
">How about some numbers here? Some kind of benchmark?\n",
|
|
||||||
"\n",
|
|
||||||
"Benchmarks are for marketing dweebs and CPU envy. OK, if it will make\n",
|
|
||||||
"you happy, the 486 is faster than the 040. BFD. Both architectures\n",
|
|
||||||
"are nearing then end of their lifetimes. And especially with the x86\n",
|
|
||||||
"architecture: good riddance.\n",
|
|
||||||
"\n",
|
|
||||||
">Besides, for 0 wait state performance, you'd need a cache anyway. I mean,\n",
|
|
||||||
">who uses a processor that runs at the speed of 80ns SIMMs? Note that this\n",
|
|
||||||
">memory speed corresponds to a clock speed of 12.5 MHz.\n",
|
|
||||||
"\n",
|
|
||||||
"The point being the processor speed is only one of many aspects of a\n",
|
|
||||||
"computers performance. Clock speed, processor, memory speed, CPU\n",
|
|
||||||
"architecture, I/O systems, even the application program all contribute \n",
|
|
||||||
"to the overall system performance.\n",
|
|
||||||
"\n",
|
|
||||||
">>And roughly, the 68040 is twice as fast at a given clock\n",
|
|
||||||
">>speed as is the 68030.\n",
|
|
||||||
">\n",
|
|
||||||
">Numbers?\n",
|
|
||||||
"\n",
|
|
||||||
"Look them up yourself.\n",
|
|
||||||
"\n",
|
|
||||||
"-- \n",
|
|
||||||
"Ray Fischer \"Convictions are more dangerous enemies of truth\n",
|
|
||||||
"ray@netcom.com than lies.\" -- Friedrich Nietzsche\n",
|
|
||||||
"\n",
|
|
||||||
"0.3491800997095306\n",
|
|
||||||
"----------------------------------------------------------------------------------------------------\n",
|
|
||||||
"----------------------------------------------------------------------------------------------------\n",
|
|
||||||
"----------------------------------------------------------------------------------------------------\n",
|
|
||||||
"From: mb4008@cehp11 (Morgan J Bullard)\n",
|
|
||||||
"Subject: Re: speeding up windows\n",
|
|
||||||
"Keywords: speed\n",
|
|
||||||
"Organization: University of Illinois at Urbana\n",
|
|
||||||
"Lines: 30\n",
|
|
||||||
"\n",
|
|
||||||
"djserian@flash.LakeheadU.Ca (Reincarnation of Elvis) writes:\n",
|
|
||||||
"\n",
|
|
||||||
">I have a 386/33 with 8 megs of memory\n",
|
|
||||||
"\n",
|
|
||||||
">I have noticed that lately when I use programs like WpfW or Corel Draw\n",
|
|
||||||
">my computer \"boggs\" down and becomes really sluggish!\n",
|
|
||||||
"\n",
|
|
||||||
">What can I do to increase performance? What should I turn on or off\n",
|
|
||||||
"\n",
|
|
||||||
">Will not loading wallpapers or stuff like that help when it comes to\n",
|
|
||||||
">the running speed of windows and the programs that run under it?\n",
|
|
||||||
"\n",
|
|
||||||
">Thanx in advance\n",
|
|
||||||
"\n",
|
|
||||||
">Derek\n",
|
|
||||||
"\n",
|
|
||||||
"1) make sure your hard drive is defragmented. This will speed up more than \n",
|
|
||||||
" just windows BTW. Use something like Norton's or PC Tools.\n",
|
|
||||||
"2) I _think_ that leaving the wall paper out will use less RAM and therefore\n",
|
|
||||||
" will speed up your machine but I could very will be wrong on this.\n",
|
|
||||||
"There's a good chance you've already done this but if not it may speed things\n",
|
|
||||||
"up. good luck\n",
|
|
||||||
"\t\t\t\tMorgan Bullard mb4008@coewl.cen.uiuc.edu\n",
|
|
||||||
"\t\t\t\t\t or mjbb@uxa.cso.uiuc.edu\n",
|
|
||||||
"\n",
|
|
||||||
">--\n",
|
|
||||||
">$_ /|$Derek J.P. Serianni $ E-Mail : djserian@flash.lakeheadu.ca $ \n",
|
|
||||||
">$\\'o.O' $Sociologist $ It's 106 miles to Chicago,we've got a full tank$\n",
|
|
||||||
">$=(___)=$Lakehead University $ of gas, half a pack of cigarettes,it's dark,and$\n",
|
|
||||||
">$ U $Thunder Bay, Ontario$ we're wearing sunglasses. -Elwood Blues $ \n",
|
|
||||||
"\n",
|
|
||||||
"0.26949927393886913\n",
|
|
||||||
"----------------------------------------------------------------------------------------------------\n",
|
|
||||||
"----------------------------------------------------------------------------------------------------\n",
|
|
||||||
"----------------------------------------------------------------------------------------------------\n"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"source": [
|
|
||||||
"for i in range (1,5):\n",
|
|
||||||
" print(newsgroups[similarities.argsort()[0][-i]])\n",
|
|
||||||
" print(np.sort(similarities)[0,-i])\n",
|
|
||||||
" print('-'*100)\n",
|
|
||||||
" print('-'*100)\n",
|
|
||||||
" print('-'*100)"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"## Zadanie domowe\n",
|
|
||||||
"\n",
|
|
||||||
"\n",
|
|
||||||
"- Wybrać zbiór tekstowy, który ma conajmniej 10000 dokumentów (inny niż w tym przykładzie).\n",
|
|
||||||
"- Na jego podstawie stworzyć wyszukiwarkę bazującą na OKAPI BM25, tzn. system który dla podanej frazy podaje kilka (5-10) posortowanych najbardziej pasujących dokumentów razem ze scorami. Należy wypisywać też ilość zwracanych dokumentów, czyli takich z niezerowym scorem. Można korzystać z gotowych bibliotek do wektoryzacji dokumentów, należy jednak samemu zaimplementować OKAPI BM25. \n",
|
|
||||||
"- Znaleźć frazę (query), dla której wynik nie jest satysfakcjonujący.\n",
|
|
||||||
"- Poprawić wyszukiwarkę (np. poprzez zmianę preprocessingu tekstu, wektoryzer, zmianę parametrów algorytmu rankującego lub sam algorytm) tak, żeby zwracała satysfakcjonujące wyniki dla poprzedniej frazy. Należy zrobić inną zmianę niż w tym przykładzie, tylko wymyślić coś własnego.\n",
|
|
||||||
"- prezentować pracę na następnych zajęciach (14.04) odpowiadając na pytania:\n",
|
|
||||||
" - jak wygląda zbiór i system wyszukiwania przed zmianami\n",
|
|
||||||
" - dla jakiej frazy wyniki są niesatysfakcjonujące (pokazać wyniki)\n",
|
|
||||||
" - jakie zmiany zostały naniesione\n",
|
|
||||||
" - jak wyglądają wyniki wyszukiwania po zmianach\n",
|
|
||||||
" - jak zmiany wpłynęły na wyniki (1-2 zdania)\n",
|
|
||||||
" \n",
|
|
||||||
"Prezentacja powinna być maksymalnie prosta i trwać maksymalnie 2-3 minuty.\n",
|
|
||||||
"punktów do zdobycia: 60\n"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": null,
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [],
|
|
||||||
"source": []
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"metadata": {
|
|
||||||
"kernelspec": {
|
|
||||||
"display_name": "Python 3",
|
|
||||||
"language": "python",
|
|
||||||
"name": "python3"
|
|
||||||
},
|
|
||||||
"language_info": {
|
|
||||||
"codemirror_mode": {
|
|
||||||
"name": "ipython",
|
|
||||||
"version": 3
|
|
||||||
},
|
|
||||||
"file_extension": ".py",
|
|
||||||
"mimetype": "text/x-python",
|
|
||||||
"name": "python",
|
|
||||||
"nbconvert_exporter": "python",
|
|
||||||
"pygments_lexer": "ipython3",
|
|
||||||
"version": "3.8.3"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"nbformat": 4,
|
|
||||||
"nbformat_minor": 4
|
|
||||||
}
|
|
||||||
@ -1,5 +1,19 @@
|
|||||||
{
|
{
|
||||||
"cells": [
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 4. <i>Wyszukiwarki</i> [ćwiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
@ -81,11 +95,14 @@
|
|||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
"kernelspec": {
|
"kernelspec": {
|
||||||
"display_name": "Python 3",
|
"display_name": "Python 3",
|
||||||
"language": "python",
|
"language": "python",
|
||||||
"name": "python3"
|
"name": "python3"
|
||||||
},
|
},
|
||||||
|
"lang": "pl",
|
||||||
"language_info": {
|
"language_info": {
|
||||||
"codemirror_mode": {
|
"codemirror_mode": {
|
||||||
"name": "ipython",
|
"name": "ipython",
|
||||||
@ -96,8 +113,11 @@
|
|||||||
"name": "python",
|
"name": "python",
|
||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.9.1"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"subtitle": "4.wyszukiwarki[ćwiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
@ -1,5 +1,19 @@
|
|||||||
{
|
{
|
||||||
"cells": [
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 5. <i>Ekstrakcja informacji z dokumentów</i> [ćwiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
@ -213,11 +227,14 @@
|
|||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
"kernelspec": {
|
"kernelspec": {
|
||||||
"display_name": "Python 3",
|
"display_name": "Python 3",
|
||||||
"language": "python",
|
"language": "python",
|
||||||
"name": "python3"
|
"name": "python3"
|
||||||
},
|
},
|
||||||
|
"lang": "pl",
|
||||||
"language_info": {
|
"language_info": {
|
||||||
"codemirror_mode": {
|
"codemirror_mode": {
|
||||||
"name": "ipython",
|
"name": "ipython",
|
||||||
@ -229,7 +246,10 @@
|
|||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.8.3"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"subtitle": "5.ekEtrakcja informacji z dokumentCCow[ćwiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
@ -2,16 +2,32 @@
|
|||||||
"cells": [
|
"cells": [
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {
|
||||||
|
"collapsed": false
|
||||||
|
},
|
||||||
"source": [
|
"source": [
|
||||||
"# Zajęcia klasyfikacja"
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 6. <i>Klasyfikacja</i> [\u0107wiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"## Zbiór kleister"
|
"# Zaj\u0119cia klasyfikacja"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"## Zbi\u00f3r kleister"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -40,7 +56,7 @@
|
|||||||
"source": [
|
"source": [
|
||||||
"### Pytanie\n",
|
"### Pytanie\n",
|
||||||
"\n",
|
"\n",
|
||||||
"Czy jurysdykcja musi być zapisana explicite w umowie?"
|
"Czy jurysdykcja musi by\u0107 zapisana explicite w umowie?"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -147,7 +163,7 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"### Czy wszystkie stany muszą występować w zbiorze trenującym w zbiorze kleister?\n",
|
"### Czy wszystkie stany musz\u0105 wyst\u0119powa\u0107 w zbiorze trenuj\u0105cym w zbiorze kleister?\n",
|
||||||
"\n",
|
"\n",
|
||||||
"https://en.wikipedia.org/wiki/U.S._state\n",
|
"https://en.wikipedia.org/wiki/U.S._state\n",
|
||||||
"\n",
|
"\n",
|
||||||
@ -400,7 +416,7 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"### Co jeżeli nazwy klas nie występują explicite w zbiorach?"
|
"### Co je\u017celi nazwy klas nie wyst\u0119puj\u0105 explicite w zbiorach?"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -457,7 +473,7 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"### Sprytne podejście do klasyfikacji tekstu? Naiwny bayess"
|
"### Sprytne podej\u015bcie do klasyfikacji tekstu? Naiwny bayess"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -863,14 +879,14 @@
|
|||||||
"metadata": {},
|
"metadata": {},
|
||||||
"outputs": [],
|
"outputs": [],
|
||||||
"source": [
|
"source": [
|
||||||
"# listing dla get_prob2, słowo 'god'"
|
"# listing dla get_prob2, s\u0142owo 'god'"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"## założenie naiwnego bayesa"
|
"## za\u0142o\u017cenie naiwnego bayesa"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -884,7 +900,7 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"**przy założeniu o niezależności zmiennych losowych $word1$, $word2$, $word3$**:\n",
|
"**przy za\u0142o\u017ceniu o niezale\u017cno\u015bci zmiennych losowych $word1$, $word2$, $word3$**:\n",
|
||||||
"\n",
|
"\n",
|
||||||
"\n",
|
"\n",
|
||||||
"$P(word1, word2, word3|class) = P(word1|class)* P(word2|class) * P(word3|class)$"
|
"$P(word1, word2, word3|class) = P(word1|class)* P(word2|class) * P(word3|class)$"
|
||||||
@ -904,18 +920,18 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"## zadania domowe naiwny bayes1 ręcznie"
|
"## zadania domowe naiwny bayes1 r\u0119cznie"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"- analogicznie zaimplementować funkcję get_prob3(index, document_tokenized), argument document_tokenized ma być zbiorem słów dokumentu. funkcja ma być naiwnym klasyfikatorem bayesowskim (w przypadku wielu słów)\n",
|
"- analogicznie zaimplementowa\u0107 funkcj\u0119 get_prob3(index, document_tokenized), argument document_tokenized ma by\u0107 zbiorem s\u0142\u00f3w dokumentu. funkcja ma by\u0107 naiwnym klasyfikatorem bayesowskim (w przypadku wielu s\u0142\u00f3w)\n",
|
||||||
"- odpalić powyższy listing prawdopodobieństw z funkcją get_prob3 dla dokumentów: {'i','love','guns'} oraz {'is','there','life','after'\n",
|
"- odpali\u0107 powy\u017cszy listing prawdopodobie\u0144stw z funkcj\u0105 get_prob3 dla dokument\u00f3w: {'i','love','guns'} oraz {'is','there','life','after'\n",
|
||||||
",'death'}\n",
|
",'death'}\n",
|
||||||
"- zadanie proszę zrobić w jupyterze, wygenerować pdf (kod + wyniki odpalenia) i umieścić go jako zadanie w teams\n",
|
"- zadanie prosz\u0119 zrobi\u0107 w jupyterze, wygenerowa\u0107 pdf (kod + wyniki odpalenia) i umie\u015bci\u0107 go jako zadanie w teams\n",
|
||||||
"- termin 12.05, punktów: 40\n"
|
"- termin 12.05, punkt\u00f3w: 40\n"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -930,14 +946,14 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"- wybrać jedno z poniższych repozytoriów i je sforkować:\n",
|
"- wybra\u0107 jedno z poni\u017cszych repozytori\u00f3w i je sforkowa\u0107:\n",
|
||||||
" - https://git.wmi.amu.edu.pl/kubapok/paranormal-or-skeptic-ISI-public\n",
|
" - https://git.wmi.amu.edu.pl/kubapok/paranormal-or-skeptic-ISI-public\n",
|
||||||
" - https://git.wmi.amu.edu.pl/kubapok/sport-text-classification-ball-ISI-public\n",
|
" - https://git.wmi.amu.edu.pl/kubapok/sport-text-classification-ball-ISI-public\n",
|
||||||
"- stworzyć klasyfikator bazujący na naiwnym bayessie (może być gotowa biblioteka), może też korzystać z gotowych implementacji tfidf\n",
|
"- stworzy\u0107 klasyfikator bazuj\u0105cy na naiwnym bayessie (mo\u017ce by\u0107 gotowa biblioteka), mo\u017ce te\u017c korzysta\u0107 z gotowych implementacji tfidf\n",
|
||||||
"- stworzyć predykcje w plikach dev-0/out.tsv oraz test-A/out.tsv\n",
|
"- stworzy\u0107 predykcje w plikach dev-0/out.tsv oraz test-A/out.tsv\n",
|
||||||
"- wynik accuracy sprawdzony za pomocą narzędzia geval (patrz poprzednie zadanie) powinien wynosić conajmniej 0.67\n",
|
"- wynik accuracy sprawdzony za pomoc\u0105 narz\u0119dzia geval (patrz poprzednie zadanie) powinien wynosi\u0107 conajmniej 0.67\n",
|
||||||
"- proszę umieścić predykcję oraz skrypty generujące (w postaci tekstowej a nie jupyter) w repo, a w MS TEAMS umieścić link do swojego repo\n",
|
"- prosz\u0119 umie\u015bci\u0107 predykcj\u0119 oraz skrypty generuj\u0105ce (w postaci tekstowej a nie jupyter) w repo, a w MS TEAMS umie\u015bci\u0107 link do swojego repo\n",
|
||||||
"termin 12.05, 40 punktów\n"
|
"termin 12.05, 40 punkt\u00f3w\n"
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
@ -958,7 +974,13 @@
|
|||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.8.3"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
|
"lang": "pl",
|
||||||
|
"subtitle": "6.Klasyfikacja[\u0107wiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
@ -2,16 +2,32 @@
|
|||||||
"cells": [
|
"cells": [
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {
|
||||||
|
"collapsed": false
|
||||||
|
},
|
||||||
"source": [
|
"source": [
|
||||||
"# Zajęcia klasyfikacja"
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 6. <i>Klasyfikacja</i> [\u0107wiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"## Zbiór kleister"
|
"# Zaj\u0119cia klasyfikacja"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"## Zbi\u00f3r kleister"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -40,7 +56,7 @@
|
|||||||
"source": [
|
"source": [
|
||||||
"### Pytanie\n",
|
"### Pytanie\n",
|
||||||
"\n",
|
"\n",
|
||||||
"Czy jurysdykcja musi być zapisana explicite w umowie?"
|
"Czy jurysdykcja musi by\u0107 zapisana explicite w umowie?"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -147,7 +163,7 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"### Czy wszystkie stany muszą występować w zbiorze trenującym w zbiorze kleister?\n",
|
"### Czy wszystkie stany musz\u0105 wyst\u0119powa\u0107 w zbiorze trenuj\u0105cym w zbiorze kleister?\n",
|
||||||
"\n",
|
"\n",
|
||||||
"https://en.wikipedia.org/wiki/U.S._state\n",
|
"https://en.wikipedia.org/wiki/U.S._state\n",
|
||||||
"\n",
|
"\n",
|
||||||
@ -400,7 +416,7 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"### Co jeżeli nazwy klas nie występują explicite w zbiorach?"
|
"### Co je\u017celi nazwy klas nie wyst\u0119puj\u0105 explicite w zbiorach?"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -457,7 +473,7 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"### Sprytne podejście do klasyfikacji tekstu? Naiwny bayess"
|
"### Sprytne podej\u015bcie do klasyfikacji tekstu? Naiwny bayess"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -1017,7 +1033,7 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"## założenie naiwnego bayesa"
|
"## za\u0142o\u017cenie naiwnego bayesa"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -1031,7 +1047,7 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"**przy założeniu o niezależności zmiennych losowych $word1$, $word2$, $word3$**:\n",
|
"**przy za\u0142o\u017ceniu o niezale\u017cno\u015bci zmiennych losowych $word1$, $word2$, $word3$**:\n",
|
||||||
"\n",
|
"\n",
|
||||||
"\n",
|
"\n",
|
||||||
"$P(word1, word2, word3|class) = P(word1|class)* P(word2|class) * P(word3|class)$"
|
"$P(word1, word2, word3|class) = P(word1|class)* P(word2|class) * P(word3|class)$"
|
||||||
@ -1051,18 +1067,18 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"## zadania domowe naiwny bayes1 ręcznie"
|
"## zadania domowe naiwny bayes1 r\u0119cznie"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"- analogicznie zaimplementować funkcję get_prob3(index, document_tokenized), argument document_tokenized ma być zbiorem słów dokumentu. funkcja ma być naiwnym klasyfikatorem bayesowskim (w przypadku wielu słów)\n",
|
"- analogicznie zaimplementowa\u0107 funkcj\u0119 get_prob3(index, document_tokenized), argument document_tokenized ma by\u0107 zbiorem s\u0142\u00f3w dokumentu. funkcja ma by\u0107 naiwnym klasyfikatorem bayesowskim (w przypadku wielu s\u0142\u00f3w)\n",
|
||||||
"- odpalić powyższy listing prawdopodobieństw z funkcją get_prob3 dla dokumentów: {'i','love','guns'} oraz {'is','there','life','after'\n",
|
"- odpali\u0107 powy\u017cszy listing prawdopodobie\u0144stw z funkcj\u0105 get_prob3 dla dokument\u00f3w: {'i','love','guns'} oraz {'is','there','life','after'\n",
|
||||||
",'death'}\n",
|
",'death'}\n",
|
||||||
"- zadanie proszę zrobić w jupyterze, wygenerować pdf (kod + wyniki odpalenia) i umieścić go jako zadanie w teams\n",
|
"- zadanie prosz\u0119 zrobi\u0107 w jupyterze, wygenerowa\u0107 pdf (kod + wyniki odpalenia) i umie\u015bci\u0107 go jako zadanie w teams\n",
|
||||||
"- termin 12.05, punktów: 40\n"
|
"- termin 12.05, punkt\u00f3w: 40\n"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -1076,14 +1092,14 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"- wybrać jedno z poniższych repozytoriów i je sforkować:\n",
|
"- wybra\u0107 jedno z poni\u017cszych repozytori\u00f3w i je sforkowa\u0107:\n",
|
||||||
" - https://git.wmi.amu.edu.pl/kubapok/paranormal-or-skeptic-ISI-public\n",
|
" - https://git.wmi.amu.edu.pl/kubapok/paranormal-or-skeptic-ISI-public\n",
|
||||||
" - https://git.wmi.amu.edu.pl/kubapok/sport-text-classification-ball-ISI-public\n",
|
" - https://git.wmi.amu.edu.pl/kubapok/sport-text-classification-ball-ISI-public\n",
|
||||||
"- stworzyć klasyfikator bazujący na naiwnym bayessie (może być gotowa biblioteka), może też korzystać z gotowych implementacji tfidf\n",
|
"- stworzy\u0107 klasyfikator bazuj\u0105cy na naiwnym bayessie (mo\u017ce by\u0107 gotowa biblioteka), mo\u017ce te\u017c korzysta\u0107 z gotowych implementacji tfidf\n",
|
||||||
"- stworzyć predykcje w plikach dev-0/out.tsv oraz test-A/out.tsv\n",
|
"- stworzy\u0107 predykcje w plikach dev-0/out.tsv oraz test-A/out.tsv\n",
|
||||||
"- wynik accuracy sprawdzony za pomocą narzędzia geval (patrz poprzednie zadanie) powinien wynosić conajmniej 0.67\n",
|
"- wynik accuracy sprawdzony za pomoc\u0105 narz\u0119dzia geval (patrz poprzednie zadanie) powinien wynosi\u0107 conajmniej 0.67\n",
|
||||||
"- proszę umieścić predykcję oraz skrypty generujące (w postaci tekstowej a nie jupyter) w repo, a w MS TEAMS umieścić link do swojego repo\n",
|
"- prosz\u0119 umie\u015bci\u0107 predykcj\u0119 oraz skrypty generuj\u0105ce (w postaci tekstowej a nie jupyter) w repo, a w MS TEAMS umie\u015bci\u0107 link do swojego repo\n",
|
||||||
"termin 12.05, 40 punktów\n"
|
"termin 12.05, 40 punkt\u00f3w\n"
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
@ -1104,7 +1120,13 @@
|
|||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.8.3"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
|
"lang": "pl",
|
||||||
|
"subtitle": "6.Klasyfikacja[\u0107wiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
@ -1,5 +1,19 @@
|
|||||||
{
|
{
|
||||||
"cells": [
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 7. <i>Regresja liniowa</i> [ćwiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
@ -1046,11 +1060,14 @@
|
|||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
"kernelspec": {
|
"kernelspec": {
|
||||||
"display_name": "Python 3",
|
"display_name": "Python 3",
|
||||||
"language": "python",
|
"language": "python",
|
||||||
"name": "python3"
|
"name": "python3"
|
||||||
},
|
},
|
||||||
|
"lang": "pl",
|
||||||
"language_info": {
|
"language_info": {
|
||||||
"codemirror_mode": {
|
"codemirror_mode": {
|
||||||
"name": "ipython",
|
"name": "ipython",
|
||||||
@ -1061,8 +1078,11 @@
|
|||||||
"name": "python",
|
"name": "python",
|
||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.8.5"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"subtitle": "7.Regresja liniowa[ćwiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
@ -1,5 +1,19 @@
|
|||||||
{
|
{
|
||||||
"cells": [
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 7. <i>Regresja liniowa</i> [ćwiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
@ -1354,11 +1368,14 @@
|
|||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
"kernelspec": {
|
"kernelspec": {
|
||||||
"display_name": "Python 3",
|
"display_name": "Python 3",
|
||||||
"language": "python",
|
"language": "python",
|
||||||
"name": "python3"
|
"name": "python3"
|
||||||
},
|
},
|
||||||
|
"lang": "pl",
|
||||||
"language_info": {
|
"language_info": {
|
||||||
"codemirror_mode": {
|
"codemirror_mode": {
|
||||||
"name": "ipython",
|
"name": "ipython",
|
||||||
@ -1369,8 +1386,11 @@
|
|||||||
"name": "python",
|
"name": "python",
|
||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.8.5"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"subtitle": "7.Regresja liniowa[ćwiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
@ -1,5 +1,19 @@
|
|||||||
{
|
{
|
||||||
"cells": [
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 8. <i>Regresja logistyczna</i> [ćwiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
@ -1024,11 +1038,14 @@
|
|||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
"kernelspec": {
|
"kernelspec": {
|
||||||
"display_name": "Python 3",
|
"display_name": "Python 3",
|
||||||
"language": "python",
|
"language": "python",
|
||||||
"name": "python3"
|
"name": "python3"
|
||||||
},
|
},
|
||||||
|
"lang": "pl",
|
||||||
"language_info": {
|
"language_info": {
|
||||||
"codemirror_mode": {
|
"codemirror_mode": {
|
||||||
"name": "ipython",
|
"name": "ipython",
|
||||||
@ -1039,8 +1056,11 @@
|
|||||||
"name": "python",
|
"name": "python",
|
||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.8.5"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"subtitle": "8.Regresja logistyczna[ćwiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
@ -1,5 +1,19 @@
|
|||||||
{
|
{
|
||||||
"cells": [
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 8. <i>Regresja logistyczna</i> [ćwiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
@ -1216,11 +1230,14 @@
|
|||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
"kernelspec": {
|
"kernelspec": {
|
||||||
"display_name": "Python 3",
|
"display_name": "Python 3",
|
||||||
"language": "python",
|
"language": "python",
|
||||||
"name": "python3"
|
"name": "python3"
|
||||||
},
|
},
|
||||||
|
"lang": "pl",
|
||||||
"language_info": {
|
"language_info": {
|
||||||
"codemirror_mode": {
|
"codemirror_mode": {
|
||||||
"name": "ipython",
|
"name": "ipython",
|
||||||
@ -1231,8 +1248,11 @@
|
|||||||
"name": "python",
|
"name": "python",
|
||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.8.5"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"subtitle": "8.Regresja logistyczna[ćwiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
@ -1,5 +1,19 @@
|
|||||||
{
|
{
|
||||||
"cells": [
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 9. <i>Sequence labeling</i> [ćwiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
@ -12357,11 +12371,14 @@
|
|||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
"kernelspec": {
|
"kernelspec": {
|
||||||
"display_name": "Python 3",
|
"display_name": "Python 3",
|
||||||
"language": "python",
|
"language": "python",
|
||||||
"name": "python3"
|
"name": "python3"
|
||||||
},
|
},
|
||||||
|
"lang": "pl",
|
||||||
"language_info": {
|
"language_info": {
|
||||||
"codemirror_mode": {
|
"codemirror_mode": {
|
||||||
"name": "ipython",
|
"name": "ipython",
|
||||||
@ -12372,8 +12389,11 @@
|
|||||||
"name": "python",
|
"name": "python",
|
||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.8.5"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"subtitle": "9.Sequence labeling[ćwiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
@ -1,5 +1,19 @@
|
|||||||
{
|
{
|
||||||
"cells": [
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 9. <i>Sequence labeling</i> [ćwiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
@ -908,11 +922,14 @@
|
|||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
"kernelspec": {
|
"kernelspec": {
|
||||||
"display_name": "Python 3",
|
"display_name": "Python 3",
|
||||||
"language": "python",
|
"language": "python",
|
||||||
"name": "python3"
|
"name": "python3"
|
||||||
},
|
},
|
||||||
|
"lang": "pl",
|
||||||
"language_info": {
|
"language_info": {
|
||||||
"codemirror_mode": {
|
"codemirror_mode": {
|
||||||
"name": "ipython",
|
"name": "ipython",
|
||||||
@ -923,8 +940,11 @@
|
|||||||
"name": "python",
|
"name": "python",
|
||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.8.5"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"subtitle": "9.Sequence labeling[ćwiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
@ -1,5 +1,19 @@
|
|||||||
{
|
{
|
||||||
"cells": [
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 10. <i>CRF</i> [ćwiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
@ -404,11 +418,14 @@
|
|||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
"kernelspec": {
|
"kernelspec": {
|
||||||
"display_name": "Python 3",
|
"display_name": "Python 3",
|
||||||
"language": "python",
|
"language": "python",
|
||||||
"name": "python3"
|
"name": "python3"
|
||||||
},
|
},
|
||||||
|
"lang": "pl",
|
||||||
"language_info": {
|
"language_info": {
|
||||||
"codemirror_mode": {
|
"codemirror_mode": {
|
||||||
"name": "ipython",
|
"name": "ipython",
|
||||||
@ -419,8 +436,11 @@
|
|||||||
"name": "python",
|
"name": "python",
|
||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.8.5"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"subtitle": "10.CRF[ćwiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
@ -1,5 +1,19 @@
|
|||||||
{
|
{
|
||||||
"cells": [
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 11. <i>NER RNN</i> [ćwiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
@ -805,11 +819,14 @@
|
|||||||
}
|
}
|
||||||
],
|
],
|
||||||
"metadata": {
|
"metadata": {
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
"kernelspec": {
|
"kernelspec": {
|
||||||
"display_name": "Python 3",
|
"display_name": "Python 3",
|
||||||
"language": "python",
|
"language": "python",
|
||||||
"name": "python3"
|
"name": "python3"
|
||||||
},
|
},
|
||||||
|
"lang": "pl",
|
||||||
"language_info": {
|
"language_info": {
|
||||||
"codemirror_mode": {
|
"codemirror_mode": {
|
||||||
"name": "ipython",
|
"name": "ipython",
|
||||||
@ -820,8 +837,11 @@
|
|||||||
"name": "python",
|
"name": "python",
|
||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.8.5"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"subtitle": "11.NER RNN[ćwiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
@ -1,10 +1,26 @@
|
|||||||
{
|
{
|
||||||
"cells": [
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {
|
||||||
|
"collapsed": false
|
||||||
|
},
|
||||||
|
"source": [
|
||||||
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 11. <i>NER RNN</i> [\u0107wiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"### Podejście softmax z embeddingami na przykładzie NER"
|
"### Podej\u015bcie softmax z embeddingami na przyk\u0142adzie NER"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -773,18 +789,18 @@
|
|||||||
"source": [
|
"source": [
|
||||||
"## pytania\n",
|
"## pytania\n",
|
||||||
"\n",
|
"\n",
|
||||||
"- co zrobić z trenowaniem na batchach > 1 ?\n",
|
"- co zrobi\u0107 z trenowaniem na batchach > 1 ?\n",
|
||||||
"- co zrobić, żeby sieć uwzględniała następne tokeny, a nie tylko poprzednie?\n",
|
"- co zrobi\u0107, \u017ceby sie\u0107 uwzgl\u0119dnia\u0142a nast\u0119pne tokeny, a nie tylko poprzednie?\n",
|
||||||
"- w jaki sposób wykorzystać taką sieć do zadania zwykłej klasyfikacji?"
|
"- w jaki spos\u00f3b wykorzysta\u0107 tak\u0105 sie\u0107 do zadania zwyk\u0142ej klasyfikacji?"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"### Zadanie na zajęcia ( 20 minut)\n",
|
"### Zadanie na zaj\u0119cia ( 20 minut)\n",
|
||||||
"\n",
|
"\n",
|
||||||
"zmodyfikować sieć tak, żeby była używała dwuwarstwowej, dwukierunkowej warstwy GRU oraz dropoutu. Dropout ma nałożony na embeddingi.\n"
|
"zmodyfikowa\u0107 sie\u0107 tak, \u017ceby by\u0142a u\u017cywa\u0142a dwuwarstwowej, dwukierunkowej warstwy GRU oraz dropoutu. Dropout ma na\u0142o\u017cony na embeddingi.\n"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -1103,12 +1119,12 @@
|
|||||||
"## Zadanie domowe\n",
|
"## Zadanie domowe\n",
|
||||||
"\n",
|
"\n",
|
||||||
"\n",
|
"\n",
|
||||||
"- sklonować repozytorium https://git.wmi.amu.edu.pl/kubapok/en-ner-conll-2003\n",
|
"- sklonowa\u0107 repozytorium https://git.wmi.amu.edu.pl/kubapok/en-ner-conll-2003\n",
|
||||||
"- stworzyć model seq labelling bazujący na sieci neuronowej opisanej w punkcie niżej (można bazować na tym jupyterze lub nie).\n",
|
"- stworzy\u0107 model seq labelling bazuj\u0105cy na sieci neuronowej opisanej w punkcie ni\u017cej (mo\u017cna bazowa\u0107 na tym jupyterze lub nie).\n",
|
||||||
"- model sieci to GRU (o dowolnych parametrach) + CRF w pytorchu korzystając z modułu CRF z poprzednich zajęć- - stworzyć predykcje w plikach dev-0/out.tsv oraz test-A/out.tsv\n",
|
"- model sieci to GRU (o dowolnych parametrach) + CRF w pytorchu korzystaj\u0105c z modu\u0142u CRF z poprzednich zaj\u0119\u0107- - stworzy\u0107 predykcje w plikach dev-0/out.tsv oraz test-A/out.tsv\n",
|
||||||
"- wynik fscore sprawdzony za pomocą narzędzia geval (patrz poprzednie zadanie) powinien wynosić conajmniej 0.65\n",
|
"- wynik fscore sprawdzony za pomoc\u0105 narz\u0119dzia geval (patrz poprzednie zadanie) powinien wynosi\u0107 conajmniej 0.65\n",
|
||||||
"- proszę umieścić predykcję oraz skrypty generujące (w postaci tekstowej a nie jupyter) w repo, a w MS TEAMS umieścić link do swojego repo\n",
|
"- prosz\u0119 umie\u015bci\u0107 predykcj\u0119 oraz skrypty generuj\u0105ce (w postaci tekstowej a nie jupyter) w repo, a w MS TEAMS umie\u015bci\u0107 link do swojego repo\n",
|
||||||
"termin 22.06, 60 punktów, za najlepszy wynik- 100 punktów\n",
|
"termin 22.06, 60 punkt\u00f3w, za najlepszy wynik- 100 punkt\u00f3w\n",
|
||||||
" "
|
" "
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
@ -1130,7 +1146,13 @@
|
|||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.8.5"
|
"version": "3.8.5"
|
||||||
}
|
},
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
|
"lang": "pl",
|
||||||
|
"subtitle": "11.NER RNN[\u0107wiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
@ -1,5 +1,21 @@
|
|||||||
{
|
{
|
||||||
"cells": [
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {
|
||||||
|
"collapsed": false
|
||||||
|
},
|
||||||
|
"source": [
|
||||||
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 12. <i>Transformery</i> [\u0107wiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
@ -79,7 +95,7 @@
|
|||||||
"metadata": {},
|
"metadata": {},
|
||||||
"outputs": [],
|
"outputs": [],
|
||||||
"source": [
|
"source": [
|
||||||
"output = tokenizer.encode(\"Nie śpiewają piosenek: pracują leniwo,\")"
|
"output = tokenizer.encode(\"Nie \u015bpiewaj\u0105 piosenek: pracuj\u0105 leniwo,\")"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -110,7 +126,7 @@
|
|||||||
{
|
{
|
||||||
"data": {
|
"data": {
|
||||||
"text/plain": [
|
"text/plain": [
|
||||||
"['Nie', ' śpie', 'wają', ' pios', 'enek', ':', ' pracują', ' leni', 'wo', ',']"
|
"['Nie', ' \u015bpie', 'waj\u0105', ' pios', 'enek', ':', ' pracuj\u0105', ' leni', 'wo', ',']"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"execution_count": 8,
|
"execution_count": 8,
|
||||||
@ -136,11 +152,11 @@
|
|||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"## ZADANIE\n",
|
"## ZADANIE\n",
|
||||||
"stworzyć BPE tokenizer na podstawie https://git.wmi.amu.edu.pl/kubapok/lalka-lm/src/branch/master/train/train.tsv\n",
|
"stworzy\u0107 BPE tokenizer na podstawie https://git.wmi.amu.edu.pl/kubapok/lalka-lm/src/branch/master/train/train.tsv\n",
|
||||||
"i stworzyć stokenizowaną listę: \n",
|
"i stworzy\u0107 stokenizowan\u0105 list\u0119: \n",
|
||||||
"https://git.wmi.amu.edu.pl/kubapok/lalka-lm/src/branch/master/test-A/in.tsv\n",
|
"https://git.wmi.amu.edu.pl/kubapok/lalka-lm/src/branch/master/test-A/in.tsv\n",
|
||||||
"\n",
|
"\n",
|
||||||
"wybrać vocab_size = 8k, uwzględnić dodatkowe tokeny: BOS oraz EOS i wpleść je do zbioru testowego"
|
"wybra\u0107 vocab_size = 8k, uwzgl\u0119dni\u0107 dodatkowe tokeny: BOS oraz EOS i wple\u015b\u0107 je do zbioru testowego"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -163,7 +179,7 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"przykłady pochodzą częściowo z: https://huggingface.co/"
|
"przyk\u0142ady pochodz\u0105 cz\u0119\u015bciowo z: https://huggingface.co/"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -931,7 +947,7 @@
|
|||||||
{
|
{
|
||||||
"data": {
|
"data": {
|
||||||
"text/plain": [
|
"text/plain": [
|
||||||
"[{'generated_text': 'Hello, I\\'m a computer science student – and there\\'s very little that I do for anything else. I need to keep doing what I do.\"'},\n",
|
"[{'generated_text': 'Hello, I\\'m a computer science student \u2013 and there\\'s very little that I do for anything else. I need to keep doing what I do.\"'},\n",
|
||||||
" {'generated_text': \"Hello, I'm a computer science student. I am a Computer Science graduate and am very looking forward to the next year. I don't get paid\"},\n",
|
" {'generated_text': \"Hello, I'm a computer science student. I am a Computer Science graduate and am very looking forward to the next year. I don't get paid\"},\n",
|
||||||
" {'generated_text': \"Hello, I'm a computer science student. I love reading and writing computer programs and then having fun with them. I'm definitely an open and interested\"},\n",
|
" {'generated_text': \"Hello, I'm a computer science student. I love reading and writing computer programs and then having fun with them. I'm definitely an open and interested\"},\n",
|
||||||
" {'generated_text': 'Hello, I\\'m a computer science student.\"\\n\\n\"Hey, I got a big question, that\\'s how much your time is going to cost'},\n",
|
" {'generated_text': 'Hello, I\\'m a computer science student.\"\\n\\n\"Hey, I got a big question, that\\'s how much your time is going to cost'},\n",
|
||||||
@ -1154,12 +1170,12 @@
|
|||||||
"evalue": "invalid syntax (<ipython-input-1-fcb19aa882d9>, line 3)",
|
"evalue": "invalid syntax (<ipython-input-1-fcb19aa882d9>, line 3)",
|
||||||
"output_type": "error",
|
"output_type": "error",
|
||||||
"traceback": [
|
"traceback": [
|
||||||
"\u001b[0;36m File \u001b[0;32m\"<ipython-input-1-fcb19aa882d9>\"\u001b[0;36m, line \u001b[0;32m3\u001b[0m\n\u001b[0;31m przewidziać <mask> token w \"The world <MASK> II started in 1939\"\" wg dowolnego anglojęzycznego modelu\u001b[0m\n\u001b[0m ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
|
"\u001b[0;36m File \u001b[0;32m\"<ipython-input-1-fcb19aa882d9>\"\u001b[0;36m, line \u001b[0;32m3\u001b[0m\n\u001b[0;31m przewidzia\u0107 <mask> token w \"The world <MASK> II started in 1939\"\" wg dowolnego angloj\u0119zycznego modelu\u001b[0m\n\u001b[0m ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"source": [
|
"source": [
|
||||||
"przewidzieć <mask> token w \"The world <MASK> II started in 1939\"\" wg dowolnego anglojęzycznego modelu"
|
"przewidzie\u0107 <mask> token w \"The world <MASK> II started in 1939\"\" wg dowolnego angloj\u0119zycznego modelu"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -1234,13 +1250,13 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"- sforkować repozytorium: https://git.wmi.amu.edu.pl/kubapok/paranormal-or-skeptic-ISI-public\n",
|
"- sforkowa\u0107 repozytorium: https://git.wmi.amu.edu.pl/kubapok/paranormal-or-skeptic-ISI-public\n",
|
||||||
"- finetunować klasyfikator bazujący na jakieś pretrenowanej sieć typu transformer (np BERT, Roberta). Można użyć dowolnej biblioteki\n",
|
"- finetunowa\u0107 klasyfikator bazuj\u0105cy na jakie\u015b pretrenowanej sie\u0107 typu transformer (np BERT, Roberta). Mo\u017cna u\u017cy\u0107 dowolnej biblioteki\n",
|
||||||
" (np hugging face, fairseq)\n",
|
" (np hugging face, fairseq)\n",
|
||||||
"- stworzyć predykcje w plikach dev-0/out.tsv oraz test-A/out.tsv\n",
|
"- stworzy\u0107 predykcje w plikach dev-0/out.tsv oraz test-A/out.tsv\n",
|
||||||
"- wynik accuracy sprawdzony za pomocą narzędzia geval (patrz poprzednie zadanie) powinien wynosić conajmniej 0.67\n",
|
"- wynik accuracy sprawdzony za pomoc\u0105 narz\u0119dzia geval (patrz poprzednie zadanie) powinien wynosi\u0107 conajmniej 0.67\n",
|
||||||
"- proszę umieścić predykcję oraz skrypty generujące (w postaci tekstowej a nie jupyter) w repo, a w MS TEAMS umieścić link do swojego repo\n",
|
"- prosz\u0119 umie\u015bci\u0107 predykcj\u0119 oraz skrypty generuj\u0105ce (w postaci tekstowej a nie jupyter) w repo, a w MS TEAMS umie\u015bci\u0107 link do swojego repo\n",
|
||||||
"termin 22.06, 60 punktów\n"
|
"termin 22.06, 60 punkt\u00f3w\n"
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
@ -1261,7 +1277,13 @@
|
|||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.8.3"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
|
"lang": "pl",
|
||||||
|
"subtitle": "12.Transformery[\u0107wiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@ -1,5 +1,21 @@
|
|||||||
{
|
{
|
||||||
"cells": [
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {
|
||||||
|
"collapsed": false
|
||||||
|
},
|
||||||
|
"source": [
|
||||||
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 13. <i>Transformery 2</i> [\u0107wiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
@ -1864,14 +1880,14 @@
|
|||||||
"source": [
|
"source": [
|
||||||
"## Zadanie (10 minut)\n",
|
"## Zadanie (10 minut)\n",
|
||||||
"\n",
|
"\n",
|
||||||
"Za pomocą modelu en-fr przetłumacz dowolne zdanie z angielskiego na język francuski i sprawdź wagi atencji dla tego tłumaczenia"
|
"Za pomoc\u0105 modelu en-fr przet\u0142umacz dowolne zdanie z angielskiego na j\u0119zyk francuski i sprawd\u017a wagi atencji dla tego t\u0142umaczenia"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"### PRZYKŁAD: GPT3"
|
"### PRZYK\u0141AD: GPT3"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -1899,7 +1915,13 @@
|
|||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.8.3"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
|
"lang": "pl",
|
||||||
|
"subtitle": "13.Transformery 2[\u0107wiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
@ -1,5 +1,21 @@
|
|||||||
{
|
{
|
||||||
"cells": [
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {
|
||||||
|
"collapsed": false
|
||||||
|
},
|
||||||
|
"source": [
|
||||||
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 13. <i>Transformery 2</i> [\u0107wiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
@ -1864,7 +1880,7 @@
|
|||||||
"source": [
|
"source": [
|
||||||
"## Zadanie (10 minut)\n",
|
"## Zadanie (10 minut)\n",
|
||||||
"\n",
|
"\n",
|
||||||
"Za pomocą modelu en-fr przetłumacz dowolne zdanie z angielskiego na język francuski i sprawdź wagi atencji dla tego tłumaczenia"
|
"Za pomoc\u0105 modelu en-fr przet\u0142umacz dowolne zdanie z angielskiego na j\u0119zyk francuski i sprawd\u017a wagi atencji dla tego t\u0142umaczenia"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -1882,7 +1898,7 @@
|
|||||||
"metadata": {},
|
"metadata": {},
|
||||||
"outputs": [],
|
"outputs": [],
|
||||||
"source": [
|
"source": [
|
||||||
"TEXT_ENCODER = \"Although I still have fresh memories of my brother the elder Hamlet’s death, and though it was proper to mourn him throughout our kingdom, life still goes on—I think it’s wise to mourn him while also thinking about my own well being.\""
|
"TEXT_ENCODER = \"Although I still have fresh memories of my brother the elder Hamlet\u2019s death, and though it was proper to mourn him throughout our kingdom, life still goes on\u2014I think it\u2019s wise to mourn him while also thinking about my own well being.\""
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -1926,7 +1942,7 @@
|
|||||||
{
|
{
|
||||||
"data": {
|
"data": {
|
||||||
"text/plain": [
|
"text/plain": [
|
||||||
"\"<pad> Bien que j'aie encore de nouveaux souvenirs de la mort de mon frère Hamlet, l'aîné, et bien qu'il fût approprié de le pleurer dans tout notre royaume,\""
|
"\"<pad> Bien que j'aie encore de nouveaux souvenirs de la mort de mon fr\u00e8re Hamlet, l'a\u00een\u00e9, et bien qu'il f\u00fbt appropri\u00e9 de le pleurer dans tout notre royaume,\""
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"execution_count": 18,
|
"execution_count": 18,
|
||||||
@ -2399,7 +2415,7 @@
|
|||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"### PRZYKŁAD: GPT3"
|
"### PRZYK\u0141AD: GPT3"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@ -2427,7 +2443,13 @@
|
|||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.8.3"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
|
"lang": "pl",
|
||||||
|
"subtitle": "13.Transformery 2[\u0107wiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
@ -1,13 +1,29 @@
|
|||||||
{
|
{
|
||||||
"cells": [
|
"cells": [
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"metadata": {
|
||||||
|
"collapsed": false
|
||||||
|
},
|
||||||
|
"source": [
|
||||||
|
"\n",
|
||||||
|
"<div class=\"alert alert-block alert-info\">\n",
|
||||||
|
"<h1> Ekstrakcja informacji </h1>\n",
|
||||||
|
"<h2> 14. <i>Ekstrakcja informacji seq2seq</i> [\u0107wiczenia]</h2> \n",
|
||||||
|
"<h3> Jakub Pokrywka (2021)</h3>\n",
|
||||||
|
"</div>\n",
|
||||||
|
"\n",
|
||||||
|
""
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"source": [
|
"source": [
|
||||||
"### SIMILARITY SEARCH\n",
|
"### SIMILARITY SEARCH\n",
|
||||||
"1. zainstaluj faiss i zrób tutorial: https://github.com/facebookresearch/faiss\n",
|
"1. zainstaluj faiss i zr\u00f3b tutorial: https://github.com/facebookresearch/faiss\n",
|
||||||
"2. wczytaj treści artykułów z BBC News Train.csv\n",
|
"2. wczytaj tre\u015bci artyku\u0142\u00f3w z BBC News Train.csv\n",
|
||||||
"3. Użyj któregoś z transformerów (możesz użyć biblioteki sentence-transformers) do stworzenia embeddingów dokumentów\n",
|
"3. U\u017cyj kt\u00f3rego\u015b z transformer\u00f3w (mo\u017cesz u\u017cy\u0107 biblioteki sentence-transformers) do stworzenia embedding\u00f3w dokument\u00f3w\n",
|
||||||
"4. wczytaj embeddingi do bazy danych faiss\n",
|
"4. wczytaj embeddingi do bazy danych faiss\n",
|
||||||
"5. wyszukaj query 'consumer electronics market'"
|
"5. wyszukaj query 'consumer electronics market'"
|
||||||
]
|
]
|
||||||
@ -234,7 +250,7 @@
|
|||||||
{
|
{
|
||||||
"data": {
|
"data": {
|
||||||
"text/plain": [
|
"text/plain": [
|
||||||
"'internet boom for gift shopping cyberspace is becoming a very popular destination for christmas shoppers. forecasts predict that british people will spend £4bn buying gifts online during the festive season an increase of 64% on 2003. surveys also show that the average amount that people are spending is rising as is the range of goods that they are happy to buy online. savvy shoppers are also using the net to find the hot presents that are all but sold out in high street stores. almost half of the uk population now shop online according to figures collected by the interactive media in retail group which represents web retailers. about 85% of this group 18m people expect to do a lot of their christmas gift buying online this year reports the industry group. on average each shopper will spend £220 and britons lead europe in their affection for online shopping. almost a third of all the money spent online this christmas will come out of british wallets and purses compared to 29% from german shoppers and only 4% from italian gift buyers. james roper director of the imrg said shoppers were now much happier to buy so-called big ticket items such as lcd television sets and digital cameras. mr roper added that many retailers were working hard to reassure consumers that online shopping was safe and that goods ordered as presents would arrive in time for christmas. he advised consumers to give shops a little more time than usual to fulfil orders given that online buying is proving so popular. a survey by hostway suggests that many men prefer to shop online to avoid the embarrassment of buying some types of presents such as lingerie for wives and girlfriends. much of this online shopping is likely to be done during work time according to research carried out by security firm saint bernard software. the research reveals that up to two working days will be lost by staff who do their shopping via their work computer. worst offenders will be those in the 18-35 age bracket suggests the research who will spend up to five hours per week in december browsing and buying at online shops. iggy fanlo chief revenue officer at shopping.com said that the growing numbers of people using broadband was driving interest in online shopping. when you consider narrowband and broadband the conversion to sale is two times higher he said. higher speeds meant that everything happened much faster he said which let people spend time browsing and finding out about products before they buy. the behaviour of online shoppers was also changing he said. the single biggest reason people went online before this year was price he said. the number one reason now is convenience. very few consumers click on the lowest price he said. they are looking for good prices and merchant reliability. consumer comments and reviews were also proving popular with shoppers keen to find out who had the most reliable customer service. data collected by ebay suggests that some smart shoppers are getting round the shortages of hot presents by buying them direct through the auction site. according to ebay uk there are now more than 150 robosapiens remote control robots for sale via the site. the robosapiens toy is almost impossible to find in online and offline stores. similarly many shoppers are turning to ebay to help them get hold of the hard-to-find slimline playstation 2 which many retailers are only selling as part of an expensive bundle. the high demand for the playstation 2 has meant that prices for it are being driven up. in shops the ps2 is supposed to sell for £104.99. in some ebay uk auctions the price has risen to more than double this figure. many people are also using ebay to get hold of gadgets not even released in this country. the portable version of the playstation has only just gone on sale in japan yet some enterprising ebay users are selling the device to uk gadget fans.'"
|
"'internet boom for gift shopping cyberspace is becoming a very popular destination for christmas shoppers. forecasts predict that british people will spend \u00a34bn buying gifts online during the festive season an increase of 64% on 2003. surveys also show that the average amount that people are spending is rising as is the range of goods that they are happy to buy online. savvy shoppers are also using the net to find the hot presents that are all but sold out in high street stores. almost half of the uk population now shop online according to figures collected by the interactive media in retail group which represents web retailers. about 85% of this group 18m people expect to do a lot of their christmas gift buying online this year reports the industry group. on average each shopper will spend \u00a3220 and britons lead europe in their affection for online shopping. almost a third of all the money spent online this christmas will come out of british wallets and purses compared to 29% from german shoppers and only 4% from italian gift buyers. james roper director of the imrg said shoppers were now much happier to buy so-called big ticket items such as lcd television sets and digital cameras. mr roper added that many retailers were working hard to reassure consumers that online shopping was safe and that goods ordered as presents would arrive in time for christmas. he advised consumers to give shops a little more time than usual to fulfil orders given that online buying is proving so popular. a survey by hostway suggests that many men prefer to shop online to avoid the embarrassment of buying some types of presents such as lingerie for wives and girlfriends. much of this online shopping is likely to be done during work time according to research carried out by security firm saint bernard software. the research reveals that up to two working days will be lost by staff who do their shopping via their work computer. worst offenders will be those in the 18-35 age bracket suggests the research who will spend up to five hours per week in december browsing and buying at online shops. iggy fanlo chief revenue officer at shopping.com said that the growing numbers of people using broadband was driving interest in online shopping. when you consider narrowband and broadband the conversion to sale is two times higher he said. higher speeds meant that everything happened much faster he said which let people spend time browsing and finding out about products before they buy. the behaviour of online shoppers was also changing he said. the single biggest reason people went online before this year was price he said. the number one reason now is convenience. very few consumers click on the lowest price he said. they are looking for good prices and merchant reliability. consumer comments and reviews were also proving popular with shoppers keen to find out who had the most reliable customer service. data collected by ebay suggests that some smart shoppers are getting round the shortages of hot presents by buying them direct through the auction site. according to ebay uk there are now more than 150 robosapiens remote control robots for sale via the site. the robosapiens toy is almost impossible to find in online and offline stores. similarly many shoppers are turning to ebay to help them get hold of the hard-to-find slimline playstation 2 which many retailers are only selling as part of an expensive bundle. the high demand for the playstation 2 has meant that prices for it are being driven up. in shops the ps2 is supposed to sell for \u00a3104.99. in some ebay uk auctions the price has risen to more than double this figure. many people are also using ebay to get hold of gadgets not even released in this country. the portable version of the playstation has only just gone on sale in japan yet some enterprising ebay users are selling the device to uk gadget fans.'"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
"execution_count": 39,
|
"execution_count": 39,
|
||||||
@ -264,7 +280,13 @@
|
|||||||
"nbconvert_exporter": "python",
|
"nbconvert_exporter": "python",
|
||||||
"pygments_lexer": "ipython3",
|
"pygments_lexer": "ipython3",
|
||||||
"version": "3.8.3"
|
"version": "3.8.3"
|
||||||
}
|
},
|
||||||
|
"author": "Jakub Pokrywka",
|
||||||
|
"email": "kubapok@wmi.amu.edu.pl",
|
||||||
|
"lang": "pl",
|
||||||
|
"subtitle": "14.Ekstrakcja informacji seq2seq[\u0107wiczenia]",
|
||||||
|
"title": "Ekstrakcja informacji",
|
||||||
|
"year": "2021"
|
||||||
},
|
},
|
||||||
"nbformat": 4,
|
"nbformat": 4,
|
||||||
"nbformat_minor": 4
|
"nbformat_minor": 4
|
||||||
|
|||||||
File diff suppressed because one or more lines are too long
Loading…
Reference in New Issue
Block a user